Commit graph

216 commits

Author SHA1 Message Date
Rob Lemley
b30e42f4ec Bug 1715111 - is_backstop updates to support non-Firefox builds. r=taskgraph-reviewers,ahal
Replace the assumptions that are Firefox-specific with arguments so that
backstop builds can be used for Thunderbird CI. Specifically, the backstop
index path and integration projects need changing.

Differential Revision: https://phabricator.services.mozilla.com/D117170
2021-06-10 17:21:00 +00:00
Steve Fink
a45b73b47e Bug 1702310 - Add --gecko-profile-features and similar options to mach try fuzzy and mach raptor etc. to make it easier to generate marker-only profiles, configure threads and sampling interval, etc. r=perftest-reviewers,sparky
Differential Revision: https://phabricator.services.mozilla.com/D110451
2021-04-26 21:59:29 +00:00
Alexandru Michis
09b48c2afd Backed out changeset aab899c222a0 (bug 1702310) for causing talos failures. 2021-04-09 05:26:23 +03:00
Steve Fink
be4388946e Bug 1702310 - Add --gecko-profile-features and similar options to mach try fuzzy and mach raptor etc. to make it easier to generate marker-only profiles, configure threads and sampling interval, etc. r=perftest-reviewers,sparky
Differential Revision: https://phabricator.services.mozilla.com/D110451
2021-04-08 20:41:34 +00:00
Tom Ritter
5ab182e7e6 Bug 1674182 - Support a regex filter for./mach try auto r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D95186
2020-11-05 17:53:31 +00:00
Ricky Stewart
02a7b4ebdf Bug 1654103: Standardize on Black for Python code in mozilla-central.
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.

To produce this patch I did all of the following:

1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.

2. Run ./mach lint --linter black --fix

3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.

4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.

5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D94045
2020-10-26 18:34:53 +00:00
Bogdan Tara
da1098d4aa Backed out 10 changesets (bug 1654103, bug 1672023, bug 1518999) for PanZoomControllerTest.touchEventForResult gv-junit failures CLOSED TREE
Backed out changeset ff3fb0b4a512 (bug 1672023)
Backed out changeset e7834b600201 (bug 1654103)
Backed out changeset 807893ca8069 (bug 1518999)
Backed out changeset 13e6b92440e9 (bug 1518999)
Backed out changeset 8b2ac5a6c98a (bug 1518999)
Backed out changeset 575748295752 (bug 1518999)
Backed out changeset 65f07ce7b39b (bug 1518999)
Backed out changeset 4bb80556158d (bug 1518999)
Backed out changeset 8ac8461d7bd7 (bug 1518999)
Backed out changeset e8ba13ee17f5 (bug 1518999)
2020-10-24 03:36:18 +03:00
Ricky Stewart
c0cea3b0fa Bug 1654103: Standardize on Black for Python code in mozilla-central. r=remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers,devtools-backward-compat-reviewers,jgilbert,preferences-reviewers,sylvestre,maja_zf,webcompat-reviewers,denschub,ntim,whimboo,sparky
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.

To produce this patch I did all of the following:

1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.

2. Run ./mach lint --linter black --fix

3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.

4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.

5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D94045
2020-10-23 20:40:42 +00:00
Dorel Luca
1ff59cb7a3 Backed out changeset 7558c8821a07 (bug 1654103) for multiple failures. CLOSED TREE 2020-10-22 03:51:06 +03:00
Ricky Stewart
50762dacab Bug 1654103: Standardize on Black for Python code in mozilla-central. r=remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers,devtools-backward-compat-reviewers,jgilbert,preferences-reviewers,sylvestre,maja_zf,webcompat-reviewers,denschub,ntim,whimboo,sparky
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.

To produce this patch I did all of the following:

1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.

2. Run ./mach lint --linter black --fix

3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.

4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D94045
2020-10-21 21:27:27 +00:00
Andrew Halberstadt
9673d50ff2 Bug 1643689 - [taskgraph] enable manifest-scheduling on autoland, r=marco
Sets autoland to use the 'bugbug' test manifest loader. This is being enabled
as part of a temporary trial to see the impact it has on sheriffing.

Differential Revision: https://phabricator.services.mozilla.com/D90160
2020-09-30 13:27:14 +00:00
Andrew Halberstadt
c288f18538 Bug 1666809 - Insert decision task indexes directly via taskluster API rather than index-task, r=taskgraph-reviewers,aki
We started using the "backstop" index added by bug 1660506 to determine whether
a push should be a backstop based on a time interval. The problem is that this
index gets added by an index-task that runs after the decision task has
completed.  Therefore, if two pushes land at roughly the same time (i.e, the
second decision task starts before the first completes), then they can both
determine themselves as backstops.

This patch gets around the problem by inserting the "backstop" index as early
as possible (immediately after resolving parameters), so the chances of this
happening become very low. It's still theoretically possible that it could
happen again, but we don't need this to be 100% perfect. As long as it is rare,
it's good enough.

Depends on D91191

Differential Revision: https://phabricator.services.mozilla.com/D91192
2020-09-24 19:19:36 +00:00
Mihai Alexandru Michis
fa05fd89fa Backed out changeset 0b196026ed59 (bug 1643689) for causing issues with manifest scheduling.
CLOSED TREE
2020-09-17 11:26:12 +03:00
Andrew Halberstadt
e8970ddcb9 Bug 1643689 - [taskgraph] enable manifest-scheduling on autoland, r=marco
Sets autoland to use the 'bugbug' test manifest loader. This is being enabled
as part of a temporary trial to see the impact it has on sheriffing.

Depends on D90159

Differential Revision: https://phabricator.services.mozilla.com/D90160
2020-09-14 23:12:31 +00:00
Mihai Tabara
e47580fde2 Bug 1661673 - remove esr68 references.r=releng-reviewers,aki DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D88618
2020-08-31 07:59:28 +00:00
Andrew Halberstadt
3c28444d2d Bug 1660506 - Move backstop determination to the decision task and store it in a parameter, r=marco,taskgraph-reviewers,jmaher
We want to be able to retroactively tell whether a push was a backstop or not.
This patch stores whether or not a push was a "backstop" directly in the
parameters. The optimization strategy now simply returns 'not
params["backstop"]'.

For simplicity, I'm not counting the 'optimized-backstop' as a backstop. It's
unclear if we'll want to be able to detect these types of the pushes in the
future or not, but we can cross that bridge when we get there.

Differential Revision: https://phabricator.services.mozilla.com/D88151
2020-08-26 21:56:05 +00:00
Nick Thomas
5d6881468d Bug 1630809 - add initial support for partner attribution in release promotion r=aki
Renames the release_enable_partners parameter to release_enable_partner_repack, and adds release_enable_partner_attribution for attribution. This it to provide support for disabling them independently in main releases, and in respins.
Adds docs for attribution, update docs for repacks.
Hardwire values for the enable params for the respin flavors, other wise read from the input (defaulting to on in promotion, off otherwise).
Fixes up the rebuild-kinds for partner repacks so that they reflect the current set, although the top level may be all that is needed.

Differential Revision: https://phabricator.services.mozilla.com/D87727
2020-08-27 22:19:41 +00:00
Butkovits Atila
b8629b8d1e Backed out 9 changesets (bug 1630809, bug 1653476) for Gecko Decision failures. CLOSED TREE
Backed out changeset 02a27bfc76dd (bug 1653476)
Backed out changeset afb5df61943a (bug 1630809)
Backed out changeset 04628c1f98e9 (bug 1630809)
Backed out changeset 4b4d50e0b1bf (bug 1630809)
Backed out changeset 2fa2deb5c993 (bug 1630809)
Backed out changeset d6652114cac3 (bug 1630809)
Backed out changeset ad5e4caa3291 (bug 1630809)
Backed out changeset d3d841cd14f3 (bug 1630809)
Backed out changeset b3746502e227 (bug 1630809)
2020-08-28 01:15:03 +03:00
Nick Thomas
ba595e786b Bug 1630809 - add initial support for partner attribution in release promotion r=aki
Renames the release_enable_partners parameter to release_enable_partner_repack, and adds release_enable_partner_attribution for attribution. This it to provide support for disabling them independently in main releases, and in respins.
Adds docs for attribution, update docs for repacks.
Hardwire values for the enable params for the respin flavors, other wise read from the input (defaulting to on in promotion, off otherwise).
Fixes up the rebuild-kinds for partner repacks so that they reflect the current set, although the top level may be all that is needed.

Differential Revision: https://phabricator.services.mozilla.com/D87727
2020-08-27 05:28:53 +00:00
Razvan Maries
6fa43f1893 Backed out changeset 9251996ce8c9 (bug 1657939) for Gecko Decision Task failures. CLOSED TREE 2020-08-11 00:32:44 +03:00
Justin Wood
b78899c21d Bug 1657939 - Output an artifact that identifies replaced taskID's. r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D86379
2020-08-10 19:20:28 +00:00
Joel Maher
abf17d06e7 Bug 1643689 - disable 1st round of manifest scheduling. r=aryx
disable 1st round of manifest scheduling

Differential Revision: https://phabricator.services.mozilla.com/D85621
2020-07-31 20:15:34 +00:00
Andrew Halberstadt
b4a179f22b Bug 1643689 - [taskgraph] enable manifest-scheduling on autoland, r=marco
Sets autoland to use the 'bugbug' test manifest loader. This is being enabled
as part of a temporary trial to see the impact it has on sheriffing.

Differential Revision: https://phabricator.services.mozilla.com/D85011
2020-07-29 17:05:48 +00:00
Bogdan Tara
244125ea67 Backed out changeset 9be5f086895c (bug 1643689) for busting gecko decision task and causig bug 1655807 CLOSED TREE 2020-07-28 19:46:47 +03:00
Andrew Halberstadt
512734fa0a Bug 1643689 - [taskgraph] enable manifest-scheduling on autoland, r=marco
Sets autoland to use the 'bugbug' test manifest loader. This is being enabled
as part of a temporary trial to see the impact it has on sheriffing.

Differential Revision: https://phabricator.services.mozilla.com/D85011
2020-07-28 14:40:40 +00:00
Andrew Halberstadt
dfdb95e779 Bug 1636271 - Fix flake8 issue arising from merge conflict CLOSED TREE
Differential Revision: https://phabricator.services.mozilla.com/D85155
2020-07-28 14:15:27 +00:00
Andrew Halberstadt
8f7d414a21 Bug 1636271 - [taskgraph] Set autoland optimizations via per-project parameter, r=tomprince
This creates a new set of optimization strategies
(taskgraph.optimize:project.autoland) to use with autoland. Among other things,
it also means there's no need for the 'test-try' optimization as the autoland
strategies are no longer the default behaviour.

Differential Revision: https://phabricator.services.mozilla.com/D79705
2020-07-28 13:22:48 +00:00
Andrew Halberstadt
1468afcc00 Bug 1636271 - [taskgraph] Move 'optimize-strategies' from try_task_config.json to a parameter, r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D79704
2020-07-28 13:21:05 +00:00
Ben Hearsum
c0a843be6b Bug 1650208: run opt builds by default on autoland; shippable builds by default on central r=tomprince,ahal,marco
Differential Revision: https://phabricator.services.mozilla.com/D83186
2020-07-27 16:25:40 +00:00
Narcis Beleuzu
ef290a86e8 Backed out 3 changesets (bug 1636271) for breaking the Gecko Decision Task. CLOSED TREE
Backed out changeset 4b0f13fcf941 (bug 1636271)
Backed out changeset 4250f49877ba (bug 1636271)
Backed out changeset d3f93bd6b1f9 (bug 1636271)
2020-07-22 23:40:32 +03:00
Andrew Halberstadt
268c833ffc Bug 1636271 - [taskgraph] Set autoland optimizations via per-project parameter, r=tomprince
This creates a new set of optimization strategies
(taskgraph.optimize:project.autoland) to use with autoland. Among other things,
it also means there's no need for the 'test-try' optimization as the autoland
strategies are no longer the default behaviour.

Differential Revision: https://phabricator.services.mozilla.com/D79705
2020-07-22 17:09:02 +00:00
Andrew Halberstadt
8d40cc05d4 Bug 1636271 - [taskgraph] Move 'optimize-strategies' from try_task_config.json to a parameter, r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D79704
2020-07-22 15:36:55 +00:00
Tom Prince
2b53c0c278 Bug 1638990: [taskgraph] Apply appropriate change from python-modernize; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D84086
2020-07-20 19:27:46 +00:00
Tom Prince
bc2b9501bb Bug 1638990: [taskgraph] Make taskgraph.decision.write_artifact work on python 3; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D84083
2020-07-20 19:27:34 +00:00
Tom Prince
847aa1938f Bug 1638990: [taskgraph] Make test_util_runnable_jobs pass on python 3; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D84077
2020-07-20 19:27:09 +00:00
Tom Prince
d64ad0967b Bug 1626058: Add support for writting out artifacts from transforms in the decision task; r=Callek,ahal
To support using kaniko[1] for building images, we need to generate the docker
contexts in a seprate task from the docker-image task. Since we use the hash of
the context as the cached-task digest, we generate the context in the decision
task already, so this adds a way to write that out to be used by downstream
tasks.

[1] https://github.com/GoogleContainerTools/kaniko

Differential Revision: https://phabricator.services.mozilla.com/D77839
2020-07-07 19:56:11 +00:00
Rob Lemley
f518507ad0 Bug 1646843 - Add comm-esr78 support to taskgraph. r=Callek
Follow-up to bug 1646588 to support comm-esr78 repository.

Differential Revision: https://phabricator.services.mozilla.com/D80271
2020-06-18 20:42:03 +00:00
Justin Wood
f66f5a235e Bug 1646588 - Add esr78 support. r=aki
Differential Revision: https://phabricator.services.mozilla.com/D79731
2020-06-18 19:57:20 +00:00
Tom Prince
ca6c9020de Bug 1641971: [taskgraph] Always pass an explicit decision task id to create_tasks; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D77551
2020-06-03 02:35:03 +00:00
Tom Prince
9f3450be95 Bug 1641971: [taskgraph] Pass explicit decision task id to action.json generation; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D77550
2020-06-03 02:35:05 +00:00
Tom Prince
581eba974b Bug 1641971: Pass mach perftest options explicitly, rather than inpsect parameters; r=tarek
Differential Revision: https://phabricator.services.mozilla.com/D77548
2020-06-03 02:35:09 +00:00
Tom Prince
6617f03f05 Bug 1641971: Allow references to decision task via task-reference/artifact-reference; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D77547
2020-06-03 02:43:34 +00:00
Narcis Beleuzu
479c45bbea Backed out 7 changesets (bug 1641971) for breaking Gecko Decision Task opt . CLOSED TREE
Backed out changeset a039e60a26d2 (bug 1641971)
Backed out changeset e7d4c490c31a (bug 1641971)
Backed out changeset f2322981b294 (bug 1641971)
Backed out changeset d123b08d2c20 (bug 1641971)
Backed out changeset 57772c0b9b3e (bug 1641971)
Backed out changeset 11a887f73dbb (bug 1641971)
Backed out changeset e9c7a59d4bc5 (bug 1641971)
2020-06-03 05:30:51 +03:00
Tom Prince
f32e4a4f4d Bug 1641971: [taskgraph] Always pass an explicit decision task id to create_tasks; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D77551
2020-06-02 18:23:19 +00:00
Tom Prince
7c6a413ca9 Bug 1641971: [taskgraph] Pass explicit decision task id to action.json generation; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D77550
2020-06-02 18:23:20 +00:00
Tom Prince
11aedca9b2 Bug 1641971: Pass mach perftest options explicitly, rather than inpsect parameters; r=tarek
Differential Revision: https://phabricator.services.mozilla.com/D77548
2020-06-02 18:23:07 +00:00
Tom Prince
602da9cdcd Bug 1641971: Allow references to decision task via task-reference/artifact-reference; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D77547
2020-06-02 18:50:10 +00:00
Andrew Halberstadt
f0e5f6ed7b Bug 1633866 - [taskgraph] Create a system to choose how test manifests are loaded via a parameter, r=tomprince
Currently test manifests are loaded by instantiating a TestResolver and
traversing moz.build files to find manifests.

With 'manifest-scheduling', we'll want to grab the manifests directly from the
bugbug service instead. Initially we'll want to enable manifest-scheduling with
|mach try auto|, but not on autoland yet.

This patch will allow |mach try auto| to set the parameter that causes bugbug
to be used (see future commits in this bug).

Differential Revision: https://phabricator.services.mozilla.com/D76522
2020-05-29 18:51:14 +00:00
Andrew Halberstadt
e5ab1dc839 Bug 1641283 - [taskgraph] Upload bugbug push schedules API data to an artifact rather than the log, r=marco
Differential Revision: https://phabricator.services.mozilla.com/D77140
2020-05-29 14:39:38 +00:00
Tom Prince
35510c1f78 Bug 1628832: [KaiOS] Add kaios specific taskgraph paramters and target tasks; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D75243
2020-05-14 18:48:00 +00:00