We only use `include_nigthly` where we are also using
`filter_beta_release_tasks`, so just change the later to include nightly.
Differential Revision: https://phabricator.services.mozilla.com/D9687
--HG--
extra : moz-landing-system : lando
The release-update-verify-config task requires that the versions passed
to it match up with what's been released. The version of Thunderbird
does not necessarily match the Gecko version it's based on.
Depends on D6509
Differential Revision: https://phabricator.services.mozilla.com/D6510
--HG--
extra : moz-landing-system : lando
Currently, release resources such as balrog, bouncer and CDN urls are
configured by project. However, all production branches use one value and all
other branches use another. Rather than duplicate lists of projects, add a
helper to identify production release branches.
Differential Revision: https://phabricator.services.mozilla.com/D5583
--HG--
extra : moz-landing-system : lando
Currently, `mach try fuzzy` generates a taskgraph that is configured exactly
like the most recent push to mozilla-central. This isn't always desirabe, so
pass some configuration down, to allow the taskgraph to behave differently.
Differential Revision: https://phabricator.services.mozilla.com/D2729
--HG--
extra : rebase_source : 99d6958b33211697227e65df17edc1eb337f63a4
extra : histedit_source : 69b5ff6805bc8409340eb71323a1f6fc637259d7
Now that 28db2c96ac69 has been uplifted, get parameters from the new index paths.
--HG--
extra : rebase_source : 17166c55ad85ee153196e1797fe027dc6895beb2
The decision task is used for everything built as part of a push (for
mozilla-central, this is firefox, devedition and fennec, as well as tasks that
aren't strictly part of any product). Thus, having `firefox` encoded as part of
the decision task doesn't make sense.
This changes the route from
index.gecko.v2.${repository.project}.latest.firefox.decision
to
index.gecko.v2.${repository.project}.latest.taskgraph.decision
while leaving the latter for backwards compatability with tools that expect it.
Differential Revision: https://phabricator.services.mozilla.com/D996
--HG--
extra : rebase_source : c4c4691bb4633225e5e37b21982b916f76353e27
extra : source : 6ef1607a3e63250eefbda5333d61fd338c23311d
This used to only be relevant to Devedition and Firefox releases.
In bug 1433536 we're going to add RC Fennec releases. Let's rename
the parameter now, for less parameters churn.
MozReview-Commit-ID: 28e1Y5FG4On
--HG--
extra : rebase_source : 59d41887d856481ab85bb8d2221dfcebca4430b0
extra : source : 4c96e312d88a3f7037c1eb39a3031d4baf997015
also clean up and move more config to the promotion config.
MozReview-Commit-ID: FmTWNNPcEaZ
--HG--
extra : rebase_source : 40431217fafb6796dbd65c7dfeab0e891ac1bbd4
extra : source : 0f5418a83477c1b6b221e4d28515792410e504d0
Add support for the three firefox and devedition relpro flavors (we
could probably reduce devedition to 2).
Also, instead of defining which kinds to use from the previous graph
in `previous_graph_kinds`, specify which kinds to rebuild (ignore)
from the previous graph in `rebuild_kinds`. This list will be much
smaller (currently empty).
MozReview-Commit-ID: 5rH1TW7GbAD
--HG--
extra : rebase_source : b4294a0d17a99b2ffd48f5d62821c724324b242c
extra : histedit_source : b3fe87101e4595f5fc5c7daaa2d4a0bd88418667
Instead of relying on environment variables, pass these in as parameters.
MozReview-Commit-ID: An58Bu2kd1g
--HG--
extra : rebase_source : 9c7b6beb13b676a3376a897f2c8143cc042b8276
Add a release promotion custom action for releng's TC release promotion migration work.
This action generates a graph dependent on previously built tasks. To track these, we add the `do_not_optimize` and `existing_tasks` parameters. The `do_not_optimize` parameter specifies tasks that we want to explicitly exclude from taskgraph optimization. The `existing_tasks` parameter specifies a label-to-taskid map for tasks from previous graphs.
MozReview-Commit-ID: 1vKrNUavM4V
--HG--
extra : rebase_source : b8ba95d270aafe1464c2b3bfc318b9568500a7a1
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
--HG--
extra : rebase_source : 25e9966696d78d899783d9f38533d5ae66f9ccb9
extra : source : b53ff084c2d7968a1d9864d1343f2d9381fb652b
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
--HG--
extra : rebase_source : 03a10610aa3337269fe76a1196bb9b1665e1ab20
extra : source : b53ff084c2d7968a1d9864d1343f2d9381fb652b
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
--HG--
extra : rebase_source : bb1d194b49caa4b69ce8faf9f324c80850d15d3f
extra : histedit_source : a933e0a6f8819b8d0e9220ecc10c34d096a0d477
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
--HG--
extra : rebase_source : fdd8c3cfb9abf759a3c43c5713e62e4772c5bd06
This provides a mechanism to modify the behaviour of tasks from a try push. The try_task_config.json
looks something like:
{
"tasks": ["build-linux64/opt", "test-linux64/opt-mochitest-e10s-1"],
"templates": {
"artifact": {"enabled": 1}
}
}
This tells taskgraph to apply the 'artifact' template to all tasks. Templates are JSONe based
.yml files that live under taskcluster/taskgraph/templates. Taskgraph will render every template
against every task definition. The templates themselves can then use JSONe condition statements to
filter out which tasks they should or shouldn't apply to.
MozReview-Commit-ID: J8HVZzOt4mX
--HG--
extra : rebase_source : 95a78bc56d3f90ff1b34aabd84ed92aff1e3d954
The purpose of this parameter has been superseded by the `include_nightly`
property.
MozReview-Commit-ID: 4iXQsv9Drqg
--HG--
extra : rebase_source : c94142282909a88c29fe6809d87721bef1f198c2
This patch enables `run-on-projects` to work appropriately for
nightly builds and tests. Initially, we were setting an empty
`run-on-projects` for nightly `build_platform`s, then explicitly
targeting the platforms in nightly-specific `target_task_method`s.
Instead, this patch enables nightlies to `run-on-projects` everywhere,
but governs the use of nightlies by either the `include_nightly`
parameter, or the `--include-nightly` try option. This lets us filter
nightly-related `target_task_method`s against `run-on-projects` without
losing all nightly tasks.
Then, enable spidermonkey tests by removing optimization from beta and
release. This patch also enables everything then disables specific
tasks, rather than disabling everything and enabling specific tasks.
Since we're beginning with a `filter_for_project` call, we should be
able to reduce these if blocks to zero over time, if desired.
MozReview-Commit-ID: A9tolynaChF
--HG--
extra : rebase_source : 3465ee2c714de3e0359f14109096fc94de27aadf
Various modules under taskcluster are doing ad-hoc url formatting or
requests to taskcluster services. While we could use the taskcluster
client python module, it's kind of overkill for the simple requests done
here. So instead of vendoring that module, create a smaller one with
a limited set of functions we need.
This changes the behavior of the get_artifact function to return a
file-like object when the file is neither a json nor a yaml, but that
branch was never used (and was actually returning an unassigned
variable, so it was broken anyways).
At the same time, make the function that does HTTP requests more
error-resistant, using urllib3's Retry with a backoff factor.
Also add a function that retrieves the list of artifacts, that while
currently unused, will be used by `mach artifact` shortly.
--HG--
extra : rebase_source : d7ef633e8e5041dc8450f3ff2f3751c85f144cdc
Various modules under taskcluster are doing ad-hoc url formatting or
requests to taskcluster services. While we could use the taskcluster
client python module, it's kind of overkill for the simple requests done
here. So instead of vendoring that module, create a smaller one with
a limited set of functions we need.
This changes the behavior of the get_artifact function to return a
file-like object when the file is neither a json nor a yaml, but that
branch was never used (and was actually returning an unassigned
variable, so it was broken anyways).
At the same time, make the function that does HTTP requests more
error-resistant, using urllib3's Retry with a backoff factor.
Also add a function that retrieves the list of artifacts, that while
currently unused, will be used by `mach artifact` shortly.
--HG--
extra : rebase_source : 06777dea62e884f546a5b951baad80fd8aec1f1e
Previously, we ran a single "target task" function to mutate the full
task graph into a subset based on input parameters (try syntax,
repository being built for, etc). This concept is useful. But
the implementation was limiting because we could only have a single
"target tasks" function.
This commit introduces the concept of "filters." They conceptually
do the same thing as "target tasks methods" but you can run more than
1 of them.
Filters are simply functions that examine an input graph+parameters
and emit nodes that should be retained. Filters, like target tasks
methods, are defined via decorated functions in a module.
TaskGraphGenerator has been converted to use filters. The list of
defined filters can be defined in the parameters dict passed into
TaskGraphGenerator. A default filter list is provided in decision.py.
The intent is to eventually convert target tasks to filters. Until
that happens, we always run the registered target tasks method via
a filter proxy function.
No new tests have been added because we don't yet have any
functionality relying explicitly on filters. Tests will be added in
a subsequent commit once we add a new filter.
While I was here, I also snuck in some logging on the size of the
graphs.
MozReview-Commit-ID: ERn2hIYbMRp
--HG--
extra : rebase_source : 36b8e86aa64b2f52b03b31b5497759b0009fb921
