We add the following command line options to Taskcluster try syntax:
--spsProfile: enable profile mode.
--rebuild-talos <N>: retrigger talos tests N times.
--setenv <VAR>=<val>: add extra environments variables.
--tag <TAG>: run tests only the tag TAG.
--no-retry: doesn't retry failed jobs.
We have a chicken-egg problem, as we first generate the full task graph
and then parse the try message. But the graph generation step needs to
know the try message to process the aforementioned options. The
solution is to parse the message before graph generation and then
pass the command line options to the transforms. Then, each transform
can look at the option that interests it and process it accordingly.
The message parse function is configured in kind.yml, which gives some
flexibility for future implementations of alternative syntaxes.
MozReview-Commit-ID: EQlE6q5E8z7
--HG--
extra : rebase_source : 4b7323cd915e8ef9820816015b4b45524811eaf1
While we don't have physical machines running taskcluster-worker, we
want to schedule talos tests on Taskcluster and run them on buildbot.
buildbot-bridge acts as a worker that runs its tasks in buildbot
machines. Any talos task is redirected to buildbot-bridge.
MozReview-Commit-ID: Im9IhhT0Mr1
--HG--
extra : rebase_source : ede38a683abdf411612e885f6985c79b0532cb46
This commit introduces support for performing build and test variations
with Stylo enabled.
From a high level, this is similar to the e10s/non-e10s variations that
we currently run. However, it is different in that Stylo is a build
time (not run time) flag. So, we need to perform separate builds for
Stylo.
In the current implementation of the task graph, we can either
introduce a new platform or a new build type/configuration. I've
chosen to introduce a new platform if nothing else so the Treeherder
results clearly separate Stylo from non-Stylo. This decision can be
revisited in the future once Stylo becomes more stable and visible
outside the "stylo" repository.
Stylo functionality is limited to layout functionality and Stylo
developers aren't very interested in many tests not related to layout
at the moment. So we introduce a new test set running a limited
functionality of tests in Stylo builds.
In the current state of this commit, Stylo builds will permafail.
This is because Stylo builds require a vendored copy of Servo. Logic
for pruning the task graph of Stylo/Servo related tasks will be
implemented in subsequent commits.
MozReview-Commit-ID: JivhkpXDjIw
--HG--
extra : rebase_source : fdc9a5300c0bfa27133478fc456bc618f26fb9b2
Job tasks are tasks that are not tied to a specific build or test. As such, they cannot be scheduled
with the regular -p/-u try syntax options. There exists a -j try syntax option, to schedule them, which
defaults to running "all" of them if not specified.
However, there is a bug here where they will only default to "all" if a try syntax exists in the commit
message. They will not be considered if a developer pushes to try without a try syntax. This happens
because self.jobs is initially initialized with '[]' and we use None to determine when to schedule "all"
later on.
I want to move towards a world without try syntax, so we should start improving the UX of the no try
syntax use case.
Note: When I say "schedule" here, I mean added to the target set. They may still be optimized away.
MozReview-Commit-ID: 4TrC84RGiaL
--HG--
extra : rebase_source : 8fd50113c37745bc10181e1311cc62d75485723a
This definitely isn't all of it. But I got the obvious chunks. Still
need to clean up mozharness.
MozReview-Commit-ID: JTZBydP3i2r
--HG--
extra : rebase_source : 782401359036751896ef6c153a81a06cb14031bb
This patch prevents tests which have the 'run_on_projects' attribute assigned to an empty set from running when the try message contains '-p all' and '-u all' together. It also makes the 'match_test' function a little more readable and updates the 'attributes.rst' document to reflect the changes that were made.
MozReview-Commit-ID: IMk0cmSza8U
--HG--
extra : rebase_source : 06d542cbec16bd8091bc282ad1b831fb4671e72a
If you want to run tests on Linux32 and Linux64, you use trychooser syntax like
"-u all[Ubuntu]" which taskcluster supports, but if you also built ASan, and
want tests on ASan, there's no supported documented way to get them. That sounds
like an edge-case, except that to get tests to run on WinXP or Win8 you have to
explicitly list them, so to get tests on every platform you build you have to
list all of them, and if ASan is one of them you have to somehow know to dig
around in the .yml and to know that linux64-asan is the thing to dig out.
MozReview-Commit-ID: 2REf0cUWmK8
--HG--
extra : rebase_source : cb77ff3e016939c94ac05964a6db809fe10aaa1a
The trychooser website advertises 'Android 4.3' as the alias to choose Android
tests in [] syntax, but without support for that in taskcluster, someone who
wants both opt and debug tests would need to guess, with no support or
documentation, that they should use [Android 4.3,android-4.3-arm7-api-15]
MozReview-Commit-ID: 4rEKcvqAfDH
--HG--
extra : rebase_source : 336d3a787569e14e5ae8608902e6e557ef2ce6c3
This leaves a lot of room for improvement, but gets the job done for now.
MozReview-Commit-ID: HmeTxZjOfOb
--HG--
extra : rebase_source : b6f55134c25cb670409e906da1acc7c0bd8d0dc0
This has some notes for future work on the task, but will work for now.
MozReview-Commit-ID: 7J4tQeKj3KJ
--HG--
extra : rebase_source : d171b5393fa340a47927e8b89ece93392ff23e56
This uses the run_on_projects attribute introduced earlier for most branches,
adjusts the `ash` method to handle that branch as the legacy implementation
did, and updates try syntax to match builds as well as tests.
In the process, this enables optimizing target tasks, meaning that tasks
specifically requested in the try syntax might be optimized. While this is
probably not ideal, it matches the existing behavior of try (where `-j all` is
the default but all jobs are set to run only when certain files have been
modified). This change can be reverted later, in a more advanced version of
try.
MozReview-Commit-ID: 5FYeUTAsafr
--HG--
extra : rebase_source : b358e0e7cd8a401c50009e63dd55c59489c9b75b
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
This adds support for the `-t`/`--talos` option, matching such jobs against
`talos_try_name`. There are no such tasks just yet.
MozReview-Commit-ID: FTEx7Nyyi9Z
--HG--
extra : rebase_source : 64f289ed18a90c4d2c6988935a5865b41367f976
The `taskgraph` package generates TaskCluster task graphs based on collections
of task "kinds". Initially, there is only one kind, the "legacy" kind, which
reads the YAML files from `testing/taskcluster/tasks` to generate the task
graph.
Try syntax is implemented by filtering the tasks in the taskgraph after it has
been created, then extending the result to include any prerequisite tasks.
A collection of `mach taskgraph` subcommands are provided for developers to
extend or debug the task-graph generation process.
MozReview-Commit-ID: 1TJCns4XxZ8
--HG--
rename : testing/taskcluster/docs/index.rst => taskcluster/docs/index.rst
extra : rebase_source : 7b9125281d66044db9bd8e4a1fade16136f384b9
extra : histedit_source : 47640d27080acda0279270babbcf33f5badb0d1c
