An error crept in, resulting in:
```
[task ...] InterpreterError: InterpreterError: infix: [..] expects integer [..] integer
```
At some point, `suite` became a string name and not an object with a
string `name` member. However, in the interim, the diversity of
`command` structures has made the template approach untenable.
Therefore, this commit converts `GeckoProfile` to a `TryConfig`. The
existing test clearly wasn't helpful, and it doesn't really map to a
`TryConfig` test, so it was removed.
Differential Revision: https://phabricator.services.mozilla.com/D41603
--HG--
extra : moz-landing-system : lando
Adds command line option for developers to run tests against experimental debian 10 image (from D42597).
This is an experimental flag and will be removed once debian 10 image is used for production CI tests.
Differential Revision: https://phabricator.services.mozilla.com/D44236
--HG--
extra : moz-landing-system : lando
This commit prepares the decks for turning specific Raptor tasks into
Raptor + browsertime tasks. The `--browsertime` flag to `mach try
...` flips the switch; eventually, the Raptor harness will recognize
the `--browsertime` flag and use browsertime to perform the pageload
measurements.
To run browsertime, we need:
1) Node.js
2) the browsertime `node_modules` (provided by the
`toolchain-browsertime` task)
3) ffmpeg (for producing videos from captured frames)
4) chromedriver (in the future, when targeting Chrome/Chromium)
5) geckodriver (provided by the `toolchain-*-geckodriver` tasks)
6) `PATH` configured
This commit arranges those things.
Since the configuration varies by test platform, and eventually we
expect the changes implemented by the flag to be moved into YAML task
definitions, we elect to use `by-test-platform` conditionals as much
as possible. The end expression is pleasant, thanks to
`evaluate_keyed_by`.
Handling PATH, however, is a rabbit hole. At this time, it's not
possible to use `fetch` task repackaging, because `releng-hardware`
doesn't support `zstandard` (Bug 1576244) and there's no appetite to
avoid `zstandard` entirely (Bug 1576698). Generally PATH is
configured using `mozharness` configuration files, which can execute
arbitrary Python and configure the PATH only for browsertime jobs.
However, the Raptor mozharness script itself runs the Raptor harness
in a stripped down environment, throwing away modifications to PATH.
It's not clear what impacts changing that has, so we leave it alone,
and add a `--browsertime-ffmpeg` flag and custom handling in the
Raptor harness. This can transition smoothly into a browsertime flag
(so that the PATH doesn't need to be set at all) and into a unified
interface for Raptor and `mach browsertime` to configure the
browsertime execution environment.
Differential Revision: https://phabricator.services.mozilla.com/D38781
--HG--
extra : moz-landing-system : lando
Stop running all Fennec functional (non-performance) tests:
- stop running all Android 4.3 tests
- switch android-em-7 cppunit and android-hw jittest from the Fennec apk to the
geckoview apk (no difference in behavior expected)
- stop running Android 7.0 marionette tests, since they also run against Fennec
- remove android-em-4.* references from taskcluster configs
- remove android instance: extra-large references from taskcluster configs,
since they only affect aws, which is no longer used for Android
Android-hw raptor tests running against Fennec remain; I will prepare a separate
patch for those.
Differential Revision: https://phabricator.services.mozilla.com/D43684
--HG--
extra : moz-landing-system : lando
For now, this just:
- updates the TreeHerder symbol
- includes the `--browsertime` extra option
The actual test jobs will react to the `--browsertime` extra option
and add dependencies, etc, as appropriate.
Differential Revision: https://phabricator.services.mozilla.com/D41605
--HG--
extra : moz-landing-system : lando
This uses the layers.d3d11.enable-blacklist pref to allow running WebRender on WARP.
Differential Revision: https://phabricator.services.mozilla.com/D42255
--HG--
extra : moz-landing-system : lando
This uses the layers.d3d11.enable-blacklist pref to allow running WebRender on WARP.
Differential Revision: https://phabricator.services.mozilla.com/D42255
--HG--
extra : moz-landing-system : lando
This uses the layers.d3d11.enable-blacklist pref to allow running WebRender on WARP.
Differential Revision: https://phabricator.services.mozilla.com/D42255
--HG--
extra : moz-landing-system : lando
This allows test tasks to declare a static optimization name, which can then be
swapped in and out from the optimize code without needing to update the
transforms. This will make it easier to change optimization strategies or run
experimental ones.
Differential Revision: https://phabricator.services.mozilla.com/D40204
--HG--
extra : moz-landing-system : lando
This uses the layers.d3d11.enable-blacklist pref to allow running WebRender on WARP.
Differential Revision: https://phabricator.services.mozilla.com/D42255
--HG--
extra : moz-landing-system : lando
This allows test tasks to declare a static optimization name, which can then be
swapped in and out from the optimize code without needing to update the
transforms. This will make it easier to change optimization strategies or run
experimental ones.
Differential Revision: https://phabricator.services.mozilla.com/D40204
--HG--
extra : moz-landing-system : lando
This was Gradle-only and then !Gradle-only. Now Gradle is required
and this is unused.
Differential Revision: https://phabricator.services.mozilla.com/D31381
--HG--
extra : moz-landing-system : lando
This turns on mochitests (except for gpu and remote) on Linux 64 (opt+debug) and
Win64 (opt only).
Differential Revision: https://phabricator.services.mozilla.com/D39917
--HG--
extra : moz-landing-system : lando
Add functionality for being able to send extra parameters for the test_url query of a test, directly from a taskcluster config.
Also, the PR adds logic in the `setup-raptor` taskgraph transform for dynamically changing the subset of youtube-playback tests based on the platform and project
Differential Revision: https://phabricator.services.mozilla.com/D39006
--HG--
extra : moz-landing-system : lando
This adds the mobile project name, like "geckoview" or "fennec" to the Android
task labels. This makes the firefox/geckoview/fennec distinction more obvious
in 'mach try fuzzy' and 'mach try chooser'; it does not appear to affect
try syntax. This also adds "geckoview"/"fennec" to the job names displayed in
the lower-left treeherder pane. Hopefully this will help to clarify which
Android application is under test for each task.
Differential Revision: https://phabricator.services.mozilla.com/D39093
--HG--
extra : moz-landing-system : lando
AWSY is built on marionette, so it inherits the option by default, we mostly
just need to propagate it properly. This also drops the --disable-webrender
option as it is now implied if --enable-webrender is not provided.
Differential Revision: https://phabricator.services.mozilla.com/D35863
--HG--
extra : moz-landing-system : lando
Changes:
- rename the task name from windows10-64-ux to `windows10-64-ref-hw-2017`
- change `hardware` worker type to use the new reference hardware
Differential Revision: https://phabricator.services.mozilla.com/D35262
--HG--
extra : moz-landing-system : lando
This drops the --disable-webrender option (which force-disables WR)
and treats the lack of an --enable-webrender as if --disable-webrender
was provided.
Differential Revision: https://phabricator.services.mozilla.com/D34622
--HG--
extra : moz-landing-system : lando
This allows us to fix a regression where -sw tasks were scheduled on autoland/inbound.
Differential Revision: https://phabricator.services.mozilla.com/D29259
--HG--
extra : moz-landing-system : lando
Currently we have the concept of a "suite" and a "flavour" in our task
configuration. Typically, the "suite" refers to the high-level test harness
like "mochitest" or "reftest", whereas the flavour is more specific, e.g
"browser-chrome-instrumentation" or "crashtest". However the line between suite
and flavour is not applied with any semblance of consistency which results in
inconsistent naming throughout the tree.
This patch gets rid of the concept of "flavours" entirely (at least when it
comes to task configuration). A suite is a type of test run, for example:
- mochitest-plain
- mochitest-devtools-chrome
- mochitest-browser-chrome-instrumentation
- jsreftest
- reftest
- firefox-ui-functional-remote
etc
There is no confusion here between suites and flavours because flavours don't
exist. However, there are a couple of places where we *do* need to know what
"test harness" is used to run a suite. These cases are:
1. For SCHEDULES moz.build rules
2. For the desktop_unittest.py mozharness script which takes arguments like
--mochitest-suite=browser (this is not a compelling use of this information
and should be refactored to work more like the android_emulator_unittest.py
script)
So to get this information, this patch introduces a new concept of a "category"
which is the overall "test harness" that runs the suite. For many suites, the
"category" is identical to the suite name. Unlike flavours, "categories" have
no bearing on how we call or refer to the suite.
Differential Revision: https://phabricator.services.mozilla.com/D27554
--HG--
extra : moz-landing-system : lando
We are starting to spin off more and more "variants" of test suites. These are
usually just duplicates of our pre-existing tasks, except with an additional
pref set.
Currently there are two variants (serviceworker-e10s and socketprocess-e10s),
but a third will be added soon (fission). This change ensures we handle these
types of requests in a consistent and well defined manner. It also splits tasks
in a loop, so we don't accidentally risk combinatorial explosion.
Variants should typically be reserved for very large changes that will impact
the entire codebase (think e10s).
Differential Revision: https://phabricator.services.mozilla.com/D28061
--HG--
extra : moz-landing-system : lando
Since e10s is the default configuration, we shouldn't explicitly mark things
with the "-e10s" suffix. Instead we should mark things that *don't* run with
'e10s. This patch removes '-e10s' from all treeherder group symbols and task
labels, adds the "-1proc" suffix to tasks that are non-e10s.
Differential Revision: https://phabricator.services.mozilla.com/D25958
--HG--
extra : moz-landing-system : lando
