This parses the metadata files without evaluating the conditional
expressions, producing a JSON summary file in which each possible
condition is represented as a string.
Not all data from the metadata files appears in the summary, only
exceptional cases likely to be of interest to engineers and unlikely
to be on wpt.fyi e.g. tests that have lsan leaks or tests that crash,
not just tests that fail. This is intended to keep the filesize down.
Depends on D24177
Depends on D24177
Differential Revision: https://phabricator.services.mozilla.com/D24178
--HG--
extra : moz-landing-system : lando
Add addtest support for per-suite arguments and multiple files. Also
support opening the created test in an editor. This allowed supporting
the wpt suite and replaces `mach wpt-create`.
# Create a wpt test
./mach addtest testing/web-platform/tests/accelerometer/test.html
# Create a wpt reftest
./mach addtest --suite wpt-reftesttesting/web-platform/tests/css/example.html --ref example-ref.html
The files created will be opened in the default editor if --editor is
supplied or a specified editor if the argument is given a value.
Differential Revision: https://phabricator.services.mozilla.com/D26339
--HG--
extra : moz-landing-system : lando
This parses the metadata files without evaluating the conditional
expressions, producing a JSON summary file in which each possible
condition is represented as a string.
Not all data from the metadata files appears in the summary, only
exceptional cases likely to be of interest to engineers and unlikely
to be on wpt.fyi e.g. tests that have lsan leaks or tests that crash,
not just tests that fail. This is intended to keep the filesize down.
Depends on D24177
Differential Revision: https://phabricator.services.mozilla.com/D24178
--HG--
extra : moz-landing-system : lando
On CI we only want to log screenshots when something unexpected happens since anything
else is rather wasteful of resources. But locally getting screenshots for expected
failures seems helpful for debugging, so worth making the default. Hopefully this isn't
too confusing for people just checking if their patch regresses anything rather than
actively working on fixing failures.
Depends on D16973
Differential Revision: https://phabricator.services.mozilla.com/D16974
--HG--
extra : moz-landing-system : lando
When the logging setup moved to earlier in the setup we ended up setting the
option to enable tbpl-style screenshots from mach after the loggers were already
initalised. Move this to earlier in the command so this option starts working again.
Differential Revision: https://phabricator.services.mozilla.com/D16973
--HG--
extra : moz-landing-system : lando
Previously we assumed that the path for the lsan_suppressions.txt file was the same as the prefs path,
which is correct in CI but wrong for local builds. This adds a --lsan-dir command line argument and sets
it in mach to the correct value for local builds and uses the prefs directory as a default for backwards
compatibility
Differential Revision: https://phabricator.services.mozilla.com/D16024
--HG--
extra : moz-landing-system : lando
When running non-Firefox products in wpt we go through the run.py
script from upstream. Therefore we need to initialise the logger in
that module as well as the one in wptrunner now that it's been
properly converted to use logging rather than print statements.
In addition, we change the run.py module to allow passing in logging
defaults for the case where nothing is specified on the command line
since the behaviour for wpt upstream is different to the behaviour for
gecko.
Differential Revision: https://phabricator.services.mozilla.com/D13547
--HG--
extra : moz-landing-system : lando
The manifest data is used to determine which tests exist, and what type they are
Differential Revision: https://phabricator.services.mozilla.com/D12408
--HG--
extra : moz-landing-system : lando
The fast internal reftest harness doesn't work yet, so in the meantime
default to the functional but slower external harness. Also ensure
that the Ahem font is installed by default.
Depends on D12035
Differential Revision: https://phabricator.services.mozilla.com/D12036
--HG--
extra : moz-landing-system : lando
We need to pass in the extra kwargs that wpt run understands but
wptrunner does not. Ideally we could switch to using the run frontend
always and allow users to actually supply these kwargs, although that
might have some unwanted behaviour, and is a larger change. For now
this gets things working again.
Depends on D9081
Differential Revision: https://phabricator.services.mozilla.com/D9082
--HG--
extra : moz-landing-system : lando
This replaces a previous Firefox-only method except on Windows 7 where
it seems that we have some issues with the registry.
Differential Revision: https://phabricator.services.mozilla.com/D9091
--HG--
extra : moz-landing-system : lando
Without this, the user must manually install fennec with expanded permissions with `adb install -g <apk>`.
With this patch, `./mach wpt` on its own handles everything from emulator setup to test running.
Differential Revision: https://phabricator.services.mozilla.com/D8333
--HG--
extra : moz-landing-system : lando
The previous code split a lot of logic between the update and download
parts, and exposed two different mach commands. In order to simplify
things it's better to have a single entry point for manifest download
and update, and ensure that's called consistently.
Differential Revision: https://phabricator.services.mozilla.com/D7497
Before we were using a different logger for the manifest download and the actual test run.
This caused timestamps to get reset in a confusing way. Now create the logger early and
share it for all the subseteps.
Depends on D7171
Differential Revision: https://phabricator.services.mozilla.com/D7172
--HG--
extra : moz-landing-system : lando
As far as I know this was unused and hasn't been well maintained since it's
not that useful.
Differential Revision: https://phabricator.services.mozilla.com/D7171
--HG--
extra : moz-landing-system : lando
This ensures that we always start from a partial manifest where possible and also ensures that the
configuration files are correctly created (a refactor to create these irrespective of whether we
do a download would make sense, but this fixes the immediate problem)
Depends on D7088
Differential Revision: https://phabricator.services.mozilla.com/D7089
Changes the wpt manifest path to the topobjdir instead so it can be moved out of tree.
Other changes so that the manifest download and update, and |mach wpt| and |mach test <wpt-test>| work with the new path.
The manifest is also downloaded and updated when creating the tests-archive to ensure that it exists when we run tests on TC.
MozReview-Commit-ID: Fp6UsKJjhTU
Differential Revision: https://phabricator.services.mozilla.com/D5312
--HG--
extra : moz-landing-system : lando
Changes the wpt manifest path to the topobjdir instead so it can be moved out of tree.
Other changes so that the manifest download and update, and |mach wpt| and |mach test <wpt-test>| work with the new path.
The manifest is also downloaded and updated when creating the tests-archive to ensure that it exists when we run tests on TC.
MozReview-Commit-ID: Fp6UsKJjhTU
Differential Revision: https://phabricator.services.mozilla.com/D5312
--HG--
extra : moz-landing-system : lando
This adds some overhead and can be annoying since it requries explicitly aborting failed jobs.
The state storage isn't very useful for just udpating metadata (the typical gecko usecase), but
is useful for performing syncs (a typical servo usecase). Therefore add a --no-store-state option
and set it by default in the gecko mach frontend.
MozReview-Commit-ID: LhEcMkyuRHD
--HG--
extra : rebase_source : 0f5c22b2a1337f7ca370cae31514792528d234a3
These files are duplicates of each other, so for simplicity we may as well use the
upstream copies to avoid falling out of sync.
MozReview-Commit-ID: JpG3joT8NNJ
--HG--
extra : rebase_source : d06c8969a9f8871b078c22e72947f7cd7584fb00
This allows running web-platform-tests on Fennec given a running emulator.
(Which is how we expect the tests to run in automation as well -- the
android_emulator_unittest mozharness script takes care of emulator
start-up.) It also hooks up ./mach wpt.
wptrunner sets up a profile for Fennec, forwards the marionette port
and starts up Fennec, etc.
= Usage =
Set your mozconfig to build fennec.
Start an emulator: `./mach android-emulator --version x86`
Install fennec: `./mach build && ./mach package && ./mach install`
Run the tests:
```
./mach wpt --product=fennec --testtype=testharness
--certutil-binary path/to/host/os/certutil path/to/some/tests
```
Differential Revision: https://phabricator.services.mozilla.com/D1587
Profiling shows that switching to this library means we no longer
spend most of the update time parsing json (vs 80% or so before),
making other optimisations worthwhile. This is never used in
automation (except wptsync) so availability of the library in the
internal pypi isn't a problem.
MozReview-Commit-ID: U5gabb5lz8
We try to find fix_macosx_stack.py on the wrong path, which leads us can't run
wpt tests without setting stack fix directory manually (--stackfix-dir).
In this patch, we fix this issue by pointing the path to @topobjdir/dist/bin.
MozReview-Commit-ID: 8JzWWgVM6fM
--HG--
extra : rebase_source : a7822998d7668b976284852bdca86e70d615b611
Upstream wpt changed from having a wptrun script to a wpt script with
a run subcommand. This involved some internal movement of code which
broke the `mach wpt` command when used with a non-firefox
product. This commit changes the mach integration to be compatible
with the new upstream API.
MozReview-Commit-ID: 1hvmZedNHSX
--HG--
extra : rebase_source : decd24551c2c5eaaae21eb64c7c376110b3de5b7
Ahem.ttf is copied to $(DIST)/bin/firefox/fonts, but this file is broken due to text mode copy. So we should use binary mode instead.
MozReview-Commit-ID: KP7yNyPiejU
--HG--
extra : rebase_source : 2de749f458a6d4650f9044f1912ff97835c5b795
Create an emmpty reference file if one doesn't exist, and open it in
the editor.
MozReview-Commit-ID: 5aBLp2AdnzY
--HG--
extra : rebase_source : bae3df50689895e6ac02cc740acea57be5c24b2e
This confuses people and isn't needed to update the manifest these days.
MozReview-Commit-ID: 4t4zdXLMob5
--HG--
extra : rebase_source : 9c43570073b15d11b3a52c830f9f3fd1f9f063c7
Allow running |mach wpt| on one click loaners in order to run
web-platform-tests tests.
This implementation is just like the one for other testsuites using
thee packaged tests rather than the checkout that we get with wpt, at
least on Linux. That's also where the tests run from so it seems
reasonable for now. Moving to the checkout in the future could remove
some of the logic here by using a fake mozbuild environment so that
the testsuite itself doesn't have to implement anything much.
MozReview-Commit-ID: CaewrdjJ2ef
--HG--
extra : rebase_source : 491b8014d48f06ff5bd41b28cc985608981fbdf4
This ensures that developers can run the majority of tests with the
default config, but makes things a little more confusing for marionette
developers.
MozReview-Commit-ID: 9wd761ZgCyx
--HG--
extra : rebase_source : d601fa10d1896afd3c450b2d44a2024b9963096f
This is required for cases where files have not changed but the
manifest logic has changed.
MozReview-Commit-ID: E46HtouILS2
--HG--
extra : rebase_source : f9d350018752f866052cf6a7a1ffd8f1d5f4ee39
For Chrome and Edge we don't have any way to set the DNS configuration
to include web-platform.test, so we need to error if this isn't already set.
MozReview-Commit-ID: BHRsTiuV28x
--HG--
extra : rebase_source : cfd3c35a513f98b47a7ffc9328058e6d104d2b2e
