Commit graph

102 commits

Author SHA1 Message Date
Marco Castelluccio
bd3f040c33 Bug 1476574 - Enable reset/dump for wpt and cleanup harness code for supporting reset/dump now that it is supported by all test suites. r=jmaher
--HG--
extra : source : c1b3950cc12f1642ad60338d2a8701e2b60131ea
extra : intermediate-source : 4ea7997194e9f995bb0b1e434524a0ea5596d758
extra : histedit_source : fffe672f456fd7c503792d2a3e99184c14bfd36c
2018-07-19 11:56:43 +02:00
Coroiu Cristina
46afadc431 Backed out 2 changesets (bug 1476574) for browser-chrome failures at browser/base/content/test/static/browser_all_files_referenced.js
Backed out changeset 4ea7997194e9 (bug 1476574)
Backed out changeset ec37892ce390 (bug 1476574)
2018-07-19 23:40:24 +03:00
Marco Castelluccio
57da0df505 Bug 1476574 - Enable reset/dump for wpt and cleanup harness code for supporting reset/dump now that it is supported by all test suites. r=jmaher
--HG--
extra : rebase_source : 554f640652e8a44e53d37ca8b4a139845efd75ec
extra : source : c1b3950cc12f1642ad60338d2a8701e2b60131ea
2018-07-19 11:56:43 +02:00
James Graham
e7b4476630 Bug 1354232 - Chunk by dir more for wpt on asan, r=maja_zf
This means that when we run these tests, the LSAN exceptions will have
the minimum possible scope.

MozReview-Commit-ID: 5aFDWqSiaNb
2018-07-19 16:32:17 +01:00
Joel Maher
8ee7eee7e1 Bug 1467837 - adjust some code for test-coverage gpu/chunks and minor cleanup in per test code. r=gbrown 2018-06-08 16:21:14 -04:00
Narcis Beleuzu
adf3d042d3 Backed out changeset a71f125a6d73 (bug 1464744) for wpt failures. CLOSED TREE 2018-06-04 02:58:20 +03:00
Xidorn Quan
8fc0bb9c61 Bug 1464744 followup - Enable wpt of css-scrollbars on all platforms. r=jgraham
--HG--
extra : source : 7e8608344bd8f270a32a9ba233d88e529365b0af
2018-06-04 09:03:19 +10:00
Gregory Mierzwinski
6b7a09faaf Bug 1465300 - Exit test coverage chunk after all suites are checked for baseline tests. r=jmaher
This patch fixes baseline test addition. It allows all baselines to run in all test suites. Previously, only the test suite which had too many tests executed in it, was searched to the end for any baselines. Now, we continue to all other test suites before we finish.

MozReview-Commit-ID: FWMAsHb22CO

--HG--
extra : rebase_source : 7dbcdb3977bd351367ff9fa7892e41c2b86a2dce
2018-05-29 22:11:00 -04:00
Joel Maher
9e5d9c55f5 Bug 1442790 - fix test-verify to not fail on: 'No checks run.'. r=ahal 2018-05-28 06:44:24 -04:00
Dorel Luca
047df7a932 Merge mozilla-inbound to mozilla-central. a=merge 2018-05-26 07:09:39 +03:00
Gregory Mierzwinski
6e9713d33a Bug 1462434 - Prevent baseline coverage tests from being skipped. r=jmaher
This patch prevents baseline coverage tests from being skipped when too many tests are being run.

MozReview-Commit-ID: JVTOYZAXbwf

--HG--
extra : rebase_source : dedd6a323445f030b60180805c6c6adf5d10771b
2018-05-23 12:41:52 -04:00
Chris AtLee
444afb9092 Bug 1237182: Remove BuildbotMixin in favour of AutomationMixin r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D1371

--HG--
rename : testing/mozharness/test/test_mozilla_buildbot.py => testing/mozharness/test/test_mozilla_automation.py
extra : rebase_source : 2fdec7a8b06c1b29ea47de1b368c08d9b8b75c3e
2018-05-16 12:51:37 -04:00
Joel Maher
8ff40400ce Bug 1461440 - test-verify should support MOZHARNESS_TEST_PATHS. r=gbrown 2018-05-23 09:59:51 -04:00
Chris AtLee
e3d628f8c0 Bug 1237182: Remove mozharness blobber mixin r=mtabara
--HG--
extra : rebase_source : 3a631d554fdd143aa5e06903c067b1907f40a82f
extra : source : aedcb763d9479b484ec4f5603e0f2c4efd0c9511
2018-05-11 18:16:08 -04:00
Chris AtLee
6e7f604585 Bug 1237182: Removing unused buildbot support r=Callek
MozReview-Commit-ID: EjclZgc864L

--HG--
extra : rebase_source : aebc03899ad88aa2838825c14a45c59e1e59be05
extra : source : ba91fec47e908619d9a8f550ee65cd78f67e3d59
2018-05-04 13:51:35 -04:00
Narcis Beleuzu
495eed2d41 Backed out changeset ba91fec47e90 (bug 1237182) for breaking the Windows 2012 Nightlies. a=backout 2018-05-12 17:33:18 +03:00
Chris AtLee
ef68c91c87 Bug 1237182: Removing unused buildbot support r=Callek
MozReview-Commit-ID: EjclZgc864L

--HG--
extra : rebase_source : 5ad7974462d14c00916fd0c88e06843e0f1f4276
2018-05-04 13:51:35 -04:00
Dorel Luca
979ac450c8 Backed out changeset 67d2aa892562 (bug 1237182) for breaking Decision task. CLOSED TREE 2018-05-10 21:20:39 +03:00
Chris AtLee
714c7498fd Bug 1237182: Removing unused buildbot support r=Callek
MozReview-Commit-ID: EjclZgc864L

--HG--
extra : rebase_source : 18ceefd30a5e5df3be7dc4152256b4407f61e8eb
2018-05-04 13:51:35 -04:00
Joel Maher
cd861869c6 Bug 1458730 - Test-Verify summarization fails on windows. r=gbrown 2018-05-10 06:00:43 -04:00
Joel Maher
91721cfdc5 Bug 1455316 - sometimes when test-verify finds a failure, all future tests are marked as failing also. r=gbrown 2018-05-01 14:04:24 -04:00
Andreea Pavel
959a710440 Backed out changeset afbec9a6802c (bug 1455316) for breaking sy jobs 2018-05-02 23:46:23 +03:00
Joel Maher
08e964fd73 Bug 1455316 - sometimes when test-verify finds a failure, all future tests are marked as failing also. r=gbrown 2018-05-01 14:04:24 -04:00
Noemi Erli
583038f191 Backed out changeset 5cdb10268f47 (bug 1455316) for breaking Users/cltbld/tasks/task_1525191282/mozharness/mozharness/base/script.py on a CLOSED TREE 2018-05-01 20:17:08 +03:00
Joel Maher
3e9caea623 Bug 1455316 - sometimes when test-verify finds a failure, all future tests are marked as failing also. r=gbrown 2018-05-01 11:50:46 -04:00
Marco Castelluccio
00d8757d68 Bug 1431753 - Upload per-test coverage reports. r=sparky
--HG--
extra : rebase_source : 52a23196a01867d1409229d6c50ce7c92b9367e6
2018-04-19 12:42:38 +02:00
Joel Maher
9077c8cf36 Bug 1453056 - test-verify should have the ability to run in chunks depending on the incoming tests. r=gbrown 2018-04-24 10:20:11 -04:00
Marco Castelluccio
7e52f8d35c Bug 1431753 - Add a test-coverage test suite, similar to test-verify, that runs tests in isolation in coverage mode. r=gbrown,jmaher
--HG--
extra : rebase_source : 1dcbefe05533b1295761a966484b2b250c3a6fa5
2018-04-19 19:19:31 +02:00
Brindusan Cristian
de7724a73a Backed out 3 changesets (bug 1431753) for android mochitest failures on mozharness/base/script.py CLOSED TREE
Backed out changeset bc062fa71ca7 (bug 1431753)
Backed out changeset df038cacff14 (bug 1431753)
Backed out changeset 381cc9c492f3 (bug 1431753)

--HG--
rename : testing/mozharness/mozharness/mozilla/testing/per_test_base.py => testing/mozharness/mozharness/mozilla/testing/verify_tools.py
2018-04-19 22:31:24 +03:00
Marco Castelluccio
ac95649c98 Bug 1431753 - Add a test-coverage test suite, similar to test-verify, that runs tests in isolation in coverage mode. r=gbrown,jmaher
--HG--
extra : rebase_source : 8e151ddc81d528f85e2a4acbd7623935d4246b4f
2018-04-19 19:19:31 +02:00
Emilio Cobos Álvarez
24cf27d33c Bug 1447611: Remove --enable-stylo and --enable-stylo-build-bindgen. r=froydnj
Will remove the prefs and stuff in a followup.

MozReview-Commit-ID: HVyfbHOEQYI
2018-03-21 19:13:26 +01:00
Csoregi Natalia
d6c6d38406 Backed out 4 changesets (bug 1447611) for mass failures due to --enable-stylo removal. CLOSED TREE
Backed out changeset c6193142bbcf (bug 1447611)
Backed out changeset 01ada1c5a95f (bug 1447611)
Backed out changeset 86c9fed44da2 (bug 1447611)
Backed out changeset bb84ac6e1468 (bug 1447611)
2018-03-21 19:01:07 +02:00
Emilio Cobos Álvarez
330ddd82bf Bug 1447611: Remove --enable-stylo and --enable-stylo-build-bindgen. r=froydnj
Will remove the prefs and stuff in a followup.

MozReview-Commit-ID: HVyfbHOEQYI
2018-03-21 17:24:41 +01:00
Emilio Cobos Álvarez
e9676561ce Bug 1446954: Remove support for running tests on automation without stylo configurations. r=froydnj
You can still run them on a --disable-stylo build, as long as that works
(presumably not for long).

I think I haven't missed anything, but please double-check.

MozReview-Commit-ID: 3BIAEjgTLo5
2018-03-20 11:29:08 +01:00
Henrik Skupin
87af8d3b07 Bug 1443446 - Enable geckodriver/Marionette trace output for wdspec test jobs in automation. r=ato
MozReview-Commit-ID: 14l4teCjrbY

--HG--
extra : rebase_source : 439b34c6076f6a5d12a5637ed592ac3d25f56adb
2018-03-06 11:41:59 +01:00
Marco Castelluccio
978b39e16d Bug 1431379 - Package lcov rewriter and its dependencies in the common test archive. r=chmanchester 2018-01-24 22:48:50 +01:00
Andrew Halberstadt
811da0a366 Bug 1413928 - [mozharness] Accept extra test harness args via environment variable r=maja_zf
When MOZHARNESS_TEST_PATHS is set, the test suite mozharness scripts
will run the paths specified there instead of the normal chunking
and/or default manifest. Paths should be separated by a ':' character.

In the case of web_platform_tests.py, we have to make the test paths
relative to 'testing/web-platform'.

MozReview-Commit-ID: IHRXXi5mB4G

--HG--
extra : rebase_source : 17b31ec19a64ab16918d0bd80d19d9bb496cbe37
2017-11-02 15:53:39 -04:00
James Graham
3f25c6dfb3 Bug 1430025 - Never pause after test in CI, r=ato
MozReview-Commit-ID: JP57dqYJHBe
2018-01-12 11:45:05 +00:00
Geoff Brown
f7891f6333 Bug 1421446 - In TVw, skip wdspec tests if geckodriver is not available; r=jgraham 2017-11-30 22:00:07 -07:00
Geoff Brown
218532c22c Bug 1411660 - mozharness changes to support test-verify-wpt; r=jmaher,jgraham 2017-11-17 14:18:57 -07:00
Steve Armand
df510b4622 Bug 1403131 - Run linters against mozharness scripts and configs. r=rail
MozReview-Commit-ID: 3dkx1SQiIns

--HG--
extra : rebase_source : 45f049032243b8bbae7d052a89cefcdb2b7e1f60
2017-10-24 20:21:41 -04:00
Sebastian Hengst
a79a507dc2 Backed out changeset 6fa6cfe0199c (bug 1403131) for mass reftest failures. r=backout on a CLOSED TREE
--HG--
extra : amend_source : 4d399778793d9239e5c441fe237a15d12c4f1360
2017-10-23 16:39:24 +02:00
Steve Armand
3be377b84e Bug 1403131 - Run linters against mozharness scripts and configs. r=rail
--HG--
extra : rebase_source : 4d9458f1819de0a73af9191d560557ab01ffa32e
extra : histedit_source : 1a3c743a9a1c6a3304e4a924be90164fcf882e1f%2C440a79ff12f256a2a79aff6b592fe838c1c9e6e3
2017-10-23 09:46:15 -04:00
James Graham
ac8d13e00d Bug 1401489 - Generate wptreport json files for web-platform tests in CI, r=maja_zf
This uploads a JSON summary of the results, without extra logging or
expectation data or anything. It is mostly useful for comparing the
results from two runs e.g. in a dashboard.

MozReview-Commit-ID: Ac45NVBxhy8

--HG--
extra : rebase_source : 2c7a82d6e46b8c9f7af8ad34559e51d84ba4e4ba
2017-09-20 10:53:51 +01:00
J. Ryan Stinnett
27242d0596 Bug 1393234 - Add --disable-stylo for WPT. r=jmaher
MozReview-Commit-ID: AUwMbjOinvY

--HG--
extra : rebase_source : 3b5e135624b77f72e82f5951b93289b1f2762ce6
2017-08-24 16:41:08 -05:00
Kartikaya Gupta
9d6afca9fb Bug 1388833 - Force-enable hardware acceleration for test suites that enable WebRender, because HWA is now a WR requirement. r=ahal
MozReview-Commit-ID: EtUdSeDCmFZ

--HG--
extra : rebase_source : 50472e3478d55bec36a1067d3e566ab2ac79c565
2017-08-09 16:23:04 -04:00
Kartikaya Gupta
8b8a999a38 Bug 1388423 - Get full rust backtraces everywhere. r=jmaher
MozReview-Commit-ID: 2Mfrpupz7As

--HG--
extra : rebase_source : 9b612430360fe46162f433f13667664188ceac55
2017-08-08 12:34:55 -04:00
Makoto Kato
2e5554c883 Bug 1386496 - Ahem.ttf is broken on web-platform-tests. r=jgraham
Ahem.ttf is copied to $(DIST)/bin/firefox/fonts, but this file is broken due to text mode copy.  So we should use binary mode instead.

MozReview-Commit-ID: KP7yNyPiejU

--HG--
extra : rebase_source : 2de749f458a6d4650f9044f1912ff97835c5b795
2017-08-02 12:35:53 +09:00
Michael Smith
b7e239e686 Bug 1373739 - Set up automated test running for WPT in headless mode. r=jgraham
Currently these are enabled for Linux and Windows, using the corresponding WPT
symbols with an "H" suffix to match the Mn->MnH convention followed by the
Marionette tests.

For Linux, the tests are set to run with a simulated 1600x1200 screen
resolution, which matches the Docker configuration. The default 1366x768 for
headless mode results in some unexpected failures due to windows not being
able to reach the full sizes the web platform tests expect.

For Windows, the simulated screen resolution is set to 1024x768, which is what
I've determined to be the resolution that Windows VMs run under within our
testing infrastructure. Setting a different screen resolution causes some web
platform tests to unexpectedly pass (in particular, several in
open-features-non-integer-height.html).

A "headless" variable is added to the environment that the WPT metadata
(.ini) files can access, indicating whether or not the test is running in
headless mode. This is used to mark the reftest bdi-neutral-wrapped.html as an
expected pass on Linux under headless mode: it normally fails (spuriously) due
to differences in scrollbar sizing, but scrollbars aren't rendered in headless
mode.

MozReview-Commit-ID: 3usazOoxx3q

--HG--
extra : rebase_source : c036070920391c30b13813e2d6282bd407a96715
2017-06-21 15:14:05 -07:00
Maja Frydrychowicz
65941f668d Bug 1383922 - Allow try-test-paths syntax to refer to wpt subtypes r=jgraham
This allows syntax like --try-test-paths web-platform-tests-reftests:path/to/test

MozReview-Commit-ID: uAet1ilPVy

--HG--
extra : rebase_source : 447277e47701435186ad87dfc089bd21f2bd1907
2017-07-26 19:40:59 -04:00