The complexity of the xpcshell test harness has grown over time. I am reluctant
to make it more complex without first trying to simplify it. Here I consolidate
some of the argument passing between functions in an attempt to simplify some
important interfaces. Changes are strictly structural and should result in no
change in behavior.
sutagent is no longer built or used; devicemanagerSUT is completely
unused. After this change, devicemanagerADB is the only implementation of
devicemanager, and test harness options like --dm_trans are eliminated.
When running xpcshell tests on my i7-6700K, the Windows Defender
MsMpEng.exe process regularly consumes up to 50% CPU - 2 whole cores.
This is because it is canning the thousands of files we create in the
temporary directory. (We create a profile for every xpcshell.exe process.)
This commit changes the behavior of `mach xpcshell-test` to use a temporary
directory under the topobjdir. We encourage people to disable file indexing
and A/V scanning in the source and object directories because this can add
overhead. So by putting temporary files in the object directory, we should
minimize the chances of file scanning slowing down test execution.
On my machine (which has Windows Defender disabled in my source and objdirs),
`mach xpcshell-test` execution time drops from ~13:40 to ~7:30. Seriously.
I'm told we have Windows Defender disabled in automation, so this hack
shouldn't need to be investigated there. i.e. this is a pure local development
win.
MozReview-Commit-ID: BMcSAZ16Yei
--HG--
extra : rebase_source : 5490be35b59ff675639180b928ee77dcc70de485
This moves test installation for test files out of the monolithic install
manifest for $objdir/_tests, and determines the test and support files
to install based on the object derived from all-tests.json. Additionally,
the files resulting from TEST_HARNESS_FILES are installed, as some tests
will depend on them.
As a result, the time to install tests when invoking the test runner will
scale with the number of tests requested to run rather than the entire set
of tests in the tree, resulting in significantly less overhead.
MozReview-Commit-ID: LeIrUVh1yD4
This also completely remove build/automationutils.py.
--HG--
extra : commitid : 50v6EAQNEHV
extra : rebase_source : 4ac1347d73498f068979514c6afb16ac50ab4033
This prevents 8 module imports from occurring at mach startup time.
As part of this, I discovered a redundant import of "sys" and eliminated
it.
--HG--
extra : commitid : AsZJHdCVbV0
extra : rebase_source : 53b1d5912ba85c7d363d9da21329a6fa65a2bce2
This removes ambiguity as to which modules are being imported, making
import slightly faster as Python doesn't need to test so many
directories for file presence.
All files should already be using absolute imports because mach command
modules aren't imported to the package they belong to: they instead
belong to the "mach" package. So relative imports shouldn't have been
used.
--HG--
extra : commitid : 6tFME1KKfTD
extra : rebase_source : 78728f82f5487281620e00c2a8004cd5e1968087
Add a `tags` attribute to a test or DEFAULT section in a manifest:
[test_foo]
tags = foo
Then run all tests with a given tag by passing in `--tag foo` to a supported test harness. So far mochitest, xpcshell and marionette are supported.
--HG--
extra : rebase_source : d75905da1ca021a15a9538117d3866425f73d962
extra : source : 3c34fd480729e3b6684fba747ff61078f672ce16
Add a `tags` attribute to a test or DEFAULT section in a manifest:
[test_foo]
tags = foo
Then run all tests with a given tag by passing in `--tag foo` to a supported test harness. So far mochitest, xpcshell and marionette are supported.
--HG--
extra : rebase_source : 68a0931c6a8ee1df4f5c09d67c396490774aa856
Make the test paths relative to topsrcdir before passing them to
TestResolver. Also do not passing cwd to TestResolver since it will
filter out tests that do not live under the directory where the mach
command is executed.
Verification steps:
Execute a mach test command from any subdirectory. For example:
$ cd testing/tps/
$ ../../mach xpcshell-test ../../services/fxaccounts/tests/xpcshell/
$ ../../mach test ../../services/fxaccounts/tests/xpcshell/
A subsequent patch will have another mach command dispatch into
`xpcshell-test`. To avoid redundant resolution of test paths, we now
support passing resolved test objects into the mach command.
--HG--
extra : rebase_source : 1eb0e7b1ef1e45ca163ddb152d1bbb7b6b4d02e6
Previously, mach xpcshell-test and mochitest-* were limited to a single
test "path" argument. This patch enables multiple arguments to be passed
in.
TestResolver in the build system has gained the ability to process
multiple paths in a single invocation. The mach commands have been
modified to utilize this new feature.
Only mach code paths that pass manifestdestiny.TestManifest instances
into the test runner can accept multiple arguments. This is because
there is no other way to pass a custom set of tests into the test
runner. If multiple test arguments are used but not supported, a warning
is emitted.
--HG--
extra : rebase_source : 1ce1328a969f654e7b43a7a0bdd15ed86f5ceb21
Manifests are now the mechanism used to specify how to run xpcshell
tests. This patch removes old code paths that allowed directories to be
specified.
runxpcshelltests.py still contains references to "testdirs." These
should likely be removed as well. This patch doesn't touch them out of
fear it may have consequences on automation.
--HG--
extra : rebase_source : 8cbfe6dc759dc0a311669c4fcba08ff7a60edc4a
extra : amend_source : 890377d9ba33d52c84e0736b0812231f3005b253
