fune/taskcluster/ci/source-test/python-tests.yml
Dustin J. Mitchell f8a58269bd Bug 1359976: base worker payload generation on worker-type; r=wcosta r=aki
To date we have variously specified both worker-type and worker-implementation,
often manually coordinated. We also embedded a few awkward assumptions such as
that the native engine only runs on OS X.

But a worker type has one and only one implementation, and that implementation
is stable over time (as changing it would require simultaneous landings on all
trees).

Instead, this change makes worker-type the primary configuration, and derives
both a worker implementation (defining the payload format) and worker OS
(determining what to include in the payload) from that value. The derivation
occurs when deciding how to implement a particular job, where the run_using
functions are distinguished by worker implementation.

The two-part logic to determine how and where to run a test task based on its
platform is combined into a single transform, `set_worker_type`.

This contains some other related changes:
 - MOZ_AUTOMATION is set in specific jobs, rather than everywhere docker-worker
   is used
 - the URL to test packages is factored out into a shared function
 - docker-worker test defaults are applied in `mozharness_test.py`
 - the WORKER_TYPE array in `task.py`, formerly mixing two types of keys, is
   split
 - the 'invalid' workerType is assigned an 'invalid' implementation
 - all tasks that do not use job descriptions but use docker-worker, etc. have
   `worker.os` added

Tested to not produce a substantially different taskgraph for a regular push, a
try push, or a nightly cron.

MozReview-Commit-ID: LDHrmrpBo7I

--HG--
extra : rebase_source : 4cdfe6b8d9874b0c156671515b213d820b48482f
2017-05-08 22:53:50 +00:00

129 lines
3.2 KiB
YAML

taskgraph-tests:
description: taskcluster/taskgraph unit tests
platform: linux64/opt
treeherder:
symbol: py(tg)
kind: test
tier: 2
worker-type: aws-provisioner-v1/gecko-t-linux-xlarge
worker:
docker-image: {in-tree: "lint"}
max-run-time: 1800
run:
using: mach
mach: taskgraph python-tests
run-on-projects:
- integration
- release
when:
files-changed:
- 'taskcluster/**/*.py'
- 'config/mozunit.py'
- 'python/mach/**/*.py'
marionette-harness:
description: testing/marionette/harness unit tests
platform: linux64/opt
treeherder:
symbol: py(mnh)
kind: test
tier: 2
worker-type:
by-platform:
linux64.*: aws-provisioner-v1/gecko-t-linux-xlarge
worker:
by-platform:
linux64.*:
docker-image: {in-tree: "lint"}
max-run-time: 3600
run:
using: mach
mach: python-test --subsuite marionette-harness
run-on-projects:
- integration
- release
when:
files-changed:
- 'testing/marionette/harness/**'
- 'testing/mozbase/mozlog/mozlog/**'
- 'testing/mozbase/mozlog/setup.py'
- 'testing/mozbase/packages.txt'
- 'python/mach_commands.py'
mozbase:
description: testing/mozbase unit tests
platform:
- linux64/opt
treeherder:
symbol: py(mb)
kind: test
tier: 2
worker-type:
by-platform:
linux64.*: aws-provisioner-v1/gecko-t-linux-xlarge
worker:
by-platform:
linux64.*:
docker-image: {in-tree: "lint"}
max-run-time: 3600
run:
using: mach
mach: python-test --subsuite mozbase
run-on-projects:
- integration
- release
when:
files-changed:
- 'testing/mozbase/**'
- 'config/mozunit.py'
- 'python/mach_commands.py'
mozharness:
description: mozharness integration tests
platform: lint/opt
treeherder:
symbol: py(MH)
kind: test
tier: 2
worker-type: aws-provisioner-v1/gecko-t-linux-xlarge
worker:
docker-image: {in-tree: "lint"}
max-run-time: 1800
run:
using: run-task
cache-dotcache: true
command: >
cd /home/worker/checkouts/gecko/testing/mozharness &&
/usr/local/bin/tox -e py27-hg4.1
run-on-projects:
- integration
- release
when:
files-changed:
- 'testing/mozharness/**'
mozlint:
description: python/mozlint unit tests
platform: linux64/opt
treeherder:
symbol: py(ml)
kind: test
tier: 2
worker-type:
by-platform:
linux64.*: aws-provisioner-v1/gecko-t-linux-xlarge
worker:
by-platform:
linux64.*:
docker-image: {in-tree: "lint"}
max-run-time: 3600
run:
using: mach
mach: python-test --subsuite mozlint
run-on-projects:
- integration
- release
when:
files-changed:
- 'python/mozlint/**'
- 'python/mach_commands.py'