nagai/fune
3
0
Fork 0
forked from mirrors/gecko-dev
Code Pull requests Activity

Bug 1619944 Migrate partials and pipfile-updates to use requirements.txt r=bhearsum

Browse source 
Differential Revision: https://phabricator.services.mozilla.com/D65323

--HG--
rename : taskcluster/ci/pipfile-update/kind.yml => taskcluster/ci/python-dependency-update/kind.yml
rename : taskcluster/docker/pipfile-updates/Dockerfile => taskcluster/docker/python-dependency-update/Dockerfile
rename : taskcluster/docker/pipfile-updates/README.md => taskcluster/docker/python-dependency-update/README.md
rename : taskcluster/docker/pipfile-updates/runme.sh => taskcluster/docker/python-dependency-update/runme.sh
rename : taskcluster/docker/pipfile-updates/scripts/update_pipfiles.sh => taskcluster/docker/python-dependency-update/scripts/update_pipfiles.sh
rename : taskcluster/docker/pipfile-updates/setup.sh => taskcluster/docker/python-dependency-update/setup.sh
rename : taskcluster/taskgraph/transforms/pipfile_update.py => taskcluster/taskgraph/transforms/python_update.py
extra : moz-landing-system : lando
This commit is contained in:
Simon Fraser 2020-03-05 13:49:24 +00:00
parent f791019f83
commit ae80827888
19 changed files with 382 additions and 513 deletions
repo.diff.show_diff_stats Download patch file Download diff file Expand all files Collapse all files

4
.cron.yml
View file

@ -197,11 +197,11 @@ jobs:
- {weekday: 'Monday', hour: 10, minute: 0} - {weekday: 'Monday', hour: 10, minute: 0}
- {weekday: 'Thursday', hour: 10, minute: 0} - {weekday: 'Thursday', hour: 10, minute: 0}
- name: pipfile-update - name: python-dependency-update
job: job:
type: decision-task type: decision-task
treeherder-symbol: Nfile treeherder-symbol: Nfile
target-tasks-method: pipfile_update target-tasks-method: python_dependency_update
run-on-projects: run-on-projects:
- mozilla-central - mozilla-central
when: when:

2
taskcluster/ci/config.yml
View file

@ -122,7 +122,7 @@ treeherder:
'langpack': 'Langpack sigatures and uploads' 'langpack': 'Langpack sigatures and uploads'
'TPS': 'Sync tests' 'TPS': 'Sync tests'
'UV': 'Update verify' 'UV': 'Update verify'
'pipfu': 'pipfile update' 'pydep': 'python dependency update'
'WR': 'WebRender standalone' 'WR': 'WebRender standalone'
'Wgpu': 'WebGPU standalone' 'Wgpu': 'WebGPU standalone'
'Gd': 'Geckodriver' 'Gd': 'Geckodriver'

2
taskcluster/ci/docker-image/kind.yml
View file

@ -236,7 +236,7 @@ jobs:
definition: partner-repack definition: partner-repack
periodic-updates: periodic-updates:
symbol: I(file) symbol: I(file)
pipfile-updates: python-dependency-update:
symbol: I(pip) symbol: I(pip)
firefox-snap: firefox-snap:
symbol: I(snap) symbol: I(snap)

16
taskcluster/ci/pipfile-update/kind.yml → taskcluster/ci/python-dependency-update/kind.yml
View file

@ -5,13 +5,13 @@
loader: taskgraph.loader.transform:loader loader: taskgraph.loader.transform:loader
transforms: transforms:
- taskgraph.transforms.pipfile_update:transforms - taskgraph.transforms.python_update:transforms
- taskgraph.transforms.task:transforms - taskgraph.transforms.task:transforms
job-defaults: job-defaults:
name: pipfile_update name: python_dependency_update
description: Update Pipfile.lock description: Update requirements.txt
run-on-projects: [] run-on-projects: []
treeherder: treeherder:
kind: build kind: build
@ -21,14 +21,14 @@ job-defaults:
worker: worker:
implementation: docker-worker implementation: docker-worker
os: linux os: linux
docker-image: {in-tree: pipfile-updates} docker-image: {in-tree: python-dependency-update}
taskcluster-proxy: true taskcluster-proxy: true
max-run-time: 2400 max-run-time: 2400
command: command:
- /runme.sh - /runme.sh
artifacts: artifacts:
- name: 'public/build/Pipfile.lock.diff' - name: 'public/build/requirements.txt.diff'
path: '/home/worker/artifacts/Pipfile.lock.diff' path: '/home/worker/artifacts/requirements.txt.diff'
type: file type: file
scopes: scopes:
- secrets:get:project/releng/gecko/build/level-{level}/arc-phabricator-token - secrets:get:project/releng/gecko/build/level-{level}/arc-phabricator-token
@ -37,9 +37,9 @@ job-defaults:
jobs: jobs:
funsize-update-generator: funsize-update-generator:
treeherder: treeherder:
symbol: pipfu(fug) symbol: pydep(fug)
worker: worker:
env: env:
PIPFILE_DIRECTORY: "taskcluster/docker/funsize-update-generator" REQUIREMENTS_FILE: "taskcluster/docker/funsize-update-generator/requirements.in"
PYTHON3: "1" PYTHON3: "1"
REVIEWERS: "sfraser, bhearsum" REVIEWERS: "sfraser, bhearsum"

26
taskcluster/docker/funsize-update-generator/Dockerfile
View file

@ -7,12 +7,22 @@ ENV DEBIAN_FRONTEND noninteractive
# to make sure that files are removed within a single docker layer # to make sure that files are removed within a single docker layer
RUN apt-get update -q && \ RUN apt-get update -q && \
apt-get install -yyq --no-install-recommends \ apt-get install -yyq --no-install-recommends \
python3.8 libpython3.8-dev python3-setuptools python3-cryptography libgetopt-simple-perl \ bzip2 \
bzip2 python3-requests python3-sh curl \ ca-certificates \
python3-dev gcc liblzma-dev xz-utils jq libdpkg-perl locales && \ curl \
apt-get clean gcc \
jq \
libdpkg-perl \
libgetopt-simple-perl \
liblzma-dev \
locales \
python3.8 \
libpython3.8-dev \
python3-dev \
xz-utils \
&& apt-get clean
RUN useradd -d /home/worker -s /bin/bash -m worker RUN useradd -d /home/worker -s /bin/bash -m worker
COPY Pipfile Pipfile.lock / COPY requirements.txt /
RUN locale-gen en_CA.UTF-8 RUN locale-gen en_CA.UTF-8
ENV LANG en_CA.UTF-8 ENV LANG en_CA.UTF-8
@ -23,7 +33,8 @@ ENV LC_ALL en_CA.UTF-8
# python-pip installs a lot of dependencies increasing the size of an image # python-pip installs a lot of dependencies increasing the size of an image
# drastically. Install it like this saves us almost 200M. # drastically. Install it like this saves us almost 200M.
RUN bash -c "curl -L https://bootstrap.pypa.io/get-pip.py | python3.8" RUN bash -c "curl -L https://bootstrap.pypa.io/get-pip.py | python3.8"
RUN ["pip", "install", "pipenv"]
RUN ["pip", "install", "-r", "/requirements.txt"]
# scripts # scripts
RUN mkdir /home/worker/bin RUN mkdir /home/worker/bin
@ -39,7 +50,6 @@ ENV SHELL /bin/bash
ENV USER worker ENV USER worker
ENV LOGNAME worker ENV LOGNAME worker
# This is done after setting environment variables to make sure the virtualenv is created in the right place. USER worker
RUN pipenv install
CMD ["/runme.sh"] CMD ["/runme.sh"]

11
taskcluster/docker/funsize-update-generator/Pipfile
View file

@ -1,11 +0,0 @@
[requires]
python_version = '3.8'
[packages]
aiohttp = "*"
mar = "*"
redo = "*"
requests = "*"
scriptworker = "*"
sh = "*"
awscli = "*"

443
taskcluster/docker/funsize-update-generator/Pipfile.lock generated
View file

@ -1,443 +0,0 @@
{
"_meta": {
"hash": {
"sha256": "802e28cd433f875179e9eca053675705f18f76f8b5b8a78b99bfbd15bcb62f86"
},
"pipfile-spec": 6,
"requires": {
"python_version": "3.8"
},
"sources": [
{
"name": "pypi",
"url": "https://pypi.org/simple",
"verify_ssl": true
}
]
},
"default": {
"aiohttp": {
"hashes": [
"sha256:1e984191d1ec186881ffaed4581092ba04f7c61582a177b187d3a2f07ed9719e",
"sha256:259ab809ff0727d0e834ac5e8a283dc5e3e0ecc30c4d80b3cd17a4139ce1f326",
"sha256:2f4d1a4fdce595c947162333353d4a44952a724fba9ca3205a3df99a33d1307a",
"sha256:32e5f3b7e511aa850829fbe5aa32eb455e5534eaa4b1ce93231d00e2f76e5654",
"sha256:344c780466b73095a72c616fac5ea9c4665add7fc129f285fbdbca3cccf4612a",
"sha256:460bd4237d2dbecc3b5ed57e122992f60188afe46e7319116da5eb8a9dfedba4",
"sha256:4c6efd824d44ae697814a2a85604d8e992b875462c6655da161ff18fd4f29f17",
"sha256:50aaad128e6ac62e7bf7bd1f0c0a24bc968a0c0590a726d5a955af193544bcec",
"sha256:6206a135d072f88da3e71cc501c59d5abffa9d0bb43269a6dcd28d66bfafdbdd",
"sha256:65f31b622af739a802ca6fd1a3076fd0ae523f8485c52924a89561ba10c49b48",
"sha256:ae55bac364c405caa23a4f2d6cfecc6a0daada500274ffca4a9230e7129eac59",
"sha256:b778ce0c909a2653741cb4b1ac7015b5c130ab9c897611df43ae6a58523cb965"
],
"index": "pypi",
"version": "==3.6.2"
},
"aiomemoizettl": {
"hashes": [
"sha256:07a6becac60f6cd2604b9f2b73bcd9a50079a0b7b55e2a4e45b1eec5a3ea9659",
"sha256:0a80d2dc765e545263f515363b6700ec8cf86fa3968b529f56390b28e34f743d"
],
"version": "==0.0.3"
},
"arrow": {
"hashes": [
"sha256:5390e464e2c5f76971b60ffa7ee29c598c7501a294bc9f5e6dadcb251a5d027b",
"sha256:70729bcc831da496ca3cb4b7e89472c8e2d27d398908155e0796179f6d2d41ee"
],
"version": "==0.15.5"
},
"asn1crypto": {
"hashes": [
"sha256:5a215cb8dc12f892244e3a113fe05397ee23c5c4ca7a69cd6e69811755efc42d",
"sha256:831d2710d3274c8a74befdddaf9f17fcbf6e350534565074818722d6d615b315"
],
"version": "==1.3.0"
},
"async-timeout": {
"hashes": [
"sha256:0c3c816a028d47f659d6ff5c745cb2acf1f966da1fe5c19c77a70282b25f4c5f",
"sha256:4291ca197d287d274d0b6cb5d6f8f8f82d434ed288f962539ff18cc9012f9ea3"
],
"version": "==3.0.1"
},
"attrs": {
"hashes": [
"sha256:08a96c641c3a74e44eb59afb61a24f2cb9f4d7188748e76ba4bb5edfa3cb7d1c",
"sha256:f7b7ce16570fe9965acd6d30101a28f62fb4a7f9e926b3bbc9b61f8b04247e72"
],
"version": "==19.3.0"
},
"awscli": {
"hashes": [
"sha256:8f587ea9b178c1eec19a676c2b9d5fa1a0564224cb3f9876a1f41d0172c86904",
"sha256:f536ea3d95c4aa1614e030e587d8e4d599302d48778492dfc4e3404ee9f551b0"
],
"index": "pypi",
"version": "==1.18.11"
},
"backports.lzma": {
"hashes": [
"sha256:16d8b68e4d3cd4e6c9ddb059850452946da3914c8a8e197a7f2b0954559f2df4"
],
"version": "==0.0.14"
},
"botocore": {
"hashes": [
"sha256:35d553b8975a3810010fabb885e42d1bbc97bdae52c873f7640a2dd87772c455",
"sha256:87a53301ee9e32e776d2d805c121753091fbddc3457dfdb8d6bdf7dbd945295a"
],
"version": "==1.15.11"
},
"certifi": {
"hashes": [
"sha256:017c25db2a153ce562900032d5bc68e9f191e44e9a0f762f373977de9df1fbb3",
"sha256:25b64c7da4cd7479594d035c08c2d809eb4aab3a26e5a990ea98cc450c320f1f"
],
"version": "==2019.11.28"
},
"cffi": {
"hashes": [
"sha256:001bf3242a1bb04d985d63e138230802c6c8d4db3668fb545fb5005ddf5bb5ff",
"sha256:00789914be39dffba161cfc5be31b55775de5ba2235fe49aa28c148236c4e06b",
"sha256:028a579fc9aed3af38f4892bdcc7390508adabc30c6af4a6e4f611b0c680e6ac",
"sha256:14491a910663bf9f13ddf2bc8f60562d6bc5315c1f09c704937ef17293fb85b0",
"sha256:1cae98a7054b5c9391eb3249b86e0e99ab1e02bb0cc0575da191aedadbdf4384",
"sha256:2089ed025da3919d2e75a4d963d008330c96751127dd6f73c8dc0c65041b4c26",
"sha256:2d384f4a127a15ba701207f7639d94106693b6cd64173d6c8988e2c25f3ac2b6",
"sha256:337d448e5a725bba2d8293c48d9353fc68d0e9e4088d62a9571def317797522b",
"sha256:399aed636c7d3749bbed55bc907c3288cb43c65c4389964ad5ff849b6370603e",
"sha256:3b911c2dbd4f423b4c4fcca138cadde747abdb20d196c4a48708b8a2d32b16dd",
"sha256:3d311bcc4a41408cf5854f06ef2c5cab88f9fded37a3b95936c9879c1640d4c2",
"sha256:62ae9af2d069ea2698bf536dcfe1e4eed9090211dbaafeeedf5cb6c41b352f66",
"sha256:66e41db66b47d0d8672d8ed2708ba91b2f2524ece3dee48b5dfb36be8c2f21dc",
"sha256:675686925a9fb403edba0114db74e741d8181683dcf216be697d208857e04ca8",
"sha256:7e63cbcf2429a8dbfe48dcc2322d5f2220b77b2e17b7ba023d6166d84655da55",
"sha256:8a6c688fefb4e1cd56feb6c511984a6c4f7ec7d2a1ff31a10254f3c817054ae4",
"sha256:8c0ffc886aea5df6a1762d0019e9cb05f825d0eec1f520c51be9d198701daee5",
"sha256:95cd16d3dee553f882540c1ffe331d085c9e629499ceadfbda4d4fde635f4b7d",
"sha256:99f748a7e71ff382613b4e1acc0ac83bf7ad167fb3802e35e90d9763daba4d78",
"sha256:b8c78301cefcf5fd914aad35d3c04c2b21ce8629b5e4f4e45ae6812e461910fa",
"sha256:c420917b188a5582a56d8b93bdd8e0f6eca08c84ff623a4c16e809152cd35793",
"sha256:c43866529f2f06fe0edc6246eb4faa34f03fe88b64a0a9a942561c8e22f4b71f",
"sha256:cab50b8c2250b46fe738c77dbd25ce017d5e6fb35d3407606e7a4180656a5a6a",
"sha256:cef128cb4d5e0b3493f058f10ce32365972c554572ff821e175dbc6f8ff6924f",
"sha256:cf16e3cf6c0a5fdd9bc10c21687e19d29ad1fe863372b5543deaec1039581a30",
"sha256:e56c744aa6ff427a607763346e4170629caf7e48ead6921745986db3692f987f",
"sha256:e577934fc5f8779c554639376beeaa5657d54349096ef24abe8c74c5d9c117c3",
"sha256:f2b0fa0c01d8a0c7483afd9f31d7ecf2d71760ca24499c8697aeb5ca37dc090c"
],
"version": "==1.14.0"
},
"chardet": {
"hashes": [
"sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
"sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
],
"version": "==3.0.4"
},
"click": {
"hashes": [
"sha256:2335065e6395b9e67ca716de5f7526736bfa6ceead690adf616d925bdc622b13",
"sha256:5b94b49521f6456670fdb30cd82a4eca9412788a93fa6dd6df72c94d5a8ff2d7"
],
"version": "==7.0"
},
"colorama": {
"hashes": [
"sha256:7d73d2a99753107a36ac6b455ee49046802e59d9d076ef8e47b61499fa29afff",
"sha256:e96da0d330793e2cb9485e9ddfd918d456036c7149416295932478192f4436a1"
],
"markers": "python_version != '3.4'",
"version": "==0.4.3"
},
"construct": {
"hashes": [
"sha256:97ba13edcd98546f10f7555af41c8ce7ae9d8221525ec4062c03f9adbf940661"
],
"version": "==2.10.56"
},
"cryptography": {
"hashes": [
"sha256:02079a6addc7b5140ba0825f542c0869ff4df9a69c360e339ecead5baefa843c",
"sha256:1df22371fbf2004c6f64e927668734070a8953362cd8370ddd336774d6743595",
"sha256:369d2346db5934345787451504853ad9d342d7f721ae82d098083e1f49a582ad",
"sha256:3cda1f0ed8747339bbdf71b9f38ca74c7b592f24f65cdb3ab3765e4b02871651",
"sha256:44ff04138935882fef7c686878e1c8fd80a723161ad6a98da31e14b7553170c2",
"sha256:4b1030728872c59687badcca1e225a9103440e467c17d6d1730ab3d2d64bfeff",
"sha256:58363dbd966afb4f89b3b11dfb8ff200058fbc3b947507675c19ceb46104b48d",
"sha256:6ec280fb24d27e3d97aa731e16207d58bd8ae94ef6eab97249a2afe4ba643d42",
"sha256:7270a6c29199adc1297776937a05b59720e8a782531f1f122f2eb8467f9aab4d",
"sha256:73fd30c57fa2d0a1d7a49c561c40c2f79c7d6c374cc7750e9ac7c99176f6428e",
"sha256:7f09806ed4fbea8f51585231ba742b58cbcfbfe823ea197d8c89a5e433c7e912",
"sha256:90df0cc93e1f8d2fba8365fb59a858f51a11a394d64dbf3ef844f783844cc793",
"sha256:971221ed40f058f5662a604bd1ae6e4521d84e6cad0b7b170564cc34169c8f13",
"sha256:a518c153a2b5ed6b8cc03f7ae79d5ffad7315ad4569b2d5333a13c38d64bd8d7",
"sha256:b0de590a8b0979649ebeef8bb9f54394d3a41f66c5584fff4220901739b6b2f0",
"sha256:b43f53f29816ba1db8525f006fa6f49292e9b029554b3eb56a189a70f2a40879",
"sha256:d31402aad60ed889c7e57934a03477b572a03af7794fa8fb1780f21ea8f6551f",
"sha256:de96157ec73458a7f14e3d26f17f8128c959084931e8997b9e655a39c8fde9f9",
"sha256:df6b4dca2e11865e6cfbfb708e800efb18370f5a46fd601d3755bc7f85b3a8a2",
"sha256:ecadccc7ba52193963c0475ac9f6fa28ac01e01349a2ca48509667ef41ffd2cf",
"sha256:fb81c17e0ebe3358486cd8cc3ad78adbae58af12fc2bf2bc0bb84e8090fa5ce8"
],
"version": "==2.8"
},
"dictdiffer": {
"hashes": [
"sha256:1adec0d67cdf6166bda96ae2934ddb5e54433998ceab63c984574d187cc563d2",
"sha256:d79d9a39e459fe33497c858470ca0d2e93cb96621751de06d631856adfd9c390"
],
"version": "==0.8.1"
},
"docutils": {
"hashes": [
"sha256:6c4f696463b79f1fb8ba0c594b63840ebd41f059e92b31957c46b74a4599b6d0",
"sha256:9e4d7ecfc600058e07ba661411a2b7de2fd0fafa17d1a7f7361cd47b1175c827",
"sha256:a2aeea129088da402665e92e0b25b04b073c04b2dce4ab65caaa38b7ce2e1a99"
],
"version": "==0.15.2"
},
"github3.py": {
"hashes": [
"sha256:15a115c18f7bfcf934dfef7ab103844eb9f620c586bad65967708926da47cbda",
"sha256:50833b5da35546b8cced0e8d7ff4c50a9afc2c8e46cc4d07dc4b66d26467c708"
],
"version": "==1.3.0"
},
"idna": {
"hashes": [
"sha256:7588d1c14ae4c77d74036e8c22ff447b26d0fde8f007354fd48a7814db15b7cb",
"sha256:a068a21ceac8a4d63dbfd964670474107f541babbd2250d61922f029858365fa"
],
"version": "==2.9"
},
"immutabledict": {
"hashes": [
"sha256:43dde3e55dcb539537ae6189fb6b09a1d01e94db304e4506e94ca2d45ec14c47",
"sha256:7881e44098f13dd12d6fec00551d564433cb46776e8b2f3453128f715df4376a"
],
"version": "==0.2.0"
},
"jmespath": {
"hashes": [
"sha256:695cb76fa78a10663425d5b73ddc5714eb711157e52704d69be03b1a02ba4fec",
"sha256:cca55c8d153173e21baa59983015ad0daf603f9cb799904ff057bfb8ff8dc2d9"
],
"version": "==0.9.5"
},
"json-e": {
"hashes": [
"sha256:d2914f785d93ecc4f0b2ad6e3f2791f33327eaa740a3c4917d68a9a485dd282d"
],
"version": "==3.0.0"
},
"jsonschema": {
"hashes": [
"sha256:4e5b3cf8216f577bee9ce139cbe72eca3ea4f292ec60928ff24758ce626cd163",
"sha256:c8a85b28d377cc7737e46e2d9f2b4f44ee3c0e1deac6bf46ddefc7187d30797a"
],
"version": "==3.2.0"
},
"jwcrypto": {
"hashes": [
"sha256:618ded1d25d3f806a1ab05cee42633a5a2787af33fca8d8f539b0aa1478b3728",
"sha256:adbe1f6266cde35d40d5de6d1419612b3bd4c869b9332c88c9d7a9163d305100"
],
"version": "==0.7"
},
"mar": {
"hashes": [
"sha256:1939df482f2d3f6221405da00d6286d77d2dd60d372a0fd37532a8f00544f64f",
"sha256:5d2904a063f0da625e37515fa9eb340082e69fa1c00fdbeb82d28b7ff3c51e28"
],
"index": "pypi",
"version": "==3.1.0"
},
"mohawk": {
"hashes": [
"sha256:3ed296a30453d0b724679e0fd41e4e940497f8e461a9a9c3b7f36e43bab0fa09",
"sha256:d2a0e3ab10a209cc79e95e28f2dd54bd4a73fd1998ffe27b7ba0f962b6be9723"
],
"version": "==1.1.0"
},
"multidict": {
"hashes": [
"sha256:317f96bc0950d249e96d8d29ab556d01dd38888fbe68324f46fd834b430169f1",
"sha256:42f56542166040b4474c0c608ed051732033cd821126493cf25b6c276df7dd35",
"sha256:4b7df040fb5fe826d689204f9b544af469593fb3ff3a069a6ad3409f742f5928",
"sha256:544fae9261232a97102e27a926019100a9db75bec7b37feedd74b3aa82f29969",
"sha256:620b37c3fea181dab09267cd5a84b0f23fa043beb8bc50d8474dd9694de1fa6e",
"sha256:6e6fef114741c4d7ca46da8449038ec8b1e880bbe68674c01ceeb1ac8a648e78",
"sha256:7774e9f6c9af3f12f296131453f7b81dabb7ebdb948483362f5afcaac8a826f1",
"sha256:85cb26c38c96f76b7ff38b86c9d560dea10cf3459bb5f4caf72fc1bb932c7136",
"sha256:a326f4240123a2ac66bb163eeba99578e9d63a8654a59f4688a79198f9aa10f8",
"sha256:ae402f43604e3b2bc41e8ea8b8526c7fa7139ed76b0d64fc48e28125925275b2",
"sha256:aee283c49601fa4c13adc64c09c978838a7e812f85377ae130a24d7198c0331e",
"sha256:b51249fdd2923739cd3efc95a3d6c363b67bbf779208e9f37fd5e68540d1a4d4",
"sha256:bb519becc46275c594410c6c28a8a0adc66fe24fef154a9addea54c1adb006f5",
"sha256:c2c37185fb0af79d5c117b8d2764f4321eeb12ba8c141a95d0aa8c2c1d0a11dd",
"sha256:dc561313279f9d05a3d0ffa89cd15ae477528ea37aa9795c4654588a3287a9ab",
"sha256:e439c9a10a95cb32abd708bb8be83b2134fa93790a4fb0535ca36db3dda94d20",
"sha256:fc3b4adc2ee8474cb3cd2a155305d5f8eda0a9c91320f83e55748e1fcb68f8e3"
],
"version": "==4.7.5"
},
"pyasn1": {
"hashes": [
"sha256:39c7e2ec30515947ff4e87fb6f456dfc6e84857d34be479c9d4a4ba4bf46aa5d",
"sha256:aef77c9fb94a3ac588e87841208bdec464471d9871bd5050a287cc9a475cd0ba"
],
"version": "==0.4.8"
},
"pycparser": {
"hashes": [
"sha256:a988718abfad80b6b157acce7bf130a30876d27603738ac39f140993246b25b3"
],
"version": "==2.19"
},
"pyrsistent": {
"hashes": [
"sha256:cdc7b5e3ed77bed61270a47d35434a30617b9becdf2478af76ad2c6ade307280"
],
"version": "==0.15.7"
},
"python-dateutil": {
"hashes": [
"sha256:73ebfe9dbf22e832286dafa60473e4cd239f8592f699aa5adaf10050e6e1823c",
"sha256:75bb3f31ea686f1197762692a9ee6a7550b59fc6ca3a1f4b5d7e32fb98e2da2a"
],
"version": "==2.8.1"
},
"pyyaml": {
"hashes": [
"sha256:0e7f69397d53155e55d10ff68fdfb2cf630a35e6daf65cf0bdeaf04f127c09dc",
"sha256:2e9f0b7c5914367b0916c3c104a024bb68f269a486b9d04a2e8ac6f6597b7803",
"sha256:35ace9b4147848cafac3db142795ee42deebe9d0dad885ce643928e88daebdcc",
"sha256:38a4f0d114101c58c0f3a88aeaa44d63efd588845c5a2df5290b73db8f246d15",
"sha256:483eb6a33b671408c8529106df3707270bfacb2447bf8ad856a4b4f57f6e3075",
"sha256:4b6be5edb9f6bb73680f5bf4ee08ff25416d1400fbd4535fe0069b2994da07cd",
"sha256:7f38e35c00e160db592091751d385cd7b3046d6d51f578b29943225178257b31",
"sha256:8100c896ecb361794d8bfdb9c11fce618c7cf83d624d73d5ab38aef3bc82d43f",
"sha256:c0ee8eca2c582d29c3c2ec6e2c4f703d1b7f1fb10bc72317355a746057e7346c",
"sha256:e4c015484ff0ff197564917b4b4246ca03f411b9bd7f16e02a2f586eb48b6d04",
"sha256:ebc4ed52dcc93eeebeae5cf5deb2ae4347b3a81c3fa12b0b8c976544829396a4"
],
"version": "==5.2"
},
"redo": {
"hashes": [
"sha256:36784bf8ae766e14f9db0e377ccfa02835d648321d2007b6ae0bf4fd612c0f94",
"sha256:71161cb0e928d824092a5f16203939bbc0867ce4c4685db263cf22c3ae7634a8"
],
"index": "pypi",
"version": "==2.0.3"
},
"requests": {
"hashes": [
"sha256:43999036bfa82904b6af1d99e4882b560e5e2c68e5c4b0aa03b655f3d7d73fee",
"sha256:b3f43d496c6daba4493e7c431722aeb7dbc6288f52a6e04e7b6023b0247817e6"
],
"index": "pypi",
"version": "==2.23.0"
},
"rsa": {
"hashes": [
"sha256:25df4e10c263fb88b5ace923dd84bf9aa7f5019687b5e55382ffcdb8bede9db5",
"sha256:43f682fea81c452c98d09fc316aae12de6d30c4b5c84226642cf8f8fd1c93abd"
],
"version": "==3.4.2"
},
"s3transfer": {
"hashes": [
"sha256:2482b4259524933a022d59da830f51bd746db62f047d6eb213f2f8855dcb8a13",
"sha256:921a37e2aefc64145e7b73d50c71bb4f26f46e4c9f414dc648c6245ff92cf7db"
],
"version": "==0.3.3"
},
"scriptworker": {
"hashes": [
"sha256:a4d428a2c9db1b0dbabb86c0badf850e919ed83ffaba701185370ebe3dd5f7aa",
"sha256:d60976a94ba86d0c4b9f23536ce3cb026281c726d00da12b6546a8dd80cb4d1b"
],
"index": "pypi",
"version": "==32.0.3"
},
"sh": {
"hashes": [
"sha256:ae3258c5249493cebe73cb4e18253a41ed69262484bad36fdb3efcb8ad8870bb",
"sha256:b52bf5833ed01c7b5c5fb73a7f71b3d98d48e9b9b8764236237bdc7ecae850fc"
],
"index": "pypi",
"version": "==1.12.14"
},
"six": {
"hashes": [
"sha256:236bdbdce46e6e6a3d61a337c0f8b763ca1e8717c03b369e87a7ec7ce1319c0a",
"sha256:8f3cd2e254d8f793e7f3d6d9df77b92252b52637291d0f0da013c76ea2724b6c"
],
"version": "==1.14.0"
},
"slugid": {
"hashes": [
"sha256:a950d98b72691178bdd4d6c52743c4a2aa039207cf7a97d71060a111ff9ba297",
"sha256:aec8b0e01c4ad32e38e12d609eab3ec912fd129aaf6b2ded0199b56a5f8fd67c"
],
"version": "==2.0.0"
},
"taskcluster": {
"hashes": [
"sha256:017e626ec7211cc250f6c59ac4fbeb3ca7d7ecf0a253eb4f21a694bc5d02ae36",
"sha256:0a8fc965441ab60aafb83bcf2b670afd3f8817b13067561e084b4d2528f1e75c",
"sha256:a9d10a794a065ea17a11fa8ee4861e57ea702362f70d00dc97d5a4f685a70d4f"
],
"version": "==25.4.0"
},
"taskcluster-urls": {
"hashes": [
"sha256:1dc740c32c7beb31e11ed7ccf9da2d47a504acdb3170c8900649433b0fd16fb2",
"sha256:4a62c776aeba6d45044789a8845ec4d8521bc1bb6ebfc86d79ee759bcdd4f2f7"
],
"version": "==12.1.0"
},
"uritemplate": {
"hashes": [
"sha256:07620c3f3f8eed1f12600845892b0e036a2420acf513c53f7de0abd911a5894f",
"sha256:5af8ad10cec94f215e3f48112de2022e1d5a37ed427fbd88652fa908f2ab7cae"
],
"version": "==3.0.1"
},
"urllib3": {
"hashes": [
"sha256:2f3db8b19923a873b3e5256dc9c2dedfa883e33d87c690d9c7913e1f40673cdc",
"sha256:87716c2d2a7121198ebcb7ce7cccf6ce5e9ba539041cfbaeecfb641dc0bf6acc"
],
"markers": "python_version != '3.4'",
"version": "==1.25.8"
},
"yarl": {
"hashes": [
"sha256:0c2ab325d33f1b824734b3ef51d4d54a54e0e7a23d13b86974507602334c2cce",
"sha256:0ca2f395591bbd85ddd50a82eb1fde9c1066fafe888c5c7cc1d810cf03fd3cc6",
"sha256:2098a4b4b9d75ee352807a95cdf5f10180db903bc5b7270715c6bbe2551f64ce",
"sha256:25e66e5e2007c7a39541ca13b559cd8ebc2ad8fe00ea94a2aad28a9b1e44e5ae",
"sha256:26d7c90cb04dee1665282a5d1a998defc1a9e012fdca0f33396f81508f49696d",
"sha256:308b98b0c8cd1dfef1a0311dc5e38ae8f9b58349226aa0533f15a16717ad702f",
"sha256:3ce3d4f7c6b69c4e4f0704b32eca8123b9c58ae91af740481aa57d7857b5e41b",
"sha256:58cd9c469eced558cd81aa3f484b2924e8897049e06889e8ff2510435b7ef74b",
"sha256:5b10eb0e7f044cf0b035112446b26a3a2946bca9d7d7edb5e54a2ad2f6652abb",
"sha256:6faa19d3824c21bcbfdfce5171e193c8b4ddafdf0ac3f129ccf0cdfcb083e462",
"sha256:944494be42fa630134bf907714d40207e646fd5a94423c90d5b514f7b0713fea",
"sha256:a161de7e50224e8e3de6e184707476b5a989037dcb24292b391a3d66ff158e70",
"sha256:a4844ebb2be14768f7994f2017f70aca39d658a96c786211be5ddbe1c68794c1",
"sha256:c2b509ac3d4b988ae8769901c66345425e361d518aecbe4acbfc2567e416626a",
"sha256:c9959d49a77b0e07559e579f38b2f3711c2b8716b8410b320bf9713013215a1b",
"sha256:d8cdee92bc930d8b09d8bd2043cedd544d9c8bd7436a77678dd602467a993080",
"sha256:e15199cdb423316e15f108f51249e44eb156ae5dba232cb73be555324a1d49c2"
],
"version": "==1.4.2"
}
},
"develop": {}
}

7
taskcluster/docker/funsize-update-generator/requirements.in Normal file
View file

@ -0,0 +1,7 @@
aiohttp
awscli
mar
redo
requests
scriptworker
sh

301
taskcluster/docker/funsize-update-generator/requirements.txt Normal file
View file

@ -0,0 +1,301 @@
#
# This file is autogenerated by pip-compile
# To update, run:
#
# pip-compile --generate-hashes requirements.in
#
aiohttp==3.6.2 \
--hash=sha256:1e984191d1ec186881ffaed4581092ba04f7c61582a177b187d3a2f07ed9719e \
--hash=sha256:259ab809ff0727d0e834ac5e8a283dc5e3e0ecc30c4d80b3cd17a4139ce1f326 \
--hash=sha256:2f4d1a4fdce595c947162333353d4a44952a724fba9ca3205a3df99a33d1307a \
--hash=sha256:32e5f3b7e511aa850829fbe5aa32eb455e5534eaa4b1ce93231d00e2f76e5654 \
--hash=sha256:344c780466b73095a72c616fac5ea9c4665add7fc129f285fbdbca3cccf4612a \
--hash=sha256:460bd4237d2dbecc3b5ed57e122992f60188afe46e7319116da5eb8a9dfedba4 \
--hash=sha256:4c6efd824d44ae697814a2a85604d8e992b875462c6655da161ff18fd4f29f17 \
--hash=sha256:50aaad128e6ac62e7bf7bd1f0c0a24bc968a0c0590a726d5a955af193544bcec \
--hash=sha256:6206a135d072f88da3e71cc501c59d5abffa9d0bb43269a6dcd28d66bfafdbdd \
--hash=sha256:65f31b622af739a802ca6fd1a3076fd0ae523f8485c52924a89561ba10c49b48 \
--hash=sha256:ae55bac364c405caa23a4f2d6cfecc6a0daada500274ffca4a9230e7129eac59 \
--hash=sha256:b778ce0c909a2653741cb4b1ac7015b5c130ab9c897611df43ae6a58523cb965 \
# via -r requirements.in, scriptworker, taskcluster
aiomemoizettl==0.0.3 \
--hash=sha256:07a6becac60f6cd2604b9f2b73bcd9a50079a0b7b55e2a4e45b1eec5a3ea9659 \
--hash=sha256:0a80d2dc765e545263f515363b6700ec8cf86fa3968b529f56390b28e34f743d \
# via scriptworker
arrow==0.15.5 \
--hash=sha256:5390e464e2c5f76971b60ffa7ee29c598c7501a294bc9f5e6dadcb251a5d027b \
--hash=sha256:70729bcc831da496ca3cb4b7e89472c8e2d27d398908155e0796179f6d2d41ee \
# via scriptworker
asn1crypto==1.3.0 \
--hash=sha256:5a215cb8dc12f892244e3a113fe05397ee23c5c4ca7a69cd6e69811755efc42d \
--hash=sha256:831d2710d3274c8a74befdddaf9f17fcbf6e350534565074818722d6d615b315 \
# via mar
async-timeout==3.0.1 \
--hash=sha256:0c3c816a028d47f659d6ff5c745cb2acf1f966da1fe5c19c77a70282b25f4c5f \
--hash=sha256:4291ca197d287d274d0b6cb5d6f8f8f82d434ed288f962539ff18cc9012f9ea3 \
# via aiohttp, taskcluster
attrs==19.3.0 \
--hash=sha256:08a96c641c3a74e44eb59afb61a24f2cb9f4d7188748e76ba4bb5edfa3cb7d1c \
--hash=sha256:f7b7ce16570fe9965acd6d30101a28f62fb4a7f9e926b3bbc9b61f8b04247e72 \
# via aiohttp, jsonschema
awscli==1.18.13 \
--hash=sha256:197a1168a2c979752bb290b09cbbf8bf836507de1b99a19431ca9b0cde8a5f81 \
--hash=sha256:d74c5b097fe9288147c6045974eca9dc369055aa6ccce05fb551b0b95e813213 \
# via -r requirements.in
backports.lzma==0.0.14 \
--hash=sha256:16d8b68e4d3cd4e6c9ddb059850452946da3914c8a8e197a7f2b0954559f2df4 \
# via mar
botocore==1.15.13 \
--hash=sha256:265bf5e902e2f0854f12032304f2559f5d042e03c10a59e4f3ec43353be5b88f \
--hash=sha256:2fce3cc8fae3bbf89984f3065d9eecf0a906b96bee491a4fb54ad7ec7f4261dc \
# via awscli, s3transfer
certifi==2019.11.28 \
--hash=sha256:017c25db2a153ce562900032d5bc68e9f191e44e9a0f762f373977de9df1fbb3 \
--hash=sha256:25b64c7da4cd7479594d035c08c2d809eb4aab3a26e5a990ea98cc450c320f1f \
# via requests
cffi==1.14.0 \
--hash=sha256:001bf3242a1bb04d985d63e138230802c6c8d4db3668fb545fb5005ddf5bb5ff \
--hash=sha256:00789914be39dffba161cfc5be31b55775de5ba2235fe49aa28c148236c4e06b \
--hash=sha256:028a579fc9aed3af38f4892bdcc7390508adabc30c6af4a6e4f611b0c680e6ac \
--hash=sha256:14491a910663bf9f13ddf2bc8f60562d6bc5315c1f09c704937ef17293fb85b0 \
--hash=sha256:1cae98a7054b5c9391eb3249b86e0e99ab1e02bb0cc0575da191aedadbdf4384 \
--hash=sha256:2089ed025da3919d2e75a4d963d008330c96751127dd6f73c8dc0c65041b4c26 \
--hash=sha256:2d384f4a127a15ba701207f7639d94106693b6cd64173d6c8988e2c25f3ac2b6 \
--hash=sha256:337d448e5a725bba2d8293c48d9353fc68d0e9e4088d62a9571def317797522b \
--hash=sha256:399aed636c7d3749bbed55bc907c3288cb43c65c4389964ad5ff849b6370603e \
--hash=sha256:3b911c2dbd4f423b4c4fcca138cadde747abdb20d196c4a48708b8a2d32b16dd \
--hash=sha256:3d311bcc4a41408cf5854f06ef2c5cab88f9fded37a3b95936c9879c1640d4c2 \
--hash=sha256:62ae9af2d069ea2698bf536dcfe1e4eed9090211dbaafeeedf5cb6c41b352f66 \
--hash=sha256:66e41db66b47d0d8672d8ed2708ba91b2f2524ece3dee48b5dfb36be8c2f21dc \
--hash=sha256:675686925a9fb403edba0114db74e741d8181683dcf216be697d208857e04ca8 \
--hash=sha256:7e63cbcf2429a8dbfe48dcc2322d5f2220b77b2e17b7ba023d6166d84655da55 \
--hash=sha256:8a6c688fefb4e1cd56feb6c511984a6c4f7ec7d2a1ff31a10254f3c817054ae4 \
--hash=sha256:8c0ffc886aea5df6a1762d0019e9cb05f825d0eec1f520c51be9d198701daee5 \
--hash=sha256:95cd16d3dee553f882540c1ffe331d085c9e629499ceadfbda4d4fde635f4b7d \
--hash=sha256:99f748a7e71ff382613b4e1acc0ac83bf7ad167fb3802e35e90d9763daba4d78 \
--hash=sha256:b8c78301cefcf5fd914aad35d3c04c2b21ce8629b5e4f4e45ae6812e461910fa \
--hash=sha256:c420917b188a5582a56d8b93bdd8e0f6eca08c84ff623a4c16e809152cd35793 \
--hash=sha256:c43866529f2f06fe0edc6246eb4faa34f03fe88b64a0a9a942561c8e22f4b71f \
--hash=sha256:cab50b8c2250b46fe738c77dbd25ce017d5e6fb35d3407606e7a4180656a5a6a \
--hash=sha256:cef128cb4d5e0b3493f058f10ce32365972c554572ff821e175dbc6f8ff6924f \
--hash=sha256:cf16e3cf6c0a5fdd9bc10c21687e19d29ad1fe863372b5543deaec1039581a30 \
--hash=sha256:e56c744aa6ff427a607763346e4170629caf7e48ead6921745986db3692f987f \
--hash=sha256:e577934fc5f8779c554639376beeaa5657d54349096ef24abe8c74c5d9c117c3 \
--hash=sha256:f2b0fa0c01d8a0c7483afd9f31d7ecf2d71760ca24499c8697aeb5ca37dc090c \
# via cryptography
chardet==3.0.4 \
--hash=sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae \
--hash=sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691 \
# via aiohttp, requests
click==7.0 \
--hash=sha256:2335065e6395b9e67ca716de5f7526736bfa6ceead690adf616d925bdc622b13 \
--hash=sha256:5b94b49521f6456670fdb30cd82a4eca9412788a93fa6dd6df72c94d5a8ff2d7 \
# via mar
colorama==0.4.3 \
--hash=sha256:7d73d2a99753107a36ac6b455ee49046802e59d9d076ef8e47b61499fa29afff \
--hash=sha256:e96da0d330793e2cb9485e9ddfd918d456036c7149416295932478192f4436a1 \
# via awscli
construct==2.10.56 \
--hash=sha256:97ba13edcd98546f10f7555af41c8ce7ae9d8221525ec4062c03f9adbf940661 \
# via mar
cryptography==2.8 \
--hash=sha256:02079a6addc7b5140ba0825f542c0869ff4df9a69c360e339ecead5baefa843c \
--hash=sha256:1df22371fbf2004c6f64e927668734070a8953362cd8370ddd336774d6743595 \
--hash=sha256:369d2346db5934345787451504853ad9d342d7f721ae82d098083e1f49a582ad \
--hash=sha256:3cda1f0ed8747339bbdf71b9f38ca74c7b592f24f65cdb3ab3765e4b02871651 \
--hash=sha256:44ff04138935882fef7c686878e1c8fd80a723161ad6a98da31e14b7553170c2 \
--hash=sha256:4b1030728872c59687badcca1e225a9103440e467c17d6d1730ab3d2d64bfeff \
--hash=sha256:58363dbd966afb4f89b3b11dfb8ff200058fbc3b947507675c19ceb46104b48d \
--hash=sha256:6ec280fb24d27e3d97aa731e16207d58bd8ae94ef6eab97249a2afe4ba643d42 \
--hash=sha256:7270a6c29199adc1297776937a05b59720e8a782531f1f122f2eb8467f9aab4d \
--hash=sha256:73fd30c57fa2d0a1d7a49c561c40c2f79c7d6c374cc7750e9ac7c99176f6428e \
--hash=sha256:7f09806ed4fbea8f51585231ba742b58cbcfbfe823ea197d8c89a5e433c7e912 \
--hash=sha256:90df0cc93e1f8d2fba8365fb59a858f51a11a394d64dbf3ef844f783844cc793 \
--hash=sha256:971221ed40f058f5662a604bd1ae6e4521d84e6cad0b7b170564cc34169c8f13 \
--hash=sha256:a518c153a2b5ed6b8cc03f7ae79d5ffad7315ad4569b2d5333a13c38d64bd8d7 \
--hash=sha256:b0de590a8b0979649ebeef8bb9f54394d3a41f66c5584fff4220901739b6b2f0 \
--hash=sha256:b43f53f29816ba1db8525f006fa6f49292e9b029554b3eb56a189a70f2a40879 \
--hash=sha256:d31402aad60ed889c7e57934a03477b572a03af7794fa8fb1780f21ea8f6551f \
--hash=sha256:de96157ec73458a7f14e3d26f17f8128c959084931e8997b9e655a39c8fde9f9 \
--hash=sha256:df6b4dca2e11865e6cfbfb708e800efb18370f5a46fd601d3755bc7f85b3a8a2 \
--hash=sha256:ecadccc7ba52193963c0475ac9f6fa28ac01e01349a2ca48509667ef41ffd2cf \
--hash=sha256:fb81c17e0ebe3358486cd8cc3ad78adbae58af12fc2bf2bc0bb84e8090fa5ce8 \
# via jwcrypto, mar, scriptworker
dictdiffer==0.8.1 \
--hash=sha256:1adec0d67cdf6166bda96ae2934ddb5e54433998ceab63c984574d187cc563d2 \
--hash=sha256:d79d9a39e459fe33497c858470ca0d2e93cb96621751de06d631856adfd9c390 \
# via scriptworker
docutils==0.15.2 \
--hash=sha256:6c4f696463b79f1fb8ba0c594b63840ebd41f059e92b31957c46b74a4599b6d0 \
--hash=sha256:9e4d7ecfc600058e07ba661411a2b7de2fd0fafa17d1a7f7361cd47b1175c827 \
--hash=sha256:a2aeea129088da402665e92e0b25b04b073c04b2dce4ab65caaa38b7ce2e1a99 \
# via awscli, botocore
github3.py==1.3.0 \
--hash=sha256:15a115c18f7bfcf934dfef7ab103844eb9f620c586bad65967708926da47cbda \
--hash=sha256:50833b5da35546b8cced0e8d7ff4c50a9afc2c8e46cc4d07dc4b66d26467c708 \
# via scriptworker
idna-ssl==1.1.0 \
--hash=sha256:a933e3bb13da54383f9e8f35dc4f9cb9eb9b3b78c6b36f311254d6d0d92c6c7c \
# via aiohttp
idna==2.9 \
--hash=sha256:7588d1c14ae4c77d74036e8c22ff447b26d0fde8f007354fd48a7814db15b7cb \
--hash=sha256:a068a21ceac8a4d63dbfd964670474107f541babbd2250d61922f029858365fa \
# via idna-ssl, requests, yarl
immutabledict==0.2.0 \
--hash=sha256:43dde3e55dcb539537ae6189fb6b09a1d01e94db304e4506e94ca2d45ec14c47 \
--hash=sha256:7881e44098f13dd12d6fec00551d564433cb46776e8b2f3453128f715df4376a \
# via scriptworker
importlib-metadata==1.5.0 \
--hash=sha256:06f5b3a99029c7134207dd882428a66992a9de2bef7c2b699b5641f9886c3302 \
--hash=sha256:b97607a1a18a5100839aec1dc26a1ea17ee0d93b20b0f008d80a5a050afb200b \
# via jsonschema
jmespath==0.9.5 \
--hash=sha256:695cb76fa78a10663425d5b73ddc5714eb711157e52704d69be03b1a02ba4fec \
--hash=sha256:cca55c8d153173e21baa59983015ad0daf603f9cb799904ff057bfb8ff8dc2d9 \
# via botocore
json-e==3.0.2 \
--hash=sha256:35dc96ce04f0caece560dc18266af7f0ffb780955273ffa20814be65447cad49 \
# via scriptworker
jsonschema==3.2.0 \
--hash=sha256:4e5b3cf8216f577bee9ce139cbe72eca3ea4f292ec60928ff24758ce626cd163 \
--hash=sha256:c8a85b28d377cc7737e46e2d9f2b4f44ee3c0e1deac6bf46ddefc7187d30797a \
# via scriptworker
jwcrypto==0.7 \
--hash=sha256:618ded1d25d3f806a1ab05cee42633a5a2787af33fca8d8f539b0aa1478b3728 \
--hash=sha256:adbe1f6266cde35d40d5de6d1419612b3bd4c869b9332c88c9d7a9163d305100 \
# via github3.py
mar==3.1.0 \
--hash=sha256:1939df482f2d3f6221405da00d6286d77d2dd60d372a0fd37532a8f00544f64f \
--hash=sha256:5d2904a063f0da625e37515fa9eb340082e69fa1c00fdbeb82d28b7ff3c51e28 \
# via -r requirements.in
mohawk==1.1.0 \
--hash=sha256:3ed296a30453d0b724679e0fd41e4e940497f8e461a9a9c3b7f36e43bab0fa09 \
--hash=sha256:d2a0e3ab10a209cc79e95e28f2dd54bd4a73fd1998ffe27b7ba0f962b6be9723 \
# via taskcluster
multidict==4.7.5 \
--hash=sha256:317f96bc0950d249e96d8d29ab556d01dd38888fbe68324f46fd834b430169f1 \
--hash=sha256:42f56542166040b4474c0c608ed051732033cd821126493cf25b6c276df7dd35 \
--hash=sha256:4b7df040fb5fe826d689204f9b544af469593fb3ff3a069a6ad3409f742f5928 \
--hash=sha256:544fae9261232a97102e27a926019100a9db75bec7b37feedd74b3aa82f29969 \
--hash=sha256:620b37c3fea181dab09267cd5a84b0f23fa043beb8bc50d8474dd9694de1fa6e \
--hash=sha256:6e6fef114741c4d7ca46da8449038ec8b1e880bbe68674c01ceeb1ac8a648e78 \
--hash=sha256:7774e9f6c9af3f12f296131453f7b81dabb7ebdb948483362f5afcaac8a826f1 \
--hash=sha256:85cb26c38c96f76b7ff38b86c9d560dea10cf3459bb5f4caf72fc1bb932c7136 \
--hash=sha256:a326f4240123a2ac66bb163eeba99578e9d63a8654a59f4688a79198f9aa10f8 \
--hash=sha256:ae402f43604e3b2bc41e8ea8b8526c7fa7139ed76b0d64fc48e28125925275b2 \
--hash=sha256:aee283c49601fa4c13adc64c09c978838a7e812f85377ae130a24d7198c0331e \
--hash=sha256:b51249fdd2923739cd3efc95a3d6c363b67bbf779208e9f37fd5e68540d1a4d4 \
--hash=sha256:bb519becc46275c594410c6c28a8a0adc66fe24fef154a9addea54c1adb006f5 \
--hash=sha256:c2c37185fb0af79d5c117b8d2764f4321eeb12ba8c141a95d0aa8c2c1d0a11dd \
--hash=sha256:dc561313279f9d05a3d0ffa89cd15ae477528ea37aa9795c4654588a3287a9ab \
--hash=sha256:e439c9a10a95cb32abd708bb8be83b2134fa93790a4fb0535ca36db3dda94d20 \
--hash=sha256:fc3b4adc2ee8474cb3cd2a155305d5f8eda0a9c91320f83e55748e1fcb68f8e3 \
# via aiohttp, yarl
pyasn1==0.4.8 \
--hash=sha256:39c7e2ec30515947ff4e87fb6f456dfc6e84857d34be479c9d4a4ba4bf46aa5d \
--hash=sha256:aef77c9fb94a3ac588e87841208bdec464471d9871bd5050a287cc9a475cd0ba \
# via rsa
pycparser==2.19 \
--hash=sha256:a988718abfad80b6b157acce7bf130a30876d27603738ac39f140993246b25b3 \
# via cffi
pyrsistent==0.15.7 \
--hash=sha256:cdc7b5e3ed77bed61270a47d35434a30617b9becdf2478af76ad2c6ade307280 \
# via jsonschema
python-dateutil==2.8.1 \
--hash=sha256:73ebfe9dbf22e832286dafa60473e4cd239f8592f699aa5adaf10050e6e1823c \
--hash=sha256:75bb3f31ea686f1197762692a9ee6a7550b59fc6ca3a1f4b5d7e32fb98e2da2a \
# via arrow, botocore, github3.py
pyyaml==5.3 \
--hash=sha256:059b2ee3194d718896c0ad077dd8c043e5e909d9180f387ce42012662a4946d6 \
--hash=sha256:1cf708e2ac57f3aabc87405f04b86354f66799c8e62c28c5fc5f88b5521b2dbf \
--hash=sha256:24521fa2890642614558b492b473bee0ac1f8057a7263156b02e8b14c88ce6f5 \
--hash=sha256:4fee71aa5bc6ed9d5f116327c04273e25ae31a3020386916905767ec4fc5317e \
--hash=sha256:70024e02197337533eef7b85b068212420f950319cc8c580261963aefc75f811 \
--hash=sha256:74782fbd4d4f87ff04159e986886931456a1894c61229be9eaf4de6f6e44b99e \
--hash=sha256:940532b111b1952befd7db542c370887a8611660d2b9becff75d39355303d82d \
--hash=sha256:cb1f2f5e426dc9f07a7681419fe39cee823bb74f723f36f70399123f439e9b20 \
--hash=sha256:dbbb2379c19ed6042e8f11f2a2c66d39cceb8aeace421bfc29d085d93eda3689 \
--hash=sha256:e3a057b7a64f1222b56e47bcff5e4b94c4f61faac04c7c4ecb1985e18caa3994 \
--hash=sha256:e9f45bd5b92c7974e59bcd2dcc8631a6b6cc380a904725fce7bc08872e691615 \
# via awscli, scriptworker
redo==2.0.3 \
--hash=sha256:36784bf8ae766e14f9db0e377ccfa02835d648321d2007b6ae0bf4fd612c0f94 \
--hash=sha256:71161cb0e928d824092a5f16203939bbc0867ce4c4685db263cf22c3ae7634a8 \
# via -r requirements.in
requests==2.23.0 \
--hash=sha256:43999036bfa82904b6af1d99e4882b560e5e2c68e5c4b0aa03b655f3d7d73fee \
--hash=sha256:b3f43d496c6daba4493e7c431722aeb7dbc6288f52a6e04e7b6023b0247817e6 \
# via -r requirements.in, github3.py, taskcluster
rsa==3.4.2 \
--hash=sha256:25df4e10c263fb88b5ace923dd84bf9aa7f5019687b5e55382ffcdb8bede9db5 \
--hash=sha256:43f682fea81c452c98d09fc316aae12de6d30c4b5c84226642cf8f8fd1c93abd \
# via awscli
s3transfer==0.3.3 \
--hash=sha256:2482b4259524933a022d59da830f51bd746db62f047d6eb213f2f8855dcb8a13 \
--hash=sha256:921a37e2aefc64145e7b73d50c71bb4f26f46e4c9f414dc648c6245ff92cf7db \
# via awscli
scriptworker==32.0.3 \
--hash=sha256:a4d428a2c9db1b0dbabb86c0badf850e919ed83ffaba701185370ebe3dd5f7aa \
--hash=sha256:d60976a94ba86d0c4b9f23536ce3cb026281c726d00da12b6546a8dd80cb4d1b \
# via -r requirements.in
sh==1.12.14 \
--hash=sha256:ae3258c5249493cebe73cb4e18253a41ed69262484bad36fdb3efcb8ad8870bb \
--hash=sha256:b52bf5833ed01c7b5c5fb73a7f71b3d98d48e9b9b8764236237bdc7ecae850fc \
# via -r requirements.in
six==1.14.0 \
--hash=sha256:236bdbdce46e6e6a3d61a337c0f8b763ca1e8717c03b369e87a7ec7ce1319c0a \
--hash=sha256:8f3cd2e254d8f793e7f3d6d9df77b92252b52637291d0f0da013c76ea2724b6c \
# via cryptography, jsonschema, mohawk, pyrsistent, python-dateutil, taskcluster
slugid==2.0.0 \
--hash=sha256:a950d98b72691178bdd4d6c52743c4a2aa039207cf7a97d71060a111ff9ba297 \
--hash=sha256:aec8b0e01c4ad32e38e12d609eab3ec912fd129aaf6b2ded0199b56a5f8fd67c \
# via taskcluster
taskcluster-urls==12.1.0 \
--hash=sha256:1dc740c32c7beb31e11ed7ccf9da2d47a504acdb3170c8900649433b0fd16fb2 \
--hash=sha256:4a62c776aeba6d45044789a8845ec4d8521bc1bb6ebfc86d79ee759bcdd4f2f7 \
# via taskcluster
taskcluster==25.4.0 \
--hash=sha256:017e626ec7211cc250f6c59ac4fbeb3ca7d7ecf0a253eb4f21a694bc5d02ae36 \
--hash=sha256:0a8fc965441ab60aafb83bcf2b670afd3f8817b13067561e084b4d2528f1e75c \
--hash=sha256:a9d10a794a065ea17a11fa8ee4861e57ea702362f70d00dc97d5a4f685a70d4f \
# via scriptworker
typing-extensions==3.7.4.1 \
--hash=sha256:091ecc894d5e908ac75209f10d5b4f118fbdb2eb1ede6a63544054bb1edb41f2 \
--hash=sha256:910f4656f54de5993ad9304959ce9bb903f90aadc7c67a0bef07e678014e892d \
--hash=sha256:cf8b63fedea4d89bab840ecbb93e75578af28f76f66c35889bd7065f5af88575 \
# via aiohttp
uritemplate==3.0.1 \
--hash=sha256:07620c3f3f8eed1f12600845892b0e036a2420acf513c53f7de0abd911a5894f \
--hash=sha256:5af8ad10cec94f215e3f48112de2022e1d5a37ed427fbd88652fa908f2ab7cae \
# via github3.py
urllib3==1.25.8 \
--hash=sha256:2f3db8b19923a873b3e5256dc9c2dedfa883e33d87c690d9c7913e1f40673cdc \
--hash=sha256:87716c2d2a7121198ebcb7ce7cccf6ce5e9ba539041cfbaeecfb641dc0bf6acc \
# via botocore, requests
yarl==1.4.2 \
--hash=sha256:0c2ab325d33f1b824734b3ef51d4d54a54e0e7a23d13b86974507602334c2cce \
--hash=sha256:0ca2f395591bbd85ddd50a82eb1fde9c1066fafe888c5c7cc1d810cf03fd3cc6 \
--hash=sha256:2098a4b4b9d75ee352807a95cdf5f10180db903bc5b7270715c6bbe2551f64ce \
--hash=sha256:25e66e5e2007c7a39541ca13b559cd8ebc2ad8fe00ea94a2aad28a9b1e44e5ae \
--hash=sha256:26d7c90cb04dee1665282a5d1a998defc1a9e012fdca0f33396f81508f49696d \
--hash=sha256:308b98b0c8cd1dfef1a0311dc5e38ae8f9b58349226aa0533f15a16717ad702f \
--hash=sha256:3ce3d4f7c6b69c4e4f0704b32eca8123b9c58ae91af740481aa57d7857b5e41b \
--hash=sha256:58cd9c469eced558cd81aa3f484b2924e8897049e06889e8ff2510435b7ef74b \
--hash=sha256:5b10eb0e7f044cf0b035112446b26a3a2946bca9d7d7edb5e54a2ad2f6652abb \
--hash=sha256:6faa19d3824c21bcbfdfce5171e193c8b4ddafdf0ac3f129ccf0cdfcb083e462 \
--hash=sha256:944494be42fa630134bf907714d40207e646fd5a94423c90d5b514f7b0713fea \
--hash=sha256:a161de7e50224e8e3de6e184707476b5a989037dcb24292b391a3d66ff158e70 \
--hash=sha256:a4844ebb2be14768f7994f2017f70aca39d658a96c786211be5ddbe1c68794c1 \
--hash=sha256:c2b509ac3d4b988ae8769901c66345425e361d518aecbe4acbfc2567e416626a \
--hash=sha256:c9959d49a77b0e07559e579f38b2f3711c2b8716b8410b320bf9713013215a1b \
--hash=sha256:d8cdee92bc930d8b09d8bd2043cedd544d9c8bd7436a77678dd602467a993080 \
--hash=sha256:e15199cdb423316e15f108f51249e44eb156ae5dba232cb73be555324a1d49c2 \
# via aiohttp
zipp==3.1.0 \
--hash=sha256:aa36550ff0c0b7ef7fa639055d797116ee891440eac1a56f378e2d3179e0320b \
--hash=sha256:c599e4d75c98f6798c509911d08a22e6c021d074469042177c8c86fb92eefd96 \
# via importlib-metadata
# WARNING: The following packages were not pinned, but pip requires them to be
# pinned when the requirements file includes hashes. Consider using the --allow-unsafe flag.
# setuptools

4
taskcluster/docker/funsize-update-generator/runme.sh
View file

@ -10,7 +10,7 @@ ARTIFACTS_DIR="/home/worker/artifacts"
mkdir -p "$ARTIFACTS_DIR" mkdir -p "$ARTIFACTS_DIR"
# duplicate the functionality of taskcluster-lib-urls, but in bash.. # duplicate the functionality of taskcluster-lib-urls, but in bash..
queue_base="$TASKCLUSTER_ROOT_URL/api/queue/v1" queue_base="${TASKCLUSTER_ROOT_URL%/}/api/queue/v1"
curl --location --retry 10 --retry-delay 10 -o /home/worker/task.json "$queue_base/task/$TASK_ID" curl --location --retry 10 --retry-delay 10 -o /home/worker/task.json "$queue_base/task/$TASK_ID"
@ -51,7 +51,7 @@ fi
# EXTRA_PARAMS is optional # EXTRA_PARAMS is optional
# shellcheck disable=SC2086 # shellcheck disable=SC2086
pipenv run /home/worker/bin/funsize.py \ python3.8 /home/worker/bin/funsize.py \
--artifacts-dir "$ARTIFACTS_DIR" \ --artifacts-dir "$ARTIFACTS_DIR" \
--task-definition /home/worker/task.json \ --task-definition /home/worker/task.json \
--sha1-signing-cert "/home/worker/keys/${SHA1_SIGNING_CERT}.pubkey" \ --sha1-signing-cert "/home/worker/keys/${SHA1_SIGNING_CERT}.pubkey" \

14
taskcluster/docker/pipfile-updates/README.md
View file

@ -1,14 +0,0 @@
==Pipfile Updates==
This docker image contains the necessary dependencies and scripts to update in-tree Pipfile.lock's,
produce a diff, and submit it to Phabricator.
==Quick Start==
```sh
docker build -t pipfile-update --no-cache --rm .
docker run -e PYTHON3="1" -e BRANCH="mozilla-central" -e PIPFILE_DIRECTORY="taskcluster/docker/funsize-update-generator" pipfile-update
```

0
taskcluster/docker/pipfile-updates/Dockerfile → taskcluster/docker/python-dependency-update/Dockerfile
View file

15
taskcluster/docker/python-dependency-update/README.md Normal file
View file

@ -0,0 +1,15 @@
==Python Dependency Updates==
This docker image contains the necessary dependencies and scripts to update
in-tree requirement.txt produced by `pip-compile --generate-hashes`, produce a
diff, and submit it to Phabricator.
==Quick Start==
```sh
docker build -t python-dependency-update --no-cache --rm .
docker run -e PYTHON3="1" -e BRANCH="mozilla-central" -e REQUIREMENTS_FILE="taskcluster/docker/funsize-update-generator/requirements.in" python-dependency-update
```

6
taskcluster/docker/pipfile-updates/runme.sh → taskcluster/docker/python-dependency-update/runme.sh
View file

@ -4,12 +4,12 @@ set -xe
# Things to be set by task definition. # Things to be set by task definition.
# -b branch # -b branch
# -p pipfile_directory # -f requirements_file
# -3 use python3 # -3 use python3
test "${BRANCH}" test "${BRANCH}"
test "${PIPFILE_DIRECTORY}" test "${REQUIREMENTS_FILE}"
PIP_ARG="-2" PIP_ARG="-2"
if [ -n "${PYTHON3}" ]; then if [ -n "${PYTHON3}" ]; then
@ -58,4 +58,4 @@ fi
export HGPLAIN=1 export HGPLAIN=1
# shellcheck disable=SC2086 # shellcheck disable=SC2086
/home/worker/scripts/update_pipfiles.sh -b "${BRANCH}" -p "${PIPFILE_DIRECTORY}" ${PIP_ARG} /home/worker/scripts/update_pipfiles.sh -b "${BRANCH}" -f "${REQUIREMENTS_FILE}" ${PIP_ARG}

30
taskcluster/docker/pipfile-updates/scripts/update_pipfiles.sh → taskcluster/docker/python-dependency-update/scripts/update_pipfiles.sh
View file

@ -7,7 +7,7 @@ function usage {
Usage: $(basename "$0") -h # Displays this usage/help text Usage: $(basename "$0") -h # Displays this usage/help text
Usage: $(basename "$0") -x # lists exit codes Usage: $(basename "$0") -x # lists exit codes
Usage: $(basename "$0") -b branch -p pipfile_directory [-2] [-3] Usage: $(basename "$0") -b branch -r REQUIREMENTS_FILE [-2] [-3]
EOF EOF
} }
@ -18,8 +18,7 @@ COMMIT_AUTHOR='ffxbld <ffxbld@mozilla.com>'
REPODIR='' REPODIR=''
HGHOST="hg.mozilla.org" HGHOST="hg.mozilla.org"
BASEDIR="${HOME}" BASEDIR="${HOME}"
PIPFILE_DIRECTORY="" REQUIREMENTS_FILE=""
DIFF_ARTIFACT="${ARTIFACTS_DIR}/Pipfile.lock.diff"
HG="$(command -v hg)" HG="$(command -v hg)"
@ -53,7 +52,7 @@ function push_repo {
# Clean up older review requests # Clean up older review requests
# Turn Needs Review D624: No bug, Automated HSTS ... # Turn Needs Review D624: No bug, Automated HSTS ...
# into D624 # into D624
for diff in $($ARC list | grep "Needs Review" | grep "${PIPFILE_DIRECTORY} pipfile-update" | awk 'match($0, /D[0-9]+[^: ]/) { print substr($0, RSTART, RLENGTH) }') for diff in $($ARC list | grep "Needs Review" | grep "${REQUIREMENTS_FILE} pip-update" | awk 'match($0, /D[0-9]+[^: ]/) { print substr($0, RSTART, RLENGTH) }')
do do
echo "Removing old request $diff" echo "Removing old request $diff"
# There is no 'arc abandon', see bug 1452082 # There is no 'arc abandon', see bug 1452082
@ -63,9 +62,9 @@ function push_repo {
$ARC diff --verbatim --reviewers "${REVIEWERS}" $ARC diff --verbatim --reviewers "${REVIEWERS}"
} }
function update_pipfile { function update_requirements {
pushd "${REPODIR}/${1}" pushd "${REPODIR}/${1}"
pipenv update pip-compile --generate-hashes "${2}"
popd popd
} }
@ -79,7 +78,7 @@ while [ $# -gt 0 ]; do
-r) REPODIR="$2"; shift ;; -r) REPODIR="$2"; shift ;;
-2) PIP="pip" ;; -2) PIP="pip" ;;
-3) PIP="pip3" ;; -3) PIP="pip3" ;;
-p) PIPFILE_DIRECTORY="$2"; shift ;; -f) REQUIREMENTS_FILE="$2"; shift ;;
-*) usage -*) usage
exit 11 ;; exit 11 ;;
*) break ;; # terminate while loop *) break ;; # terminate while loop
@ -102,7 +101,7 @@ if [ "${BRANCH}" == "" ]; then
fi fi
if [ "${REPODIR}" == "" ]; then if [ "${REPODIR}" == "" ]; then
REPODIR="$(basename "${BRANCH}")" REPODIR="${BASEDIR}/$(basename "${BRANCH}")"
fi fi
if [ "${BRANCH}" == "mozilla-central" ]; then if [ "${BRANCH}" == "mozilla-central" ]; then
@ -115,13 +114,18 @@ fi
clone_repo clone_repo
${PIP} install pipenv ${PIP} install pip-tools
update_pipfile "${PIPFILE_DIRECTORY}" requirements_basefile="$(basename "${REQUIREMENTS_FILE}")"
echo "INFO: diffing old/new Pipfile.lock into ${DIFF_ARTIFACT}" requirements_dir="$(dirname "${REQUIREMENTS_FILE}")"
${HG} -R "${REPODIR}" diff "${BASEDIR}/${BRANCH}/${PIPFILE_DIRECTORY}/Pipfile.lock" | tee "${DIFF_ARTIFACT}" update_requirements "${requirements_dir}" "${requirements_basefile}"
requirements_newfile="${requirements_basefile%%.in}.txt"
DIFF_ARTIFACT="${ARTIFACTS_DIR}/${requirements_newfile}.diff"
COMMIT_MESSAGE="No Bug, ${PIPFILE_DIRECTORY} pipfile-update." echo "INFO: diffing old/new ${requirements_newfile} into ${DIFF_ARTIFACT}"
${HG} -R "${REPODIR}" diff "${BASEDIR}/${BRANCH}/${requirements_dir}/${requirements_newfile}" | tee "${DIFF_ARTIFACT}"
COMMIT_MESSAGE="No Bug, ${requirements_dir}/${requirements_newfile} pip-update."
if ${HG} -R "${REPODIR}" commit -u "${COMMIT_AUTHOR}" -m "${COMMIT_MESSAGE}" if ${HG} -R "${REPODIR}" commit -u "${COMMIT_AUTHOR}" -m "${COMMIT_MESSAGE}"
then then

0
taskcluster/docker/pipfile-updates/setup.sh → taskcluster/docker/python-dependency-update/setup.sh
View file

8
taskcluster/docs/kinds.rst
View file

@ -570,10 +570,10 @@ repo-update
Repo-Update tasks are tasks that perform some action on the project repo itself, Repo-Update tasks are tasks that perform some action on the project repo itself,
in order to update its state in some way. in order to update its state in some way.
pipfile-update python-dependency-update
-------------- ------------------------
Pipfile-update tasks generate update Pipfile.lock for in-tree Pipfiles, and attach Python-dependency-update runs `pip-compile --generate-hashes` against the specified `requirements.in` and
patches with the updates to Phabricator. submits patches to Phabricator.
partials partials
-------- --------

6
taskcluster/taskgraph/target_tasks.py
View file

@ -651,13 +651,13 @@ def target_tasks_chromium_update(full_task_graph, parameters, graph_config):
'fetch-mac-chromium'] 'fetch-mac-chromium']
@_target_task('pipfile_update') @_target_task('python_dependency_update')
def target_tasks_pipfile_update(full_task_graph, parameters, graph_config): def target_tasks_python_update(full_task_graph, parameters, graph_config):
"""Select the set of tasks required to perform nightly in-tree pipfile updates """Select the set of tasks required to perform nightly in-tree pipfile updates
""" """
def filter(task): def filter(task):
# For now any task in the repo-update kind is ok # For now any task in the repo-update kind is ok
return task.kind in ['pipfile-update'] return task.kind in ['python-dependency-update']
return [l for l, t in full_task_graph.tasks.iteritems() if filter(t)] return [l for l, t in full_task_graph.tasks.iteritems() if filter(t)]

0
taskcluster/taskgraph/transforms/pipfile_update.py → taskcluster/taskgraph/transforms/python_update.py
View file