There was quite a bit of discussion of this in `#build` on IRC,
and the consensus was that geckodriver should be built as a
stand-alone Rust crate and not as part of Firefox/Gecko (say, as a new
--enable-project target). This follows that approach, and the
expression, modeled off of cbindgen but updated to cross compile from
a Linux host to all targets, is pretty straight-forward.
A sparse profile would be nice, but the way that the Gecko Cargo
workspace works means that the profile must accumulate Rust code from
many locations.
If we want to, eventually testing/geckodriver can be removed from the
top-level Rust workspace, the geckodriver-signing tasks migrated to
these toolchain tasks, consumers migrated to the signing tasks, and
geckodriver removed from the "common" test archive.
Differential Revision: https://phabricator.services.mozilla.com/D43646
--HG--
rename : taskcluster/scripts/misc/vs-setup.sh => taskcluster/scripts/misc/vs-setup32.sh
extra : moz-landing-system : lando
This version includes several fixes for sccache-dist in addition
to changes that make sccache compatible with cargo pipelining.
Differential Revision: https://phabricator.services.mozilla.com/D36075
--HG--
extra : moz-landing-system : lando
this change comprises the in-tree changes required to make use of sccache in gcp.
specifically:
- a gcp metadata lookup for availability-zone is added to mozconfig, enabling a build to determine its regional gcp sccache bucket
- the sccache cargo build command is modified to include the gcs feature when the environment contains gcs configuration
note that further changes are required on infra to support sccache use. the required changes already [exist](https://github.com/mozilla-releng/OpenCloudConfig/commit/1d515dc) and are enabled for gcp windows infra, including:
- a json credential file on the build instance filesystem, containing credentials valid for the appropriate scm level bucket for the gcp region
- an `SCCACHE_GCS_KEY_PATH` env variable containing the path to the json credential file
- an `SCCACHE_GCS_RW_MODE` env variable containg the text `READ_WRITE`
- sccache buckets must exist for each region and scm levels 1 & 3
- credentials for scm level 1 buckets **must not** be valid for scm level 3 buckets
on gcp systems which do not contain credential files and the above mentioned env variables (eg gecko-[1-3]-b-linux), sccache should fail gracefully without breaking builds.
Differential Revision: https://phabricator.services.mozilla.com/D29622
--HG--
extra : moz-landing-system : lando
It turns out version 15.4.2 spawns a vctip process that sticks after the
build, and since bug 1527798, that breaks unmounting caches because the
process has a handle on msvcp140.dll, which lies in the source
directory. The problem goes away with 15.8.4, so upgrade all toolchain
tasks to that.
That's the same version as we're using on x86/x86-64 MSVC builds.
Differential Revision: https://phabricator.services.mozilla.com/D19752
to give time to docker images and toolchains to build.
--HG--
rename : taskcluster/docker/debian-raw/cloud-mirror-workaround.sh => taskcluster/docker/debian-base/cloud-mirror-workaround.sh
rename : taskcluster/docker/debian-raw/setup_packages.sh => taskcluster/docker/debian-base/setup_packages.sh
It turns out version 15.4.2 spawns a vctip process that sticks after the
build, and since bug 1527798, that breaks unmounting caches because the
process has a handle on msvcp140.dll, which lies in the source
directory. The problem goes away with 15.8.4, so upgrade all toolchain
tasks to that.
That's the same version as we're using on x86/x86-64 MSVC builds.
Differential Revision: https://phabricator.services.mozilla.com/D19752
It turns out version 15.4.2 spawns a vctip process that sticks after the
build, and since bug 1527798, that breaks unmounting caches because the
process has a handle on msvcp140.dll, which lies in the source
directory. The problem goes away with 15.8.4, so upgrade all toolchain
tasks to that.
That's the same version as we're using on x86/x86-64 MSVC builds.
Differential Revision: https://phabricator.services.mozilla.com/D19752
Bug 1476604 tried the same thing for different reasons, but ultimately
failed because of unsupported flags for PGO with clang-cl. However, to
unblock switching to clang for Linux nightlies, we go ahead and upgrade
only the Linux sccache.
Differential Revision: https://phabricator.services.mozilla.com/D5375
--HG--
extra : moz-landing-system : lando
The URL is now being redirected to
https://www.openssl.org/source/old/1.1.0/openssl-1.1.0g.tar.gz. Let's
add a -L so we follow redirects automatically.
MozReview-Commit-ID: AuZ98jGidzl
--HG--
extra : rebase_source : 07e61558024e789df45d8e2ab67ab5ad9d3d355b
It was failing to build with the GCC/binutils on the CentOS-based docker
image, but it doesn't with the Debian-based one, so we can remove the
dependency on the gcc toolchain task. This allows sccache to remain
untouched when we change the gcc build scripts, and more importantly,
this allows it to depend on no toolchain that requires building things.
This makes it now possible to use sccache as a dependency for all other
toolchains jobs that compile, if that's beneficial (which might not be
the case, given the current sccache retention time, but at least it's a
viable option, now)
When adding sccache toolchain jobs in bug 1381772, building with gcc
failed, and building with clang worked, so I just went with the path of
least resistance. That's however a suboptimal position in the dependency
graph, so it's still preferable to use gcc if possible.
Looking exactly how it fails, it turns out it's because without CC being
set, ring wants to build with "cc", which ends up being the system gcc
instead of ours (our gcc archive doesn't provide "cc", only "gcc"), and
it is too old to support the compiler flags ring uses.
So setting CC does the trick.
--HG--
extra : rebase_source : 4c657664957dff1f7aebe470e0440a52c9e280e5
Our current sccache build links in openssl's libraries dynamically. The
sonames of the dynamic libraries linked in are specific to the
CentOS/Fedora-ish systems that we build on; attempting to run the
generated sccache binaries on different systems (e.g. Debian-ish) will
result in failure. All of our current automation images are
CentOS-based, but for various reasons, Debian-based images may be used
in the future, and it would be great to have an sccache binary to run on
such systems as well. (It might also be interesting to distribute the
sccache binary we use to local developers as well, but that's a bit
further off.)
Therefore, this patch alters the sccache build on Linux to use static
linking for openssl. We cannot use the system openssl we build on
because the system openssl links to libkrb5, and the distribution we use
for the system images does not provide static libraries of libkrb5.
Building openssl ourself enables us to eliminate the libkrb5 dependency.
An sccache binary from builds with this patch depends on the following
libraries:
froydnj@hawkeye:~$ ldd sccache2/sccache
linux-vdso.so.1 => (0x00007ffe02b39000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ff0e7403000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007ff0e71fb000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ff0e6fdd000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ff0e6dc6000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff0e69fc000)
/lib64/ld-linux-x86-64.so.2 (0x0000557c8540b000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff0e66f2000)
which are standard on any Linux distribution.
This picks up a fix we need to update the OS X SDK we build with.
MozReview-Commit-ID: 8dvq4JV1o7q
--HG--
extra : rebase_source : a07f13992f30a29ede29a2167e7f1da8d533fd09
