This adds a --enable-bootstrap build flag that will automatically update
cbindgen, node, clang, sccache, nasm, wine, lucetc, dump_syms, pdbstr,
and winchecksec if they are already installed in ~/.mozbuild.
Eventually, we'll want to allow to install toolchains that weren't
already install, but one step at a time.
This explicitly doesn't cover rustc, which is its own can of worms, or
android-{ndk,sdk}, which are not installed via toolchain artifacts
currently.
Differential Revision: https://phabricator.services.mozilla.com/D101723
Now that we use an external dump_syms, we don't need to build
breakpad's.
This means we also don't need the dump_syms_rust_demangle crate anymore.
Differential Revision: https://phabricator.services.mozilla.com/D101865
These are the minimum changes that we need to make to common build system code
to allow us to generate files during pre-export.
We add a `required_before_export` flag to `GeneratedFile` to indicate when a
particular file must be generated in `pre-export`. We set that flag when there
are `.jinja` input files and we're configured for a GeckoView build, otherwise
it is set to `False`.
Then the recursive `make` backend assigns any `GeneratedFile`s that have
`required_before_export` set to run in the `pre-export` tier.
Differential Revision: https://phabricator.services.mozilla.com/D82576
These are the minimum changes that we need to make to common build system code
to allow us to generate files during pre-export.
We add a `required_before_export` flag to `GeneratedFile` to indicate when a
particular file must be generated in `pre-export`. We set that flag when there
are `.jinja` input files and we're configured for a GeckoView build, otherwise
it is set to `False`.
Then the recursive `make` backend assigns any `GeneratedFile`s that have
`required_before_export` set to run in the `pre-export` tier.
Differential Revision: https://phabricator.services.mozilla.com/D82576
There's a macOS-specific wrinkle for browser/ that populates the
`.app` directory. This makes that happen as part of `mach
package-multi-locale`. It's the equivalent, I suppose, of `mach
android assemble-app` for Desktop.
Differential Revision: https://phabricator.services.mozilla.com/D101502
Since zstandard has native code that must be compiled, and that code
uses Python headers, we should be installing those headers as part
of bootstrap.
Most users will have these packages on their machines through various
other means (notably installing `pip`, ie `sudo apt install python3-pip`),
but since it is possible to avoid a pip installation (for example
by installing Mercurial through `yum` and then running bootstrap
immediately after cloning) we should specify these packages as required
by bootstrap.
Differential Revision: https://phabricator.services.mozilla.com/D101479
The current recommendation fails while waiting on user input. Instead, just
save the script to disk as an intermediate step, then invoke it.
Differential Revision: https://phabricator.services.mozilla.com/D101228
Knowing whether `brew` or `macports` is available isn't necessary
to generate the android mozconfig.
This should fix the generation of android mozconfig when a package
manager isn't available.
Differential Revision: https://phabricator.services.mozilla.com/D99496
pylint_requirements.txt fail to install with the new pip resolver due
to a conflict between astroid and lazy-object-proxy.
Rather than bumping those packages and handling the potential fallout,
the package-upgrade has been deferred and we will use the legacy
resolver in the interrim.
Differential Revision: https://phabricator.services.mozilla.com/D99940
The `AUTOCLOBBER` mozconfig option is reliably causing builds to fail when
a clobber is triggered. When we auto-clobber a build we do so after running
`configure` but before running `make client.mk`. This means we destroy all
the gathered information from the `configure` step in the objdir and then
attempt to run `make` using the previously destroyed information.
This commit moves the call to `_check_clobber` to an earlier stage in the
build process, before `configure` is called, so any clobber that takes place
will happen before setting up the objdir via `configure`.
Since `_check_clobber` is only called once in the codebase, and both cases
are now adding clobber metrics one after another, we remove the metrics
gathering from `_check_clobber` and rely on callers to set metrics instead.
Also clean up some nested `if` statements that can be flattened.
Differential Revision: https://phabricator.services.mozilla.com/D100794
This function has a few code paths and has a slightly confusing return value.
Add a comment describing what it does and what the return value actually
means.
Differential Revision: https://phabricator.services.mozilla.com/D100793
Since the printed value is a `str` anyways, this causes the converted `bytes`
to be printed to the terminal as `b'/path/to/topobjdir'`. Just print the `str`
version to the screen instead.
Differential Revision: https://phabricator.services.mozilla.com/D100792
When mach errors out, an error report is sent to Sentry. This error
report contains information about the state of the interpreter during
the failure, details about the environment, installed packages and more.
Having this information available immediately when attempting to resolve
a bug report is generally desirable, instead of going through a back-and-forth
needinfo tag on Bugzilla or spending time asking the reporter questions on
Matrix.
This commit captures the Sentry ID returned from `sentry_sdk.capture_exception`
and prints it to the screen. If a user adds this line to their bug report (as
the error messages suggest) a build team member can enter this number into
Sentry to identify the exact report and debug the error. At minimum this will
reduce the amount of back-and-forth between the reporter and the assignee
required to resolve a bug. Optimally it should make bugs easier to spot and
reduce the time spent on end user support requests.
To use the Sentry ID to identify information about a specific bug report, the
bug assignee should open the Mozilla Sentry page for the `mach` project and
paste the ID into the search box, which will produce the full stack trace with
all submitted information.
Differential Revision: https://phabricator.services.mozilla.com/D100247
Knowing whether `brew` or `macports` is available isn't necessary
to generate the android mozconfig.
This should fix the generation of android mozconfig when a package
manager isn't available.
Differential Revision: https://phabricator.services.mozilla.com/D99496
pylint_requirements.txt fail to install with the new pip resolver due
to a conflict between astroid and lazy-object-proxy.
Rather than bumping those packages and handling the potential fallout,
the package-upgrade has been deferred and we will use the legacy
resolver in the interrim.
Differential Revision: https://phabricator.services.mozilla.com/D99940
Some distros include flags when they specify the location of a binary,
such as: `XARGS=xargs -r`.
This was confusing in `configure`, since:
* We require that environment variables contain only path
overrides (without flags).
* The error message thrown when configure chokes was unclear: "why
would `$ xargs -r` fail?"
This patch should make our "path-only" requirement more clear.
Differential Revision: https://phabricator.services.mozilla.com/D100044
At the moment installing Python packages with native code is done by calling
`pip install <package>` and does not enforce any SHA hash for installed
dependencies, nor does it enforce a specific version to be installed.
This commit adds `requirements.in` and `requirements.txt` files for native
packages and changes these packages to be installed by running `pip install`
and passing the requirements file for the package. This allows us to pin the
SHA of the various dependencies. The `.txt` files are generated using
`pip-compile`.
We also add the new requirements files to the sparse profile for `mach`.
Differential Revision: https://phabricator.services.mozilla.com/D99912
Knowing whether `brew` or `macports` is available isn't necessary
to generate the android mozconfig.
This should fix the generation of android mozconfig when a package
manager isn't available.
Differential Revision: https://phabricator.services.mozilla.com/D99496
Knowing whether `brew` or `macports` is available isn't necessary
to generate the android mozconfig.
This should fix the generation of android mozconfig when a package
manager isn't available.
Depends on D99482
Differential Revision: https://phabricator.services.mozilla.com/D99496
Our old version of `glean_sdk` only had pip wheels, which meant that
un-wheeled machines had no option for building the package.
Differential Revision: https://phabricator.services.mozilla.com/D98795
Native python packages will probably require python headers. For some
systems, these are available in a separate system package (such as
python[3]-dev).
If the user is bootstrapping with "--no-system-changes", we just
try to install these native packages and fail gracefully if required
headers aren't installed.
Differential Revision: https://phabricator.services.mozilla.com/D98530
The CI environment doesn't bootstrap/`create-mach-environment`, so
native modules (such as zstandard) aren't installed.
`./mach artifact install` is run in CI and needs `zstandard`, so we
must install it ad-hoc.
Differential Revision: https://phabricator.services.mozilla.com/D98794
This patch enables PSM and Firefox to use TLS 1.3 Encrypted Client Hello (draft -08). Specifically:
- Compile NSS with NSS_ENABLE_DRAFT_HPKE=1
- Add ECH "public_name" handling in SSLServerCertVerification.cpp (see: https://tools.ietf.org/html/draft-ietf-tls-esni-08#section-6.3.2)
- Adds `mIsAcceptedEch` to TransportSecurityInfo, and xpcshell tests for ECH use cases
- Adds EncryptedClientHelloServer to facilitate the xpcshell tests
- Un-ifdef Set/GetEchConfigs code in nsNSSIOLayer.cpp. Also reverted the Base64 encoding and decoding, as the data returned from DNS is already decoded (wire-format).
Differential Revision: https://phabricator.services.mozilla.com/D92651
