The base compiler check in python configure does some preprocessing,
which ensures the compiler works to some extent. Autoconf used to have
a more complete test, doing a compile/link. We do have plenty of tests
afterwards that do that anyways, but it's better if we fail early if
the toolchain fails somehow.
This refactors try_compile such that the *_compiler variable themselves
can be used to trigger compiler tests. Eventually, we'll want something
similar for preprocessing and possibly other invocations.
This also removes similar tests from build/autoconf/toolchain.m4.
--HG--
extra : rebase_source : c60d1d6e39b6bd2a377516687affd9b8932ebc12
While on automation, there is no MSVC to find through the registry, the
story is different on local builds, and that can interfere with tests.
Specifically, it breaks test_toolchain_configure.py because it's not
expecting the registry to provide a valid path to an almost valid
compiler, and then fails because that compiler doesn't match the
expected target CPU.
And because build/moz.configure/toolchain.configure also affects the PATH
environment variable, subsequent tests end up failing even earlier
because executing the empty mozconfig with the modified environment then
fails because of the unicode value of PATH.
This change implements enough of _winreg to make the get_registry_values
function return nothing.
--HG--
extra : rebase_source : f70e40ddabaed1197f6cddce67832bb112f1969d
Back when it was added, it was used, but it is not anymore, outside
test_base.py.
--HG--
extra : rebase_source : f0b9a4dab2985e89e9950eda774ae853c7de764c
We've been reading the mozconfig in MozbuildObject.from_environment to
check whether the mozconfig topobjdir matches the detected topobjdir.
Since bug 1278415, everything using the buildconfig python module now
calls MozbuildObject.from_environment, which reads the mozconfig. A lot
of things to that during the build. But none of them actually need the
data from the mozconfig, and the topobjdir match test has been breaking
things randomly on multiple occasions.
The topobjdir match test, however, really only needs to happen once:
when a mach command starts. So we can move the test to MachCommandBase,
where it belongs, and anything actively using MozbuildObject.mozconfig
will have the mozconfig read, but everything else won't.
On a Linux64 opt build, this brings down the number of times the
mozconfig is read during `mach build` from 979 to 9.
--HG--
extra : rebase_source : 6b340f1fcf73a3c3987033c37f8f14ef06a44f04
The base compiler check in python configure does some preprocessing,
which ensures the compiler works to some extent. Autoconf used to have
a more complete test, doing a compile/link. We do have plenty of tests
afterwards that do that anyways, but it's better if we fail early if
the toolchain fails somehow.
This refactors try_compile such that the *_compiler variable themselves
can be used to trigger compiler tests. Eventually, we'll want something
similar for preprocessing and possibly other invocations.
This also removes similar tests from build/autoconf/toolchain.m4 and
old-configure.in.
--HG--
extra : rebase_source : 4f6f84e5ad220386e9edf82d19cc2cd6c1f4c43e
This patch is really two separate changes.
The first change is that rust crates are large, standalone entities that
may contain multitudes of source files. It therefore doesn't make sense
to keep them in SOURCES, as we have been doing. Moving to use cargo
will require a higher-level approach, which suggests that we need a
different, higher-level representation for Rust sources in the build
system.
The representation here is to have the build system refer to things
defined in Cargo.toml files as the entities dealt with in the build
system, and let Cargo deal with the details of actually building things.
This approach means that adding a new crate to an existing library just
requires editing Rust and Cargo.toml files, rather than dealing with
moz.build, which seems more natural to Rust programmers. By having the
source files for libraries (and binaries in subsequent iterations of
this support) checked in to the tree, we can also take advantage of
Cargo.lock files.
The second is that we switch the core build system over to building via
cargo, rather than invoking rustc directly.
We also clean up a number of leftover things from the Old Way of doing
things. A number of tests are added to confirm that we'll only permit
crates to be built that have dependencies in-tree.
rlibs are going to be less important once we start building with cargo:
the focus will move to crates instead, so that's what we should call the
moz.build object.
Through an oversight, we listed librul.a twice when linking libxul: once
as part of the "objects" we were linking, and once as a static library.
This duplication is unnecessary and would cause problems later when we
try to generate librul.a via cargo, as cargo will put it someplace
different from where we expect and the two names will conflict. Let's
have rules.mk be the single source of truth for how librul.a is named,
and then the code to link libxul can simply refer to that name.
This patch is really two separate changes.
The first change is that rust crates are large, standalone entities that
may contain multitudes of source files. It therefore doesn't make sense
to keep them in SOURCES, as we have been doing. Moving to use cargo
will require a higher-level approach, which suggests that we need a
different, higher-level representation for Rust sources in the build
system.
The representation here is to have the build system refer to things
defined in Cargo.toml files as the entities dealt with in the build
system, and let Cargo deal with the details of actually building things.
This approach means that adding a new crate to an existing library just
requires editing Rust and Cargo.toml files, rather than dealing with
moz.build, which seems more natural to Rust programmers. By having the
source files for libraries (and binaries in subsequent iterations of
this support) checked in to the tree, we can also take advantage of
Cargo.lock files.
The second is that we switch the core build system over to building via
cargo, rather than invoking rustc directly.
We also clean up a number of leftover things from the Old Way of doing
things. A number of tests are added to confirm that we'll only permit
crates to be built that have dependencies in-tree.
rlibs are going to be less important once we start building with cargo:
the focus will move to crates instead, so that's what we should call the
moz.build object.
Through an oversight, we listed librul.a twice when linking libxul: once
as part of the "objects" we were linking, and once as a static library.
This duplication is unnecessary and would cause problems later when we
try to generate librul.a via cargo, as cargo will put it someplace
different from where we expect and the two names will conflict. Let's
have rules.mk be the single source of truth for how librul.a is named,
and then the code to link libxul can simply refer to that name.
In the `python-test` mach command and the mozharness script for
the Marionette harness tests, use the vendored-in Pytest
instead of installing from pip.
Add the Marionette harness test requirements file to the
file_patterns in the definition of the marionette-harness taskcluster
job, as changes to the requirements should trigger the job to run.
MozReview-Commit-ID: J5pln2WB4GY
--HG--
extra : rebase_source : 5144ccfabb84eb0da244b24f6d27b59ae183c174
Vendor in Pytest (2.9.2) and its requirement Py (1.4.31),
so that it can be used for e.g. the Marionette
harness unit tests and a pytest plugin for mozlog.
Copy pytest and py package directories (extracted from
tars on Pip) into `mozilla-central/python/`, removing
some support files (e.g. changelog, docs, tests).
Add both `.pth` entries to `virtualenv_packages.txt`.
Add both paths to `SEARCH_PATHS` in `mach_bootstrap.py`.
MozReview-Commit-ID: IOTCOUxX8R9
--HG--
extra : rebase_source : e03d8a4be084062c0055b365bcc18da6dbb0b7a7
Until now, HAVE_64BIT_BUILD was entirely determined by a compiler check.
But we didn't run the check on e.g. artifact builds, while relying on
its result for some non-compilation related things, leading to subtle
discrepancies.
This changes the configure check to derive HAVE_64BIT_BUILD from bitness
determined by the target CPU, and double checked with a compiler check.
--HG--
extra : rebase_source : 5dc0cf2369ed4457bdd9a15736a70265a771d919
Currently, it returns either None or the contents of the compiler's stdout,
which is always expected to be an empty string, and is not very useful. So
instead, return True in the latter case.
--HG--
extra : rebase_source : ee69cdeab38d27178ce759591fb394da65e694ac
Mingw python has a different os.path setup from native python, and has
os.sep and os.altsep reversed. In that case, the normsep function was
doing the wrong thing, leading to all sorts of problems.
While fixing this, also ensure the corresponding unit test covers this
peculiarity, even when running under the native win32 python.
--HG--
extra : rebase_source : 8fb18e0d4dc669c1d7e069f73fc44c22d419d43c
This never was a problem since auto-logging was broken on Windows, but
having mach clobber auto-log is a problem on Windows since it tries to
remove the log file while itself has it open, which fails.
--HG--
extra : rebase_source : 6da9f3e6148c201eade01d4598ce9c4becd051a0
We were relying on the log manager's terminal to know whether we're
running on a terminal to turn on auto-logging, but the log manager's
terminal is always None on Windows because blessings imports curses,
which doesn't work on Windows.
So instead, just use a plain os.isatty() call.
The same problem also applies to the show-log command to trigger the
pager (less). At the same time, fix the environment setting for LESS,
as on Windows, unicode literals are not allowed in the environment.
--HG--
extra : rebase_source : 1462849608c8cb86afcb080184efb50caf6d2f9e
Some linters, such as flake8, will lint invalid file extensions if you explicitly pass them in. E.g,
|flake8 foobar.js| will result in flake8 attempting to lint a JS file. This is a problem because passing
in files explicitly is exactly what the --rev/--workdir options do. If a developer modifies a JS file
then runs |mach lint -l flake8 -w|, that JS file will get linted.
To prevent this, mozlint needs to handle file extensions instead of relying on the underlying linter to
do it. This patch adds an "extensions" config option to the LINTER dict, and will filter these files out
as part of the 'filterpaths' steps.
MozReview-Commit-ID: KYhC6SEySC3
--HG--
extra : rebase_source : 46807a4913660f33e691b864c2c59c448902dfa5
The best kind of patch: bulk deletion. This removes two separate but
similar build flags, and an unsupported integration piece. The first
packaged Fennec's resources into an ill-defined GeckoView archive; the
second built on the first to produce an Android ARchive for external
consumption. Neither of these artifacts are supported or have
consumers; in fact, they mislead potential consumers, since they're
known to be broken. The Gradle build work under the --with-gradle
flag, and significant follow-up, is the path forward if Mozilla wants
to invest in packaging GeckoView on Android for external consumption.
That is, rather than hacking together AAR files, we'll use the
well-supported Gradle mechanisms for building and publishing such
libraries.
MozReview-Commit-ID: 4Z1jJ8z0cyJ
--HG--
extra : rebase_source : b8e65f39c286313fe8963bf3832d9b6977ef188f
I have a need to create tar archives deterministically
and reproducibly. Since we already have similar functionality in
mozpack for producting zip/jar archives, I figured it made sense for
this functionality to live in mozpack.
I made the functionality as simple as possible: we only accept
files from the filesystem and the set of files must be known in
advance. No class to hold/buffer state: just a simple function
that takes a mapping of files and writes to a stream.
MozReview-Commit-ID: If0NTcA7wpc
--HG--
extra : rebase_source : 9cbea36347ba2840dd5bff9dffefd994a73a0725
Prevailing style in the file for when read_topsrcdir is being tested for
exception-throwingness is to omit the assignment, so let's make
everything consistent.
And for GCC and clang, try to see if adding -m32, -m64 or --target
works if they don't.
--HG--
extra : rebase_source : 874bc2404a5ccc48e938bc7d9b2fe67ba625cb3e
This removes the unnecessary setting of c-basic-offset from all
python-mode files.
This was automatically generated using
perl -pi -e 's/; *c-basic-offset: *[0-9]+//'
... on the affected files.
The bulk of these files are moz.build files but there a few others as
well.
MozReview-Commit-ID: 2pPf3DEiZqx
--HG--
extra : rebase_source : 0a7dcac80b924174a2c429b093791148ea6ac204
This makes building on msys2 easier since its pip is broken.
MozReview-Commit-ID: 1hQHeu3BKOd
--HG--
extra : rebase_source : 5447d96893a502225980d1dab7b4f89b888ad661
Added a quick for MinGW64 then adds the default location of system modules to
sys.path.
MozReview-Commit-ID: IubUhpB3rOf
--HG--
extra : rebase_source : 95f2245fb254473fb9977b37b60f087c05a7eedc
This is going to be necessary for MSYS2 which needs a custom site.py since
the MinGW64 version of Python does not import the system packages.
MozReview-Commit-ID: EfHRmRSmvlV
--HG--
extra : rebase_source : 6ffbcda31318d3581ded3bf860bbd0db2eec8780
This adds the 'xpcshell-test' command to the mach environment found in the test
package. This will allow developers to easily run xpcshell after checking out
and interactive worker.
MozReview-Commit-ID: fBAbfuG5XQ
--HG--
extra : rebase_source : 71077c9142f33843ed87d4bc4617a780f775939b
Also fix that the default merge dir in the mach command creates a directory
that's the merge make target, and thus keeps that make target from actually
running.
MozReview-Commit-ID: HWZBPxWuHSy
--HG--
extra : rebase_source : a39157ad9eb99f3eef5d149d003e62a235f92fc1
We need this to avoid weird site.py hacks to get msys2 to play nicely with virtualenv.
MozReview-Commit-ID: 9lJSo6MistO
--HG--
extra : rebase_source : 598f25a207dc4307ed531f4e4e5dcfb951ae8283
Before, virtualenv would try to use symlinks based entirely on the value given by the options parser.
Unfortunately, some versions of Python cannot symlink (specifically the MinGW version of Python).
Now, virtualenv ensures the os module has the symlink funciton before attempting to use the symlink function.
MozReview-Commit-ID: DFJ59AaPNpB
--HG--
extra : rebase_source : 4a92c9cbfbfe205e18b890e100fe549d78cebc81
By placing the functools module in the REQUIRED_MODULES list, the environment will not throw an ImportError when the build systemimports functools.
MozReview-Commit-ID: 8DjzXqxvxXd
--HG--
extra : rebase_source : 47ceae1e8b852e0b8bf61f9b34cc953db42e393e
This is going to be necessary in the following commits to ensure that virtualenv has access to native libraries.
MozReview-Commit-ID: FwxOoCWnQMW
--HG--
extra : rebase_source : c4cf829231757b3dc2c5ca821c7248dc3fb0d763
Before bin_path would detect msys2 as if it were a windows system which would cause a virtualenv problem since the binaries should go in a /bin folder not a /Scripts folder for msys2.
Based on a patch by :vlad 89b21efe93
MozReview-Commit-ID: IyAukszHGmN
--HG--
extra : rebase_source : 9ada9cf61b76c85cf483ad8212c95d17070a7025
Technically, this includes the mingw64 version of make, but it is still called mingw32-make when installed from pacman with msys2.
This is a necessary step to make Firefox build on a pure msys2 environment.
Also, unlike mozmake, because this make is installed from a package manager. It can be automatically updated.
MozReview-Commit-ID: EAN6xmIgYWd
--HG--
extra : rebase_source : 43fe0d4b35e91f1143ce2ea1d20f5f3c5f1f7c86
Before, mach would only detect MINGW32 (used with msys). Now it also detects MINGW64 (used with msys2).
MozReview-Commit-ID: 6IrNNmkLKgB
--HG--
extra : rebase_source : e84ee16b5bbc1b6b113163ba5b61759f763645c1
The mozconfig detection logic has bitten us on many occasions in the
past. The following changes are made to tentatively improve the
situation:
- The API is modified such that autodetection of the mozconfig has
to be a conscious decision made by the caller, and not triggered
any time there is no mozconfig given, which could be a conscious
decision of the opposite.
- mozinfo.json now stores the actual mozconfig (or lack thereof) used
during configure.
--HG--
extra : rebase_source : c7a632afd414f25daf7bbe7e1a66c3736c26e039
Per froydnj in bug 1186064 comment #23, "it makes sense to proceed with removing
MSVC 2013 support." This commit does that.
We also go a step further and require VS2015 Update 2 instead of just
update 1. This temporarily brings us down to just 1 officially supported
Visual Studio version. However, VS2015u3 was just released and is
unofficially supported.
Since MOZ_CRT is no longer set, references to it have been removed.
MozReview-Commit-ID: 8MUR6qLzQA5
--HG--
extra : rebase_source : 8f5061080a3d56dd484f9be03649fb65f0145f67
Per froydnj in bug 1186064 comment #23, "it makes sense to proceed with removing
MSVC 2013 support." This commit does that.
We also go a step further and require VS2015 Update 2 instead of just
update 1. This temporarily brings us down to just 1 officially supported
Visual Studio version. However, VS2015u3 was just released and is
unofficially supported.
Since MOZ_CRT is no longer set, references to it have been removed.
MozReview-Commit-ID: 8MUR6qLzQA5
--HG--
extra : rebase_source : 22ab4f47661ead4995d0c5261104abfb02b82aa2
This adds two parameters, --rev and --workdir. Each works both with mercurial and git (though the syntax for
specifying revisions is different between them). The value is simply forwarded to either |hg log| or |git diff|
so syntax like |mach lint -r .~4::.| or |mach lint -r "HEAD~4 HEAD"| will work as expected.
MozReview-Commit-ID: aOGp2Yrncs
--HG--
extra : rebase_source : d2cb834d4cc1a083171a3551af4e72c8a7d14021
There is currently no built-in user interface to mozlint. The only existing interface is the
external cli provided by |mach lint|. However, in the future mozlint may need to be used in a
context where mach isn't readily available (i.e version-control-tools). This patch basically
just moves the cli logic out of mach_commands.py, and into mozlint core. That way it can be
re-used in other places without needing to be re-implemented.
The |mach lint setup| subcommand was removed because apparently subcommands don't work with
the parser attribute. Nothing was using it yet anyway, so I removed it for now. It may get
re-added in some form in the future.
MozReview-Commit-ID: aOGp2Yrncs
--HG--
extra : rebase_source : 8f7530de96e5c131d2ed5bfcdd7a159329401e5b
Since it conflicts with moz.configure which resolves shortnames since
parts of the build system don't like paths with spaces.
MozReview-Commit-ID: nJkRV4Nafo
--HG--
extra : rebase_source : 9f2db54ec38eb0f05b478b445c0a883ca69b0127
Should be a drop-in replacement.
MozReview-Commit-ID: 2ebRTXwCBfM
--HG--
rename : python/virtualenv/virtualenv_support/setuptools-20.3-py2.py3-none-any.whl => python/virtualenv/virtualenv_support/setuptools-25.2.0-py2.py3-none-any.whl
extra : rebase_source : d8cef816464fb494a8134967bf912d9dabe41b84
Should be a drop-in replacement.
MozReview-Commit-ID: 5934X9tS29g
--HG--
rename : python/virtualenv/virtualenv_support/pip-8.1.1-py2.py3-none-any.whl => python/virtualenv/virtualenv_support/pip-8.1.2-py2.py3-none-any.whl
extra : rebase_source : 01d234565de37041f79f3f4498eec1d6fadfe005
This matches the implementation from mach_bootstrap.py.
MozReview-Commit-ID: 8kZCKuIsAMQ
--HG--
extra : rebase_source : 59b1f3d595e1663603363bb712da9cb74c3ba0e0
extra : amend_source : e95774ade1c7d28ba1d880b6b9ef1d64eaa618a8
We'll be consolidating code from mach_bootstrap.py and mozboot.
We don't want mach_bootstrap.py to import bootstrap.py because it
imports nearly every module under mozboot. So establish a standalone
module with minimal dependencies to hold utility code. Move
get_state_dir() there.
MozReview-Commit-ID: Hw5VB5OZGCi
--HG--
extra : rebase_source : e083f9a5d2fabea308b7b884e9830f800758ae17
extra : amend_source : 0a7b5f42a937430170fdc16909c559f720085668
