Bug 1703105 - wasm: Move generate-spectest tool into tree. r=lth

This commit moves the wasm-generate-testuite [1] tool into the
tree so that we have everything we need to update spectests in
one place. Documentation is updated.

Differential Revision: https://phabricator.services.mozilla.com/D111306
This commit is contained in:
Ryan Hunt 2021-04-13 16:41:46 +00:00
parent 8a858ee54b
commit 8610f775db
16 changed files with 2356 additions and 59 deletions

View file

@ -60,8 +60,10 @@ _OPT\.OBJ/
# SpiderMonkey test result logs
^js/src/tests/results-.*\.(html|txt)$
^js/src/devtools/rootAnalysis/t/out
# SpiderMonkey clone of the wasm-generate-testsuite repository
^js/src/jit-test/etc/wasm/wasm-generate-testsuite/
# SpiderMonkey wasm/generate-spectests artifacts
^js/src/jit-test/etc/wasm/generate-spectests/specs/
^js/src/jit-test/etc/wasm/generate-spectests/tests/
^js/src/jit-test/etc/wasm/generate-spectests/target/
# Java HTML5 parser classes
^parser/html/java/(html|java)parser/

View file

@ -1,23 +1,10 @@
.PHONY: update run expectations
.PHONY: update
warning = '\# Wasm Spec Tests\n\nThese tests are autogenerated using a tool, do not edit.\n\nSee `jit-test/etc/wasm/` for more information.'
update:
[ -d ./wasm-generate-testsuite ] || git clone https://github.com/eqrion/wasm-generate-testsuite ./wasm-generate-testsuite
cp ./config.toml ./wasm-generate-testsuite/config.toml
cp ./config-lock.toml ./wasm-generate-testsuite/config-lock.toml
(cd ./wasm-generate-testsuite && RUST_LOG=info cargo run)
cp ./wasm-generate-testsuite/config-lock.toml ./config-lock.toml
(cd ./generate-spectests && RUST_LOG=info cargo run)
rm -r ../../tests/wasm/spec
cp -R wasm-generate-testsuite/tests/js ../../tests/wasm/spec
cp -R generate-spectests/tests/js ../../tests/wasm/spec
echo $(warning) > ../../tests/wasm/spec/README.md
[ ! -d ./spec-tests.patch ] || (cd ../../tests/wasm/spec && patch -u -p7 < ../../../etc/wasm/spec-tests.patch)
run:
@[ -z $(MOZCONFIG) ] && echo "You need to define the MOZCONFIG env variable first."
@[ -z $(MOZCONFIG) ] || ../../../../../mach wpt /_mozilla/wasm
expectations:
@[ -z $(MOZCONFIG) ] && echo "You need to define the MOZCONFIG env variable first." || true
@[ -z $(MOZCONFIG) ] || ../../../../../mach wpt /_mozilla/wasm --log-raw /tmp/expectations.log || true
@[ -z $(MOZCONFIG) ] || ../../../../../mach wpt-update /tmp/expectations.log --no-patch

View file

@ -9,16 +9,14 @@ Spec tests are given in `test/core` of the `spec` repository as `.wast` files.
A `.wast` file is a superset of the `.wat` format with commands for running
tests.
The spec interpreter can natively consume `.wast` files to run the tests, or
generate `.js` files which rely on the WebAssembly JS-API plus a harness file
to implement unit-test functionality.
We rely on the spec interpreter to generate `.js` files for us to run.
The spec interpreter can natively consume `.wast` files to run the tests, but
we cannot run them directly ourselves. To workaround this, we have a tool which
can convert `.wast` files to `.js` that can be run efficiently in our jit-test
harness.
## Running tests
Tests are imported to `jit-test` and `wpt` to be run in both the JS shell or the
browser.
Tests are imported to `jit-test` to be run in the JS shell.
To run under `jit-test`:
```bash
@ -26,11 +24,6 @@ cd js/src
./jit-test.py path_to_build/dist/bin/js wasm/spec/
```
To run under `wpt` (generally not necessary):
```bash
./mach web-platform-test testing/web-platform/mozilla/tests/wasm/
```
## Test importing
There are many proposals in addition to the canonical spec. Each proposal is a
@ -44,12 +37,9 @@ Testing each proposal separately in full isn't an attractive option either, as
most tests are unchanged and that would cause a significant amount of wasted
computation time.
For this reason, we use a tool [1] to generate a set of separate test-suites
that are 'pruned' to obtain a minimal set of tests. The tool works by merging
each proposal with the proposal it is based off of and removing tests that
have not changed.
[1] https://github.com/eqrion/wasm-generate-testsuite
For this reason, we generate a set of separate test-suites that are 'pruned' to
obtain a minimal set of tests. The tool works by merging each proposal with the
proposal it is based off of and removing tests that have not changed.
### Configuration
@ -63,9 +53,9 @@ useful as proposals often make inconvenient and breaking changes.
```bash
# Add, remove, or modify proposals
vim config.toml
vim generate-spectests/config.toml
# Remove locks for any proposals you wish to pull the latest changes on
vim config-lock.toml
vim generate-spectests/config-lock.toml
# Import the tests
make update
# View the tests that were imported
@ -73,29 +63,13 @@ hg stat
# Run the imported tests and note failures
./jit-test.py dist/bin/js wasm/spec/
# Exclude test failures
vim config.toml
vim generate-spectests/config.toml
# Re-import the tests to exclude failing tests
make update
# Commit the changes
hg commit
```
### Debugging test failures
Imported tests use the binary format, which is inconvenient for understanding
why a test is failing.
Luckily, each assertion in an imported test contains the line of the source file
that it corresponds with.
Unfortunately, the '.wast' files are not commited in the tree and so you must
use the import tool to get the original source.
Follow the steps in 'Operation' to get a `wasm-generate-testsuite` repo with
the '.wast' files of each proposal. All proposals are stored in a single git
repo named `specs/`. Each proposal is a branch, and you can find all tests
under `test/core`.
### Debugging import failures
Proposals can often have conflicts with their upstream proposals. This is okay,
@ -104,10 +78,6 @@ and the test importer will fallback to building tests on an unmerged tree.
This will likely result in extra tests being imported due to spurious
differences between the proposal and upstream, but generally is okay.
It's also possible that a proposal is 'broken' and fails to generate '.js' test
files with the spec interpreter. The best way to debug this is to check out the
proposal repo and debug why `./test/build.py` is failing.
The import tool uses `RUST_LOG` to output debug information. `Makefile`
automatically uses `RUST_LOG=info`. Change that to `RUST_LOG=debug` to get
verbose output of all the commands run.

View file

@ -0,0 +1,826 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
[[package]]
name = "Inflector"
version = "0.11.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fe438c63458706e03479442743baae6c88256498e6431708f6dfc520a26515d3"
dependencies = [
"lazy_static",
"regex",
]
[[package]]
name = "aho-corasick"
version = "0.7.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d5e63fd144e18ba274ae7095c0197a870a7b9468abc801dd62f190d80817d2ec"
dependencies = [
"memchr",
]
[[package]]
name = "anyhow"
version = "1.0.40"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "28b2cd92db5cbd74e8e5028f7e27dd7aa3090e89e4f2a197cc7c8dfb69c7063b"
[[package]]
name = "ast_node"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7c84c445d38f7f29c82ed56c2cfae4885e5e6d9fb81b956ab31430757ddad5d7"
dependencies = [
"darling",
"pmutil",
"proc-macro2",
"quote",
"swc_macros_common",
"syn",
]
[[package]]
name = "atty"
version = "0.2.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9b39be18770d11421cdb1b9947a45dd3f37e93092cbf377614828a319d5fee8"
dependencies = [
"hermit-abi",
"libc",
"winapi",
]
[[package]]
name = "autocfg"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cdb031dd78e28731d87d56cc8ffef4a8f36ca26c38fe2de700543e627f8a464a"
[[package]]
name = "bumpalo"
version = "3.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "63396b8a4b9de3f4fdfb320ab6080762242f66a8ef174c49d8e19b674db4cdbe"
[[package]]
name = "byteorder"
version = "1.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "14c189c53d098945499cdfa7ecc63567cf3886b3332b312a5b4585d8d3a6a610"
[[package]]
name = "cfg-if"
version = "0.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4785bdd1c96b2a846b2bd7cc02e86b6b3dbf14e7e53446c4f54c92a361040822"
[[package]]
name = "cfg-if"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd"
[[package]]
name = "darling"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0d706e75d87e35569db781a9b5e2416cff1236a47ed380831f959382ccd5f858"
dependencies = [
"darling_core",
"darling_macro",
]
[[package]]
name = "darling_core"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0c960ae2da4de88a91b2d920c2a7233b400bc33cb28453a2987822d8392519b"
dependencies = [
"fnv",
"ident_case",
"proc-macro2",
"quote",
"strsim",
"syn",
]
[[package]]
name = "darling_macro"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9b5a2f4ac4969822c62224815d069952656cadc7084fdca9751e6d959189b72"
dependencies = [
"darling_core",
"quote",
"syn",
]
[[package]]
name = "dprint-core"
version = "0.35.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "93bd44f40b1881477837edc7112695d4b174f058c36c1cbc4c50f8d0482e2ac8"
dependencies = [
"bumpalo",
"fnv",
"serde",
]
[[package]]
name = "dprint-plugin-typescript"
version = "0.42.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fd74250d3285dbd39de6f721c5975a1a78ff66cb4062a0053ab358522d582f4d"
dependencies = [
"dprint-core",
"dprint-swc-ecma-ast-view",
"fnv",
"serde",
"swc_common",
"swc_ecmascript",
]
[[package]]
name = "dprint-swc-ecma-ast-view"
version = "0.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5f8e29f47721be08f3e2568786d44820a6d7e68667da9e1968e706a75de89a0"
dependencies = [
"bumpalo",
"fnv",
"num-bigint",
"swc_atoms",
"swc_common",
"swc_ecmascript",
]
[[package]]
name = "either"
version = "1.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e78d4f1cc4ae33bbfc157ed5d5a5ef3bc29227303d595861deb238fcec4e9457"
[[package]]
name = "enum_kind"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78b940da354ae81ef0926c5eaa428207b8f4f091d3956c891dfbd124162bed99"
dependencies = [
"pmutil",
"proc-macro2",
"swc_macros_common",
"syn",
]
[[package]]
name = "env_logger"
version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "44533bbbb3bb3c1fa17d9f2e4e38bbbaf8396ba82193c4cb1b6445d711445d36"
dependencies = [
"atty",
"humantime",
"log",
"regex",
"termcolor",
]
[[package]]
name = "fnv"
version = "1.0.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1"
[[package]]
name = "from_variant"
version = "0.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0951635027ca477be98f8774abd6f0345233439d63f307e47101acb40c7cc63d"
dependencies = [
"pmutil",
"proc-macro2",
"swc_macros_common",
"syn",
]
[[package]]
name = "fxhash"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c31b6d751ae2c7f11320402d34e41349dd1016f8d5d45e48c4312bc8625af50c"
dependencies = [
"byteorder",
]
[[package]]
name = "getrandom"
version = "0.1.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fc3cb4d91f53b50155bdcfd23f6a4c39ae1969c2ae85982b135750cccaf5fce"
dependencies = [
"cfg-if 1.0.0",
"libc",
"wasi",
]
[[package]]
name = "hermit-abi"
version = "0.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "725cf19794cf90aa94e65050cb4191ff5d8fa87a498383774c47b332e3af952e"
dependencies = [
"libc",
]
[[package]]
name = "humantime"
version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df004cfca50ef23c36850aaaa59ad52cc70d0e90243c3c7737a4dd32dc7a3c4f"
dependencies = [
"quick-error",
]
[[package]]
name = "ident_case"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9e0384b61958566e926dc50660321d12159025e767c18e043daf26b70104c39"
[[package]]
name = "is-macro"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a322dd16d960e322c3d92f541b4c1a4f0a2e81e1fdeee430d8cecc8b72e8015f"
dependencies = [
"Inflector",
"pmutil",
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "lazy_static"
version = "1.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e2abad23fbc42b3700f2f279844dc832adb2b2eb069b2df918f455c4e18cc646"
[[package]]
name = "leb128"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3576a87f2ba00f6f106fdfcd16db1d698d648a26ad8e0573cad8537c3c362d2a"
[[package]]
name = "libc"
version = "0.2.68"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dea0c0405123bba743ee3f91f49b1c7cfb684eef0da0a50110f758ccf24cdff0"
[[package]]
name = "log"
version = "0.4.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "14b6052be84e6b71ab17edffc2eeabf5c2c3ae1fdb464aae35ac50c67a44e1f7"
dependencies = [
"cfg-if 0.1.10",
]
[[package]]
name = "memchr"
version = "2.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3728d817d99e5ac407411fa471ff9800a778d88a24685968b36824eaf4bee400"
[[package]]
name = "new_debug_unreachable"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e4a24736216ec316047a1fc4252e27dabb04218aa4a3f37c6e7ddbf1f9782b54"
[[package]]
name = "num-bigint"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "090c7f9998ee0ff65aa5b723e4009f7b217707f1fb5ea551329cc4d6231fb304"
dependencies = [
"autocfg",
"num-integer",
"num-traits",
"serde",
]
[[package]]
name = "num-integer"
version = "0.1.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2cc698a63b549a70bc047073d2949cce27cd1c7b0a4a862d08a8031bc2801db"
dependencies = [
"autocfg",
"num-traits",
]
[[package]]
name = "num-traits"
version = "0.2.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a64b1ec5cda2586e284722486d802acf1f7dbdc623e2bfc57e65ca1cd099290"
dependencies = [
"autocfg",
]
[[package]]
name = "once_cell"
version = "1.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "af8b08b04175473088b46763e51ee54da5f9a164bc162f615b91bc179dbf15a3"
[[package]]
name = "owning_ref"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ff55baddef9e4ad00f88b6c743a2a8062d4c6ade126c2a528644b8e444d52ce"
dependencies = [
"stable_deref_trait",
]
[[package]]
name = "phf_generator"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "17367f0cc86f2d25802b2c26ee58a7b23faeccf78a396094c13dced0d0182526"
dependencies = [
"phf_shared",
"rand",
]
[[package]]
name = "phf_shared"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c00cf8b9eafe68dde5e9eaa2cef8ee84a9336a47d566ec55ca16589633b65af7"
dependencies = [
"siphasher",
]
[[package]]
name = "pmutil"
version = "0.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3894e5d549cccbe44afecf72922f277f603cd4bb0219c8342631ef18fffbe004"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "ppv-lite86"
version = "0.2.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac74c624d6b2d21f425f752262f42188365d7b8ff1aff74c82e45136510a4857"
[[package]]
name = "precomputed-hash"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "925383efa346730478fb4838dbe9137d2a47675ad789c546d150a6e1dd4ab31c"
[[package]]
name = "proc-macro2"
version = "1.0.26"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a152013215dca273577e18d2bf00fa862b89b24169fb78c4c95aeb07992c9cec"
dependencies = [
"unicode-xid",
]
[[package]]
name = "quick-error"
version = "1.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a1d01941d82fa2ab50be1e79e6714289dd7cde78eba4c074bc5a4374f650dfe0"
[[package]]
name = "quote"
version = "1.0.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3d0b9745dc2debf507c8422de05d7226cc1f0644216dfdfead988f9b1ab32a7"
dependencies = [
"proc-macro2",
]
[[package]]
name = "rand"
version = "0.7.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a6b1679d49b24bbfe0c803429aa1874472f50d9b363131f0e89fc356b544d03"
dependencies = [
"getrandom",
"libc",
"rand_chacha",
"rand_core",
"rand_hc",
"rand_pcg",
]
[[package]]
name = "rand_chacha"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f4c8ed856279c9737206bf725bf36935d8666ead7aa69b52be55af369d193402"
dependencies = [
"ppv-lite86",
"rand_core",
]
[[package]]
name = "rand_core"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "90bde5296fc891b0cef12a6d03ddccc162ce7b2aff54160af9338f8d40df6d19"
dependencies = [
"getrandom",
]
[[package]]
name = "rand_hc"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ca3129af7b92a17112d59ad498c6f81eaf463253766b90396d39ea7a39d6613c"
dependencies = [
"rand_core",
]
[[package]]
name = "rand_pcg"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "16abd0c1b639e9eb4d7c50c0b8100b0d0f849be2349829c740fe8e6eb4816429"
dependencies = [
"rand_core",
]
[[package]]
name = "regex"
version = "1.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "322cf97724bea3ee221b78fe25ac9c46114ebb51747ad5babd51a2fc6a8235a8"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax",
"thread_local",
]
[[package]]
name = "regex-syntax"
version = "0.6.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b28dfe3fe9badec5dbf0a79a9cccad2cfc2ab5484bdb3e44cbd1ae8b3ba2be06"
[[package]]
name = "scoped-tls"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ea6a9290e3c9cf0f18145ef7ffa62d68ee0bf5fcd651017e586dc7fd5da448c2"
[[package]]
name = "serde"
version = "1.0.125"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "558dc50e1a5a5fa7112ca2ce4effcb321b0300c0d4ccf0776a9f60cd89031171"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.125"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b093b7a2bb58203b5da3056c05b4ec1fed827dcfdb37347a8841695263b3d06d"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "siphasher"
version = "0.3.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cbce6d4507c7e4a3962091436e56e95290cb71fa302d0d270e32130b75fbff27"
[[package]]
name = "smallvec"
version = "1.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fe0f37c9e8f3c5a4a66ad655a93c74daac4ad00c441533bf5c6e7990bb42604e"
[[package]]
name = "stable_deref_trait"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a8f112729512f8e442d81f95a8a7ddf2b7c6b8a1a6f509a95864142b30cab2d3"
[[package]]
name = "string_cache"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ddb1139b5353f96e429e1a5e19fbaf663bddedaa06d1dbd49f82e352601209a"
dependencies = [
"lazy_static",
"new_debug_unreachable",
"phf_shared",
"precomputed-hash",
"serde",
]
[[package]]
name = "string_cache_codegen"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f24c8e5e19d22a726626f1a5e16fe15b132dcf21d10177fa5a45ce7962996b97"
dependencies = [
"phf_generator",
"phf_shared",
"proc-macro2",
"quote",
]
[[package]]
name = "string_enum"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f584cc881e9e5f1fd6bf827b0444aa94c30d8fe6378cf241071b5f5700b2871f"
dependencies = [
"pmutil",
"proc-macro2",
"quote",
"swc_macros_common",
"syn",
]
[[package]]
name = "strsim"
version = "0.9.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6446ced80d6c486436db5c078dde11a9f73d42b57fb273121e160b84f63d894c"
[[package]]
name = "swc_atoms"
version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "762f5c66bf70e6f96db67808b5ad783c33a72cc3e0022cd04b41349231cdbe6c"
dependencies = [
"string_cache",
"string_cache_codegen",
]
[[package]]
name = "swc_common"
version = "0.10.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1c46e9d1414d5a361a4eb548e3f76bee3e2d9f8f4333907881fe681a59a829aa"
dependencies = [
"ast_node",
"cfg-if 0.1.10",
"either",
"from_variant",
"fxhash",
"log",
"num-bigint",
"once_cell",
"owning_ref",
"scoped-tls",
"serde",
"string_cache",
"swc_eq_ignore_macros",
"swc_visit",
"unicode-width",
]
[[package]]
name = "swc_ecma_ast"
version = "0.40.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b73b90f45238ac4b29e264591cfb34b0df32fb336f74a12a369678c7d5e906b"
dependencies = [
"is-macro",
"num-bigint",
"serde",
"string_enum",
"swc_atoms",
"swc_common",
]
[[package]]
name = "swc_ecma_parser"
version = "0.50.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ace7b320cac00adc4bf907e93d76c67ca758ac601aaf67db25f82f59252988be"
dependencies = [
"either",
"enum_kind",
"fxhash",
"log",
"num-bigint",
"serde",
"smallvec",
"swc_atoms",
"swc_common",
"swc_ecma_ast",
"swc_ecma_visit",
"unicode-xid",
]
[[package]]
name = "swc_ecma_visit"
version = "0.26.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dd01a0c58d20627f2fa29f44c5c6f68ae7d2ca6e2de0447429f2db65e4482a4e"
dependencies = [
"num-bigint",
"swc_atoms",
"swc_common",
"swc_ecma_ast",
"swc_visit",
]
[[package]]
name = "swc_ecmascript"
version = "0.27.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4bed22af200d57b250e38824c0db9c9f0401ee534d4eb58d693bf0ff2952e6dc"
dependencies = [
"swc_ecma_ast",
"swc_ecma_parser",
]
[[package]]
name = "swc_eq_ignore_macros"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8c8f200a2eaed938e7c1a685faaa66e6d42fa9e17da5f62572d3cbc335898f5e"
dependencies = [
"pmutil",
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "swc_macros_common"
version = "0.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08ed2e930f5a1a4071fe62c90fd3a296f6030e5d94bfe13993244423caf59a78"
dependencies = [
"pmutil",
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "swc_visit"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "583cfe83f6002e1118559308b88181f34b5936b403b72548cd0259bfcf0ca39e"
dependencies = [
"either",
"swc_visit_macros",
]
[[package]]
name = "swc_visit_macros"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3b2825fee79f10d0166e8e650e79c7a862fb991db275743083f07555d7641f0"
dependencies = [
"Inflector",
"pmutil",
"proc-macro2",
"quote",
"swc_macros_common",
"syn",
]
[[package]]
name = "syn"
version = "1.0.65"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f3a1d708c221c5a612956ef9f75b37e454e88d1f7b899fbd3a18d4252012d663"
dependencies = [
"proc-macro2",
"quote",
"unicode-xid",
]
[[package]]
name = "termcolor"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb6bfa289a4d7c5766392812c0a1f4c1ba45afa1ad47803c11e1f407d846d75f"
dependencies = [
"winapi-util",
]
[[package]]
name = "thread_local"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d40c6d1b69745a6ec6fb1ca717914848da4b44ae29d9b3080cbee91d72a69b14"
dependencies = [
"lazy_static",
]
[[package]]
name = "toml"
version = "0.5.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ffc92d160b1eef40665be3a05630d003936a3bc7da7421277846c2613e92c71a"
dependencies = [
"serde",
]
[[package]]
name = "unicode-width"
version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9337591893a19b88d8d87f2cec1e73fad5cdfd10e5a6f349f498ad6ea2ffb1e3"
[[package]]
name = "unicode-xid"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "826e7639553986605ec5979c7dd957c7895e93eabed50ab2ffa7f6128a75097c"
[[package]]
name = "wasi"
version = "0.9.0+wasi-snapshot-preview1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cccddf32554fecc6acb585f82a32a72e28b48f8c4c1883ddfeeeaa96f7d8e519"
[[package]]
name = "wasm-generate-spectests"
version = "0.1.0"
dependencies = [
"anyhow",
"env_logger",
"log",
"regex",
"serde",
"serde_derive",
"toml",
"wast2js",
]
[[package]]
name = "wast"
version = "35.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a5800e9f86a1eae935e38bea11e60fd253f6d514d153fb39b3e5535a7b37b56"
dependencies = [
"leb128",
]
[[package]]
name = "wast2js"
version = "0.1.0"
dependencies = [
"anyhow",
"dprint-plugin-typescript",
"wast",
]
[[package]]
name = "winapi"
version = "0.3.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8093091eeb260906a183e6ae1abdba2ef5ef2257a21801128899c3fc699229c6"
dependencies = [
"winapi-i686-pc-windows-gnu",
"winapi-x86_64-pc-windows-gnu",
]
[[package]]
name = "winapi-i686-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6"
[[package]]
name = "winapi-util"
version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fa515c5163a99cc82bab70fd3bfdd36d827be85de63737b40fcef2ce084a436e"
dependencies = [
"winapi",
]
[[package]]
name = "winapi-x86_64-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"

View file

@ -0,0 +1,19 @@
[package]
name = "wasm-generate-spectests"
version = "0.1.0"
authors = ["Ryan Hunt <rhunt@eqrion.net>"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
regex = "1"
serde = "1"
serde_derive = "1"
toml = "0.5.6"
log = "0.4"
env_logger = "0.7"
anyhow = "1.0.19"
wast2js = { path = "./wast2js" }
[workspace]

View file

@ -0,0 +1,62 @@
# generate-spectests
A tool to generate a combined testsuite for the wasm spec and all proposals.
This tool tries to be as robust as possible to deal with upstream breakage, while still generating a minimal testsuite for proposals that don't change many tests.
## Usage
```bash
# Configure the tests you want
vim config.toml
# Generate the tests
# This will create a `repos/` and `tests/` in your working directory
cargo run
```
## config.toml
```toml
# (optional) Text to add to a 'directives.txt' file put in 'js/${repo}/harness'
harness_directive = ""
# (optional) Text to add to a 'directives.txt' file put in 'js/${repo}'
directive = ""
# (optional) Tests to include even if they haven't changed with respect to their parent repository
included_tests = ["test.wast"]
# (optional) Tests to exclude
excluded_tests = ["test.wast"]
[[repos]]
# Name of the repository
name = "sign-extension-ops"
# Url of the repository
url = "https://github.com/WebAssembly/sign-extension-ops"
# (optional) Name of the repository that is the upstream for this repository.
# This repository will attempt to merge with this upstream when generating
# tests. The parent repository must be specified before this repository in the
# 'config.toml'. If you change this, you must delete the 'repos' directory
# before generating tests again.
parent = "spec"
# (optional) Whether to skip merging with upstream, if it exists.
skip_merge = "false"
# (optional) The commit to checkout when generating tests. If not specified,
# defaults to the latest 'origin/master'.
commit = "df34ea92"
# (optional) Text to add to a 'directives.txt' file put in 'js/{$repo}'
directive = ""
# (optional) Tests to include even if they haven't changed with respect to their parent repository
included_tests = ["test.wast"]
# (optional) Tests to exclude
excluded_tests = ["test.wast"]
```

View file

@ -0,0 +1,486 @@
use std::env;
use std::ffi::OsStr;
use std::fs;
use std::path::{Path, PathBuf};
use std::process::Command;
use anyhow::{bail, Result};
use regex::{RegexSet, RegexSetBuilder};
use serde_derive::{Deserialize, Serialize};
use toml;
use wast2js;
use log::{debug, info, warn};
// Data structures
#[derive(Debug, Default, Serialize, Deserialize)]
struct Config {
#[serde(default)]
harness_directive: Option<String>,
#[serde(default)]
directive: Option<String>,
#[serde(default)]
included_tests: Vec<String>,
#[serde(default)]
excluded_tests: Vec<String>,
repos: Vec<Repo>,
}
impl Config {
fn find_repo_mut(&mut self, name: &str) -> Option<&mut Repo> {
self.repos.iter_mut().find(|x| &x.name == name)
}
}
#[derive(Debug, Default, Serialize, Deserialize)]
struct Repo {
name: String,
url: String,
#[serde(default)]
branch: Option<String>,
#[serde(default)]
parent: Option<String>,
#[serde(default)]
directive: Option<String>,
#[serde(default)]
included_tests: Vec<String>,
#[serde(default)]
excluded_tests: Vec<String>,
#[serde(default)]
skip_wast: bool,
#[serde(default)]
skip_js: bool,
}
#[derive(Debug, Default, Serialize, Deserialize)]
struct Lock {
repos: Vec<LockRepo>,
}
impl Lock {
fn find_commit(&self, name: &str) -> Option<&str> {
self.repos
.iter()
.find(|x| &x.name == name)
.map(|x| x.commit.as_ref())
}
fn set_commit(&mut self, name: &str, commit: &str) {
if let Some(lock) = self.repos.iter_mut().find(|x| &x.name == name) {
lock.commit = commit.to_owned();
} else {
self.repos.push(LockRepo {
name: name.to_owned(),
commit: commit.to_owned(),
});
}
}
}
#[derive(Debug, Default, Serialize, Deserialize)]
struct LockRepo {
name: String,
commit: String,
}
#[derive(Debug)]
enum Merge {
Standalone,
Merged,
Conflicted,
}
#[derive(Debug)]
struct Status {
commit_base_hash: String,
commit_final_message: String,
merged: Merge,
built: bool,
}
// Roll-your-own CLI utilities
fn run(name: &str, args: &[&str]) -> Result<String> {
debug!("{} {:?}", name, args);
let output = Command::new(name).args(args).output()?;
let stdout = String::from_utf8(output.stdout)?.trim().to_owned();
let stderr = String::from_utf8(output.stderr)?.trim().to_owned();
if !stdout.is_empty() {
debug!("{}", stdout);
}
if !stderr.is_empty() {
debug!("{}", stderr);
}
if output.status.success() {
Ok(stdout)
} else {
bail!("{}: {}\n{}", name.to_owned(), stdout, stderr)
}
}
fn change_dir(dir: &str) -> impl Drop {
#[must_use]
struct Reset {
previous: PathBuf,
}
impl Drop for Reset {
fn drop(&mut self) {
debug!("cd {}", self.previous.display());
env::set_current_dir(&self.previous).unwrap()
}
}
let previous = Reset {
previous: env::current_dir().unwrap(),
};
debug!("cd {}", dir);
env::set_current_dir(dir).unwrap();
previous
}
fn find(dir: &str) -> Vec<PathBuf> {
let mut paths = Vec::new();
fn find(dir: &str, paths: &mut Vec<PathBuf>) {
for entry in fs::read_dir(dir).unwrap().map(|x| x.unwrap()) {
let path = entry.path();
if entry.file_type().unwrap().is_dir() {
find(path.to_str().unwrap(), paths);
} else {
paths.push(path);
}
}
}
find(dir, &mut paths);
paths
}
fn write_string<P: AsRef<Path>>(path: P, text: &str) -> Result<()> {
let path = path.as_ref();
if let Some(dir) = path.parent() {
let _ = fs::create_dir_all(dir);
}
fs::write(path, text.as_bytes())?;
Ok(())
}
// The main script
fn main() {
env_logger::init();
// Load the config
let mut config: Config =
toml::from_str(&fs::read_to_string("config.toml").expect("failed to read config.toml"))
.expect("invalid config.toml");
// Load the lock file, or default to no pinned commits
let mut lock: Lock = if Path::new("config-lock.toml").exists() {
toml::from_str(
&fs::read_to_string("config-lock.toml").expect("failed to read config-lock.toml"),
)
.expect("invalid config-lock.toml")
} else {
Lock::default()
};
// Clean old tests and initialize the repo if it doesn't exist
let specs_dir = "specs/";
clean_and_init_dirs(specs_dir);
// Generate the tests
let mut successes = Vec::new();
let mut failures = Vec::new();
{
// Change to the `specs/` dir where all the work happens
let _cd = change_dir(specs_dir);
for repo in &config.repos {
info!("Processing {:#?}", repo);
match build_repo(repo, &config, &lock) {
Ok(status) => successes.push((repo.name.clone(), status)),
Err(err) => failures.push((repo.name.clone(), err)),
};
}
}
// Abort if we had a failure
if !failures.is_empty() {
warn!("Failed.");
for (name, err) in &failures {
warn!("{}: (failure) {:?}", name, err);
}
std::process::exit(1);
}
// Display successful results
info!("Done.");
for (name, status) in &successes {
let repo = config.find_repo_mut(&name).unwrap();
lock.set_commit(&name, &status.commit_base_hash);
info!(
"{}: ({} {}) {}",
repo.name,
match status.merged {
Merge::Standalone => "standalone",
Merge::Merged => "merged",
Merge::Conflicted => "conflicted",
},
if status.built { "building" } else { "broken" },
status.commit_final_message.trim_end()
);
}
// Commit the new lock file
write_string("config-lock.toml", &toml::to_string_pretty(&lock).unwrap()).unwrap();
}
fn clean_and_init_dirs(specs_dir: &str) {
if !Path::new(specs_dir).exists() {
fs::create_dir(specs_dir).unwrap();
run("git", &["-C", specs_dir, "init"]).unwrap();
}
let _ = fs::remove_dir_all("./tests");
}
fn build_repo(repo: &Repo, config: &Config, lock: &Lock) -> Result<Status> {
let remote_name = &repo.name;
let remote_url = &repo.url;
let remote_branch = repo.branch.as_ref().map(|x| x.as_str()).unwrap_or("master");
let branch_upstream = format!("{}/{}", repo.name, remote_branch);
let branch_base = repo.name.clone();
// Initialize our remote and branches if they don't exist
let remotes = run("git", &["remote"])?;
if !remotes.lines().any(|x| x == repo.name) {
run("git", &["remote", "add", remote_name, &remote_url])?;
run("git", &["fetch", remote_name])?;
run("git", &["branch", &branch_base, &branch_upstream])?;
}
// Set the upstream to the correct branch
run(
"git",
&[
"branch",
&branch_base,
"--set-upstream-to",
&branch_upstream,
],
)?;
// Fetch the latest changes for this repo
run("git", &["fetch", remote_name])?;
// Checkout the pinned commit, if any, and get the absolute commit hash
let base_treeish = lock.find_commit(&repo.name).unwrap_or(&branch_upstream);
run("git", &["checkout", &branch_base])?;
run("git", &["reset", base_treeish, "--hard"])?;
let commit_base_hash = run("git", &["log", "--pretty=%h", "-n", "1"])?
.trim()
.to_owned();
// Try to merge with parent repo, if specified
let merged = try_merge_parent(repo, &commit_base_hash)?;
// Try to build the test suite on this commit. This may fail due to merging
// with a parent repo, in which case we will try again in an unmerged state.
let mut built = false;
match try_build_tests() {
Ok(()) => built = true,
Err(err) => warn!("Failed to build tests: {:?}", err),
};
// if try_build_tests().is_err() {
// if repo.parent.is_some() {
// warn!(
// "Failed to build interpreter. Retrying on unmerged commit ({})",
// &commit_base_hash
// );
// run("git", &["reset", &commit_base_hash, "--hard"])?;
// built = try_build_tests().is_ok();
// } else {
// built = false;
// }
// }
// if !built {
// warn!("Failed to build interpreter, Won't emit js/html");
// }
// Get the final commit message we ended up on
let commit_final_message = run("git", &["log", "--oneline", "-n", "1"])?;
// Compute the source files that changed, and use that to filter the files
// we copy over. We can't compare the generated tests, because for a
// generated WPT we need to copy both the .js and .html even if only
// one of those is different from the master.
let tests_changed = find_tests_changed(repo)?;
info!("Changed tests: {:#?}", tests_changed);
// Include the changed tests, specified files, and `harness/` directory
let mut included_files = Vec::new();
included_files.extend_from_slice(&tests_changed);
included_files.extend_from_slice(&config.included_tests);
included_files.extend_from_slice(&repo.included_tests);
included_files.push("harness/".to_owned());
// Exclude files specified from the config and repo
let mut excluded_files = Vec::new();
excluded_files.extend_from_slice(&config.excluded_tests);
excluded_files.extend_from_slice(&repo.excluded_tests);
// Generate a regex set of the files to include or exclude
let include = RegexSetBuilder::new(&included_files).build().unwrap();
let exclude = RegexSetBuilder::new(&excluded_files).build().unwrap();
// Copy over all the desired test-suites
if !repo.skip_wast {
copy_tests(repo, "test/core", "../tests", "wast", &include, &exclude);
}
if built && !repo.skip_js {
copy_tests(repo, "js", "../tests", "js", &include, &exclude);
copy_directives(repo, config)?;
}
Ok(Status {
commit_final_message,
commit_base_hash,
merged,
built,
})
}
fn try_merge_parent(repo: &Repo, commit_base_hash: &str) -> Result<Merge> {
if !repo.parent.is_some() {
return Ok(Merge::Standalone);
}
let parent = repo.parent.as_ref().unwrap();
// Try to merge with the parent branch.
let message = format!("Merging {}:{}with {}", repo.name, commit_base_hash, parent);
Ok(
if !run("git", &["merge", "-q", parent, "-m", &message]).is_ok() {
// Ignore merge conflicts in the document directory.
if !run("git", &["checkout", "--ours", "document"]).is_ok()
|| !run("git", &["add", "document"]).is_ok()
|| !run("git", &["-c", "core.editor=true", "merge", "--continue"]).is_ok()
{
// Reset to master if we failed
warn!(
"Failed to merge {}, falling back to {}.",
repo.name, &commit_base_hash
);
run("git", &["merge", "--abort"])?;
run("git", &["reset", &commit_base_hash, "--hard"])?;
Merge::Conflicted
} else {
Merge::Merged
}
} else {
Merge::Merged
},
)
}
fn try_build_tests() -> Result<()> {
let _ = fs::remove_dir_all("./js");
fs::create_dir("./js")?;
let paths = find("./test/core/");
for path in paths {
if path.extension() != Some(OsStr::new("wast")) {
continue;
}
let source = std::fs::read_to_string(&path)?;
let script = wast2js::convert(&path, &source)?;
std::fs::write(
Path::new("./js").join(&path.with_extension("wast.js").file_name().unwrap()),
&script,
)?;
}
fs::create_dir("./js/harness")?;
write_string("./js/harness/harness.js", &wast2js::harness())?;
Ok(())
}
fn copy_tests(
repo: &Repo,
src_dir: &str,
dst_dir: &str,
test_name: &str,
include: &RegexSet,
exclude: &RegexSet,
) {
for path in find(src_dir) {
let stripped_path = path.strip_prefix(src_dir).unwrap();
let stripped_path_str = stripped_path.to_str().unwrap();
if !include.is_match(stripped_path_str) || exclude.is_match(stripped_path_str) {
continue;
}
let out_path = Path::new(dst_dir)
.join(test_name)
.join(&repo.name)
.join(&stripped_path);
let out_dir = out_path.parent().unwrap();
let _ = fs::create_dir_all(out_dir);
fs::copy(path, out_path).unwrap();
}
}
fn copy_directives(repo: &Repo, config: &Config) -> Result<()> {
// Write directives files
if let Some(harness_directive) = &config.harness_directive {
let directives_path = Path::new("../tests/js")
.join(&repo.name)
.join("harness/directives.txt");
write_string(&directives_path, harness_directive)?;
}
let directives = format!(
"{}{}",
config.directive.as_ref().map(|x| x.as_str()).unwrap_or(""),
repo.directive.as_ref().map(|x| x.as_str()).unwrap_or("")
);
if !directives.is_empty() {
let directives_path = Path::new("../tests/js")
.join(&repo.name)
.join("directives.txt");
write_string(&directives_path, &directives)?;
}
Ok(())
}
fn find_tests_changed(repo: &Repo) -> Result<Vec<String>> {
let files_changed = if let Some(parent) = repo.parent.as_ref() {
run(
"git",
&["diff", "--name-only", &repo.name, &parent, "test/core"],
)?
.lines()
.map(|x| PathBuf::from(x))
.collect()
} else {
find("test/core")
};
let mut tests_changed = Vec::new();
for path in files_changed {
if path.extension().map(|x| x.to_str().unwrap()) != Some("wast") {
continue;
}
let name = path.file_name().unwrap().to_str().unwrap().to_owned();
tests_changed.push(name);
}
Ok(tests_changed)
}

View file

@ -0,0 +1,12 @@
[package]
name = "wast2js"
version = "0.1.0"
authors = ["Ryan Hunt <rhunt@eqrion.net>"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
anyhow = "1.0.19"
wast = { version = "35.0.0" }
dprint-plugin-typescript = { version = "0.42.0" }

View file

@ -0,0 +1,3 @@
# wast2js
Mozilla specific converter from `.wast` to `.js` for SpiderMonkey jit-tests.

View file

@ -0,0 +1,574 @@
/* Copyright 2021 Mozilla Foundation
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
use anyhow::{bail, Context as _, Result};
use std::fmt::Write;
use std::path::Path;
use std::str;
const HARNESS: &'static str = include_str!("./harness.js");
const LICENSE: &'static str = include_str!("./license.js");
pub fn harness() -> String {
format_js(Path::new("harness.js"), HARNESS)
}
pub fn convert<P: AsRef<Path>>(path: P, wast: &str) -> Result<String> {
let filename = path.as_ref();
let adjust_wast = |mut err: wast::Error| {
err.set_path(filename);
err.set_text(wast);
err
};
let buf = wast::parser::ParseBuffer::new(wast).map_err(adjust_wast)?;
let ast = wast::parser::parse::<wast::Wast>(&buf).map_err(adjust_wast)?;
let mut out = String::new();
writeln!(&mut out, "{}", LICENSE)?;
writeln!(&mut out, "// {}\n", filename.display())?;
let mut current_instance: Option<usize> = None;
for directive in ast.directives {
let sp = directive.span();
let (line, col) = sp.linecol_in(wast);
convert_directive(
directive,
&mut current_instance,
filename,
line,
col,
wast,
&mut out,
)
.with_context(|| {
format!(
"failed to convert directive on {}:{}:{}",
filename.display(),
line + 1,
col
)
})?;
}
Ok(format_js(Path::new(filename), &out))
}
fn format_js(filename: &Path, text: &str) -> String {
let config = dprint_plugin_typescript::configuration::ConfigurationBuilder::new()
.deno()
.build();
std::panic::catch_unwind(|| {
dprint_plugin_typescript::format_text(filename, text, &config).unwrap()
})
.unwrap_or_else(|_| text.to_string())
}
fn convert_directive(
directive: wast::WastDirective,
current_instance: &mut Option<usize>,
filename: &Path,
line: usize,
col: usize,
wast: &str,
out: &mut String,
) -> Result<()> {
use wast::WastDirective::*;
if col == 1 {
writeln!(out, "// {}:{}", filename.display(), line + 1)?;
} else {
writeln!(out, "// {}:{}:{}", filename.display(), line + 1, col)?;
}
match directive {
Module(module) => {
let next_instance = current_instance.map(|x| x + 1).unwrap_or(0);
let module_text = module_to_js_string(&module, wast)?;
writeln!(
out,
"let ${} = instantiate(`{}`);",
next_instance, module_text
)?;
if let Some(id) = module.id {
writeln!(
out,
"register(${}, {});",
next_instance,
format!("`{}`", escape_template_string(id.name()))
)?;
}
*current_instance = Some(next_instance);
}
QuoteModule { span: _, source: _ } => {
write!(out, "// unsupported quote module")?;
}
Register {
span: _,
name,
module,
} => {
let instanceish = module
.map(|x| format!("`{}`", escape_template_string(x.name())))
.unwrap_or_else(|| format!("${}", current_instance.unwrap()));
writeln!(
out,
"register({}, `{}`);",
instanceish,
escape_template_string(name)
)?;
}
Invoke(i) => {
writeln!(out, "{};", invoke_to_js(current_instance, i)?)?;
}
AssertReturn {
span: _,
exec,
results,
} => {
writeln!(
out,
"assert_return(() => {}, {});",
execute_to_js(current_instance, exec, wast)?,
to_js_value_array(&results, |x| assert_expression_to_js_value(x))?
)?;
}
AssertTrap {
span: _,
exec,
message,
} => {
writeln!(
out,
"assert_trap(() => {}, `{}`);",
execute_to_js(current_instance, exec, wast)?,
escape_template_string(message)
)?;
}
AssertExhaustion {
span: _,
call,
message,
} => {
writeln!(
out,
"assert_exhaustion(() => {}, `{}`);",
invoke_to_js(current_instance, call)?,
escape_template_string(message)
)?;
}
AssertInvalid {
span: _,
module,
message,
} => {
let text = module_to_js_string(&module, wast)?;
writeln!(
out,
"assert_invalid(() => instantiate(`{}`), `{}`);",
text,
escape_template_string(message)
)?;
}
AssertMalformed {
module,
span: _,
message,
} => {
let text = match module {
wast::QuoteModule::Module(m) => module_to_js_string(&m, wast)?,
wast::QuoteModule::Quote(source) => quote_module_to_js_string(source)?,
};
writeln!(
out,
"assert_malformed(() => instantiate(`{}`), `{}`);",
text,
escape_template_string(message)
)?;
}
AssertUnlinkable {
span: _,
module,
message,
} => {
let text = module_to_js_string(&module, wast)?;
writeln!(
out,
"assert_unlinkable(() => instantiate(`{}`), `{}`);",
text,
escape_template_string(message)
)?;
}
}
writeln!(out, "")?;
Ok(())
}
fn escape_template_string(text: &str) -> String {
text.replace("$", "$$")
.replace("\\", "\\\\")
.replace("`", "\\`")
}
fn span_to_offset(span: wast::Span, text: &str) -> Result<usize> {
let (span_line, span_col) = span.linecol_in(text);
let mut cur = 0;
// Use split_terminator instead of lines so that if there is a `\r`,
// it is included in the offset calculation. The `+1` values below
// account for the `\n`.
for (i, line) in text.split_terminator('\n').enumerate() {
if span_line == i {
assert!(span_col < line.len());
return Ok(cur + span_col);
}
cur += line.len() + 1;
}
bail!("invalid line/col");
}
fn closed_module(module: &str) -> Result<&str> {
enum State {
Module,
QStr,
EscapeQStr,
}
let mut i = 0;
let mut level = 1;
let mut state = State::Module;
let mut chars = module.chars();
while level != 0 {
let next = match chars.next() {
Some(next) => next,
None => bail!("was unable to close module"),
};
match state {
State::Module => match next {
'(' => level += 1,
')' => level -= 1,
'"' => state = State::QStr,
_ => {}
},
State::QStr => match next {
'"' => state = State::Module,
'\\' => state = State::EscapeQStr,
_ => {}
},
State::EscapeQStr => match next {
_ => state = State::QStr,
},
}
i += next.len_utf8();
}
return Ok(&module[0..i]);
}
fn module_to_js_string(module: &wast::Module, wast: &str) -> Result<String> {
let offset = span_to_offset(module.span, wast)?;
let opened_module = &wast[offset..];
if !opened_module.starts_with("module") {
return Ok(escape_template_string(opened_module));
}
Ok(escape_template_string(&format!(
"({}",
closed_module(opened_module)?
)))
}
fn quote_module_to_js_string(quotes: Vec<&[u8]>) -> Result<String> {
let mut text = String::new();
for src in quotes {
text.push_str(str::from_utf8(src)?);
text.push_str(" ");
}
let escaped = escape_template_string(&text);
Ok(escaped)
}
fn invoke_to_js(current_instance: &Option<usize>, i: wast::WastInvoke) -> Result<String> {
let instanceish = i
.module
.map(|x| format!("`{}`", escape_template_string(x.name())))
.unwrap_or_else(|| format!("${}", current_instance.unwrap()));
Ok(format!(
"invoke({}, `{}`, {})",
instanceish,
escape_template_string(i.name),
to_js_value_array(&i.args, |x| expression_to_js_value(x))?
))
}
fn execute_to_js(
current_instance: &Option<usize>,
exec: wast::WastExecute,
wast: &str,
) -> Result<String> {
match exec {
wast::WastExecute::Invoke(invoke) => invoke_to_js(current_instance, invoke),
wast::WastExecute::Module(module) => {
let text = module_to_js_string(&module, wast)?;
Ok(format!("instantiate(`{}`)", text))
}
wast::WastExecute::Get { module, global } => {
let instanceish = module
.map(|x| format!("`{}`", escape_template_string(x.name())))
.unwrap_or_else(|| format!("${}", current_instance.unwrap()));
Ok(format!("get({}, `{}`)", instanceish, global))
}
}
}
fn to_js_value_array<T, F: Fn(&T) -> Result<String>>(vals: &[T], func: F) -> Result<String> {
let mut out = String::new();
let mut first = true;
write!(out, "[")?;
for val in vals {
if !first {
write!(out, ", ")?;
}
first = false;
write!(out, "{}", (func)(val)?)?;
}
write!(out, "]")?;
Ok(out)
}
fn f32_needs_bits(a: f32) -> bool {
if a.is_infinite() {
return false;
}
return a.is_nan()
|| ((a as f64) as f32).to_bits() != a.to_bits()
|| (format!("{:.}", a).parse::<f64>().unwrap() as f32).to_bits() != a.to_bits();
}
fn f64_needs_bits(a: f64) -> bool {
return a.is_nan();
}
fn f32x4_needs_bits(a: &[wast::Float32; 4]) -> bool {
a.iter().any(|x| {
let as_f32 = f32::from_bits(x.bits);
f32_needs_bits(as_f32)
})
}
fn f64x2_needs_bits(a: &[wast::Float64; 2]) -> bool {
a.iter().any(|x| {
let as_f64 = f64::from_bits(x.bits);
f64_needs_bits(as_f64)
})
}
fn f32_to_js_value(val: f32) -> String {
if val == f32::INFINITY {
format!("Infinity")
} else if val == f32::NEG_INFINITY {
format!("-Infinity")
} else if val.is_sign_negative() && val == 0f32 {
format!("-0")
} else {
format!("{:.}", val)
}
}
fn f64_to_js_value(val: f64) -> String {
if val == f64::INFINITY {
format!("Infinity")
} else if val == f64::NEG_INFINITY {
format!("-Infinity")
} else if val.is_sign_negative() && val == 0f64 {
format!("-0")
} else {
format!("{:.}", val)
}
}
fn float32_to_js_value(val: &wast::Float32) -> String {
let as_f32 = f32::from_bits(val.bits);
if f32_needs_bits(as_f32) {
format!(
"bytes('f32', {})",
to_js_value_array(&val.bits.to_le_bytes(), |x| Ok(format!("0x{:x}", x))).unwrap()
)
} else {
format!("value('f32', {})", f32_to_js_value(as_f32))
}
}
fn float64_to_js_value(val: &wast::Float64) -> String {
let as_f64 = f64::from_bits(val.bits);
if f64_needs_bits(as_f64) {
format!(
"bytes('f64', {})",
to_js_value_array(&val.bits.to_le_bytes(), |x| Ok(format!("0x{:x}", x))).unwrap()
)
} else {
format!("value('f64', {})", f64_to_js_value(as_f64))
}
}
fn f32_pattern_to_js_value(pattern: &wast::NanPattern<wast::Float32>) -> String {
use wast::NanPattern::*;
match pattern {
CanonicalNan => format!("`f32_canonical_nan`"),
ArithmeticNan => format!("`f32_arithmetic_nan`"),
Value(x) => float32_to_js_value(x),
}
}
fn f64_pattern_to_js_value(pattern: &wast::NanPattern<wast::Float64>) -> String {
use wast::NanPattern::*;
match pattern {
CanonicalNan => format!("`f64_canonical_nan`"),
ArithmeticNan => format!("`f64_arithmetic_nan`"),
Value(x) => float64_to_js_value(x),
}
}
fn assert_expression_to_js_value(v: &wast::AssertExpression<'_>) -> Result<String> {
use wast::AssertExpression::*;
Ok(match &v {
I32(x) => format!("value('i32', {})", x),
I64(x) => format!("value('i64', {}n)", x),
F32(x) => f32_pattern_to_js_value(x),
F64(x) => f64_pattern_to_js_value(x),
RefNull(x) => match x {
Some(wast::HeapType::Func) => format!("value('funcref', null)"),
Some(wast::HeapType::Extern) => format!("value('externref', null)"),
other => bail!(
"couldn't convert ref.null {:?} to a js assertion value",
other
),
},
RefExtern(x) => format!("value('externref', externref({}))", x),
V128(x) => {
use wast::V128Pattern::*;
match x {
I8x16(elements) => format!(
"i8x16({})",
to_js_value_array(elements, |x| Ok(format!("0x{:x}", x)))?
),
I16x8(elements) => format!(
"i16x8({})",
to_js_value_array(elements, |x| Ok(format!("0x{:x}", x)))?
),
I32x4(elements) => format!(
"i32x4({})",
to_js_value_array(elements, |x| Ok(format!("0x{:x}", x)))?
),
I64x2(elements) => format!(
"i64x2({})",
to_js_value_array(elements, |x| Ok(format!("0x{:x}n", x)))?
),
F32x4(elements) => {
let elements: Vec<String> = elements
.iter()
.map(|x| f32_pattern_to_js_value(x))
.collect();
format!(
"new F32x4Pattern({}, {}, {}, {})",
elements[0], elements[1], elements[2], elements[3]
)
}
F64x2(elements) => {
let elements: Vec<String> = elements
.iter()
.map(|x| f64_pattern_to_js_value(x))
.collect();
format!("new F64x2Pattern({}, {})", elements[0], elements[1])
}
}
}
other => bail!("couldn't convert {:?} to a js assertion value", other),
})
}
fn expression_to_js_value(v: &wast::Expression<'_>) -> Result<String> {
use wast::Instruction::*;
if v.instrs.len() != 1 {
bail!("too many instructions in {:?}", v);
}
Ok(match &v.instrs[0] {
I32Const(x) => format!("{}", *x),
I64Const(x) => format!("{}n", *x),
F32Const(x) => float32_to_js_value(x),
F64Const(x) => float64_to_js_value(x),
V128Const(x) => {
use wast::V128Const::*;
match x {
I8x16(elements) => format!(
"i8x16({})",
to_js_value_array(elements, |x| Ok(format!("0x{:x}", x)))?
),
I16x8(elements) => format!(
"i16x8({})",
to_js_value_array(elements, |x| Ok(format!("0x{:x}", x)))?
),
I32x4(elements) => format!(
"i32x4({})",
to_js_value_array(elements, |x| Ok(format!("0x{:x}", x)))?
),
I64x2(elements) => format!(
"i64x2({})",
to_js_value_array(elements, |x| Ok(format!("0x{:x}n", x)))?
),
F32x4(elements) => {
if f32x4_needs_bits(elements) {
format!(
"bytes('v128', {})",
to_js_value_array(&x.to_le_bytes(), |x| Ok(format!("0x{:x}", x)))?
)
} else {
format!(
"f32x4({})",
to_js_value_array(elements, |x| Ok(f32_to_js_value(f32::from_bits(
x.bits
))))?
)
}
}
F64x2(elements) => {
if f64x2_needs_bits(elements) {
format!(
"bytes('v128', {})",
to_js_value_array(&x.to_le_bytes(), |x| Ok(format!("0x{:x}", x)))?
)
} else {
format!(
"f64x2({})",
to_js_value_array(elements, |x| Ok(f64_to_js_value(f64::from_bits(
x.bits
))))?
)
}
}
}
}
RefNull(_) => format!("null"),
RefExtern(x) => format!("externref({})", x),
other => bail!("couldn't convert {:?} to a js value", other),
})
}

View file

@ -0,0 +1,286 @@
'use strict';
/* Copyright 2021 Mozilla Foundation
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
if (!wasmIsSupported()) {
quit();
}
function bytes(type, bytes) {
var typedBuffer = new Uint8Array(bytes);
return wasmGlobalFromArrayBuffer(type, typedBuffer.buffer);
}
function value(type, value) {
return new WebAssembly.Global({
value: type,
mutable: false,
}, value);
}
function i8x16(elements) {
let typedBuffer = new Uint8Array(elements);
return wasmGlobalFromArrayBuffer('v128', typedBuffer.buffer);
}
function i16x8(elements) {
let typedBuffer = new Uint16Array(elements);
return wasmGlobalFromArrayBuffer('v128', typedBuffer.buffer);
}
function i32x4(elements) {
let typedBuffer = new Uint32Array(elements);
return wasmGlobalFromArrayBuffer('v128', typedBuffer.buffer);
}
function i64x2(elements) {
let typedBuffer = new BigUint64Array(elements);
return wasmGlobalFromArrayBuffer('v128', typedBuffer.buffer);
}
function f32x4(elements) {
let typedBuffer = new Float32Array(elements);
return wasmGlobalFromArrayBuffer('v128', typedBuffer.buffer);
}
function f64x2(elements) {
let typedBuffer = new Float64Array(elements);
return wasmGlobalFromArrayBuffer('v128', typedBuffer.buffer);
}
class F32x4Pattern {
constructor(x, y, z, w) {
this.x = x;
this.y = y;
this.z = z;
this.w = w;
}
}
class F64x2Pattern {
constructor(x, y) {
this.x = x;
this.y = y;
}
}
let externrefs = {};
let externsym = Symbol("externref");
function externref(s) {
if (! (s in externrefs)) externrefs[s] = {[externsym]: s};
return externrefs[s];
}
function is_externref(x) {
return (x !== null && externsym in x) ? 1 : 0;
}
function is_funcref(x) {
return typeof x === "function" ? 1 : 0;
}
function eq_externref(x, y) {
return x === y ? 1 : 0;
}
function eq_funcref(x, y) {
return x === y ? 1 : 0;
}
let spectest = {
externref: externref,
is_externref: is_externref,
is_funcref: is_funcref,
eq_externref: eq_externref,
eq_funcref: eq_funcref,
print: console.log.bind(console),
print_i32: console.log.bind(console),
print_i32_f32: console.log.bind(console),
print_f64_f64: console.log.bind(console),
print_f32: console.log.bind(console),
print_f64: console.log.bind(console),
global_i32: 666,
global_i64: 666n,
global_f32: 666,
global_f64: 666,
table: new WebAssembly.Table({initial: 10, maximum: 20, element: 'anyfunc'}),
memory: new WebAssembly.Memory({initial: 1, maximum: 2})
};
let linkage = {
spectest,
};
function getInstance(instanceish) {
if (typeof instanceish === 'string') {
assertEq(instanceish in linkage, true, `'${instanceish}'' must be registered`);
return linkage[instanceish];
}
return instanceish;
}
function instantiate(source) {
let bytecode = wasmTextToBinary(source);
let module = new WebAssembly.Module(bytecode);
let instance = new WebAssembly.Instance(module, linkage);
return instance.exports;
}
function register(instanceish, name) {
linkage[name] = getInstance(instanceish);
}
function invoke(instanceish, field, params) {
let func = getInstance(instanceish)[field];
assertEq(func instanceof Function, true, 'expected a function');
return wasmLosslessInvoke(func, ...params);
}
function get(instanceish, field) {
let global = getInstance(instanceish)[field];
assertEq(global instanceof WebAssembly.Global, true, 'expected a WebAssembly.Global');
return global;
}
function assert_trap(thunk, message) {
try {
thunk();
assertEq("normal return", "trap");
} catch (err) {
assertEq(
err instanceof WebAssembly.RuntimeError, true, "expected trap");
}
}
let StackOverflow;
try { (function f() { 1 + f() })() } catch (e) { StackOverflow = e.constructor }
function assert_exhaustion(thunk, message) {
try {
thunk();
assertEq("normal return", "exhaustion");
} catch (err) {
assertEq(
err instanceof StackOverflow, true, "expected exhaustion");
}
}
function assert_invalid(thunk, message) {
try {
thunk();
assertEq("valid module", "invalid module");
} catch (err) {
assertEq(err instanceof WebAssembly.LinkError ||
err instanceof WebAssembly.CompileError, true, "expected an invalid module");
}
}
function assert_unlinkable(thunk, message) {
try {
thunk();
assertEq(true, false, "expected an unlinkable module");
} catch (err) {
assertEq(err instanceof WebAssembly.LinkError ||
err instanceof WebAssembly.CompileError, true, "expected an unlinkable module");
}
}
function assert_malformed(thunk, message) {
try {
thunk();
assertEq("valid module", "malformed module");
} catch (err) {
assertEq(err instanceof TypeError ||
err instanceof SyntaxError ||
err instanceof WebAssembly.CompileError ||
err instanceof WebAssembly.LinkError, true,
`expected a malformed module`);
}
}
function assert_return(thunk, expected) {
let results = thunk();
if (results === undefined) {
results = [];
} else if (!Array.isArray(results)) {
results = [results];
}
if (!Array.isArray(expected)) {
expected = [expected];
}
if (!compareResults(results, expected)) {
let got = results.map((x) => formatResult(x)).join(', ');
let wanted = expected.map((x) => formatExpected(x)).join(', ');
assertEq(
`[${got}]`,
`[${wanted}]`);
assertEq(true, false, `${got} !== ${wanted}`);
}
}
function formatResult(result) {
if (typeof(result) === 'object') {
return wasmGlobalToString(result);
} else {
return `${result}`;
}
}
function formatExpected(expected) {
if (expected === `f32_canonical_nan` ||
expected === `f32_arithmetic_nan` ||
expected === `f64_canonical_nan` ||
expected === `f64_arithmetic_nan`) {
return expected;
} else if (expected instanceof F32x4Pattern) {
return `f32x4(${formatExpected(expected.x)}, ${formatExpected(expected.y)}, ${formatExpected(expected.z)}, ${formatExpected(expected.w)})`
} else if (expected instanceof F64x2Pattern) {
return `f64x2(${formatExpected(expected.x)}, ${formatExpected(expected.y)})`
} else if (typeof(expected) === 'object') {
return wasmGlobalToString(expected);
} else {
throw new Error('unknown expected result');
}
}
function compareResults(results, expected) {
if (results.length !== expected.length) {
return false;
}
for (let i in results) {
if (!compareResult(results[i], expected[i])) {
return false;
}
}
return true;
}
function compareResult(result, expected) {
if (expected === `f32_canonical_nan` ||
expected === `f32_arithmetic_nan`) {
// TODO: compare exact NaN bits
return wasmGlobalsEqual(result, value('f32', NaN));
} else if (expected === `f64_canonical_nan` ||
expected === `f64_arithmetic_nan`) {
// TODO: compare exact NaN bits
return wasmGlobalsEqual(result, value('f64', NaN));
} else if (expected instanceof F32x4Pattern) {
return compareResult(wasmGlobalExtractLane(result, 'f32x4', 0), expected.x) &&
compareResult(wasmGlobalExtractLane(result, 'f32x4', 1), expected.y) &&
compareResult(wasmGlobalExtractLane(result, 'f32x4', 2), expected.z) &&
compareResult(wasmGlobalExtractLane(result, 'f32x4', 3), expected.w);
} else if (expected instanceof F64x2Pattern) {
return compareResult(wasmGlobalExtractLane(result, 'f64x2', 0), expected.x) &&
compareResult(wasmGlobalExtractLane(result, 'f64x2', 1), expected.y);
} else if (typeof(expected) === 'object') {
return wasmGlobalsEqual(result,
expected
);
} else {
throw new Error('unknown expected result');
}
}

View file

@ -0,0 +1,17 @@
/* Copyright 2021 Mozilla Foundation
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
mod convert;
pub use convert::*;

View file

@ -0,0 +1,14 @@
/* Copyright 2021 Mozilla Foundation
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

View file

@ -0,0 +1,39 @@
/* Copyright 2021 Mozilla Foundation
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
use anyhow::{Context as _, Result};
use std::env;
use std::path::Path;
mod convert;
fn main() -> Result<()> {
let files = env::args().collect::<Vec<String>>();
for path in &files[1..] {
let source =
std::fs::read_to_string(path).with_context(|| format!("failed to read `{}`", path))?;
let mut full_script = String::new();
full_script.push_str(&convert::harness());
full_script.push_str(&convert::convert(path, &source)?);
std::fs::write(
Path::new(path).with_extension("js").file_name().unwrap(),
&full_script,
)?;
}
Ok(())
}