Auto merge of #122036 - alexcrichton:test-wasm-with-wasi, r=oli-obk

Test wasm32-wasip1 in CI, not wasm32-unknown-unknown

This commit changes CI to no longer test the `wasm32-unknown-unknown` target and instead test the `wasm32-wasip1` target. There was some discussion of this in a [Zulip thread], and the motivations for this PR are:

* Runtime failures on `wasm32-unknown-unknown` print nothing, meaning all you get is "something failed". In contrast `wasm32-wasip1` can print to stdout/stderr.

* The unknown-unknown target is missing lots of pieces of libstd, and while `wasm32-wasip1` is also missing some pieces (e.g. threads) it's missing fewer pieces. This means that many more tests can be run.

Overall my hope is to improve the debuggability of wasm failures on CI and ideally be a bit less of a maintenance burden.

This commit specifically removes the testing of `wasm32-unknown-unknown` and replaces it with testing of `wasm32-wasip1`. Along the way there were a number of other archiectural changes made as well, including:

* A new `target.*.runtool` option can now be specified in `config.toml` which is passed as `--runtool` to `compiletest`. This is used to reimplement execution of WebAssembly in a less-wasm-specific fashion.

* The default value for `runtool` is an ambiently located WebAssembly runtime found on the system, if any. I've implemented logic for Wasmtime.

* Existing testing support for `wasm32-unknown-unknown` and Emscripten has been removed. I'm not aware of Emscripten testing being run any time recently and otherwise `wasm32-wasip1` is in theory the focus now.

* I've added a new `//@ needs-threads` directive for `compiletest` and classified a bunch of wasm-ignored tests as needing threads. In theory these tests can run on `wasm32-wasi-preview1-threads`, for example.

* I've tried to audit all existing tests that are either `ignore-emscripten` or `ignore-wasm*`. Many now run on `wasm32-wasip1` due to being able to emit error messages, for example. Many are updated with comments as to why they can't run as well.

* The `compiletest` output matching for `wasm32-wasip1` automatically uses "match a subset" mode implemented in `compiletest`. This is because WebAssembly runtimes often add extra information on failure, such as the `unreachable` instruction in `panic!`, which isn't able to be matched against the golden output from native platforms.

* I've ported most existing `run-make` tests that use custom Node.js wrapper scripts to the new run-make-based-in-Rust infrastructure. To do this I added `wasmparser` as a dependency of `run-make-support` for the various wasm tests to use that parse wasm files. The one test that executed WebAssembly now uses `wasmtime`-the-CLI to execute the test instead. I have not ported over an exception-handling test as Wasmtime doesn't implement this yet.

* I've updated the `test` crate to print out timing information for WASI targets as it can do that (gets a previously ignored test now passing).

* The `test-various` image now builds a WASI sysroot for the WASI target and additionally downloads a fixed release of Wasmtime, currently the latest one at 18.0.2, and uses that for testing.

[Zulip thread]: https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/Have.20wasm.20tests.20ever.20caused.20problems.20on.20CI.3F/near/424317944

18 files changed, 12 insertions, 9 deletions

diff --git a/tests/codegen/abi-main-signature-32bit-c-int.rs b/tests/codegen/abi-main-signature-32bit-c-int.rs
index 52db3d893e1..7684024a2e3 100644
--- a/tests/codegen/abi-main-signature-32bit-c-int.rs
+++ b/tests/codegen/abi-main-signature-32bit-c-int.rs
@@ -4,6 +4,7 @@
 // This test is for targets with 32bit c_int only.
 //@ ignore-msp430
 //@ ignore-avr
+//@ ignore-wasi wasi codegens the main symbol differently
 
 fn main() {
 }
diff --git a/tests/codegen/drop.rs b/tests/codegen/drop.rs
index 93e54979a05..1e80247ba8a 100644
--- a/tests/codegen/drop.rs
+++ b/tests/codegen/drop.rs
@@ -1,4 +1,3 @@
-//@ ignore-wasm32-bare compiled with panic=abort by default
 //@ needs-unwind - this test verifies the amount of drop calls when unwinding is used
 //@ compile-flags: -C no-prepopulate-passes
 
diff --git a/tests/codegen/enum/enum-debug-clike.rs b/tests/codegen/enum/enum-debug-clike.rs
index 205c57d1456..59ad5878443 100644
--- a/tests/codegen/enum/enum-debug-clike.rs
+++ b/tests/codegen/enum/enum-debug-clike.rs
@@ -3,6 +3,7 @@
 
 //
 //@ ignore-msvc
+//@ ignore-wasi wasi codegens the main symbol differently
 
 //@ compile-flags: -g -C no-prepopulate-passes
 
diff --git a/tests/codegen/enum/enum-debug-niche.rs b/tests/codegen/enum/enum-debug-niche.rs
index fc6a73e8472..90de928bced 100644
--- a/tests/codegen/enum/enum-debug-niche.rs
+++ b/tests/codegen/enum/enum-debug-niche.rs
@@ -2,6 +2,7 @@
 // This is ignored for the fallback mode on MSVC due to problems with PDB.
 
 //@ ignore-msvc
+//@ ignore-wasi wasi codegens the main symbol differently
 
 //@ compile-flags: -g -C no-prepopulate-passes
 
diff --git a/tests/codegen/enum/enum-debug-tagged.rs b/tests/codegen/enum/enum-debug-tagged.rs
index 87a6ccae291..f13922ee33b 100644
--- a/tests/codegen/enum/enum-debug-tagged.rs
+++ b/tests/codegen/enum/enum-debug-tagged.rs
@@ -2,6 +2,7 @@
 // This is ignored for the fallback mode on MSVC due to problems with PDB.
 
 //@ ignore-msvc
+//@ ignore-wasi wasi codegens the main symbol differently
 
 //@ compile-flags: -g -C no-prepopulate-passes
 
diff --git a/tests/codegen/enum/enum-u128.rs b/tests/codegen/enum/enum-u128.rs
index 94e80e34068..ecdff3c5ce3 100644
--- a/tests/codegen/enum/enum-u128.rs
+++ b/tests/codegen/enum/enum-u128.rs
@@ -3,6 +3,7 @@
 
 //
 //@ ignore-msvc
+//@ ignore-wasi wasi codegens the main symbol differently
 
 //@ compile-flags: -g -C no-prepopulate-passes
 
diff --git a/tests/codegen/fn-impl-trait-self.rs b/tests/codegen/fn-impl-trait-self.rs
index 9f10762d8fa..5799d23b5a0 100644
--- a/tests/codegen/fn-impl-trait-self.rs
+++ b/tests/codegen/fn-impl-trait-self.rs
@@ -1,4 +1,5 @@
 //@ compile-flags: -g
+//@ ignore-wasi wasi codegens the main symbol differently
 //
 // CHECK-LABEL: @main
 // MSVC: {{.*}}DIDerivedType(tag: DW_TAG_pointer_type, name: "recursive_type$ (*)()",{{.*}}
diff --git a/tests/codegen/generic-debug.rs b/tests/codegen/generic-debug.rs
index 87a0ddfea93..0f289026396 100644
--- a/tests/codegen/generic-debug.rs
+++ b/tests/codegen/generic-debug.rs
@@ -1,4 +1,5 @@
 //@ ignore-windows
+//@ ignore-wasi wasi codegens the main symbol differently
 
 //@ compile-flags: -g -C no-prepopulate-passes
 
diff --git a/tests/codegen/link_section.rs b/tests/codegen/link_section.rs
index 6747feba211..281d3fb99d4 100644
--- a/tests/codegen/link_section.rs
+++ b/tests/codegen/link_section.rs
@@ -1,4 +1,4 @@
-//@ ignore-emscripten default visibility is hidden
+//@ ignore-wasm32 custom sections work differently on wasm
 //@ compile-flags: -C no-prepopulate-passes
 
 #![crate_type = "lib"]
diff --git a/tests/codegen/mainsubprogram.rs b/tests/codegen/mainsubprogram.rs
index 8e173df0e86..c1933b2b390 100644
--- a/tests/codegen/mainsubprogram.rs
+++ b/tests/codegen/mainsubprogram.rs
@@ -3,6 +3,7 @@
 
 //@ ignore-windows
 //@ ignore-macos
+//@ ignore-wasi
 
 //@ compile-flags: -g -C no-prepopulate-passes
 
diff --git a/tests/codegen/mainsubprogramstart.rs b/tests/codegen/mainsubprogramstart.rs
index db2c1466bf5..84d680b9bff 100644
--- a/tests/codegen/mainsubprogramstart.rs
+++ b/tests/codegen/mainsubprogramstart.rs
@@ -1,5 +1,6 @@
 //@ ignore-windows
 //@ ignore-macos
+//@ ignore-wasi wasi codegens the main symbol differently
 
 //@ compile-flags: -g -C no-prepopulate-passes
 
diff --git a/tests/codegen/personality_lifetimes.rs b/tests/codegen/personality_lifetimes.rs
index 06389688e0e..f2ab9c3bb82 100644
--- a/tests/codegen/personality_lifetimes.rs
+++ b/tests/codegen/personality_lifetimes.rs
@@ -1,5 +1,4 @@
 //@ ignore-msvc
-//@ ignore-wasm32-bare compiled with panic=abort by default
 //@ needs-unwind
 
 //@ compile-flags: -O -C no-prepopulate-passes
diff --git a/tests/codegen/unwind-abis/nounwind-on-stable-panic-abort.rs b/tests/codegen/unwind-abis/nounwind-on-stable-panic-abort.rs
index 0b3bfd567aa..d27cbd60437 100644
--- a/tests/codegen/unwind-abis/nounwind-on-stable-panic-abort.rs
+++ b/tests/codegen/unwind-abis/nounwind-on-stable-panic-abort.rs
@@ -1,5 +1,4 @@
 //@ compile-flags: -C opt-level=0 -Cpanic=abort
-//@ ignore-wasm32-bare compiled with panic=abort by default
 
 #![crate_type = "lib"]
 
diff --git a/tests/codegen/unwind-abis/nounwind-on-stable-panic-unwind.rs b/tests/codegen/unwind-abis/nounwind-on-stable-panic-unwind.rs
index 1e6f8c9ede9..a7f7f2fb77f 100644
--- a/tests/codegen/unwind-abis/nounwind-on-stable-panic-unwind.rs
+++ b/tests/codegen/unwind-abis/nounwind-on-stable-panic-unwind.rs
@@ -1,5 +1,4 @@
 //@ compile-flags: -C opt-level=0
-//@ ignore-wasm32-bare compiled with panic=abort by default
 //@ needs-unwind
 
 #![crate_type = "lib"]
diff --git a/tests/codegen/unwind-abis/nounwind.rs b/tests/codegen/unwind-abis/nounwind.rs
index ac53cd7bed3..80bf8d67091 100644
--- a/tests/codegen/unwind-abis/nounwind.rs
+++ b/tests/codegen/unwind-abis/nounwind.rs
@@ -1,5 +1,5 @@
 //@ compile-flags: -C opt-level=0 -Cpanic=abort
-//@ ignore-wasm32-bare compiled with panic=abort by default
+//@ needs-unwind
 
 #![crate_type = "lib"]
 #![feature(c_unwind)]
diff --git a/tests/codegen/unwind-extern-exports.rs b/tests/codegen/unwind-extern-exports.rs
index d670a776ac6..ea59748b3bc 100644
--- a/tests/codegen/unwind-extern-exports.rs
+++ b/tests/codegen/unwind-extern-exports.rs
@@ -1,5 +1,4 @@
 //@ compile-flags: -C opt-level=0
-//@ ignore-wasm32-bare compiled with panic=abort by default
 //@ needs-unwind
 
 #![crate_type = "lib"]
diff --git a/tests/codegen/unwind-extern-imports.rs b/tests/codegen/unwind-extern-imports.rs
index 7386704b430..790e4def8b3 100644
--- a/tests/codegen/unwind-extern-imports.rs
+++ b/tests/codegen/unwind-extern-imports.rs
@@ -1,5 +1,4 @@
 //@ compile-flags: -C no-prepopulate-passes
-//@ ignore-wasm32-bare compiled with panic=abort by default
 //@ needs-unwind
 
 #![crate_type = "lib"]
diff --git a/tests/codegen/wasm_exceptions.rs b/tests/codegen/wasm_exceptions.rs
index 66d2bbed709..a53722f834c 100644
--- a/tests/codegen/wasm_exceptions.rs
+++ b/tests/codegen/wasm_exceptions.rs
@@ -1,4 +1,4 @@
-//@ only-wasm32-bare
+//@ only-wasm32
 //@ compile-flags: -C panic=unwind
 
 #![crate_type = "lib"]