diff options
Diffstat (limited to 'src/doc/rustc-dev-guide/src/tests')
| -rw-r--r-- | src/doc/rustc-dev-guide/src/tests/directives.md | 10 | ||||
| -rw-r--r-- | src/doc/rustc-dev-guide/src/tests/intro.md | 6 | ||||
| -rw-r--r-- | src/doc/rustc-dev-guide/src/tests/misc.md | 2 | ||||
| -rw-r--r-- | src/doc/rustc-dev-guide/src/tests/suggest-tests.md | 59 | ||||
| -rw-r--r-- | src/doc/rustc-dev-guide/src/tests/ui.md | 20 |
5 files changed, 27 insertions, 70 deletions
diff --git a/src/doc/rustc-dev-guide/src/tests/directives.md b/src/doc/rustc-dev-guide/src/tests/directives.md index 839076b809d..5c3ae359ba0 100644 --- a/src/doc/rustc-dev-guide/src/tests/directives.md +++ b/src/doc/rustc-dev-guide/src/tests/directives.md @@ -59,7 +59,7 @@ not be exhaustive. Directives can generally be found by browsing the | `aux-crate` | Like `aux-build` but makes available as extern prelude | All except `run-make` | `<extern_prelude_name>=<path/to/aux/file.rs>` | | `aux-codegen-backend` | Similar to `aux-build` but pass the compiled dylib to `-Zcodegen-backend` when building the main file | `ui-fulldeps` | Path to codegen backend file | | `proc-macro` | Similar to `aux-build`, but for aux forces host and don't use `-Cprefer-dynamic`[^pm]. | All except `run-make` | Path to auxiliary proc-macro `.rs` file | -| `build-aux-docs` | Build docs for auxiliaries as well | All except `run-make` | N/A | +| `build-aux-docs` | Build docs for auxiliaries as well. Note that this only works with `aux-build`, not `aux-crate`. | All except `run-make` | N/A | [^pm]: please see the Auxiliary proc-macro section in the [compiletest](./compiletest.md) chapter for specifics. @@ -75,8 +75,10 @@ expectations](ui.md#controlling-passfail-expectations). | `check-fail` | Building (no codegen) should fail | `ui`, `crashes` | N/A | | `build-pass` | Building should pass | `ui`, `crashes`, `codegen`, `incremental` | N/A | | `build-fail` | Building should fail | `ui`, `crashes` | N/A | -| `run-pass` | Running the test binary should pass | `ui`, `crashes`, `incremental` | N/A | -| `run-fail` | Running the test binary should fail | `ui`, `crashes` | N/A | +| `run-pass` | Program must exit with code `0` | `ui`, `crashes`, `incremental` | N/A | +| `run-fail` | Program must exit with code `1..=127` | `ui`, `crashes` | N/A | +| `run-crash` | Program must crash | `ui` | N/A | +| `run-fail-or-crash` | Program must `run-fail` or `run-crash` | `ui` | N/A | | `ignore-pass` | Ignore `--pass` flag | `ui`, `crashes`, `codegen`, `incremental` | N/A | | `dont-check-failure-status` | Don't check exact failure status (i.e. `1`) | `ui`, `incremental` | N/A | | `failure-status` | Check | `ui`, `crashes` | Any `u16` | @@ -203,6 +205,8 @@ settings: on `wasm32-unknown-unknown` target because the target does not support the `proc-macro` crate type. - `needs-target-std` — ignores if target platform does not have std support. +- `ignore-backends` — ignores the listed backends, separated by whitespace characters. +- `needs-backends` — only runs the test if current codegen backend is listed. The following directives will check LLVM support: diff --git a/src/doc/rustc-dev-guide/src/tests/intro.md b/src/doc/rustc-dev-guide/src/tests/intro.md index c55d60f4a5c..79b96c450a8 100644 --- a/src/doc/rustc-dev-guide/src/tests/intro.md +++ b/src/doc/rustc-dev-guide/src/tests/intro.md @@ -111,12 +111,14 @@ and it can be invoked so: This requires building all of the documentation, which might take a while. -### Dist check +### `distcheck` `distcheck` verifies that the source distribution tarball created by the build system will unpack, build, and run all tests. -> Example: `./x test distcheck` +```console +./x test distcheck +``` ### Tool tests diff --git a/src/doc/rustc-dev-guide/src/tests/misc.md b/src/doc/rustc-dev-guide/src/tests/misc.md index c0288b3dd10..39f88174879 100644 --- a/src/doc/rustc-dev-guide/src/tests/misc.md +++ b/src/doc/rustc-dev-guide/src/tests/misc.md @@ -9,7 +9,7 @@ for testing: - `RUSTC_BOOTSTRAP=1` will "cheat" and bypass usual stability checking, allowing you to use unstable features and cli flags on a stable `rustc`. -- `RUSTC_BOOTSTRAP=-1` will force a given `rustc` to pretend that is a stable +- `RUSTC_BOOTSTRAP=-1` will force a given `rustc` to pretend it is a stable compiler, even if it's actually a nightly `rustc`. This is useful because some behaviors of the compiler (e.g. diagnostics) can differ depending on whether the compiler is nightly or not. diff --git a/src/doc/rustc-dev-guide/src/tests/suggest-tests.md b/src/doc/rustc-dev-guide/src/tests/suggest-tests.md deleted file mode 100644 index 663e8a5af3b..00000000000 --- a/src/doc/rustc-dev-guide/src/tests/suggest-tests.md +++ /dev/null @@ -1,59 +0,0 @@ -# Suggest tests tool - -This chapter is about the internals of and contribution instructions for the -`suggest-tests` tool. For a high-level overview of the tool, see [this -section](../building/suggested.md#x-suggest). This tool is currently in a beta -state and is tracked by [this](https://github.com/rust-lang/rust/issues/109933) -issue on Github. Currently the number of tests it will suggest are very limited -in scope, we are looking to expand this (contributions welcome!). - -## Internals - -The tool is defined in a separate crate -([`src/tools/suggest-tests`](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests)) -which outputs suggestions which are parsed by a shim in bootstrap -([`src/bootstrap/src/core/build_steps/suggest.rs`](https://github.com/rust-lang/rust/blob/master/src/bootstrap/src/core/build_steps/suggest.rs)). -The only notable thing the bootstrap shim does is (when invoked with the `--run` -flag) use bootstrap's internal mechanisms to create a new `Builder` and uses it -to invoke the suggested commands. The `suggest-tests` crate is where the fun -happens, two kinds of suggestions are defined: "static" and "dynamic" -suggestions. - -### Static suggestions - -Defined -[here](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests/src/static_suggestions.rs). -Static suggestions are simple: they are just -[globs](https://crates.io/crates/glob) which map to a `x` command. In -`suggest-tests`, this is implemented with a simple `macro_rules` macro. - -### Dynamic suggestions - -Defined -[here](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests/src/dynamic_suggestions.rs). -These are more complicated than static suggestions and are implemented as -functions with the following signature: `fn(&Path) -> Vec<Suggestion>`. In other -words, each suggestion takes a path to a modified file and (after running -arbitrary Rust code) can return any number of suggestions, or none. Dynamic -suggestions are useful for situations where fine-grained control over -suggestions is needed. For example, modifications to the `compiler/xyz/` path -should trigger the `x test compiler/xyz` suggestion. In the future, dynamic -suggestions might even read file contents to determine if (what) tests should -run. - -## Adding a suggestion - -The following steps should serve as a rough guide to add suggestions to -`suggest-tests` (very welcome!): - -1. Determine the rules for your suggestion. Is it simple and operates only on a - single path or does it match globs? Does it need fine-grained control over - the resulting command or does "one size fit all"? -2. Based on the previous step, decide if your suggestion should be implemented - as either static or dynamic. -3. Implement the suggestion. If it is dynamic then a test is highly recommended, - to verify that your logic is correct and to give an example of the - suggestion. See the - [tests.rs](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests/src/tests.rs) - file. -4. Open a PR implementing your suggestion. **(TODO: add example PR)** diff --git a/src/doc/rustc-dev-guide/src/tests/ui.md b/src/doc/rustc-dev-guide/src/tests/ui.md index 4fce5838b6e..9bfc60e08a6 100644 --- a/src/doc/rustc-dev-guide/src/tests/ui.md +++ b/src/doc/rustc-dev-guide/src/tests/ui.md @@ -448,7 +448,7 @@ even run the resulting program. Just add one of the following - `//@ build-pass` — compilation and linking should succeed but do not run the resulting binary. - `//@ run-pass` — compilation should succeed and running the resulting - binary should also succeed. + binary should make it exit with code 0 which indicates success. - Fail directives: - `//@ check-fail` — compilation should fail (the codegen phase is skipped). This is the default for UI tests. @@ -457,10 +457,20 @@ even run the resulting program. Just add one of the following - First time is to ensure that the compile succeeds without the codegen phase - Second time is to ensure that the full compile fails - `//@ run-fail` — compilation should succeed, but running the resulting - binary should fail. - -For `run-pass` and `run-fail` tests, by default the output of the program itself -is not checked. + binary should make it exit with a code in the range `1..=127` which + indicates regular failure. On targets without unwind support, crashes + are also accepted. + - `//@ run-crash` — compilation should succeed, but running the resulting + binary should fail with a crash. Crashing is defined as "not exiting with + a code in the range `0..=127`". Example on Linux: Termination by `SIGABRT` + or `SIGSEGV`. Example on Windows: Exiting with the code for + `STATUS_ILLEGAL_INSTRUCTION` (`0xC000001D`). + - `//@ run-fail-or-crash` — compilation should succeed, but running the + resulting binary should either `run-fail` or `run-crash`. Useful if a test + crashes on some targets but just fails on others. + +For `run-pass`. `run-fail`, `run-crash` and `run-fail-or-crash` tests, by +default the output of the program itself is not checked. If you want to check the output of running the program, include the `check-run-results` directive. This will check for a `.run.stderr` and |