| Age | Commit message (Collapse) | Author | Lines |
|---|
|
Include test suite metadata in the build metrics
This PR enhances the build metadata to include structured information about the test suites being executed, allowing external tools consuming the metadata to understand what was being tested.
The included metadata is:
* Target triple
* Host triple
* Stage number
* For compiletest tests:
* Suite name
* Mode
* Comparing mode
* For crate tests:
* List of crate names
This is implemented by replacing the `test` JSON node with a new `test_suite` node, which contains the metadata and the list of tests. This change also improves the handling of multiple test suites executed in the same step (for example in compiletest tests with a compare mode), as the multiple test suite executions will now be tracked in separate `test_suite` nodes.
This included a breaking change in the build metrics metadata format. To better handle this, in the second commit this PR introduces the `metadata_version` top-level field. The old version is considered to be `0`, while the new one `1`. Bootstrap will also gracefully handle existing metadata of a different version.
r? `@jyn514`
|
|
r=albertlarsan68,oli-obk
new tool `rustdoc-gui-test`
Implements new tool `rustdoc-gui-test` that allows using compiletest headers for `rustdoc-gui` tests.
|
|
|
|
|
|
|
|
|
|
Generate shell completions for bootstrap with Clap
Now that #110693 has been merged, we can look at generating shell completions for x.py with `clap_complete`. Leaving this as draft for now as I'm not sure of the best way to integration the completion generator. Additionally, the generated completions for zsh are completely broken (will need to be resolved upstream, it doesn't seem to handle subcommands + global arguments well).
I don't have Fish installed and would be interested to know how well completions work there.
Alternative to #107827
|
|
|
|
|
|
Migrate bootstrap to Clap-based argument parsing
Supercedes #108083
I chose to re-do the work rather than rebase the onto the large changes since the original PR. If it's preferred I can instead force-push the original PR to this version.
cc `@jyn514` `@albertlarsan68`
|
|
|
|
Previously, this was only passed to miri and compiletest. Extended it to
all other tests and binaries as well.
|
|
Add a `sysroot` crate to represent the standard library crates
This adds a dummy crate named `sysroot` to represent the standard library target instead of using the `test` crate. This allows the removal of `proc_macro` as a dependency of `test` allowing these 2 crates to build in parallel saving around 9 seconds locally.
|
|
- Use `cargo metadata` to determine whether a crate has a library
package or not
- Collect metadata for all workspaces, not just the root workspace and
cargo
- Don't pass `--lib` for crates without a library
- Use `run_cargo_test` for rust-installer
- Don't build documentation in `lint-docs` if `--no-doc` is passed
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Support `x test --stage 1 ui-fulldeps`
`@Nilstrieb` had an excellent idea the other day: the same way that rustdoc is able to load `rustc_driver` from the sysroot, ui-fulldeps tests should also be able to load it from the sysroot. That allows us to run fulldeps tests with stage1, without having to fully rebuild the compiler twice. It does unfortunately have the downside that we're building the tests with the *bootstrap* compiler, not the in-tree sources, but since most of the fulldeps tests are for the *API* of the compiler, that seems ok.
I think it's possible to extend this to `run-make-fulldeps`, but I've run out of energy for tonight.
- Move `plugin` tests into a subdirectory.
Plugins are loaded at runtime with `dlopen` and so require the ABI of the running compile to match the ABI of the compiler linked with `rustc_driver`. As a result they can't be supported in stage 1 and have to use `// ignore-stage1`.
- Remove `ignore-stage1` from most non-plugin tests
- Ignore diagnostic tests in stage 1. Even though this requires a stage 2 build to load rustc_driver, it's primarily testing the error message that the *running* compiler emits when the diagnostic struct is malformed.
- Pass `-Zforce-unstable-if-unmarked` in stage1, not just stage2. That allows running `hash-stable-is-unstable` in stage1, since it now suggests adding `rustc_private` to enable loading the crates.
- Add libLLVM.so to the stage0 target sysroot, to allow fulldeps tests that act as custom drivers to load it at runtime.
- Pass `--sysroot stage0-sysroot` in compiletest so that we use the correct version of std.
- Move a few lint tests from ui-fulldeps to ui
These had an `aux-build:lint-group-plugin-test.rs` that they never actually loaded with `feature(plugin)` nor tested. I removed the unused aux-build and they pass fine with stage 1.
Fixes https://github.com/rust-lang/rust/issues/75905.
|
|
Nils had an excellent idea the other day: the same way that rustdoc is
able to load `rustc_driver` from the sysroot, ui-fulldeps tests should
also be able to load it from the sysroot. That allows us to run fulldeps
tests with stage1, without having to fully rebuild the compiler twice.
It does unfortunately have the downside that we're running the tests on
the *bootstrap* compiler, not the in-tree sources, but since most of the
fulldeps tests are for the *API* of the compiler, that seems ok.
I think it's possible to extend this to `run-make-fulldeps`, but I've
run out of energy for tonight.
- Move `plugin` tests into a subdirectory.
Plugins are loaded at runtime with `dlopen` and so require the ABI of
the running compile to match the ABI of the compiler linked with
`rustc_driver`. As a result they can't be supported in stage 1 and have
to use `// ignore-stage1`.
- Remove `ignore-stage1` from most non-plugin tests
- Ignore diagnostic tests in stage 1. Even though this requires a stage
2 build to load rustc_driver, it's primarily testing the error message
that the *running* compiler emits when the diagnostic struct is malformed.
- Pass `-Zforce-unstable-if-unmarked` in stage1, not just stage2. That
allows running `hash-stable-is-unstable` in stage1, since it now
suggests adding `rustc_private` to enable loading the crates.
- Add libLLVM.so to the stage0 target sysroot, to allow fulldeps tests
that act as custom drivers to load it at runtime.
- Pass `--sysroot stage0-sysroot` in compiletest so that we use the
correct version of std.
|
|
|
|
This header can be used for tests which check the output of `--version --verbose` commands.
|
|
Rollup of 7 pull requests
Successful merges:
- #103682 (Stabilize rustdoc `--test-run-directory`)
- #106249 (Create "suggested tests" tool in `rustbuild`)
- #110047 (Add link to `collections` docs to `extend` trait)
- #110269 (Add `tidy-alphabetical` to features in `core`)
- #110292 (Add `tidy-alphabetical` to features in `alloc` & `std`)
- #110305 (rustdoc-search: use ES6 `Map` and `Set` where they make sense)
- #110315 (Add a stable MIR way to get the main function)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
Create "suggested tests" tool in `rustbuild`
Not the claimed person in #97339 but:
I've done a very rough implementation of this feature in-tree. I'm very new to `rustc` development (outside of docs) so some help would be greatly appreciated. The UI of this new subcommand obviously will change and I need some mentoring with the `--run` flag.
r? ```@jyn514```
|
|
fix running Miri tests
This partially reverts https://github.com/rust-lang/rust/pull/108659 to fix https://github.com/rust-lang/rust/issues/110102: the Miri test runner does not support any flags, they are interpreted as filters instead which leads to no tests being run.
I have not checked any of the other test runners for whether they are having any trouble with these flags.
Cc `@pietroalbini` `@Mark-Simulacrum` `@jyn514`
|
|
Fix `--extend-css` option
Fixes #110002.
The file was generated in the wrong folder so I moved it into `static.files` as it made more sense to have there instead of changing the path in the templates.
I also added a GUI test to ensure that this option won't break unexpectedly again.
|
|
Fix x check --stage 1 when download-ci-llvm=false
Bootstrap tries to avoid building LLVM unless it needs to; in particular we only build it for `x build`, not `x check`. Unfortunately, the check forgot about existence of stages - it would break if you used `x check --stage 1`:
```
= note: /usr/bin/ld: cannot find -lPolly: No such file or directory
/usr/bin/ld: cannot find -lPollyISL: No such file or directory
```
Fix it to work for stage 1.
I recommend reading this commit-by-commit; the first one makes a bunch of whitespace changes but otherwise doesn't change the logic.
|
|
|
|
Pass host linker to compiletest.
Tests marked `// force-host` were using the default linker, even if a custom linker was configured in config.toml.
This change adds a new flag, --host-linker, to compiletest, and renames --linker to --target-linker.
|
|
|
|
|
|
|
|
r=Mark-Simulacrum
Download beta compiler toolchain in bootstrap if it doesn't yet exist
Blocker for #107812 and #99989
See: https://github.com/rust-lang/rust/pull/107812#discussion_r1143492202
r? `@jyn514`
|
|
This is needed for when the shell scripts bypass python altogether and run the downloaded
bootstrap directly. Changes are mainly provided from @jyn514, I just fixed the review notes.
Signed-off-by: ozkanonur <work@onurozkan.dev>
|
|
Tests marked `// force-host` were using the default linker, even if a
custom linker was configured in config.toml.
This change adds a new flag, --host-linker, to compiletest, and renames
--linker to --target-linker.
|
|
|
|
Renamed 'native.rs' to 'llvm.rs', also moved `TestHelpers` to `test.rs`.Replaced all the `native.rs` ocurrences at `src/bootstrap` files to `llvm.rs`
|
|
Include executed tests in the build metrics (and use a custom test display impl)
The main goal of this PR is to include all tests executed in CI inside the build metrics JSON files. I need this for Ferrocene, and `@Mark-Simulacrum` expressed desire to have this as well to ensure all tests are executed at least once somewhere in CI.
Unfortunately implementing this required rewriting inside of bootstrap all of the code to render the test output to console. libtest supports outputting JSON instead of raw text, which we can indeed use to populate the build metrics. Doing that suppresses the console output though, and compared to rustc and Cargo the console output is not included as a JSON field.
Because of that, this PR had to reimplement both the "pretty" format (one test per line, with `rust.verbose-tests = true`), and the "terse" format (the wall of dots, with `rust.verbose-tests = false`). The current implementation should have the exact same output as libtest, except for the benchmark output. libtest's benchmark output is broken in the "terse" format, so since that's our default I slightly improved how it's rendered.
Also, to bring parity with libtest I had to introduce support for coloring output from bootstrap, using the same dependencies `annotate-snippets` uses. It's now possible to use `builder.color_for_stdout(Color::Red, "text")` and `builder.color_for_stderr(Color::Green, "text")` across all of bootstrap, automatically respecting the `--color` flag and whether the stream is a terminal or not.
I recommend reviewing the PR commit-by-commit.
r? `@Mark-Simulacrum`
|
|
This has a significant speedup for me locally, from about 1.3 seconds to
.9 seconds.
|
|
|
|
Behind the scenes Clippy uses compiletest-rs, which doesn't support the
--json flag we added to Rust's compiletest.
|
|
|
|
|
|
This commit adds `--no-fail-fast` flag to each `cargo test`
command in each tool Step trait implementation.
Fixes #108261
|
|
Pass arguments to `x` subcommands with `--`
Fixes #107375
Any arguments passed to `x` following `--` are removed and not parsed, instead passed down to subcommands (just the ones listed in the issue, not sure if more are needed). This does not remove the existing `--args` and `--test-args` behaviour, just extends it. It's also not documented in the help, not sure of the best way to format it.
r? `@jyn514`
|
|
Add only modified subcommand for compiletest
r? `@jyn514`
From [discussion](https://rust-lang.zulipchat.com/#narrow/stream/326414-t-infra.2Fbootstrap/topic/Add.20subcommand.20to.20retest.20all.20tests.20with.20different.20results)
|
|
|
