about summary refs log tree commit diff
path: root/src/bootstrap/test.rs
AgeCommit message (Collapse)AuthorLines
2023-04-29Remove unnecessary `test_kind` field and `TestKind` structjyn-65/+13
2023-04-25Add a `sysroot` crate to represent the standard library cratesJohn Kåre Alsaker-1/+1
2023-04-22Group entire build steps in the gha logsOli Scherer-24/+25
2023-04-18Auto merge of #110478 - jyn514:stage1-fulldeps, r=albertlarsan68bors-6/+19
Support `x test --stage 1 ui-fulldeps` `@Nilstrieb` had an excellent idea the other day: the same way that rustdoc is able to load `rustc_driver` from the sysroot, ui-fulldeps tests should also be able to load it from the sysroot. That allows us to run fulldeps tests with stage1, without having to fully rebuild the compiler twice. It does unfortunately have the downside that we're building the tests with the *bootstrap* compiler, not the in-tree sources, but since most of the fulldeps tests are for the *API* of the compiler, that seems ok. I think it's possible to extend this to `run-make-fulldeps`, but I've run out of energy for tonight. - Move `plugin` tests into a subdirectory. Plugins are loaded at runtime with `dlopen` and so require the ABI of the running compile to match the ABI of the compiler linked with `rustc_driver`. As a result they can't be supported in stage 1 and have to use `// ignore-stage1`. - Remove `ignore-stage1` from most non-plugin tests - Ignore diagnostic tests in stage 1. Even though this requires a stage 2 build to load rustc_driver, it's primarily testing the error message that the *running* compiler emits when the diagnostic struct is malformed. - Pass `-Zforce-unstable-if-unmarked` in stage1, not just stage2. That allows running `hash-stable-is-unstable` in stage1, since it now suggests adding `rustc_private` to enable loading the crates. - Add libLLVM.so to the stage0 target sysroot, to allow fulldeps tests that act as custom drivers to load it at runtime. - Pass `--sysroot stage0-sysroot` in compiletest so that we use the correct version of std. - Move a few lint tests from ui-fulldeps to ui These had an `aux-build:lint-group-plugin-test.rs` that they never actually loaded with `feature(plugin)` nor tested. I removed the unused aux-build and they pass fine with stage 1. Fixes https://github.com/rust-lang/rust/issues/75905.
2023-04-17Support `x test --stage 1 ui-fulldeps`jyn-6/+19
Nils had an excellent idea the other day: the same way that rustdoc is able to load `rustc_driver` from the sysroot, ui-fulldeps tests should also be able to load it from the sysroot. That allows us to run fulldeps tests with stage1, without having to fully rebuild the compiler twice. It does unfortunately have the downside that we're running the tests on the *bootstrap* compiler, not the in-tree sources, but since most of the fulldeps tests are for the *API* of the compiler, that seems ok. I think it's possible to extend this to `run-make-fulldeps`, but I've run out of energy for tonight. - Move `plugin` tests into a subdirectory. Plugins are loaded at runtime with `dlopen` and so require the ABI of the running compile to match the ABI of the compiler linked with `rustc_driver`. As a result they can't be supported in stage 1 and have to use `// ignore-stage1`. - Remove `ignore-stage1` from most non-plugin tests - Ignore diagnostic tests in stage 1. Even though this requires a stage 2 build to load rustc_driver, it's primarily testing the error message that the *running* compiler emits when the diagnostic struct is malformed. - Pass `-Zforce-unstable-if-unmarked` in stage1, not just stage2. That allows running `hash-stable-is-unstable` in stage1, since it now suggests adding `rustc_private` to enable loading the crates. - Add libLLVM.so to the stage0 target sysroot, to allow fulldeps tests that act as custom drivers to load it at runtime. - Pass `--sysroot stage0-sysroot` in compiletest so that we use the correct version of std.
2023-04-16Fix error due to renaming of ignore_gitErik Hofmayer-1/+1
2023-04-16Add needs-git-hash header to compiletestErik Hofmayer-0/+4
This header can be used for tests which check the output of `--version --verbose` commands.
2023-04-14Auto merge of #110324 - JohnTitor:rollup-jq31pd1, r=JohnTitorbors-0/+36
Rollup of 7 pull requests Successful merges: - #103682 (Stabilize rustdoc `--test-run-directory`) - #106249 (Create "suggested tests" tool in `rustbuild`) - #110047 (Add link to `collections` docs to `extend` trait) - #110269 (Add `tidy-alphabetical` to features in `core`) - #110292 (Add `tidy-alphabetical` to features in `alloc` & `std`) - #110305 (rustdoc-search: use ES6 `Map` and `Set` where they make sense) - #110315 (Add a stable MIR way to get the main function) Failed merges: r? `@ghost` `@rustbot` modify labels: rollup
2023-04-14Rollup merge of #106249 - Ezrashaw:suggest-test-tool, r=jyn514,albertlarsan68Yuki Okushi-0/+36
Create "suggested tests" tool in `rustbuild` Not the claimed person in #97339 but: I've done a very rough implementation of this feature in-tree. I'm very new to `rustc` development (outside of docs) so some help would be greatly appreciated. The UI of this new subcommand obviously will change and I need some mentoring with the `--run` flag. r? ```@jyn514```
2023-04-14Auto merge of #110177 - RalfJung:miri-tests, r=oli-obkbors-1/+4
fix running Miri tests This partially reverts https://github.com/rust-lang/rust/pull/108659 to fix https://github.com/rust-lang/rust/issues/110102: the Miri test runner does not support any flags, they are interpreted as filters instead which leads to no tests being run. I have not checked any of the other test runners for whether they are having any trouble with these flags. Cc `@pietroalbini` `@Mark-Simulacrum` `@jyn514`
2023-04-12Rollup merge of #110235 - GuillaumeGomez:fix-extend-css, r=notriddleMatthias Krüger-0/+2
Fix `--extend-css` option Fixes #110002. The file was generated in the wrong folder so I moved it into `static.files` as it made more sense to have there instead of changing the path in the templates. I also added a GUI test to ensure that this option won't break unexpectedly again.
2023-04-12Rollup merge of #110122 - jyn514:check-stage1-llvm, r=ozkanonurMatthias Krüger-1/+1
Fix x check --stage 1 when download-ci-llvm=false Bootstrap tries to avoid building LLVM unless it needs to; in particular we only build it for `x build`, not `x check`. Unfortunately, the check forgot about existence of stages - it would break if you used `x check --stage 1`: ``` = note: /usr/bin/ld: cannot find -lPolly: No such file or directory /usr/bin/ld: cannot find -lPollyISL: No such file or directory ``` Fix it to work for stage 1. I recommend reading this commit-by-commit; the first one makes a bunch of whitespace changes but otherwise doesn't change the logic.
2023-04-12Add/update tests for `--extend-css` optionGuillaume Gomez-0/+2
2023-04-11Rollup merge of #110018 - jfgoog:host-and-target-linker, r=wesleywiserMichael Goulet-1/+4
Pass host linker to compiletest. Tests marked `// force-host` were using the default linker, even if a custom linker was configured in config.toml. This change adds a new flag, --host-linker, to compiletest, and renames --linker to --target-linker.
2023-04-11fix running Miri testsRalf Jung-1/+4
2023-04-10Fix `x check --stage 1` when `download-ci-llvm=false`Joshua Nelson-1/+1
2023-04-09feat: implement basic suggest-tests toolEzra Shaw-0/+36
2023-04-07Auto merge of #109448 - ozkanonur:download-beta-compiler-toolchain, ↵bors-1/+1
r=Mark-Simulacrum Download beta compiler toolchain in bootstrap if it doesn't yet exist Blocker for #107812 and #99989 See: https://github.com/rust-lang/rust/pull/107812#discussion_r1143492202 r? `@jyn514`
2023-04-08download beta compiler toolchain in bootstrap if it doesn't yet existozkanonur-1/+1
This is needed for when the shell scripts bypass python altogether and run the downloaded bootstrap directly. Changes are mainly provided from @jyn514, I just fixed the review notes. Signed-off-by: ozkanonur <work@onurozkan.dev>
2023-04-06Pass host linker to compiletest.James Farrell-1/+4
Tests marked `// force-host` were using the default linker, even if a custom linker was configured in config.toml. This change adds a new flag, --host-linker, to compiletest, and renames --linker to --target-linker.
2023-04-03fix remaining testsPietro Albini-1/+1
2023-03-24Rename 'src/bootstrap/native.rs' to llvm.rsRobin Hafid-7/+72
Renamed 'native.rs' to 'llvm.rs', also moved `TestHelpers` to `test.rs`.Replaced all the `native.rs` ocurrences at `src/bootstrap` files to `llvm.rs`
2023-03-21Auto merge of #108659 - ferrocene:pa-test-metrics, r=Mark-Simulacrumbors-27/+20
Include executed tests in the build metrics (and use a custom test display impl) The main goal of this PR is to include all tests executed in CI inside the build metrics JSON files. I need this for Ferrocene, and `@Mark-Simulacrum` expressed desire to have this as well to ensure all tests are executed at least once somewhere in CI. Unfortunately implementing this required rewriting inside of bootstrap all of the code to render the test output to console. libtest supports outputting JSON instead of raw text, which we can indeed use to populate the build metrics. Doing that suppresses the console output though, and compared to rustc and Cargo the console output is not included as a JSON field. Because of that, this PR had to reimplement both the "pretty" format (one test per line, with `rust.verbose-tests = true`), and the "terse" format (the wall of dots, with `rust.verbose-tests = false`). The current implementation should have the exact same output as libtest, except for the benchmark output. libtest's benchmark output is broken in the "terse" format, so since that's our default I slightly improved how it's rendered. Also, to bring parity with libtest I had to introduce support for coloring output from bootstrap, using the same dependencies `annotate-snippets` uses. It's now possible to use `builder.color_for_stdout(Color::Red, "text")` and `builder.color_for_stderr(Color::Green, "text")` across all of bootstrap, automatically respecting the `--color` flag and whether the stream is a terminal or not. I recommend reviewing the PR commit-by-commit. r? `@Mark-Simulacrum`
2023-03-18Let tidy use more threadsJoshua Nelson-1/+5
This has a significant speedup for me locally, from about 1.3 seconds to .9 seconds.
2023-03-07Add bootstrap support for rust-installerMark Rousskov-0/+55
2023-03-02do not use render_tests for clippyPietro Albini-1/+1
Behind the scenes Clippy uses compiletest-rs, which doesn't support the --json flag we added to Rust's compiletest.
2023-03-02switch all tests to use render_testsPietro Albini-23/+18
2023-03-02render compiletest output with render_testsPietro Albini-5/+3
2023-02-22Add no-fail-fast support to miri, rustfmt and clippy testsuitesJesus Checa Hidalgo-0/+12
This commit adds `--no-fail-fast` flag to each `cargo test` command in each tool Step trait implementation. Fixes #108261
2023-02-17Rollup merge of #107905 - clubby789:x-free-args, r=albertlarsan68Matthias Krüger-0/+1
Pass arguments to `x` subcommands with `--` Fixes #107375 Any arguments passed to `x` following `--` are removed and not parsed, instead passed down to subcommands (just the ones listed in the issue, not sure if more are needed). This does not remove the existing `--args` and `--test-args` behaviour, just extends it. It's also not documented in the help, not sure of the best way to format it. r? `@jyn514`
2023-02-11Rollup merge of #107657 - chenyukang:yukang/add-only-modified, r=albertlarsan68Matthias Krüger-0/+4
Add only modified subcommand for compiletest r? `@jyn514` From [discussion](https://rust-lang.zulipchat.com/#narrow/stream/326414-t-infra.2Fbootstrap/topic/Add.20subcommand.20to.20retest.20all.20tests.20with.20different.20results)
2023-02-10Pass arguments to `x` subcommands with `--`clubby789-0/+1
2023-02-10add only modified for compiletestyukang-0/+4
2023-02-05Run `expand-yaml-anchors` in `x test tidy`Joshua Nelson-3/+5
Previously, the pre-commit hook which runs `x test tidy` could pass only to have CI fail within the first 30 seconds. This adds about 30 seconds to `test tidy` (for an initial run, much less after the tool is built the first time) in exchange for catching errors in `.github/workflows/ci.yml` before they're pushed.
2023-01-14Auto merge of #106520 - ehuss:update-mdbook, r=Mark-Simulacrumbors-1/+3
Update mdbook This updates mdbook from 0.4.21 to 0.4.25. The list of changes is [here](https://github.com/rust-lang/mdBook/blob/master/CHANGELOG.md#mdbook-0425). The only user-visible changes are some changes around the theme picker, and change to the copy-to-clipboard ignoring hidden lines. Internally there were some dependency updates and small fixes. This also updates `clap` from 4.0.15 to 4.0.32 whose changelog is [here](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md#4032---2022-12-22). This impacts tools like cargo. I don't see anything particularly noteworthy there, though there are some small user-visible changes. Unfortunately this required adding a hack for building `rustix` with a bootstrap tool. The comment explains why. I am unable to think of some other workaround (or even a cleaner way to set the rustflag). Ideas are welcome if you can think of alternatives. I'm struggling to even think of a long-term solution, other than asking projects not to do auto-nightly feature detection. One medium-term solution is to avoid the clap dependency for the mdbook library (which is how rustix gets pulled in). That is one of my goals for the 0.5 release of mdbook, but that probably won't happen until later this year. It would also require dropping clap from `rustbook` and using some other means to parse arguments (there's only two options, so it can probably be done manually).
2023-01-11Change `src/test` to `tests` in source files, fix tidy and testsAlbert Larsan-38/+30
2023-01-09Update mdbookEric Huss-1/+3
2022-12-31Run `cargo test` on tidy itself.Eric Huss-0/+34
2022-12-30Use more consistent progress messages in bootstrapJoshua Nelson-2/+7
Before: ``` Testing ["rustc_interface"] stage0 (aarch64-unknown-linux-gnu -> aarch64-unknown-linux-gnu) ``` After: ``` Testing {rustc_interface} stage0 (aarch64-unknown-linux-gnu -> aarch64-unknown-linux-gnu) ``` Note there is a slight consistency between `build` and `test`: The former doesn't print "compiler artifacts". It would be annoying to fix and doesn't hurt anything, so I left it be. ``` ; x t rustc_interface --stage 0 --dry-run Testing {rustc_interface} stage0 (aarch64-unknown-linux-gnu -> aarch64-unknown-linux-gnu) ; x b rustc_interface --stage 0 --dry-run Building {rustc_interface} stage0 compiler artifacts (aarch64-unknown-linux-gnu -> aarch64-unknown-linux-gnu) ```
2022-12-28-Zrustdoc-scrape-examples no longer takes argumentsMark Rousskov-1/+1
2022-12-22Use LLVM_CMAKE_DIR for lld buildNikita Popov-1/+2
LLVM_CONFIG_PATH is no longer supported as of LLVM 16, switch to using the cmake module instead. We separately return the llvm-config and cmake directory paths, because llvm-config always refers to the host binary, while the cmake directory is for the target triple.
2022-11-26Support unit tests for jsondoclintNixon Enraght-Moony-0/+36
2022-11-24Fix typo in miri sysrootJoshua Nelson-1/+1
2022-11-20Rollup merge of #104611 - notriddle:notriddle/scrape-examples-button, ↵Matthias Krüger-0/+2
r=GuillaumeGomez rustdoc: use real buttons for scrape examples controls This makes the expand and switch controls keyboard-accessible. Preview: https://notriddle.com/notriddle-rustdoc-demos/scrape-examples-button/test_dingus/fn.test.html
2022-11-19rustdoc: add test case for scraped example expand GUIMichael Howell-0/+2
2022-11-16normalize download-rustc's prefix when running compiletestsPietro Albini-0/+4
2022-11-16normalize source paths from sysroot in compiletestPietro Albini-0/+1
2022-11-12Distinguish `--dry-run` from the automatic dry run checkJoshua Nelson-5/+5
2022-11-09Rollup merge of #104046 - RalfJung:run-miri-run, r=oli-obkMichael Goulet-43/+55
bootstrap: add support for running Miri on a file This enables: ``` ./x.py run src/tools/miri --stage 0 --args src/tools/miri/tests/pass/hello.rs ``` That can be super helpful for debugging. Also avoid sharing the Miri sysroot dir with a system-wide (rustup-managed) installation of Miri. Fixes https://github.com/rust-lang/rust/issues/76666
2022-11-06bootstrap: put Miri sysroot into local build dirRalf Jung-1/+9