summary refs log tree commit diff
path: root/src/tools/compiletest
AgeCommit message (Collapse)AuthorLines
2024-06-07Rename `S` environment variable to `SOURCE_ROOT` in run-make testsJakub Beránek-1/+1
2024-06-07Do not pass source root when building run-make testsJakub Beránek-1/+0
2024-06-06Rollup merge of #126072 - Zalathar:run-flags, r=jieyouxuJubilee-7/+11
compiletest: Allow multiple `//@ run-flags:` headers While working on some tests, I was annoyed to find that multiple `// `@run-flags:`` headers do not combine with each other (as `//@ compile-flags:` headers do), and instead all but one are silently discarded. This makes it impossible to split long flag lists into multiple lines. Fortunately it's easy to just recycle the existing logic from the other command-line-flags headers.
2024-06-07compiletest: Allow multiple `//@ run-flags:` headersZalathar-7/+11
2024-06-06Make the panic info more usefulOli Scherer-0/+1
2024-06-05Rollup merge of #126008 - Zalathar:fulldeps-19371, r=jieyouxuMatthias Krüger-0/+10
Port `tests/run-make-fulldeps/issue-19371` to ui-fulldeps This test can run as an ordinary `tests/ui-fulldeps` test, with the help of some additional header variable substitutions to supply a sysroot and linker. --- Unlike #125973, this test appears to be testing something vaguely useful and breakable, which is why I didn't just delete it.
2024-06-05Rollup merge of #125683 - Oneirical:patience-testing-test, r=jieyouxuJubilee-0/+1
Rewrite `suspicious-library`, `resolve-rename` and `incr-prev-body-beyond-eof` `run-make` tests in `rmake.rs` format Part of #121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html). Some oddly specific ignore flags in `incr-prev-body-beyond-eof`: ```rs // ignore-none // ignore-nvptx64-nvidia-cuda ``` it could be interesting to run a try job, but it seems there is no nvidia-cuda in the CI settings (`jobs.yml`). try-job: armhf-gnu
2024-06-05Port `tests/run-make-fulldeps/issue-19371` to ui-fulldepsZalathar-0/+10
2024-06-04rewrite suspicious-library in rmakeOneirical-0/+1
2024-06-02Rollup merge of #125896 - jieyouxu:compiletest-rmake-comment, r=compiler-errorsJubilee-5/+5
compiletest: fix outdated rmake.rs comment Noticed in https://github.com/rust-lang/rust/pull/125827#discussion_r1623420820. I fixed the PR description but forgot to update the comment.
2024-06-02Rollup merge of #125890 - Nilstrieb:gay-compiletest, r=jieyouxuJubilee-6/+34
Improve compiletest expected/not found formatting compiletest, oh compiletest, you are truly one of the tools in this repository. You're the omnipresent gatekeeper, ensuring that every new change works, doesn't break the world, and is nice. We thank you for your work, for your tests, for your test runs, for your features that help writing tests, for all the stability and and good you have caused. Without you, Rust wouldn't exist as it does, without you, nothing would work, without you, we would all go insane from having changes break and having to test them all by hand. Thank you, compiletest. but holy shit i fucking hate your stupid debug output so much i simply cannot take this anymore aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa By changing a few magic lines in this file called "runtest.rs", we can cause compiletest to emit nicer messages. This is widely regarded as a good thing. We stop wasting vertical space, allowing more errors to be displayed at once. Additionally, we add colors, which make it so much more pretty *and* gay, both of which are very good and useful. There's a bit of fuckery needed to get the colors to work. `colored` checks whether stdout is a terminal. We also print to stdout, so that works well. But.... for some stupid reason that I absolutely refuse to even attempt to debug, stdout is *not* a terminal when executing tests *in a terminal*. But stderr is >:). So this just checks whether stderr is a terminal. If you have a use case where you dump compiletest stdout into a place where colors are not supported while having stderr be a terminal, then I'm sorry for you, but you are gonna get colors and you're gonna like it. Stop it with the usual environment variable, which `colored` also respects by default. ### before (bad, hurts your brain, makes you want to cry) ![image](https://github.com/rust-lang/rust/assets/48135649/cbeecb5d-fc25-460b-b192-9808f8fa2079) ## after (good, gay, makes you want to cry) ![image](https://github.com/rust-lang/rust/assets/48135649/a655b220-8841-443e-a825-72a835d56882) r? jieyouxu said he wants to review the PR
2024-06-02compiletest: fix outdated rmake.rs comment许杰友 Jieyou Xu (Joe)-5/+5
2024-06-02Improve compiletest expected/not found formattingNilstrieb-6/+34
compiletest, oh compiletest, you are truly one of the tools in this repository. You're the omnipresent gatekeeper, ensuring that every new change works, doesn't break the world, and is nice. We thank you for your work, for your tests, for your test runs, for your features that help writing tests, for all the stability and and good you have caused. Without you, Rust wouldn't exist as it does, without you, nothing would work, without you, we would all go insane from having changes break and having to test them all by hand. Thank you, compiletest. but holy shit i fucking hate your stupid debug output so much i simply cannot take this anymore aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa By changing a few magic lines in this file called "runtest.rs", we can cause compiletest to emit nicer messages. This is widely regarded as a good thing. We stop wasting vertical space, allowing more errors to be displayed at once. Additionally, we add colors, which make it so much more pretty *and* gay, both of which are very good and useful. There's a bit of fuckery needed to get the colors to work. `colored` checks whether stdout is a terminal. We also print to stdout, so that works well. But.... for some stupid reason that I absolutely refuse to even attempt to debug, stdout is *not* a terminal when executing tests *in a terminal*. But stderr is >:). So this just checks whether stderr is a terminal. If you have a use case where you dump compiletest stdout into a place where colors are not supported while having stderr be a terminal, then I'm sorry for you, but you are gonna get colors and you're gonna like it. Stop it with the usual environment variable, which `colored` also respects by default.
2024-06-02compiletest: split rmake executable from scratch dir许杰友 Jieyou Xu (Joe)-9/+24
When implementing support for rmake.rs, I copied over the `$TMPDIR` directory logic from the legacy Makefile setup. In doing so, I also compiled recipe `rmake.rs` into executables which unfortunately are placed into `$TMPDIR` as well. This causes a problem on Windows where: - The `rmake.exe` executable is placed in `$TMPDIR`. - We run the `rmake.exe` as a process. - The process uses `rmake.exe` inside `$TMPDIR`. - Windows prevents the .exe file from being deleted when the process is still alive. - The recipe test code tries to `remove_dir_all($TMPDIR)`, which fails with access denied because `rmake.exe` is still being used. We fix this by separating the recipe executable and the sratch directory: ``` base_dir/ rmake.exe scratch/ ``` We construct a base directory, unique to each run-make test, under which we place rmake.exe alongside a `scratch/` directory. This `scratch/` directory is what is passed to rmake.rs tests as `$TMPDIR`, so now `remove_dir_all($TMPDIR)` has a chance to succeed because it no longer contains `rmake.exe`. Oops. This was a fun one to try figure out.
2024-05-31Remove unused rust-demangler support from compiletestZalathar-10/+0
2024-05-31Use `coverage-dump --demangle` as the demangler for coverage-run testsZalathar-5/+12
This avoids the need to build `rust-demangler` when running coverage tests, since we typically need to build `coverage-dump` anyway.
2024-05-30Auto merge of #125710 - RalfJung:compiletest-components, r=workingjubileebors-3/+6
compiletest: clarify COMPILETEST_NEEDS_ALL_LLVM_COMPONENTS error COMPILETEST_NEEDS_ALL_LLVM_COMPONENTS is a confusing name because elsewhere "needs" means "ignore when requirement not met", but here it means "fail when requirement not met".
2024-05-30compiletest: clarify COMPILETEST_NEEDS_ALL_LLVM_COMPONENTS errorRalf Jung-3/+6
also improve wording for an ignore reason
2024-05-30compiletest: Unify `cmd2procres` with `run_command_to_procres`Zalathar-29/+19
2024-05-29Extract coverage-specific code out of `compiletest::runtest`Zalathar-426/+372
This moves a few hundred lines of coverage-specific code out of the main module, making navigation a bit easier.
2024-05-28Make more of the test suite run on Mac CatalystMads Marquart-3/+10
This adds the `only-apple`/`ignore-apple` compiletest directive, and uses that basically everywhere instead of `only-macos`/`ignore-macos`. Some of the updates in `run-make` are a bit redundant, as they use `ignore-cross-compile` and won't run on iOS - but using Apple in these is still more correct, so I've made that change anyhow.
2024-05-23Rollup merge of #125156 - zachs18:for_loops_over_fallibles_behind_refs, ↵Matthias Krüger-2/+2
r=Nilstrieb Expand `for_loops_over_fallibles` lint to lint on fallibles behind references. Extends the scope of the (warn-by-default) lint `for_loops_over_fallibles` from just `for _ in x` where `x: Option<_>/Result<_, _>` to also cover `x: &(mut) Option<_>/Result<_>` ```rs fn main() { // Current lints for _ in Some(42) {} for _ in Ok::<_, i32>(42) {} // New lints for _ in &Some(42) {} for _ in &mut Some(42) {} for _ in &Ok::<_, i32>(42) {} for _ in &mut Ok::<_, i32>(42) {} // Should not lint for _ in Some(42).into_iter() {} for _ in Some(42).iter() {} for _ in Some(42).iter_mut() {} for _ in Ok::<_, i32>(42).into_iter() {} for _ in Ok::<_, i32>(42).iter() {} for _ in Ok::<_, i32>(42).iter_mut() {} } ``` <details><summary><code>cargo build</code> diff</summary> ```diff diff --git a/old.out b/new.out index 84215aa..ca195a7 100644 --- a/old.out +++ b/new.out `@@` -1,33 +1,93 `@@` warning: for loop over an `Option`. This is more readably written as an `if let` statement --> src/main.rs:3:14 | 3 | for _ in Some(42) {} | ^^^^^^^^ | = note: `#[warn(for_loops_over_fallibles)]` on by default help: to check pattern in a loop use `while let` | 3 | while let Some(_) = Some(42) {} | ~~~~~~~~~~~~~~~ ~~~ help: consider using `if let` to clear intent | 3 | if let Some(_) = Some(42) {} | ~~~~~~~~~~~~ ~~~ warning: for loop over a `Result`. This is more readably written as an `if let` statement --> src/main.rs:4:14 | 4 | for _ in Ok::<_, i32>(42) {} | ^^^^^^^^^^^^^^^^ | help: to check pattern in a loop use `while let` | 4 | while let Ok(_) = Ok::<_, i32>(42) {} | ~~~~~~~~~~~~~ ~~~ help: consider using `if let` to clear intent | 4 | if let Ok(_) = Ok::<_, i32>(42) {} | ~~~~~~~~~~ ~~~ -warning: `for-loops-over-fallibles` (bin "for-loops-over-fallibles") generated 2 warnings - Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.04s +warning: for loop over a `&Option`. This is more readably written as an `if let` statement + --> src/main.rs:7:14 + | +7 | for _ in &Some(42) {} + | ^^^^^^^^^ + | +help: to check pattern in a loop use `while let` + | +7 | while let Some(_) = &Some(42) {} + | ~~~~~~~~~~~~~~~ ~~~ +help: consider using `if let` to clear intent + | +7 | if let Some(_) = &Some(42) {} + | ~~~~~~~~~~~~ ~~~ + +warning: for loop over a `&mut Option`. This is more readably written as an `if let` statement + --> src/main.rs:8:14 + | +8 | for _ in &mut Some(42) {} + | ^^^^^^^^^^^^^ + | +help: to check pattern in a loop use `while let` + | +8 | while let Some(_) = &mut Some(42) {} + | ~~~~~~~~~~~~~~~ ~~~ +help: consider using `if let` to clear intent + | +8 | if let Some(_) = &mut Some(42) {} + | ~~~~~~~~~~~~ ~~~ + +warning: for loop over a `&Result`. This is more readably written as an `if let` statement + --> src/main.rs:9:14 + | +9 | for _ in &Ok::<_, i32>(42) {} + | ^^^^^^^^^^^^^^^^^ + | +help: to check pattern in a loop use `while let` + | +9 | while let Ok(_) = &Ok::<_, i32>(42) {} + | ~~~~~~~~~~~~~ ~~~ +help: consider using `if let` to clear intent + | +9 | if let Ok(_) = &Ok::<_, i32>(42) {} + | ~~~~~~~~~~ ~~~ + +warning: for loop over a `&mut Result`. This is more readably written as an `if let` statement + --> src/main.rs:10:14 + | +10 | for _ in &mut Ok::<_, i32>(42) {} + | ^^^^^^^^^^^^^^^^^^^^^ + | +help: to check pattern in a loop use `while let` + | +10 | while let Ok(_) = &mut Ok::<_, i32>(42) {} + | ~~~~~~~~~~~~~ ~~~ +help: consider using `if let` to clear intent + | +10 | if let Ok(_) = &mut Ok::<_, i32>(42) {} + | ~~~~~~~~~~ ~~~ + +warning: `for-loops-over-fallibles` (bin "for-loops-over-fallibles") generated 6 warnings + Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.02s ``` </details> ----- Question: * ~~Currently, the article `an` is used for `&Option`, and `&mut Option` in the lint diagnostic, since that's what `Option` uses. Is this okay or should it be changed? (likewise, `a` is used for `&Result` and `&mut Result`)~~ The article `a` is used for `&Option`, `&mut Option`, `&Result`, `&mut Result` and (as before) `Result`. Only `Option` uses `an` (as before). `@rustbot` label +A-lint
2024-05-18Fix typos (taking into account review comments)blyxyas-3/+3
2024-05-16Fix `for_loops_over_fallibles` hits in compiletest/src/json.rszachs18-2/+2
2024-05-11Make crashes dump mir to build dirMichael Goulet-8/+14
2024-05-09Tidy check for test revisions that are mentioned but not declaredZalathar-0/+2
If a `[revision]` name appears in a test header directive or error annotation, but isn't declared in the `//@ revisions:` header, that is almost always a mistake. In cases where a revision needs to be temporarily disabled, adding it to an `//@ unused-revision-names:` header will suppress these checks for that name. Adding the wildcard name `*` to the unused list will suppress these checks for the entire file.
2024-05-04compiletest: add enable-by-default check-cfgUrgau-3/+22
2024-05-04compiletest: add no-auto-check-cfg directiveUrgau-0/+7
this directive prevents compiletest from adding any implicit and automatic --check-cfg arguments
2024-05-02Rollup merge of #124138 - mati865:ignore-llvm-abi-in-dlltool-tests, r=davidtwcoMatthias Krüger-46/+24
Ignore LLVM ABI in dlltool tests since those targets don't use dlltool Otherwise those two tests fail when running `./x.py test` with this target.
2024-05-01Auto merge of #124491 - madsmtm:target_vendor-apple, r=workingjubileebors-2/+2
Use `target_vendor = "apple"` instead of `target_os = "..."` Use `target_vendor = "apple"` instead of `all(target_os = "macos", target_os = "ios", target_os = "tvos", target_os = "watchos", target_os = "visionos")`. The apple targets are quite close to being identical, with iOS, tvOS, watchOS and visionOS being even closer, so using `target_vendor` when possible makes it clearer when something is actually OS-specific, or just Apple-specific. Note that `target_vendor` will [be deprecated in the future](https://github.com/rust-lang/rust/issues/100343), but not before an alternative (like `target_family = "apple"`) is available. While doing this, I found various inconsistencies and small mistakes in the standard library, see the commits for details. Will follow-up with an extra PR for a similar issue that need a bit more discussion. EDIT: https://github.com/rust-lang/rust/pull/124494 Since you've talked about using `target_vendor = "apple"` in the past: r? workingjubilee CC `@simlay,` `@thomcc` `@rustbot` label O-macos O-ios O-tvos O-watchos O-visionos
2024-04-30Rollup merge of #124280 - beetrees:repr128-test-rmake, r=jieyouxu许杰友 Jieyou Xu (Joe)-0/+1
Port repr128-dwarf run-make test to rmake This PR ports the repr128-dwarf run-make test to rmake, using the `gimli` crate instead of the `llvm-dwarfdump` command. Note that this PR changes `rmake.rs` files to be compiled with the 2021 edition (previously no edition was passed to `rustc`, meaning they were compiled with the 2015 edition). This means that `panic!("{variable}")` will now work as expected in `rmake.rs` files (there's already a usage in the [wasm-symbols-not-exported test](https://github.com/rust-lang/rust/blob/aca749eefceaed0cda19a7ec5e472fce9387bc00/tests/run-make/wasm-symbols-not-exported/rmake.rs#L34) that this will fix). Tracking issue: #121876
2024-04-30Port repr128-dwarf run-make test to rmakebeetrees-0/+1
2024-04-29aux-bin: Avoid old .so files from old tests; clean auxiliary dir rootMartin Nordholts-0/+1
2024-04-28Also raise fd limit on tvOS when testingMads Marquart-2/+2
2024-04-27Remove lazycell and once_cell from compiletest dependenciesGeorge Bateman-104/+80
2024-04-24Skip dlltool tests on non `*windows-gnu` targetsMateusz Mikuła-0/+5
2024-04-24Refactor dlltool searching code into separate functionMateusz Mikuła-46/+19
2024-04-20Rollup merge of #124196 - RalfJung:mir-opt-tests, r=Mark-Simulacrum许杰友 Jieyou Xu (Joe)-3/+3
mir-opt tests: rename unit-test -> test-mir-pass "unit-test" is extremely non-descriptive, no idea how one is supposed to read that and know that this specifies the MIR pass being tested.
2024-04-20mir-opt tests: rename unit-test -> test-mir-passRalf Jung-3/+3
2024-04-19Match hyphen in multi-revision comment matchersTrevor Gross-1/+16
Currently, the matcher `//[rev-foo,rev-bar]~` does not get selected by the regex. Change the matcher to also match strings that contain a `-`.h
2024-04-17Rollup merge of #123975 - lqd:rust-lld-tests, r=jieyouxuGuillaume Gomez-0/+1
Port the 2 `rust-lld` run-make tests to `rmake` In preparation for finalizing most of the `rust-lld` work, this PR ports the following tests to `rmake`: - `tests/run-make/rust-lld` - `tests/run-make/rust-lld-custom-target` As they use `$(CGREP) -e` I added `regex` as an exported dependency to the `run_make_support` library. Unfortunately, the most recent versions depend on `memchr` 2.6.0 but it's currently pinned at 2.5.0 in the workspace, and therefore had to settle for the older `regex-1.8.0`. r? `@jieyouxu`
2024-04-16Make ui_test backtraces short by defaultOli Scherer-1/+4
2024-04-16Unset test env vars before setting new ones.Oli Scherer-2/+2
If you want to override an env var, don't unset it, just set it
2024-04-15add missing lld directive to compiletestRémy Rakic-0/+1
2024-04-15Auto merge of #122997 - matthiaskrgr:compiletest_ices, r=oli-obkbors-7/+54
compiletest ice tracking see https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/where.20to.20mass-add.20known.20ices.20.2F.20merging.20glacier.20into.20rust/near/429082963 This will allow us to sunset most of https://github.com/rust-lang/glacier The rustc ices will be tracked directly inside the rust testsuite There are a couple of .sh tests remaining that I have not ported over yet. This adds `tests/crashes`, a file inside this directory MUST ice, otherwise it is considered test-fail. This will be used to track ICEs from glacier and the bugtracker. When someones pr accidentally fixes one of these ICEs, they can move the test from `crashes` into `ui` for example. I also added a new tidy lint that warns when a test inside `tests/crashes` does not have a `//@ known-bug: ` line the env var `COMPILETEST_VERBOSE_CRASHES` can be set to get exit code, stderr and stdout of a crash-test to aid debugging/adding tests.
2024-04-14update README and add COMPILETEST_VERBOSE_CRASHES env var which when set ↵Matthias Krüger-7/+12
print stdout, stderr and exit code of "crashes" tests, useful for debugging or adding new tests
2024-04-14crashes: fix ice detection which did not trigger if code compiled without ↵Matthias Krüger-8/+8
error by accident
2024-04-14compiletest: switch crash detection logic for run_crash_test aroundMatthias Krüger-5/+6
previously we would explicitly look for exit code 101 and call it a crash, however in case of stack overflows for example, exit code could differ due to the process being killed by a signal which is not easy to detect on none-unix. So now we reject everything that exits with 0 (no error) or 1 (compiler failed to compile code) and "accept" everyhing else as an internal compiler error.
2024-04-14run_crash_test(): better error message when we get a none-iceMatthias Krüger-1/+5
2024-04-14compiletest: crashes: turn off backtraces for faster testsMatthias Krüger-2/+13