about summary refs log tree commit diff
path: root/src/tools/compiletest
AgeCommit message (Collapse)AuthorLines
2024-06-23test: ignore force-frame-pointers test on some targetsJubilee Young-0/+2
all of these currently force stronger frame pointers, and currently the CLI does not override the target
2024-06-15Auto merge of #122613 - Zalathar:profiler, r=nnethercotebors-3/+0
Don't build a broken/untested profiler runtime on mingw targets Context: https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/Why.20build.20a.20broken.2Funtested.20profiler.20runtime.20on.20mingw.3F #75872 added `--enable-profiler` to the `x86_64-mingw` job (to cause some additional tests to run), but had to also add `//@ ignore-windows-gnu` to all of the tests that rely on the profiler runtime actually *working*, because it's broken on that target. We can achieve a similar outcome by going through all the `//@ needs-profiler-support` tests that don't actually need to produce/run a binary, and making them use `-Zno-profiler-runtime` instead, so that they can run even in configurations that don't have the profiler runtime available. Then we can remove `--enable-profiler` from `x86_64-mingw`, and still get the same amount of testing. This PR also removes `--enable-profiler` from the mingw dist builds, since it is broken/untested on that target. Those builds have had that flag for a very long time.
2024-06-14Remove `//@ ignore-windows-gnu` from tests that need the profilerZalathar-3/+0
The profiler runtime is no longer built in mingw test jobs, so these tests should naturally be skipped by `//@ needs-profiler-support`.
2024-06-13compiletest: apply considerable clippy suggestionsonur-ozkan-56/+51
Signed-off-by: onur-ozkan <work@onurozkan.dev>
2024-06-13Rollup merge of #126370 - Zalathar:normalize, r=oli-obkLeón Orell Valerian Liehr-61/+74
compiletest: Stricter parsing of `//@ normalize-*` headers I noticed some problems with the existing parser for these headers: - It is extremely lax, and basically ignores everything other than the text between two pairs of double-quote characters. - Unlike other name-value headers, it doesn't even check for a colon after the header name, so the test suite contains a mixture of with-colon and without-colon normalization rules. - If parsing fails, the header is silently ignored. The latter is especially bad for platform-specific normalization rules, because the lack of normalization probably won't be noticed until the test mysteriously fails in one of the full CI jobs.
2024-06-13Auto merge of #126197 - jieyouxu:rmake-must-use, r=Kobzolbors-0/+4
run-make: annotate library with `#[must_use]` and enforce `unused_must_use` in rmake.rs This PR adds `#[must_use]` annotations to functions of the `run_make_support` library where it makes sense, and adjusts compiletest to compile rmake.rs with `-Dunused_must_use`. The rationale is that it's highly likely that unused `#[must_use]` values in rmake.rs test files are bugs. For example, unused fs/io results are often load-bearing to the correctness of the test and often unchecked fs/io results allow the test to silently pass where it would've failed if the result was checked. This PR is best reviewed commit-by-commit. try-job: test-various try-job: x86_64-msvc
2024-06-13compiletest: Stricter parsing of `//@ normalize-*` headersZalathar-51/+63
2024-06-13compiletest: Move `static_regex!` into `compiletest::util`Zalathar-10/+11
2024-06-12Rollup merge of #126256 - ferrocene:lw-target-subst, r=albertlarsan68Jubilee-0/+5
Add {{target}} substitution to compiletest In ferrocene we have ui tests testing the cli interface of the compiler, one of which tests the `--target` flag. To be able to run this on all targets we require a way to specify a valid target in the `compile-flags` directive that is target independent, as otherwise we can only run the test against the one target we choose to supply in the flags. See https://github.com/ferrocene/ferrocene/blob/383cbc80f4e85859a4055f121f15dac329908346/tests/ui/ferrocene/compiler-arguments/target/target.rs We figured the project might be able to make use of this substitution as well in the future. try-job: dist-x86_64-msvc
2024-06-11compiletest: compile rmake.rs with -Dunused_must_use许杰友 Jieyou Xu (Joe)-0/+4
2024-06-11Rename `needs-matching-clang` to `needs-force-clang-based-tests`Zalathar-3/+3
This header is much more restrictive than its old name would suggest. As a result, most of the tests that use it don't actually run in any CI jobs.
2024-06-11Add {{target}} substitution to compiletestLukas Wirth-0/+5
2024-06-08Auto merge of #126097 - Kobzol:runmake-change-cwd, r=jieyouxubors-19/+34
Change how runmake v2 tests are executed This PR makes execution of v2 runmake tests more sane, by executing each test in a temporary directory by default, rather than running it inside `tests/run-make`. This will have.. a lot of conflicts. Fixes: https://github.com/rust-lang/rust/issues/126080 Closes https://github.com/rust-lang/rust/issues/125726, because it removes `tmp_dir`, lol. r? `@jieyouxu` try-job: x86_64-msvc
2024-06-08Auto merge of #126111 - Zalathar:fulldeps-hotplug, r=jieyouxubors-0/+22
Port `tests/run-make-fulldeps/hotplug_codegen_backend` to ui-fulldeps This is the last remaining run-make-fulldeps test, which means I actually had to leave behind a dummy README file to prevent compiletest from complaining about a missing directory. (Removing the run-make-fulldeps suite entirely is non-trivial, so I intend to do so in a separate PR after this one.) --- I wasn't sure about adding a new kind of aux build just for this one test, so I also tried to just port this test from Makefile to [rmake](https://github.com/rust-lang/rust/issues/121876) instead. But I found that I couldn't get rmake to fully work for a run-make-fulldeps test, which convinced me that getting rid of run-make-fulldeps is worthwhile. r? `@jieyouxu`
2024-06-07Port `tests/run-make-fulldeps/hotplug_codegen_backend` to ui-fulldepsZalathar-0/+19
2024-06-07Address review commentsJakub Beránek-1/+1
2024-06-07Remove dependence on `tmp_dir`Jakub Beránek-16/+2
And also remove some environment variables passed to compilation of `rmake.rs`.
2024-06-07Do not run run-make tests in the test source directoryJakub Beránek-3/+32
2024-06-07Rename `S` environment variable to `SOURCE_ROOT` in run-make testsJakub Beránek-1/+1
2024-06-07Do not pass source root when building run-make testsJakub Beránek-1/+0
2024-06-07compiletest: Don't pass `--out-dir` if the compile flags include `-o`Zalathar-0/+3
2024-06-06Rollup merge of #126072 - Zalathar:run-flags, r=jieyouxuJubilee-7/+11
compiletest: Allow multiple `//@ run-flags:` headers While working on some tests, I was annoyed to find that multiple `// `@run-flags:`` headers do not combine with each other (as `//@ compile-flags:` headers do), and instead all but one are silently discarded. This makes it impossible to split long flag lists into multiple lines. Fortunately it's easy to just recycle the existing logic from the other command-line-flags headers.
2024-06-07compiletest: Allow multiple `//@ run-flags:` headersZalathar-7/+11
2024-06-06Make the panic info more usefulOli Scherer-0/+1
2024-06-05Rollup merge of #126008 - Zalathar:fulldeps-19371, r=jieyouxuMatthias Krüger-0/+10
Port `tests/run-make-fulldeps/issue-19371` to ui-fulldeps This test can run as an ordinary `tests/ui-fulldeps` test, with the help of some additional header variable substitutions to supply a sysroot and linker. --- Unlike #125973, this test appears to be testing something vaguely useful and breakable, which is why I didn't just delete it.
2024-06-05Rollup merge of #125683 - Oneirical:patience-testing-test, r=jieyouxuJubilee-0/+1
Rewrite `suspicious-library`, `resolve-rename` and `incr-prev-body-beyond-eof` `run-make` tests in `rmake.rs` format Part of #121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html). Some oddly specific ignore flags in `incr-prev-body-beyond-eof`: ```rs // ignore-none // ignore-nvptx64-nvidia-cuda ``` it could be interesting to run a try job, but it seems there is no nvidia-cuda in the CI settings (`jobs.yml`). try-job: armhf-gnu
2024-06-05Port `tests/run-make-fulldeps/issue-19371` to ui-fulldepsZalathar-0/+10
2024-06-04rewrite suspicious-library in rmakeOneirical-0/+1
2024-06-02Rollup merge of #125896 - jieyouxu:compiletest-rmake-comment, r=compiler-errorsJubilee-5/+5
compiletest: fix outdated rmake.rs comment Noticed in https://github.com/rust-lang/rust/pull/125827#discussion_r1623420820. I fixed the PR description but forgot to update the comment.
2024-06-02Rollup merge of #125890 - Nilstrieb:gay-compiletest, r=jieyouxuJubilee-6/+34
Improve compiletest expected/not found formatting compiletest, oh compiletest, you are truly one of the tools in this repository. You're the omnipresent gatekeeper, ensuring that every new change works, doesn't break the world, and is nice. We thank you for your work, for your tests, for your test runs, for your features that help writing tests, for all the stability and and good you have caused. Without you, Rust wouldn't exist as it does, without you, nothing would work, without you, we would all go insane from having changes break and having to test them all by hand. Thank you, compiletest. but holy shit i fucking hate your stupid debug output so much i simply cannot take this anymore aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa By changing a few magic lines in this file called "runtest.rs", we can cause compiletest to emit nicer messages. This is widely regarded as a good thing. We stop wasting vertical space, allowing more errors to be displayed at once. Additionally, we add colors, which make it so much more pretty *and* gay, both of which are very good and useful. There's a bit of fuckery needed to get the colors to work. `colored` checks whether stdout is a terminal. We also print to stdout, so that works well. But.... for some stupid reason that I absolutely refuse to even attempt to debug, stdout is *not* a terminal when executing tests *in a terminal*. But stderr is >:). So this just checks whether stderr is a terminal. If you have a use case where you dump compiletest stdout into a place where colors are not supported while having stderr be a terminal, then I'm sorry for you, but you are gonna get colors and you're gonna like it. Stop it with the usual environment variable, which `colored` also respects by default. ### before (bad, hurts your brain, makes you want to cry) ![image](https://github.com/rust-lang/rust/assets/48135649/cbeecb5d-fc25-460b-b192-9808f8fa2079) ## after (good, gay, makes you want to cry) ![image](https://github.com/rust-lang/rust/assets/48135649/a655b220-8841-443e-a825-72a835d56882) r? jieyouxu said he wants to review the PR
2024-06-02compiletest: fix outdated rmake.rs comment许杰友 Jieyou Xu (Joe)-5/+5
2024-06-02Improve compiletest expected/not found formattingNilstrieb-6/+34
compiletest, oh compiletest, you are truly one of the tools in this repository. You're the omnipresent gatekeeper, ensuring that every new change works, doesn't break the world, and is nice. We thank you for your work, for your tests, for your test runs, for your features that help writing tests, for all the stability and and good you have caused. Without you, Rust wouldn't exist as it does, without you, nothing would work, without you, we would all go insane from having changes break and having to test them all by hand. Thank you, compiletest. but holy shit i fucking hate your stupid debug output so much i simply cannot take this anymore aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa By changing a few magic lines in this file called "runtest.rs", we can cause compiletest to emit nicer messages. This is widely regarded as a good thing. We stop wasting vertical space, allowing more errors to be displayed at once. Additionally, we add colors, which make it so much more pretty *and* gay, both of which are very good and useful. There's a bit of fuckery needed to get the colors to work. `colored` checks whether stdout is a terminal. We also print to stdout, so that works well. But.... for some stupid reason that I absolutely refuse to even attempt to debug, stdout is *not* a terminal when executing tests *in a terminal*. But stderr is >:). So this just checks whether stderr is a terminal. If you have a use case where you dump compiletest stdout into a place where colors are not supported while having stderr be a terminal, then I'm sorry for you, but you are gonna get colors and you're gonna like it. Stop it with the usual environment variable, which `colored` also respects by default.
2024-06-02compiletest: split rmake executable from scratch dir许杰友 Jieyou Xu (Joe)-9/+24
When implementing support for rmake.rs, I copied over the `$TMPDIR` directory logic from the legacy Makefile setup. In doing so, I also compiled recipe `rmake.rs` into executables which unfortunately are placed into `$TMPDIR` as well. This causes a problem on Windows where: - The `rmake.exe` executable is placed in `$TMPDIR`. - We run the `rmake.exe` as a process. - The process uses `rmake.exe` inside `$TMPDIR`. - Windows prevents the .exe file from being deleted when the process is still alive. - The recipe test code tries to `remove_dir_all($TMPDIR)`, which fails with access denied because `rmake.exe` is still being used. We fix this by separating the recipe executable and the sratch directory: ``` base_dir/ rmake.exe scratch/ ``` We construct a base directory, unique to each run-make test, under which we place rmake.exe alongside a `scratch/` directory. This `scratch/` directory is what is passed to rmake.rs tests as `$TMPDIR`, so now `remove_dir_all($TMPDIR)` has a chance to succeed because it no longer contains `rmake.exe`. Oops. This was a fun one to try figure out.
2024-05-31Remove unused rust-demangler support from compiletestZalathar-10/+0
2024-05-31Use `coverage-dump --demangle` as the demangler for coverage-run testsZalathar-5/+12
This avoids the need to build `rust-demangler` when running coverage tests, since we typically need to build `coverage-dump` anyway.
2024-05-30Auto merge of #125710 - RalfJung:compiletest-components, r=workingjubileebors-3/+6
compiletest: clarify COMPILETEST_NEEDS_ALL_LLVM_COMPONENTS error COMPILETEST_NEEDS_ALL_LLVM_COMPONENTS is a confusing name because elsewhere "needs" means "ignore when requirement not met", but here it means "fail when requirement not met".
2024-05-30compiletest: clarify COMPILETEST_NEEDS_ALL_LLVM_COMPONENTS errorRalf Jung-3/+6
also improve wording for an ignore reason
2024-05-30compiletest: Unify `cmd2procres` with `run_command_to_procres`Zalathar-29/+19
2024-05-29Extract coverage-specific code out of `compiletest::runtest`Zalathar-426/+372
This moves a few hundred lines of coverage-specific code out of the main module, making navigation a bit easier.
2024-05-28Make more of the test suite run on Mac CatalystMads Marquart-3/+10
This adds the `only-apple`/`ignore-apple` compiletest directive, and uses that basically everywhere instead of `only-macos`/`ignore-macos`. Some of the updates in `run-make` are a bit redundant, as they use `ignore-cross-compile` and won't run on iOS - but using Apple in these is still more correct, so I've made that change anyhow.
2024-05-23Rollup merge of #125156 - zachs18:for_loops_over_fallibles_behind_refs, ↵Matthias Krüger-2/+2
r=Nilstrieb Expand `for_loops_over_fallibles` lint to lint on fallibles behind references. Extends the scope of the (warn-by-default) lint `for_loops_over_fallibles` from just `for _ in x` where `x: Option<_>/Result<_, _>` to also cover `x: &(mut) Option<_>/Result<_>` ```rs fn main() { // Current lints for _ in Some(42) {} for _ in Ok::<_, i32>(42) {} // New lints for _ in &Some(42) {} for _ in &mut Some(42) {} for _ in &Ok::<_, i32>(42) {} for _ in &mut Ok::<_, i32>(42) {} // Should not lint for _ in Some(42).into_iter() {} for _ in Some(42).iter() {} for _ in Some(42).iter_mut() {} for _ in Ok::<_, i32>(42).into_iter() {} for _ in Ok::<_, i32>(42).iter() {} for _ in Ok::<_, i32>(42).iter_mut() {} } ``` <details><summary><code>cargo build</code> diff</summary> ```diff diff --git a/old.out b/new.out index 84215aa..ca195a7 100644 --- a/old.out +++ b/new.out `@@` -1,33 +1,93 `@@` warning: for loop over an `Option`. This is more readably written as an `if let` statement --> src/main.rs:3:14 | 3 | for _ in Some(42) {} | ^^^^^^^^ | = note: `#[warn(for_loops_over_fallibles)]` on by default help: to check pattern in a loop use `while let` | 3 | while let Some(_) = Some(42) {} | ~~~~~~~~~~~~~~~ ~~~ help: consider using `if let` to clear intent | 3 | if let Some(_) = Some(42) {} | ~~~~~~~~~~~~ ~~~ warning: for loop over a `Result`. This is more readably written as an `if let` statement --> src/main.rs:4:14 | 4 | for _ in Ok::<_, i32>(42) {} | ^^^^^^^^^^^^^^^^ | help: to check pattern in a loop use `while let` | 4 | while let Ok(_) = Ok::<_, i32>(42) {} | ~~~~~~~~~~~~~ ~~~ help: consider using `if let` to clear intent | 4 | if let Ok(_) = Ok::<_, i32>(42) {} | ~~~~~~~~~~ ~~~ -warning: `for-loops-over-fallibles` (bin "for-loops-over-fallibles") generated 2 warnings - Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.04s +warning: for loop over a `&Option`. This is more readably written as an `if let` statement + --> src/main.rs:7:14 + | +7 | for _ in &Some(42) {} + | ^^^^^^^^^ + | +help: to check pattern in a loop use `while let` + | +7 | while let Some(_) = &Some(42) {} + | ~~~~~~~~~~~~~~~ ~~~ +help: consider using `if let` to clear intent + | +7 | if let Some(_) = &Some(42) {} + | ~~~~~~~~~~~~ ~~~ + +warning: for loop over a `&mut Option`. This is more readably written as an `if let` statement + --> src/main.rs:8:14 + | +8 | for _ in &mut Some(42) {} + | ^^^^^^^^^^^^^ + | +help: to check pattern in a loop use `while let` + | +8 | while let Some(_) = &mut Some(42) {} + | ~~~~~~~~~~~~~~~ ~~~ +help: consider using `if let` to clear intent + | +8 | if let Some(_) = &mut Some(42) {} + | ~~~~~~~~~~~~ ~~~ + +warning: for loop over a `&Result`. This is more readably written as an `if let` statement + --> src/main.rs:9:14 + | +9 | for _ in &Ok::<_, i32>(42) {} + | ^^^^^^^^^^^^^^^^^ + | +help: to check pattern in a loop use `while let` + | +9 | while let Ok(_) = &Ok::<_, i32>(42) {} + | ~~~~~~~~~~~~~ ~~~ +help: consider using `if let` to clear intent + | +9 | if let Ok(_) = &Ok::<_, i32>(42) {} + | ~~~~~~~~~~ ~~~ + +warning: for loop over a `&mut Result`. This is more readably written as an `if let` statement + --> src/main.rs:10:14 + | +10 | for _ in &mut Ok::<_, i32>(42) {} + | ^^^^^^^^^^^^^^^^^^^^^ + | +help: to check pattern in a loop use `while let` + | +10 | while let Ok(_) = &mut Ok::<_, i32>(42) {} + | ~~~~~~~~~~~~~ ~~~ +help: consider using `if let` to clear intent + | +10 | if let Ok(_) = &mut Ok::<_, i32>(42) {} + | ~~~~~~~~~~ ~~~ + +warning: `for-loops-over-fallibles` (bin "for-loops-over-fallibles") generated 6 warnings + Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.02s ``` </details> ----- Question: * ~~Currently, the article `an` is used for `&Option`, and `&mut Option` in the lint diagnostic, since that's what `Option` uses. Is this okay or should it be changed? (likewise, `a` is used for `&Result` and `&mut Result`)~~ The article `a` is used for `&Option`, `&mut Option`, `&Result`, `&mut Result` and (as before) `Result`. Only `Option` uses `an` (as before). `@rustbot` label +A-lint
2024-05-18Fix typos (taking into account review comments)blyxyas-3/+3
2024-05-16Fix `for_loops_over_fallibles` hits in compiletest/src/json.rszachs18-2/+2
2024-05-11Make crashes dump mir to build dirMichael Goulet-8/+14
2024-05-09Tidy check for test revisions that are mentioned but not declaredZalathar-0/+2
If a `[revision]` name appears in a test header directive or error annotation, but isn't declared in the `//@ revisions:` header, that is almost always a mistake. In cases where a revision needs to be temporarily disabled, adding it to an `//@ unused-revision-names:` header will suppress these checks for that name. Adding the wildcard name `*` to the unused list will suppress these checks for the entire file.
2024-05-04compiletest: add enable-by-default check-cfgUrgau-3/+22
2024-05-04compiletest: add no-auto-check-cfg directiveUrgau-0/+7
this directive prevents compiletest from adding any implicit and automatic --check-cfg arguments
2024-05-02Rollup merge of #124138 - mati865:ignore-llvm-abi-in-dlltool-tests, r=davidtwcoMatthias Krüger-46/+24
Ignore LLVM ABI in dlltool tests since those targets don't use dlltool Otherwise those two tests fail when running `./x.py test` with this target.
2024-05-01Auto merge of #124491 - madsmtm:target_vendor-apple, r=workingjubileebors-2/+2
Use `target_vendor = "apple"` instead of `target_os = "..."` Use `target_vendor = "apple"` instead of `all(target_os = "macos", target_os = "ios", target_os = "tvos", target_os = "watchos", target_os = "visionos")`. The apple targets are quite close to being identical, with iOS, tvOS, watchOS and visionOS being even closer, so using `target_vendor` when possible makes it clearer when something is actually OS-specific, or just Apple-specific. Note that `target_vendor` will [be deprecated in the future](https://github.com/rust-lang/rust/issues/100343), but not before an alternative (like `target_family = "apple"`) is available. While doing this, I found various inconsistencies and small mistakes in the standard library, see the commits for details. Will follow-up with an extra PR for a similar issue that need a bit more discussion. EDIT: https://github.com/rust-lang/rust/pull/124494 Since you've talked about using `target_vendor = "apple"` in the past: r? workingjubilee CC `@simlay,` `@thomcc` `@rustbot` label O-macos O-ios O-tvos O-watchos O-visionos
2024-04-30Rollup merge of #124280 - beetrees:repr128-test-rmake, r=jieyouxu许杰友 Jieyou Xu (Joe)-0/+1
Port repr128-dwarf run-make test to rmake This PR ports the repr128-dwarf run-make test to rmake, using the `gimli` crate instead of the `llvm-dwarfdump` command. Note that this PR changes `rmake.rs` files to be compiled with the 2021 edition (previously no edition was passed to `rustc`, meaning they were compiled with the 2015 edition). This means that `panic!("{variable}")` will now work as expected in `rmake.rs` files (there's already a usage in the [wasm-symbols-not-exported test](https://github.com/rust-lang/rust/blob/aca749eefceaed0cda19a7ec5e472fce9387bc00/tests/run-make/wasm-symbols-not-exported/rmake.rs#L34) that this will fix). Tracking issue: #121876