about summary refs log tree commit diff
path: root/src/tools/compiletest
AgeCommit message (Collapse)AuthorLines
2025-04-20Remove `setup-upstream-remote.sh` and upstream handling.Jakub Beránek-1/+1
It shouldn't be needed anymore.
2025-04-18Rollup merge of #139393 - ↵Matthias Krüger-0/+3
willglynn:rustdoc_output_target_feature_information, r=aDotInTheVoid rustdoc-json: Output target feature information `#[target_feature]` attributes refer to a target-specific list of features. Enabling certain features can imply enabling other features. Certain features are always enabled on certain targets, since they are required by the target's ABI. Features can also be enabled indirectly based on other compiler flags. Feature information is ultimately known to `rustc`. Rather than force external tools to track it – which may be wildly impractical due to `-C target-cpu` – have `rustdoc` output `rustc`'s feature data. This change is motivated by https://github.com/obi1kenobi/cargo-semver-checks/issues/1246, which intends to detect semver hazards caused by `#[target_feature]`. try-job: aarch64-gnu try-job: armhf-gnu try-job: test-various try-job: x86_64-msvc-1 try-job: i686-msvc-1 try-job: x86_64-mingw-1 try-job: aarch64-apple
2025-04-17Rollup merge of #139967 - jieyouxu:auxiliary, r=wesleywiserMatthias Krüger-0/+11
Introduce and use specialized `//@ ignore-auxiliary` for test support files instead of using `//@ ignore-test` ### Summary Add a semantically meaningful directive for ignoring test *auxiliary* files. This is for auxiliary files that *participate* in actual tests but should not be built by `compiletest` (i.e. these files are involved through `mod xxx;` or `include!()` or `#[path = "xxx"]`, etc.). ### Motivation A specialized directive like `//@ ignore-auxiliary` makes it way easier to audit disabled tests via `//@ ignore-test`. - These support files cannot use the canonical `auxiliary/` dir because they participate in module resolution or are included, or their relative paths can be important for test intention otherwise. Follow-up to: - #139705 - #139783 - #139740 See also discussions in: - [#t-compiler > Directive name for non-test aux files?](https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/Directive.20name.20for.20non-test.20aux.20files.3F/with/512773817) - [#t-compiler > Handling disabled &#96;//@ ignore-test&#96; tests](https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/Handling.20disabled.20.60.2F.2F.40.20ignore-test.60.20tests/with/512005974) - [#t-compiler/meetings > &#91;steering&#93; 2025-04-11 Dealing with disabled tests](https://rust-lang.zulipchat.com/#narrow/channel/238009-t-compiler.2Fmeetings/topic/.5Bsteering.5D.202025-04-11.20Dealing.20with.20disabled.20tests/with/511717981) ### Remarks on remaining unconditionally disabled tests under `tests/` After this PR, against commit 79a272c6402, only **14** remaining test files are disabled through `//@ ignore-test`: <details> <summary>Remaining `//@ ignore-test` files under `tests/`</summary> ``` tests/debuginfo/drop-locations.rs 4://@ ignore-test (broken, see #128971) tests/rustdoc/macro-document-private-duplicate.rs 1://@ ignore-test (fails spuriously, see issue #89228) tests/rustdoc/inline_cross/assoc-const-equality.rs 3://@ ignore-test (FIXME: #125092) tests/ui/match/issue-27021.rs 7://@ ignore-test (#54987) tests/ui/match/issue-26996.rs 7://@ ignore-test (#54987) tests/ui/issues/issue-49298.rs 9://@ ignore-test (#54987) tests/ui/issues/issue-59756.rs 2://@ ignore-test (rustfix needs multiple suggestions) tests/ui/precondition-checks/write.rs 5://@ ignore-test (unimplemented) tests/ui/precondition-checks/read.rs 5://@ ignore-test (unimplemented) tests/ui/precondition-checks/write_bytes.rs 5://@ ignore-test (unimplemented) tests/ui/explicit-tail-calls/drop-order.rs 2://@ ignore-test: tail calls are not implemented in rustc_codegen_ssa yet, so this causes 🧊 tests/ui/panics/panic-short-backtrace-windows-x86_64.rs 3://@ ignore-test (#92000) tests/ui/json/json-bom-plus-crlf-multifile-aux.rs 3://@ ignore-test Not a test. Used by other tests tests/ui/traits/next-solver/object-soundness-requires-generalization.rs 2://@ ignore-test (see #114196) ``` </details> Of these, most are either **unimplemented**, or **spurious**, or **known-broken**. The outstanding one is `tests/ui/json/json-bom-plus-crlf-multifile-aux.rs` which I did not want to touch in *this* PR -- that aux file has load-bearing BOM and carriage returns and byte offset matters. I think those test files that require special encoding / BOM probably are better off as `run-make` tests. See #139968 for that aux file. ### Review advice - Best reviewed commit-by-commit. - The directive name diverged from the most voted `//@ auxiliary` because I think that's easy to confuse with `//@ aux-{crate,dir}`. r? compiler
2025-04-17Rollup merge of #139870 - ↵Matthias Krüger-14/+25
Shourya742:2025-04-15-add-retries-to-remove_and_create_dir_all, r=jieyouxu add retries to remove and create dir all closes: #139230 r? ```@jieyouxu```
2025-04-17compiletest: add specific `//@ ignore-auxiliary` for test support filesJieyou Xu-0/+11
This is for files that *participate* in actual tests but should not be built by `compiletest` (i.e. these files are involved through `mod xxx;` or `include!()` or `#[path = "xxx"]`, etc.). A specialized directive like `//@ ignore-auxiliary` makes it way easier to audit disabled tests via `//@ ignore-test`.
2025-04-16remove old remove_and_create_dir_all and use build_helpers ↵bit-aloo-14/+25
remove_and_create_dir_all
2025-04-16Rollup merge of #139823 - WaffleLapkin:bootpaper, r=jieyouxuMatthias Krüger-1/+8
Fix some bootstrap papercuts ... related to jj and my `./build` symlink setup[^1]. I'm not sure if these are good solutions, but they seem to work. See commits for a bit more info. r? ``@jieyouxu`` [^1]: see #139804
2025-04-15rustdoc: Output target feature informationWill Glynn-0/+3
`#[target_feature]` attributes refer to a target-specific list of features. Enabling certain features can imply enabling other features. Certain features are always enabled on certain targets, since they are required by the target's ABI. Features can also be enabled indirectly based on other compiler flags. Feature information is ultimately known to `rustc`. Rather than force external tools to track it -- which may be wildly impractical due to `-C target-cpu` -- have `rustdoc` output `rustc`'s feature data.
2025-04-15normalize canonical and non-canonical paths in compiletestWaffle Lapkin-5/+8
Apparently there are tests that print canonical paths *and* tests which print non-canonical paths. An example of the latter is `tests/ui/type_length_limit.rs`.
2025-04-15canonicalize test build dir before normalizing itWaffle Lapkin-1/+5
Fix fixes failures of the following tests when build directory is a symlink: - `tests/ui/error-codes/E{0464,0523}.rs` - `tests/ui/crate-loading/crateresolve{1,2}.rs` (those are the same tests)
2025-04-15fix typoWaffle Lapkin-1/+1
2025-04-15compiletest: Add an experimental new executor to replace libtestZalathar-11/+451
The new executor can be enabled by passing `--new-executor` or `-n` to compiletest. For example: `./x test ui -- -n`
2025-04-15compiletest: Extract libtest-specific executor code to a submoduleZalathar-103/+113
2025-04-13Rollup merge of #139695 - jieyouxu:compiletest-utf8path, r=KobzolJacob Pratt-433/+404
compiletest: consistently use `camino::{Utf8Path,Utf8PathBuf}` throughout compiletest already practically assumes UTF-8 paths everywhere. Use `camino`'s `{Utf8Path,Utf8PathBuf}` consistently throughout compiletest to enforce UTF-8 path assumptions. r? ```@Kobzol``` (or compiler/bootstrap)
2025-04-13compiletest: consistently use `{Utf8Path,Utf8PathBuf}`Jieyou Xu-433/+403
Since compiletest already assumes UTF-8 paths and does not try to be robust against non-UTF-8 paths.
2025-04-13compiletest: add `camino` for UTF-8 path handlingJieyou Xu-0/+1
2025-04-13Rollup merge of #139618 - petrochenkov:virsugg, r=jieyouxuChris Denton-38/+18
compiletest: Make `SUGGESTION` annotations viral If one of them is expected in a test file, then others should be annotated as well, in the same way as with `HELP`s and `NOTE`s. This doesn't require much of an additional annotation burden, but simplifies the rules. r? ```@jieyouxu```
2025-04-11Auto merge of #139578 - ferrocene:pa-compiletest-edition, r=jieyouxubors-3/+14
Fix breakage when running compiletest with `--test-args=--edition=2015` Compiletest has an `--edition` flag to change the default edition tests are run with. Unfortunately no test suite successfully executes when that flag is passed. If the edition is set to something greater than 2015 the breakage is expected, since the test suite currently supports only edition 2015 (Ferrous Systems will open an MCP about fixing that soonish). Surprisingly, the test suite is also broken if `--edition=2015` is passed to compiletest. This PR focuses on fixing the latter. This PR fixes the two categories of failures happening when `--edition=2015` is passed: * Some edition-specific tests set their edition through `//@ compile-flags` instead of `//@ edition`. Compiletest doesn't parse the compile flags, so it would see no `//@ edition` and add another `--edition` flag, leading to a rustc error. * Compiletest would add the edition after `//@ compile-flags`, while some tests depend on flags passed to `//@ compile-flags` being the last flags in the rustc invocation. Note that for the first category, I opted to manually go and replace all `//@ compile-flags` setting an edition with an explicit `//@ edition`. We could've changed compiletest to instead check whether an edition was set in `//@ compile-flags`, but I thought it was better to enforce a consistent way to set the edition in tests. I also added the edition to the stamp, so that changing `--edition` results in tests being re-executed. r? `@jieyouxu`
2025-04-11Rollup merge of #139469 - jieyouxu:compiletest-supports-crate-type, r=onur-ozkanStuart Cook-2/+117
Introduce a `//@ needs-crate-type` compiletest directive The `//@ needs-crate-type: $crate_types...` directive takes a comma-separated list of crate types that the target platform must support in order for the test to be run. This allows the test writer to semantically convey that the ignore condition is based on target crate type needs, instead of using a general purpose `//@ ignore-$target` directive (often without comment). Fixes #132309. ### Example ```rs //@ needs-crate-type: dylib (ignored on e.g. wasm32-unknown-unknown) //@ compile-flags: --crate-type=dylib fn foo() {} ``` ### Review advice - Best reviewed commit-by-commit. - The impl is not very clean, I briefly attempted to clean up the directive handling but found that more invasive changes are needed, so I'd like to not block on the cleanup for now. try-job: test-various try-job: armhf-gnu
2025-04-10compiletest: Trim the value of `dont-require-annotations`Vadim Petrochenkov-1/+1
2025-04-10compiletest: Turn `TestProps::require_annotations` into a setVadim Petrochenkov-46/+19
and further simplify its checking in runtest
2025-04-10compiletest: Make `SUGGESTION` annotations viralVadim Petrochenkov-3/+10
2025-04-10Rollup merge of #139609 - jieyouxu:compiletest-path-misc, r=KobzolMatthias Krüger-22/+23
compiletest: don't use stringly paths for `compose_and_run` Eventually I'd like to fully migrate to `camino`'s `{Utf8Path,Utf8PathBuf}` because compiletest assumes UTF-8 paths all over the place, so this is an precursor change to make the migration diff cleaner. r? `@Kobzol` (or bootstrap/compiler)
2025-04-10Rollup merge of #139606 - jieyouxu:compiletest-edition2024, r=compiler-errorsMatthias Krüger-15/+44
Update compiletest to Edition 2024 r? bootstrap (or compiler) try-job: x86_64-apple-1 try-job: x86_64-msvc-1 try-jbo: x86_64-mingw-1
2025-04-10Rollup merge of #139507 - Zalathar:trim-env-name, r=jieyouxuMatthias Krüger-16/+13
compiletest: Trim whitespace from environment variable names When a test contains a directive like `//@ exec-env: FOO=bar`, compiletest currently includes that leading space in the name of the environment variable, so it is defined as ` FOO` instead of `FOO`. This is an annoying footgun that is pretty much never intended, especially since most other directives *do* trim whitespace. So let's get rid of it by trimming the environment variable name. Values remain untrimmed, since there could conceivably be a use-case for values with leading space, but perhaps we'll end up trimming values too in the future. Recently observed in https://github.com/rust-lang/rust/pull/138603#issuecomment-2783709359. Fixes #132990. Supersedes #133148. --- try-job: test-various
2025-04-10include --edition in the compiletest stampPietro Albini-0/+1
2025-04-10prevent configuring the edition with //@compile-flagsPietro Albini-1/+7
2025-04-10ensure //@compile-flags are last even when passing --editionPietro Albini-2/+6
2025-04-10compiletest: update to Edition 2024Jieyou Xu-15/+44
2025-04-10compiletest: add `needs-crate-type` directiveJieyou Xu-2/+117
The `//@ needs-crate-type: $crate_types...` directive takes a comma-separated list of crate types that the target platform must support in order for the test to be run.
2025-04-10compiletest: don't use stringly paths for `compose_and_run`Jieyou Xu-22/+23
2025-04-10Auto merge of #139279 - BoxyUwU:bump-boostrap, r=jieyouxubors-3/+0
Bump boostrap compiler to new beta try-job: `*msvc*`
2025-04-09Rollup merge of #139489 - petrochenkov:noreqann, r=jieyouxuMatthias Krüger-15/+41
compiletest: Add directive `dont-require-annotations` for making matching on specific diagnostic kinds non-exhaustive. E.g. `//@ dont-require-annotations:ERROR`, like in the examples in this PR. cc https://github.com/rust-lang/rust/pull/139427#issuecomment-2782827583 Closes #132647 FYI `@BoxyUwU` since you've wanted this. r? `@jieyouxu`
2025-04-09update cfgsBoxy-3/+0
2025-04-09compiletest: Add directive `dont-require-annotations`Vadim Petrochenkov-15/+41
for making matching on specific diagnostic kinds non-exhaustive
2025-04-09Auto merge of #139552 - matthiaskrgr:rollup-b194mk8, r=matthiaskrgrbors-20/+15
Rollup of 10 pull requests Successful merges: - #139494 (Restrict some queries by def-kind more) - #139496 (Revert r-a changes of rust-lang/rust#139455) - #139506 (add missing word in doc comment (part 2)) - #139515 (Improve presentation of closure signature mismatch from `Fn` trait goal) - #139520 (compiletest maintenance: sort deps and drop dep on `anyhow`) - #139523 (Rustc dev guide subtree update) - #139526 (Fix deprecation note for std::intrinsics) - #139528 (compiletest: Remove the `--logfile` flag) - #139541 (Instantiate higher-ranked transmute goal w/ placeholders before emitting sub-obligations) - #139547 (Update library tracking issue template to set S-tracking-unimplemented) r? `@ghost` `@rustbot` modify labels: rollup
2025-04-08Rollup merge of #139528 - Zalathar:no-logfile, r=jieyouxuMatthias Krüger-6/+1
compiletest: Remove the `--logfile` flag This flag is deprecated in libtest (#134283), and there's no evidence in-tree of this flag actually being passed to compiletest. For detailed information about test results, bootstrap parses JSON output from compiletest instead (#108659). As part of my experimental work on removing the libtest dependency from compiletest, it's useful to be able to disconnect libtest functionality that isn't needed.
2025-04-08compiletest: Remove the `--logfile` flagZalathar-6/+1
This flag is deprecated in libtest, and there's no evidence in-tree of this flag actually being passed to compiletest. (For detailed information about test results, bootstrap parses JSON output from compiletest instead.)
2025-04-08compiletest: Trim whitespace from environment variable namesZalathar-16/+13
2025-04-08compiletest: drop dependency on `anyhow`Jieyou Xu-7/+5
Currently `compiletest` panics all over the place but doesn't really use `anyhow` anyway. I'd like to introduce some more principled error handling and disciplined diagnostic reporting in the near future.
2025-04-08compiletest: sort dependencies alphabeticallyJieyou Xu-8/+10
2025-04-07compiletest: Avoid ignoring empty diagnostics in one more placeVadim Petrochenkov-2/+1
This catches some silly notes emitted by rustc, which should ideally be fixed
2025-04-07compiletest: Stricter parsing for diagnostic kindsVadim Petrochenkov-35/+37
2025-04-07compiletest: Always preserve kind for compiler diagnosticsVadim Petrochenkov-22/+36
Those that didn't previously preserved kind are now marked as not requiring annotations to keep the previous behavior. Also, do not lose diagnostics with an empty message.
2025-04-07compiletest: Cleanup collection of actual errorsVadim Petrochenkov-48/+35
2025-04-04Rollup merge of #139317 - Zalathar:hide-libtest, r=jieyouxuMatthias Krüger-105/+208
compiletest: Encapsulate all of the code that touches libtest Compiletest currently relies on unstable libtest APIs in order to actually execute tests. That's unfortunate, but removing the dependency isn't trivial. However, we can make a small step towards removing the libtest dependency by encapsulating the libtest interactions into a single dedicated module. That makes it easier to see what parts of libtest are actually used. --- As a side-effect of moving the `test_opts` function into that dedicated module, this PR also ends up allowing `--fail-fast` to be passed on the command line, instead of requiring an environment variable. --- There is still (at least) one other aspect of the libtest dependency that this PR does not address, namely the fact that we rely on libtest's output capture (via unstable std APIs) to capture the output that we print during individual tests. I hope to do something about that at some point. r? jieyouxu
2025-04-03compiletest: Encapsulate all of the code that touches libtestZalathar-98/+194
2025-04-03compiletest: Allow `--fail-fast` as a command-line optionZalathar-8/+15
2025-04-03compiletest: Require `//~` annotations even if `error-pattern` is specifiedVadim Petrochenkov-10/+6
2025-03-29Rollup merge of #139100 - petrochenkov:errbelow, r=jieyouxuMatthias Krüger-1/+7
compiletest: Support matching diagnostics on lines below Using `//~vvv ERROR`. This is not needed often, but it's easy to support, and it allows to eliminate a class of `error-pattern`s that cannot be eliminated in any other way. See the diff for the examples of such patterns coming from parser. Some of them can be matched by `//~ ERROR` or `//~^ ERROR` as well (when the final newline is allowed), but it changes the shape of reported spans, so I chose to keep the spans by using `//~v ERROR`.