| Age | Commit message (Collapse) | Author | Lines |
|---|
|
- `run-make` test suite will now no longer receive a `cargo`.
- NOTE: the user could technically still write
`Command::new("cargo")` which might find *a* cargo from the
environment, but that is not a supported case.
- `run-make-cargo` will receive a built in-tree `cargo`.
|
|
compiletest: Implement an experimental `--new-output-capture` mode
Thanks to the efforts on rust-lang/rust#140192, compiletest no longer has an unstable dependency on libtest, but it still has an unstable dependency on `#![feature(internal_output_capture)]`. That makes building compiletest more complicated than for most other bootstrap tools.
This PR therefore adds opt-in support for an experimental compiletest mode that avoids the use of `internal_output_capture` APIs, and instead uses more mundane means to capture the output of individual test runners.
Each `TestCx` now contains `&dyn ConsoleOut` references for stdout and stderr. All print statements in `compiletests::runtest` have been replaced with `write!` or `writeln!` calls that explicitly write to one of those trait objects. The underlying implementation then forwards to `print!` or `eprint!` (for `--no-capture` or old-output-capture mode), or writes to a separate buffer (in new-output-capture mode).
---
Currently, new-output-capture is disabled by default. It can be explicitly enabled in one of two ways:
- When running `x test`, pass `--new-output-capture=on` as a *compiletest* argument (after `--`).
- E.g. `x test ui -- --new-output-capture=on`.
- The short form is `-Non` or `-Ny`.
- Set environment variable `COMPILETEST_NEW_OUTPUT_CAPTURE=on`.
After some amount of opt-in testing, new-output-capture will become the default (with a temporary opt-out). Eventually, old-output-capture and `#![feature(internal_output_capture)]` will be completely removed from compiletest.
r? jieyouxu
|
|
|
|
|
|
|
|
|
|
|
|
This was done in #145740 and #145947. It is causing problems for people
using r-a on anything that uses the rustc-dev rustup package, e.g. Miri,
clippy.
This repository has lots of submodules and subtrees and various
different projects are carved out of pieces of it. It seems like
`[workspace.dependencies]` will just be more trouble than it's worth.
|
|
|
|
Make target pointer width in target json an integer
r? Noratrieb
cc `@RalfJung` (https://github.com/rust-lang/rust/pull/142352/files#r2230380120)
try-job: x86_64-rust-for-linux
|
|
compiletest: Reduce the number of `println!` calls that don't have access to `TestCx`
In order to stop using `#![feature(internal_output_capture)]` in compiletest, we need to be able to capture the console output of individual tests run by the executor.
The approach I have planned is to have all test runners print “console” output into a trait object that is passed around as part of `TestCx`, since almost all test-runner code has easy access to that context. So `println!("foo")` will become `writeln!(self.stdout, "foo")`, and so on.
In order to make that viable, we need to avoid unnecessary printing in places that don't have easy access to `TestCx`. To do so, we can either get rid of unnecessary print statements, or rearrange the code to make the context available. This PR uses both approaches.
r? jieyouxu
|
|
This appears to have been leftover debugging code.
If the capture information turns out to have still been useful, we can find a
way to emit it in a way that doesn't interfere with overhauling compiletests's
output capture system.
|
|
The code in this module is always called in the context of running an
individual tests, and sometimes prints output that needs to be captured.
Moving this module into `runtest` will make it easier to find and audit all of
the print statements that need to be updated when overhauling output-capture.
|
|
When working on a new output-capture system, this will make it easier to obtain
a capturing stream from the test context.
|
|
Running `./x --verbose` will still print out the command-line arguments, and
setting `RUST_LOG=compiletest` will now log the full config instead of a
subset.
|
|
Add more to the `[workspace.dependencies]` section in the top-level `Cargo.toml`
Following on from rust-lang/rust#145740.
r? `@Kobzol`
|
|
compiletest: Remove several remnants of the old libtest-based executor
I noticed a few bits of low-hanging cleanup that are possible now that the non-libtest executor is well and truly established.
|
|
|
|
|
|
|
|
|
|
|
|
Inherit TCC in debuginfo tests on macOS
macOS has a system for propagating folder permissions, which LLDB disables when spawning processes, which in turn causes debuginfo tests to spam the user with repeated pop-ups asking for permissions. See the code comment for details, as well as the following video for an example of how this looks in practice:
https://github.com/user-attachments/assets/1e54f5b8-9130-4b59-8e92-1db1e58fb361
I stumbled upon the incantation to fix this (`settings set target.inherit-tcc true`) while investigating slowdowns when spawning newly created binaries due to XprotectService, see [this Zulip thread](https://rust-lang.zulipchat.com/#narrow/channel/246057-t-cargo/topic/build.20scripts.20slow.20on.20macOS.3F).
This would allow me to no longer have a `build.build-dir = "/Users/madsmtm/rust-build"` workaround in my `bootstrap.toml`.
|
|
|
|
|
|
compiletest: if a compiler fails, show its output
Before, when working on something like a `rustdoc-js` test, if you made a syntax error in a rust file, you would not get that error output unless you ran with `--verbose`, which would also cause an enormous amount of other output to be printed as well. This can also lead to frustration in new contributors who don't think to run with `--verbose`.
Now, if rustc or rustdoc is run by compiletest and produces an non-zero exit code, its output will be printed.
|
|
|
|
Add more tests for the parallel rustc
At the moment, the parallel frontend test cases are severely lacking. Althought some reported issues have been resolved, they haven't been added into the tests.
This PR arranges the resolved ICE issues and adds tests for them.
Whether it is worthwhile to add a separate test suite for the paralel frontend still requires futher discussion. But we are trying coveraging issues being resolved through capability of the existing UI test suite.
Discussion: [Zulip](https://rust-lang.zulipchat.com/#narrow/channel/233931-t-compiler.2Fmajor-changes/topic/Proposal.20for.20a.20dedicated.20test.20suite.20for.20t.E2.80.A6.20compiler-team.23906)
Related issues:
- rust-lang/rust#120760
- rust-lang/rust#124423 fixed by rust-lang/rust#140358
- rust-lang/rust#127971 fxied by rust-lang/rust#140358
- rust-lang/rust#120601 fixed by rust-lang/rust#127311
cc `@jieyouxu`
|
|
folkertdev:c-variadic-same-program-multiple-abis-arm, r=RalfJung,davidtwco
c-variadic: multiple ABIs in the same program for arm
similar to https://github.com/rust-lang/rust/pull/144379, but for arm, requested in https://github.com/rust-lang/rust/pull/144066.
Quoting https://github.com/rust-lang/reference/issues/1946#issuecomment-3124555690
> `"aapcs"` specifically refers to the soft-float ABI where floating-point values are passed in integer registers.
However for c-variadic functions, `aapcs` behaves the same as `C`:
https://github.com/ARM-software/abi-aa/blob/main/aapcs32/aapcs32.rst#65parameter-passing
> A variadic function is always marshaled as for the base standard.
https://github.com/ARM-software/abi-aa/blob/main/aapcs32/aapcs32.rst#7the-standard-variants
> This section applies only to non-variadic functions. For a variadic function the base standard is always used both for argument passing and result return.
---
I also noticed that rustc currently emit more instructions than clang for c-variadic functions on arm, see https://godbolt.org/z/hMce9rnTh. I'll fix that separately. (edit: https://github.com/rust-lang/rust/pull/144549)
try-job: armhf-gnu
r? `@RalfJung`
|
|
`--default-codegen-backend` and `--override-codegen-backend`
|
|
|
|
r=lolbinarycat,GuillaumeGomez
rustdoc-search: search backend with partitioned suffix tree
Before:
- https://notriddle.com/windows-docs-rs/doc-old/windows/
- https://doc.rust-lang.org/1.89.0/std/index.html
- https://doc.rust-lang.org/1.89.0/nightly-rustc/rustc_hir/index.html
After:
- https://notriddle.com/windows-docs-rs/doc/windows/
- https://notriddle.com/rustdoc-html-demo-12/stringdex/doc/std/index.html
- https://notriddle.com/rustdoc-html-demo-12/stringdex/compiler-doc/rustc_hir/index.html
## Summary
Rewrites the rustdoc search engine to use an indexed data structure, factored out as a crate called [stringdex](https://crates.io/crates/stringdex), that allows it to perform modified-levenshtein distance calculations, substring matches, and prefix matches in a reasonably efficient, and, more importantly, *incremental* algorithm.
## Motivation
Fixes https://github.com/rust-lang/rust/issues/131156
While the windows-rs crate is definitely the worst offender, I've noticed performance problems with the compiler crates as well. It makes no sense for rustdoc-search to have poor performance: it's basically a spell checker, and those have been usable since the 90's.
Stringdex is particularly designed to quickly return exact matches, to always report those matches first, and to never, ever [place new matches on top of old ones](https://web.dev/articles/cls). It also tries to yield to the event loop occasionally as it runs. This way, you can click the exactly-matched result before the rest of the search finishes running.
## Explanation
A longer description of how name search works can be found in stringdex's [HACKING.md](https://gitlab.com/notriddle/stringdex/-/blob/main/HACKING.md) file.
Type search is done by performing a name search on each element, then performing bitmap operations to narrow down a list of potential matches, then performing type unification on each match.
## Drawbacks
It's rather complex, and takes up more disk space than the current flat list of strings.
## Rationale and alternatives
Instead of a suffix tree, I could've used a different [approximate matching data structure](https://en.wikipedia.org/wiki/Approximate_string_matching). I didn't do that because I wanted to keep the current behavior (for example, a straightforward trigram index won't match [oepn](https://doc.rust-lang.org/nightly/std/?search=oepn) like the current system does).
## Prior art
[Sherlodoc](https://github.com/art-w/sherlodoc) is based on a similar concept, but they:
- use edit distance over a suffix tree for type-based search, instead of the binary matching that's implemented here
- use substring matching for name-based search, [but not fuzzy name matching](https://github.com/art-w/sherlodoc/issues/21)
- actually implement body text search, which is a potential-future feature, but not implemented in this PR
## Future possibilities
### Low-level optimization in stringdex
There are half a dozen low-level optimizations that I still need to explore. I haven't done them yet, because I've been working on bug fixes and rebasing on rustdoc's side, and a more solid and diverse test suite for stringdex itself.
- Stringdex decides whether to bundle two nodes into the same file based on size. To figure out a node's size, I have to run compression on it. This is probably slower than it needs to be.
- Stack compression is limited to the same 256-slot sliding windows as backref compression, and it doesn't have to be. (stack and backref compression are used to optimize the representation of the edge pointer from a parent node to its child; backref uses one byte, while stack is entirely implicit)
- The JS-side decoder is pretty naive. It performs unnecessary hash table lookups when decoding compressed nodes, and retains a list of hashes that it doesn't need. It needs to calculate the hashes in order to construct the merkle tree correctly, but it doesn't need to keep them.
- Data compression happens at the end, while emitting the node. This means it's not being counted when deciding on how to bundle, which is pretty dumb.
### Improved recall in type-driven search
Right now, type-driven search performs very strict matching. It's very precise, but misses a lot of things people would want.
What I'm not sure about is whether to focus more on edit-distance-based approaches, or to focus on type-theoretical approaches. Both gives avenues to improve, but edit distance is going to be faster while type checking is going to be more precise.
For example, a type theoretical improvement would fix [`Iterator<T>, (T -> U) -> Iterator<U>`](https://doc.rust-lang.org/nightly/std/?search=Iterator%3CT%3E%2C%20(T%20-%3E%20U)%20-%3E%20Iterator%3CU%3E) to give [`Iterator::map`](https://doc.rust-lang.org/nightly/std/iter/trait.Iterator.html#method.map), because it would recognize that the Map struct implements the Iterator trait. I don't know of any clean way to get this result to work without implementing significant type checking logic in search.js, and an edit-distance-based "dirty" approach would likely give a bunch of other results on top of this one.
## Full-text search
Once you've got this fuzzy dictionary matching to work, the logical next step is to implement some kind of information retrieval-based approach to phrase matching.
Like applying edit distance to types, phrase search gets you significantly better recall, but with a few major drawbacks:
- You have to pick between index bloat and the use of stopwords. Stopwords are bad because they might actually be important (try searching "if let" in mdBook if you're feeling brave), but without them, you spend a lot of space on text that doesn't matter.
- Example code also tends to have a lot of irrelevant stuff in it. Like stop words, we'd have to pick potentially-confusing or bloat.
Neither of these problems are deal-breakers, but they're worth keeping in mind.
|
|
|
|
When a UI test runs a compiled binary and an error/forbid pattern
check fails, the failure message previously only showed compiler output,
hiding the executed programs stdout/stderr. This makes it harder to
see near-miss or unexpected runtime lines.
|
|
|
|
|
|
Simplify polonius location-sensitive analysis
This PR reworks the location-sensitive analysis into what we think is a worthwhile subset of the datalog analysis. A sort of polonius alpha analysis that handles NLL problem case 3 and more, but is still using the faster "reachability as an approximation of liveness", as well as the same loans-in-scope computation as NLLs -- and thus doesn't handle full flow-sensitivity like the datalog implementation.
In the last few months, we've identified this subset as being actionable:
- we believe we can make a stabilizable version of this analysis
- it is an improvement over the status quo
- it can also be modeled in a-mir-formality, or some other formalism, for assurances about soundness, and I believe ````````@nikomatsakis```````` is interested in looking into this during H2.
- and we've identified the areas of work we wish to explore later to gradually expand the supported cases: the differences between reachability and liveness, support of kills, and considerations of time-traveling, for example.
The approach in this PR is to try less to have the graph only represent live paths, by checking whether we reach a live region during traversal and recording the loan as live there, instead of equating traversal with liveness like today because it has subtleties with the typeck edges in statements (that could forward loans to the successor point without ensuring their liveness). We can then also simplify these typeck stmt edges. And we also can simplify traversal by removing looking at kills, because that's enough to handle a bunch of NLL problem 3 cases -- and we can gradually support them more and more in traversal in the future, to reduce the approximation of liveness.
There's still some in-progress pieces of work w/r/t opaque types that I'm expecting [lcnr's opaque types rework](https://github.com/rust-lang/rust/pull/139587), and [amanda's SCCs rework](https://github.com/rust-lang/rust/pull/130227) to handle. That didn't seem to show up in tests until I rebased today (and shows lack of test coverage once again) when https://github.com/rust-lang/rust/pull/142255 introduced a couple of test failures with the new captures rules from edition 2024. It's not unexpected since we know more work is needed with member constraints (and we're not even using SCCs in this prototype yet)
I'll look into these anyways, both for future work, and checking how these other 2 PRs would change things.
---
I'm not sure the following means a lot until we have some formalism in-place, but:
- I've changed the polonius compare-mode to use this analysis: the tests pass with it, except 2 cases with minor diagnostics differences, and the 2 edition 2024 opaque types one I mentioned above and need to investigate
- things that are expected to work still do work: it bootstraps, can run our rustc-perf benchmarks (and the results are not even that bad), and a crater run didn't find any regressions (forgetting that crater currently fails to test around a quarter of all crates 👼)
- I've added tests with improvements, like the NLL problem case 3 and others, as well as some that behave the same as NLLs today and are thus worse than the datalog implementation
r? ````````@jackh726````````
(no rush I know you're deep in phd work and "implmentating" the new trait solver for r-a :p <3)
This also fixes rust-lang/rust#135646, a diagnostics ICE from the previous implementation.
|
|
|
|
|
|
Revert "Preserve the .debug_gdb_scripts section"
https://github.com/rust-lang/rust/pull/143679 introduces a significant build time perf regression for ripgrep. Let's revert it such that we can investigate it without pressure.
|
|
So we don't need to add normalization to every test that includes a
panic message, add a global normalization to compiletest.
|
|
This reverts commit b4d923cea0509933b1fb859930cb20784251f9be.
|
|
Rollup of 12 pull requests
Successful merges:
- rust-lang/rust#144552 (Rehome 33 `tests/ui/issues/` tests to other subdirectories under `tests/ui/`)
- rust-lang/rust#144676 (Add documentation for unstable_feature_bound)
- rust-lang/rust#144836 (Change visibility of Args new function)
- rust-lang/rust#144910 (Add regression tests for seemingly fixed issues)
- rust-lang/rust#144913 ([rustdoc] Fix wrong `i` tooltip icon)
- rust-lang/rust#144924 (compiletest: add hint for when a ui test produces no errors)
- rust-lang/rust#144926 (Correct the use of `must_use` on btree::IterMut)
- rust-lang/rust#144928 (Drop `rust-version` from `rustc_thread_pool`)
- rust-lang/rust#144945 (Autolabel PRs that change explicit tail call tests as `F-explicit_tail_calls`)
- rust-lang/rust#144954 (run-make: Allow blessing snapshot files that don't exist yet)
- rust-lang/rust#144971 (num: Rename `isolate_most_least_significant_one` functions)
- rust-lang/rust#144978 (Fix some doc links for intrinsics)
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
Instead of collecting pretty printers transitively when building
executables/staticlibs/cdylibs, let the debugger find each crate's
pretty printers via its .debug_gdb_scripts section. This covers the case
where libraries defining custom pretty printers are loaded dynamically.
|
|
|
|
For "stage 1" ui-fulldeps, use the stage 1 compiler to query target info
Testing ui-fulldeps in "stage 1" actually uses the stage 0 compiler, so that test programs can link against stage 1 rustc crates.
Unfortunately, using the stage 0 compiler causes problems when compiletest tries to obtain target information from the compiler, but the output format has changed since the last bootstrap beta bump.
We can work around this by also providing compiletest with a stage 1 compiler, and having it use that compiler to query for target information.
---
This fixes the stage 1 ui-fulldeps failure seen at https://github.com/rust-lang/rust/pull/144443#issuecomment-3146992771.
|
|
compiletest: Preliminary cleanup of `ProcRes` printing/unwinding
While experimenting with changes to how compiletest handles output capture, error reporting, and unwinding, I repeatedly ran in to difficulties with this core code for reporting test failures caused by a subprocess.
There should be no change in compiletest output.
r? jieyouxu
|
|
r=jieyouxu
Remove the omit_gdb_pretty_printer_section attribute
Disabling loading of pretty printers in the debugger itself is more reliable. Before this commit the .gdb_debug_scripts section couldn't be included in dylibs or rlibs as otherwise there is no way to disable the section anymore without recompiling the entire standard library.
|
|
Testing ui-fulldeps in "stage 1" actually uses the stage 0 compiler, so that
test programs can link against stage 1 rustc crates.
Unfortunately, using the stage 0 compiler causes problems when compiletest
tries to obtain target information from the compiler, but the output format has
changed since the last bootstrap beta bump.
We can work around this by also providing compiletest with a stage 1 compiler,
and having it use that compiler to query for target information.
|
|
This reduces the amount of "hidden" printing in error-reporting code, which
will be helpful when overhauling compiletest's error handling and output
capture.
|
