| Age | Commit message (Collapse) | Author | Lines |
|---|
|
Rollup of 7 pull requests
Successful merges:
- #109526 (LIBPATH is used as dylib's path environment variable on AIX)
- #109642 (check for missing codegen backeng config)
- #109722 (Implement read_buf for RustHermit)
- #109856 (fix(middle): emit error rather than delay bug when reaching limit)
- #109868 (Improve PR job names in Github Actions preview)
- #109871 (Include invocation start times)
- #109873 (Move some UI tests into subdirectories)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
Include invocation start times
For multi-invocation builders (e.g., dist-x86_64-linux) this timestamp is necessary to correlate the data in the metrics JSON with other data sources (e.g., logs, cpu-usage CSV, etc.). Such correlation may not be perfect but is sometimes helpful and awkward to do otherwise.
|
|
lenko-d:rust_codegen-backends_interacts_confusingly_with_paths, r=Mark-Simulacrum
check for missing codegen backeng config
Fixes [#109610](https://github.com/rust-lang/rust/issues/109610)
|
|
LIBPATH is used as dylib's path environment variable on AIX
See https://4js.com/online_documentation/fjs-fgl-3.00.05-manual-html/c_fgl_installation_017.html.
|
|
fix `build --stage 2 compiler/rustc` panic
Skip assembling(which causes panic due to not found `.librustc.stamp` file) process
for stage3(since it has problems with sysroot) if full-bootstrap isn't used.
Resolves #90244
|
|
For multi-invocation builders (e.g., dist-x86_64-linux) this timestamp
is necessary to correlate the data in the metrics JSON with other data
sources (e.g., logs, cpu-usage CSV, etc.). Such correlation may not be
perfect but is sometimes helpful and awkward to do otherwise.
|
|
Signed-off-by: ozkanonur <work@onurozkan.dev>
|
|
Replace any existing `build/host` symlink
This has two advantages:
1. If `build.build` changes between runs, the symlink is no longer silently wrong.
2. If the entire build directory is moved, the symlink is no longer broken because it points to the wrong absolute path.
|
|
This has two advantages:
1. If `build.build` changes between runs, the symlink is no longer
silently wrong.
2. If the entire build directory is moved, the symlink is no longer
broken because it points to the wrong absolute path.
|
|
|
|
FreeBSD 13.1 and 13.2 can build Rust with LLD just fine on powerpc.
|
|
Add OpenHarmony targets
- `aarch64-unknown-linux-ohos`
- `armv7-unknown-linux-ohos`
Compiler team MCP: https://github.com/rust-lang/compiler-team/issues/568
|
|
- `aarch64-unknown-linux-ohos`
- `armv7-unknown-linux-ohos`
|
|
socket ancillary data implementation for FreeBSD (from 13 and above).
introducing new build config as well.
|
|
Always set `RUSTC_BOOTSTRAP` with `x doc`
Fixes #100060
Note that there is still a warning - the `unused_allocation` lint does not fire in stage 0, but that's just a matter of waiting for #104363 to land in beta
|
|
|
|
|
|
introducing new build config as well.
|
|
r=Mark-Simulacrum
Rename 'src/bootstrap/native.rs' to llvm.rs
Fixed #108240
Renamed 'native.rs' to 'llvm.rs', also moved `TestHelpers` to `test.rs`. Replaced all the `native.rs` occurrences at `src/bootstrap` files to `llvm.rs`
|
|
- Link to more documentation
- Move `changelog-seen` into the "Global Settings" section
- Update incorrect comments on `llvm.link-shared` and
`rust.debug-assertions`
- Use the correct default in the commented-out example more often
- Clarify that `docs` and `compiler-docs` only control the default,
they're not a hard-off switch.
- Document `-vvv` and `local-rebuild`
- Minor improvements to doc-comments in config.toml.example
This also sets `download-rustc = false`; that was already the default,
but it will be helpful in case the default changes
(https://jyn.dev/2023/01/12/Bootstrapping-Rust-in-2023.html).
|
|
|
|
Permit the MIR inliner to inline diverging functions
This heuristic prevents inlining of `hint::unreachable_unchecked`, which in turn makes `Option/Result::unwrap_unchecked` a bad inlining candidate. I looked through the changes to `core`, `alloc`, `std`, and `hashbrown` by hand and they all seem reasonable. Let's see how this looks in perf...
---
Based on rustc-perf it looks like this regresses ctfe-stress, and the cachegrind diff indicates that this regression is in `InterpCx::statement`. I don't know how to do any deeper analysis because that function is _enormous_ in the try toolchain, which has no debuginfo in it. And a local build produces significantly different codegen for that function, even with LTO.
|
|
r=pietroalbini
Bugfix: avoid panic on invalid json output from libtest
#108659 introduces a custom test display implementation. It does so by using libtest to output json. The stdout is read and parsed; The code trims the line read and checks whether it starts with a `{` and ends with a `}`. If so, it concludes that it must be a json encoded `Message`. Unfortunately, this does not work in all cases:
- This assumes that tests running with `--nocapture` will never start and end lines with `{` and `}` characters
- Output is generated by issuing multiple `write_message` [statements](https://github.com/rust-lang/rust/blob/master/library/test/src/formatters/json.rs#L33-L60). Where only the last one issues a `\n`. This likely results in a race condition as we see multiple json outputs on the same line when running tests for the `x86_64-fortanix-unknown-sgx` target:
```
10:21:04 [0m[0m[1m[32m Running[0m tests/run-time-detect.rs (build/x86_64-unknown-linux-gnu/stage1-std/x86_64-fortanix-unknown-sgx/release/deps/run_time_detect-8c66026bd4b1871a)
10:21:04
10:21:04 running 1 tests
10:21:04 test x86_all ... ok
10:21:04 [0m[0m[1m[32m Running[0m tests/thread.rs (build/x86_64-unknown-linux-gnu/stage1-std/x86_64-fortanix-unknown-sgx/release/deps/thread-ed5456a7d80a6193)
10:21:04 thread 'main' panicked at 'failed to parse libtest json output; error: trailing characters at line 1 column 135, line: "{ \"type\": \"suite\", \"event\": \"ok\", \"passed\": 1, \"failed\": 0, \"ignored\": 0, \"measured\": 0, \"filtered_out\": 0, \"exec_time\": 0.000725911 }{ \"type\": \"suite\", \"event\": \"started\", \"test_count\": 1 }\n"', render_tests.rs:108:25
```
This PR implements a partial fix by being much more conservative of what it asserts is a valid json encoded `Message`. This prevents panics, but still does not resolve the race condition. A discussion is needed where this race condition comes from exactly and how it best can be avoided.
cc: `@jethrogb,` `@pietroalbini`
|
|
Renamed 'native.rs' to 'llvm.rs', also moved `TestHelpers` to `test.rs`.Replaced all the `native.rs` ocurrences at `src/bootstrap` files to `llvm.rs`
|
|
|
|
Rollup of 10 pull requests
Successful merges:
- #109373 (Set LLVM `LLVM_UNREACHABLE_OPTIMIZE` to `OFF`)
- #109392 (Custom MIR: Allow optional RET type annotation)
- #109394 (adapt tests/codegen/vec-shrink-panik for LLVM 17)
- #109412 (rustdoc: Add GUI test for "Auto-hide item contents for large items" setting)
- #109452 (Ignore the vendor directory for tidy tests.)
- #109457 (Remove comment about reusing rib allocations)
- #109461 (rustdoc: remove redundant `.content` prefix from span/a colors)
- #109477 (`HirId` to `LocalDefId` cleanup)
- #109489 (More general captures)
- #109494 (Do not feed param_env for RPITITs impl side)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
Set LLVM `LLVM_UNREACHABLE_OPTIMIZE` to `OFF`
This option was added to LLVM in https://reviews.llvm.org/D121750?id=416339. It makes `llvm_unreachable` in builds without assertions compile to an `LLVM_BUILTIN_TRAP` instead of `LLVM_BUILTIN_UNREACHABLE` (which causes undefined behavior and is equivalent to `std::hint::unreachable_unchecked`).
Having compiler bugs triggering undefined behavior generally seems undesirable and inconsistent with Rust's goals. There is a check in `src/tools/tidy/src/style.rs` to reject code using `llvm_unreachable`. But it is used a lot within LLVM itself.
For instance, this changes a failure I get compiling `libcore` for m68k from a `SIGSEGV` to `SIGILL`, which seems better though it still doesn't provide a useful message without switching to an LLVM build with asserts.
It may be best not to do this if it noticeably degrades compiler performance, but worthwhile if it doesn't do so in any significant way. I haven't looked into what benchmarks there are for Rustc. That should be considered before merging.
|
|
refactor `fn bootstrap::builder::Builder::compiler_for` logic
- check compiler stage before forcing for stage2.
- check if download_rustc is not set before forcing for stage1.
resolves #109286
|
|
|
|
This option was added to LLVM in
https://reviews.llvm.org/D121750?id=416339. It makes `llvm_unreachable`
in builds without assertions compile to an `LLVM_BUILTIN_TRAP` instead
of `LLVM_BUILTIN_UNREACHABLE` (which causes undefined behavior and is
equivalent to `std::hint::unreachable_unchecked`).
Having compiler bugs triggering undefined behavior generally seems
undesirable and inconsistent with Rust's goals. There is a check in
`src/tools/tidy/src/style.rs` to reject code using `llvm_unreachable`.
But it is used a lot within LLVM itself.
For instance, this changes a failure I get compiling `libcore` for m68k
from a `SIGSEGV` to `SIGILL`, which seems better though it still doesn't
provide a useful message without switching to an LLVM build with asserts.
It may be best not to do this if it noticeably degrades compiler
performance, but worthwhile if it doesn't do so in any significant way. I
haven't looked into what benchmarks there are for Rustc. That should be
considered before merging.
|
|
Include executed tests in the build metrics (and use a custom test display impl)
The main goal of this PR is to include all tests executed in CI inside the build metrics JSON files. I need this for Ferrocene, and `@Mark-Simulacrum` expressed desire to have this as well to ensure all tests are executed at least once somewhere in CI.
Unfortunately implementing this required rewriting inside of bootstrap all of the code to render the test output to console. libtest supports outputting JSON instead of raw text, which we can indeed use to populate the build metrics. Doing that suppresses the console output though, and compared to rustc and Cargo the console output is not included as a JSON field.
Because of that, this PR had to reimplement both the "pretty" format (one test per line, with `rust.verbose-tests = true`), and the "terse" format (the wall of dots, with `rust.verbose-tests = false`). The current implementation should have the exact same output as libtest, except for the benchmark output. libtest's benchmark output is broken in the "terse" format, so since that's our default I slightly improved how it's rendered.
Also, to bring parity with libtest I had to introduce support for coloring output from bootstrap, using the same dependencies `annotate-snippets` uses. It's now possible to use `builder.color_for_stdout(Color::Red, "text")` and `builder.color_for_stderr(Color::Green, "text")` across all of bootstrap, automatically respecting the `--color` flag and whether the stream is a terminal or not.
I recommend reviewing the PR commit-by-commit.
r? `@Mark-Simulacrum`
|
|
|
|
Add `dist.compression-profile` option to control compression speed
PR #108534 reduced the size of compressed archives, but (as expected) it also resulted in way longer compression times and memory usage during compression.
It's desirable to keep status quo (smaller archives but more CI usage), but it should also be configurable so that downstream users don't have to waste that much time on CI. As a data point, this resulted in doubling the time of Ferrocene's dist jobs, and required us to increase the RAM allocation for one of such jobs.
This PR adds a new `config.toml` setting, `dist.compression-profile`. The values can be:
* `fast`: equivalent to the gzip and xz preset of "1"
* `balanced`: equivalent to the gzip and xz preset of "6" (the CLI defaults as far as I'm aware)
* `best`: equivalent to the gzip present of "9", and our custom xz profile
The default has also been moved back to `balanced`, to try and avoid the compression time regression for downstream users. I don't feel too strongly on the default, and I'm open to changing it.
Also, for the `best` profile the XZ settings do not match the "9" preset used by the CLI, and it might be confusing. Should we create a `custom-rustc-ci`/`ultra` profile for that?
r? ``@Mark-Simulacrum``
|
|
|
|
Distribute libntdll.a with windows-gnu toolchains
This allows the OS loader to load essential functions (e.g. read/write file) at load time instead of lazily doing so at runtime.
r? libs
|
|
migrate compiler, bootstrap and compiletest to windows-rs
This PR migrates the compiler, bootstrap, and compiletest to use [windows-rs](https://github.com/microsoft/windows-rs) instead of winapi-rs. windows-rs is the bindings crate provided by Microsoft, and is actively maintained compared to winapi-rs. Not all ecosystem crates have migrated over yet, so there will be a period of time where both crates are used.
windows-rs also provides some nice ergonomics over winapi-rs to convert return values to `Result`s (which found a case where we forgot to check the return value of `CreateFileW`).
|
|
|
|
Add tests for configure.py
I highly recommend reviewing this with whitespace disabled.
Notably, verifying that we generate valid toml relies on python 3.11 so
we can use `tomllib`, so this also switches`x86_64-gnu-llvm-14` (one of the PR builders) to use 3.11.
While fixing that, I noticed that we stopped testing python2.7 support on PR CI in https://github.com/rust-lang/rust/pull/106085. `@fee1-dead` `@pietroalbini` please be more careful in the future, there is no CI for CI itself that verifies we are testing everything we should be.
- Separate out functions so that each unit test doesn't create a file on disk
- Add a few unit tests
|
|
Set `CMAKE_SYSTEM_NAME` for Linux targets
When bootstrap compiles native dependencies like LLVM, it should set `CMAKE_SYSTEM_NAME` for the target system; otherwise cmake may not identify that it is cross-compiling.
In particular, when building a Linux rustc on a macOS host, cmake was including `-isysroot /path/to/macOS.sdk` options that caused things to break. By setting `CMAKE_SYSTEM_NAME=Linux` when building for Linux targets, the macOS SDK is no longer passed as sysroot to the compiler.
r? bootstrap
|
|
|
|
Upgrade to LLVM 16
This updates Rust to LLVM 16. It also updates our host compiler for dist-x86_64-linux to LLVM 16. The reason for that is that Bolt from LLVM 15 is not capable of compiling LLVM 16 (https://github.com/llvm/llvm-project/issues/61114).
LLVM 16.0.0 has been [released](https://discourse.llvm.org/t/llvm-16-0-0-release/69326) on March 18, while Rust 1.70 will become stable on June 1.
Tested images: `dist-x86_64-linux`, `dist-riscv64-linux` (alt), `dist-x86_64-illumos`, `dist-various-1`, `dist-various-2`, `dist-powerpc-linux`, `wasm32`, `armhf-gnu`
Tested images until the usual IPv6 failures: `test-various`
|
|
- check compiler stage before forcing for stage2.
- check if download_rustc is not set before forcing for stage1.
Signed-off-by: ozkanonur <work@onurozkan.dev>
|
|
Speed up tidy quite a lot
I highly recommend reviewing this commit-by-commit. Based on #106440 for convenience.
## Timings
These were collected by running `x test tidy -v` to copy paste the command, then using [`samply record`](https://github.com/mstange/samply).
before (8 threads)
after (8 threads) 
before (64 threads) 
after (64 threads) 
The last commit makes tidy use more threads, so comparing "before (8 threads)" to "after (64 threads)" is IMO the most realistic comparison. Locally, that brings the time for me to run tidy down from 4 to .9 seconds, i.e. the majority of the time for `x test tidy` is now spend running `fmt --check`.
r? `@the8472`
|
|
This has a significant speedup for me locally, from about 1.3 seconds to
.9 seconds.
|
|
Check for llvm-tools before install
Fixes #108948
````@jpalus```` Please review
|
|
This also fixes a regression from
https://github.com/rust-lang/rust/pull/106085 which stopped testing that
we support python2 in PR CI.
|
|
- Separate out functions so that each unit test doesn't create a file on disk
- Add a few unit tests
Notably, verifying that we generate valid toml relies on python 3.11 so
we can use `tomllib`.
|
|
|
|
|
|
Prevent stable `libtest` from supporting `-Zunstable-options`
Took a while for me to get around to this but seems trivial (unless I'm missing some reason this will break all our tests). Fixes #75526
Basically `libtest` already tries to handle this in https://github.com/rust-lang/rust/blob/501ad021b9a4fb2cd6a39e0302d22f169f6166b0/library/test/src/cli.rs#L310-L318
But that env var was not passed. I'm guessing at one point [this code](https://github.com/rust-lang/rust/blob/501ad021b9a4fb2cd6a39e0302d22f169f6166b0/src/bootstrap/compile.rs#L842-L844) (or a common ancestor) was used to compile the standard library/libtest, but that is no longer the case (or perhaps it never worked, I don't have time to go digging).
I don't love that this is a "allow unstable by default" situation, as it means things like [`rustc-build-sysroot`](https://github.com/RalfJung/rustc-build-sysroot) could accidentally get unstable (CC ````@RalfJung)```` even if this is fixed here, but it's consistent with what happens in `rustc_feature`, so... yeah.
This is user-facing after all, even if it's hard to imagine the outcome of that conversation being "lets continue allowing use of `-Zunstable-features` from stable rust" (especially since a `RUSTC_BOOTSTRAP=1`-shaped loophole remains)... I think it probably should get a vibe check in the t-libs meeting (and plausibly a relnote along the lines of "hey `cargo test -- -Zunstable-options --some --unstable --stuff=here` used to work on stable, that's been fixed, sorry").
I'll nominate it for that after CI comes up green (I've done a smoke check but don't know what (if anything) will need `bootstrap` to enable `RUSTC_BOOTSTRAP=1` when running tests)
r? ````@jyn514````
|
