| Age | Commit message (Collapse) | Author | Lines |
|---|
|
bootstrap: don't use `--merges` to look for commit hashes for downloading artifacts
Shallow clones (and possibly worktrees, though I can't seem to reproduce the problem there) can cause `git rev-list --merges` to falsely return no results, even if a merge commit is present. Stop using the `--merges` option when looking for commit hashes that have build artifacts. `--first-parent` and `--author=bors@rust-lang.org` should be sufficient.
Also exit with an error if the configuration asks for artifacts to be downloaded and we can't determine an appropriate commit hash to use to download artifacts.
Fixes #87890.
r? ``@jyn514``
``@rustbot`` label +A-rustbuild +A-contributor-roadblock
|
|
r=Mark-Simulacrum
Don't rebuild GUI test crates every time you run test src/test/rustdoc-gui
This method has multiple advantages:
* It'll completely remove the rustdoc-GUI test doc folder if rustdoc was updated
* It'll rebuild GUI test crates only they have been updated
All in all, it's quite convenient! (even more with https://github.com/rust-lang/rust/pull/88816)
r? ```@Mark-Simulacrum```
|
|
Remove unused clippy bootstrap env vars
Continues rust-lang/rust-clippy#7646
|
|
Exit with an error if we can't find a commit hash for downloading
LLVM or rustc snapshots.
|
|
Shallow clones can cause `git rev-list --merges` to miss merge
commits. Omit it, because the most recent bors commit is
almost always a merge commit.
|
|
|
|
The wrapper is installed as `ld` and `ld64` in the `lib\rustlib\<host_target>\bin\gcc-ld`
directory and its sole purpose is to invoke `rust-lld` in the parent directory with
the correct flavor.
|
|
|
|
On NixOS systems, bootstrap will patch rustc used in bootstrapping after
checking `/etc/os-release` (to confirm the current distribution is NixOS).
However, when using Nix on a non-NixOS system, it can be desirable for
bootstrap to patch rustc. In this commit, a `patch-binaries-for-nix`
option is added to `config.toml`, which allows for user opt-in to
bootstrap's Nix patching.
Signed-off-by: David Wood <david.wood@huawei.com>
|
|
|
|
Downloading LLVM from CI works for all platforms now.
|
|
x.py: run `rustup toolchain link` in setup
Addresses #89206
r? ``@jyn514``
|
|
Libgccjit codegen
This PR introduces a subtree for a gcc-based codegen backend to the repository, per decision in https://github.com/rust-lang/compiler-team/issues/442. We do not yet expect to ship this backend on nightly or run tests in CI, but we do verify that the backend checks (i.e., `cargo check`) successfully.
Work is expected to progress primarily in https://github.com/rust-lang/rustc_codegen_gcc, with semi-regular upstreaming, like with other subtrees.
|
|
r=Mark-Simulacrum
Simplify explicit request check and allow to run "doc src/librustdoc" even without config set
Originally I wanted to allow the command `doc src/librustdoc` to work when passed explicitly but then `@Mark-Simulacrum` recommended me to generalize it, so here we are!
r? `@Mark-Simulacrum`
|
|
Fixed types
Add checks for rustup and if toolchain is linked
Fortified rustup/directory checks; made other suggested changes
Added check for output status
Remove output of rustup from console
Made suggested change
Deleted confusing comment
Fixed compiler error; removed extra declaration
Refactored to smaller components; made suggested changes
Automate toolchain linking for stage 1 builds
|
|
|
|
|
|
Rollup of 12 pull requests
Successful merges:
- #88795 (Print a note if a character literal contains a variation selector)
- #89015 (core::ascii::escape_default: reduce struct size)
- #89078 (Cleanup: Remove needless reference in ParentHirIterator)
- #89086 (Stabilize `Iterator::map_while`)
- #89096 ([bootstrap] Improve the error message when `ninja` is not found to link to installation instructions)
- #89113 (dont `.ensure()` the `thir_abstract_const` query call in `mir_build`)
- #89114 (Fixes a technicality regarding the size of C's `char` type)
- #89115 (:arrow_up: rust-analyzer)
- #89126 (Fix ICE when `indirect_structural_match` is allowed)
- #89141 (Impl `Error` for `FromSecsError` without foreign type)
- #89142 (Fix match for placeholder region)
- #89147 (add case for checking const refs in check_const_value_eq)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
[bootstrap] Improve the error message when `ninja` is not found to link to installation instructions
fixes #89091
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
|
|
|
|
These were left over in migrations to subtrees, which should generally be treated
as-if it was local.
Also fixes a warning caused by this change.
|
|
Add initial support for m68k
This patch series adds initial support for m68k making use of the new M68k
backend introduced with LLVM-13. Additional changes will be needed to be
able to actually use the backend for this target.
|
|
|
|
installation instructions.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
|
|
Improve build command for compiler docs
It was rather complicated to document rustc crates. With this, you can directly run:
```console
x.py doc compiler
x.py doc compiler/rustc_hir_pretty
```
The second commit adds the handling of the `--open` flag.
r? `@Mark-Simulacrum`
|
|
|
|
|
|
|
|
|
|
|
|
Fix jump def background
Fixes #88870.
I somehow badly wrote the color in #88111.
r? ``@camelid``
|
|
* Add test for jump-to-def links background color
|
|
|
|
Pin bootstrap checksums and add a tool to update it automatically
:warning: :warning: This is just a proactive hardening we're performing on the build system, and it's not prompted by any known compromise. If you're aware of security issues being exploited please [check out our responsible disclosure page](https://www.rust-lang.org/policies/security). :warning: :warning:
---
This PR aims to improve Rust's supply chain security by pinning the checksums of the bootstrap compiler downloaded by `x.py`, preventing a compromised `static.rust-lang.org` from affecting building the compiler. The checksums are stored in `src/stage0.json`, which replaces `src/stage0.txt`. This PR also adds a tool to automatically update the bootstrap compiler.
The changes in this PR were originally discussed in [Zulip](https://zulip-archive.rust-lang.org/stream/241545-t-release/topic/pinning.20stage0.20hashes.html).
## Potential attack
Before this PR, an attacker who wanted to compromise the bootstrap compiler would "just" need to:
1. Gain write access to `static.rust-lang.org`, either by compromising DNS or the underlying storage.
2. Upload compromised binaries and corresponding `.sha256` files to `static.rust-lang.org`.
There is no signature verification in `x.py` as we don't want the build system to depend on GPG. Also, since the checksums were not pinned inside the repository, they were downloaded from `static.rust-lang.org` too: this only protected from accidental changes in `static.rust-lang.org` that didn't change the `*.sha256` files. The attack would allow the attacker to compromise past and future invocations of `x.py`.
## Mitigations introduced in this PR
This PR adds pinned checksums for all the bootstrap components in `src/stage0.json` instead of downloading the checksums from `static.rust-lang.org`. This changes the attack scenario to:
1. Gain write access to `static.rust-lang.org`, either by compromising DNS or the underlying storage.
2. Upload compromised binaries to `static.rust-lang.org`.
3. Land a (reviewed) change in the `rust-lang/rust` repository changing the pinned hashes.
Even with a successful attack, existing clones of the Rust repository won't be affected, and once the attack is detected reverting the pinned hashes changes should be enough to be protected from the attack. This also enables further mitigations to be implemented in following PRs, such as verifying signatures when pinning new checksums (removing the trust on first use aspect of this PR) and adding a check in CI making sure a PR updating the checksum has not been tampered with (see the future improvements section).
## Additional changes
There are additional changes implemented in this PR to enable the mitigation:
* The `src/stage0.txt` file has been replaced with `src/stage0.json`. The reasoning for the change is that there is existing tooling to read and manipulate JSON files compared to the custom format we were using before, and the slight challenge of manually editing JSON files (no comments, no trailing commas) are not a problem thanks to the new `bump-stage0`.
* A new tool has been added to the repository, `bump-stage0`. When invoked, the tool automatically calculates which release should be used as the bootstrap compiler given the current version and channel, gathers all the relevant checksums and updates `src/stage0.json`. The tool can be invoked by running:
```
./x.py run src/tools/bump-stage0
```
* Support for downloading releases from `https://dev-static.rust-lang.org` has been removed, as it's not possible to verify checksums there (it's customary to replace existing artifacts there if a rebuild is warranted). This will require a change to the release process to avoid bumping the bootstrap compiler on beta before the stable release.
## Future improvements
* Add signature verification as part of `bump-stage0`, which would require the attacker to also obtain the release signing keys in order to successfully compromise the bootstrap compiler. This would be fine to add now, as the burden of installing the tool to verify signatures would only be placed on whoever updates the bootstrap compiler, instead of everyone compiling Rust.
* Add a check on CI that ensures the checksums in `src/stage0.json` are the expected ones. If a PR changes the stage0 file CI should also run the `bump-stage0` tool and fail if the output in CI doesn't match the committed file. This prevents the PR author from tweaking the output of the tool manually, which would otherwise be close to impossible for a human to detect.
* Automate creating the PRs bumping the bootstrap compiler, by setting up a scheduled job in GitHub Actions that runs the tool and opens a PR.
* Investigate whether a similar mitigation can be done for "download from CI" components like the prebuilt LLVM.
r? `@Mark-Simulacrum`
|
|
x.py clippy: don't run with --all-targets by default
this caused a lot of noise because benchmarks and tests were also checked
before:
`./x.py clippy |& wc -l`
`74026`
with change:
`./x.py clippy |& wc -l`
`43269`
Zulip thread: https://rust-lang.zulipchat.com/#narrow/stream/122651-general/topic/x.2Epy.20check.20default.20targets
r? `@jyn514` or `@Mark-Simulacrum`
|
|
Make sure FileCheck is copied in the LLVM output directory
The tool, which is needed by parts of our test suite, is built as part of LLVM but is *not* copied to the directory containing the output LLVM binaries. This adds a flag to ensure the binary is copied. This shouldn't add any extra built time, as the flag just installs extra binaries that were already compiled.
This is not strictly needed for the test suite to work (as it also checks `build/$target/llvm/build/bin` for the binary), but it allows deleting the `build/$TARGET/llvm/build` directory (which also contains the intermediary build artifacts) without affecting the test suite, saving disk space.
|
|
sunos systems add sanitizer supported.
|
|
this caused a lot of noise because benchmarks and tests were also checked
|
|
|
|
Broken in #85600
|
|
Make `-Z gcc-ld=lld` work for Apple targets
`-Z gcc-ld=lld` was introduced in #85961. It does not work on Macos because lld needs be either named `ld64` or passed `-flavor darwin` as the first two arguments in order to select the Mach-O flavor. Rust invokes cc (=clang) on Macos for linking which calls `ld` as linker binary and not `ld64`, so just creating an `ld64` binary and modifying the search path with `-B` does not work.
In order to solve this patch does:
* Set the `lld_flavor` for all Apple-derived targets to `LldFlavor::Ld64`. As far as I can see this actually works towards fixing `-Xlinker=rust-lld` as all those targets use the Mach-O object format.
* Copy/hardlink rust-lld to the gcc-ld subdirectory as ld64 next to ld.
* If `-Z gcc-ld=lld` is used and the target lld flavor is Ld64 add `-fuse-ld=/path/to/ld64` to the linker invocation.
Fixes #86945.
|
|
|
|
The tool, which is needed by parts of our test suite, is built as part
of LLVM but is *not* copied to the directory containing the output LLVM
binaries. This adds a flag to ensure the binary is copied.
This shouldn't add any extra built time, as the flag just installs extra
binaries that were already compiled.
|
|
|
|
|
|
PGO for LLVM builds on x86_64-unknown-linux-gnu in CI
This shows up to 6% less instruction counts with larger - up to 18% - wins on cycles
on multiple benchmarks, and up to 19% wins on the -j1 wall times for rustc self-compilation.
We can afford to spend the extra cycles building LLVM essentially once more for
the x86_64-unknown-linux-gnu CI build today. The builder finishes in around 50
minutes on average, and this adds just 10 more minutes. Given the sizeable
improvements in compiler performance, this is definitely worth it.
|
|
This will ensure the optimized LLVM is used for local builds after rebasing
immediately, rather than needing to wait for a future LLVM bump.
|
|
bootstrap.py: recognize riscv64 when auto-detect
The architecture auto-detect table has no entry for riscv64 (which rustc
uses riscv64gc for the first part of triplet, assuming it's a generic
Linux distro).
Add it to the table to allow riscv64 systems to bootstrap Rust.
Signed-off-by: Icenowy Zheng <icenowy@aosc.io>
|
|
This shows up to 5% less instruction counts on multiple benchmarks, and up to
19% wins on the -j1 wall times for rustc self-compilation.
We can afford to spend the extra cycles building LLVM essentially once more for
the x86_64-unknown-linux-gnu CI build today. The builder finishes in around 50
minutes on average, and this adds just 10 more minutes. Given the sizeable
improvements in compiler performance, this is definitely worth it.
|
|
|
