| Age | Commit message (Collapse) | Author | Lines |
|---|
|
x.py: run `rustup toolchain link` in setup
Addresses #89206
r? ``@jyn514``
|
|
Libgccjit codegen
This PR introduces a subtree for a gcc-based codegen backend to the repository, per decision in https://github.com/rust-lang/compiler-team/issues/442. We do not yet expect to ship this backend on nightly or run tests in CI, but we do verify that the backend checks (i.e., `cargo check`) successfully.
Work is expected to progress primarily in https://github.com/rust-lang/rustc_codegen_gcc, with semi-regular upstreaming, like with other subtrees.
|
|
r=Mark-Simulacrum
Simplify explicit request check and allow to run "doc src/librustdoc" even without config set
Originally I wanted to allow the command `doc src/librustdoc` to work when passed explicitly but then `@Mark-Simulacrum` recommended me to generalize it, so here we are!
r? `@Mark-Simulacrum`
|
|
Fixed types
Add checks for rustup and if toolchain is linked
Fortified rustup/directory checks; made other suggested changes
Added check for output status
Remove output of rustup from console
Made suggested change
Deleted confusing comment
Fixed compiler error; removed extra declaration
Refactored to smaller components; made suggested changes
Automate toolchain linking for stage 1 builds
|
|
|
|
|
|
Rollup of 12 pull requests
Successful merges:
- #88795 (Print a note if a character literal contains a variation selector)
- #89015 (core::ascii::escape_default: reduce struct size)
- #89078 (Cleanup: Remove needless reference in ParentHirIterator)
- #89086 (Stabilize `Iterator::map_while`)
- #89096 ([bootstrap] Improve the error message when `ninja` is not found to link to installation instructions)
- #89113 (dont `.ensure()` the `thir_abstract_const` query call in `mir_build`)
- #89114 (Fixes a technicality regarding the size of C's `char` type)
- #89115 (:arrow_up: rust-analyzer)
- #89126 (Fix ICE when `indirect_structural_match` is allowed)
- #89141 (Impl `Error` for `FromSecsError` without foreign type)
- #89142 (Fix match for placeholder region)
- #89147 (add case for checking const refs in check_const_value_eq)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
[bootstrap] Improve the error message when `ninja` is not found to link to installation instructions
fixes #89091
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
|
|
|
|
These were left over in migrations to subtrees, which should generally be treated
as-if it was local.
Also fixes a warning caused by this change.
|
|
Add initial support for m68k
This patch series adds initial support for m68k making use of the new M68k
backend introduced with LLVM-13. Additional changes will be needed to be
able to actually use the backend for this target.
|
|
|
|
installation instructions.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
|
|
Improve build command for compiler docs
It was rather complicated to document rustc crates. With this, you can directly run:
```console
x.py doc compiler
x.py doc compiler/rustc_hir_pretty
```
The second commit adds the handling of the `--open` flag.
r? `@Mark-Simulacrum`
|
|
|
|
|
|
|
|
|
|
|
|
Fix jump def background
Fixes #88870.
I somehow badly wrote the color in #88111.
r? ``@camelid``
|
|
* Add test for jump-to-def links background color
|
|
|
|
Pin bootstrap checksums and add a tool to update it automatically
:warning: :warning: This is just a proactive hardening we're performing on the build system, and it's not prompted by any known compromise. If you're aware of security issues being exploited please [check out our responsible disclosure page](https://www.rust-lang.org/policies/security). :warning: :warning:
---
This PR aims to improve Rust's supply chain security by pinning the checksums of the bootstrap compiler downloaded by `x.py`, preventing a compromised `static.rust-lang.org` from affecting building the compiler. The checksums are stored in `src/stage0.json`, which replaces `src/stage0.txt`. This PR also adds a tool to automatically update the bootstrap compiler.
The changes in this PR were originally discussed in [Zulip](https://zulip-archive.rust-lang.org/stream/241545-t-release/topic/pinning.20stage0.20hashes.html).
## Potential attack
Before this PR, an attacker who wanted to compromise the bootstrap compiler would "just" need to:
1. Gain write access to `static.rust-lang.org`, either by compromising DNS or the underlying storage.
2. Upload compromised binaries and corresponding `.sha256` files to `static.rust-lang.org`.
There is no signature verification in `x.py` as we don't want the build system to depend on GPG. Also, since the checksums were not pinned inside the repository, they were downloaded from `static.rust-lang.org` too: this only protected from accidental changes in `static.rust-lang.org` that didn't change the `*.sha256` files. The attack would allow the attacker to compromise past and future invocations of `x.py`.
## Mitigations introduced in this PR
This PR adds pinned checksums for all the bootstrap components in `src/stage0.json` instead of downloading the checksums from `static.rust-lang.org`. This changes the attack scenario to:
1. Gain write access to `static.rust-lang.org`, either by compromising DNS or the underlying storage.
2. Upload compromised binaries to `static.rust-lang.org`.
3. Land a (reviewed) change in the `rust-lang/rust` repository changing the pinned hashes.
Even with a successful attack, existing clones of the Rust repository won't be affected, and once the attack is detected reverting the pinned hashes changes should be enough to be protected from the attack. This also enables further mitigations to be implemented in following PRs, such as verifying signatures when pinning new checksums (removing the trust on first use aspect of this PR) and adding a check in CI making sure a PR updating the checksum has not been tampered with (see the future improvements section).
## Additional changes
There are additional changes implemented in this PR to enable the mitigation:
* The `src/stage0.txt` file has been replaced with `src/stage0.json`. The reasoning for the change is that there is existing tooling to read and manipulate JSON files compared to the custom format we were using before, and the slight challenge of manually editing JSON files (no comments, no trailing commas) are not a problem thanks to the new `bump-stage0`.
* A new tool has been added to the repository, `bump-stage0`. When invoked, the tool automatically calculates which release should be used as the bootstrap compiler given the current version and channel, gathers all the relevant checksums and updates `src/stage0.json`. The tool can be invoked by running:
```
./x.py run src/tools/bump-stage0
```
* Support for downloading releases from `https://dev-static.rust-lang.org` has been removed, as it's not possible to verify checksums there (it's customary to replace existing artifacts there if a rebuild is warranted). This will require a change to the release process to avoid bumping the bootstrap compiler on beta before the stable release.
## Future improvements
* Add signature verification as part of `bump-stage0`, which would require the attacker to also obtain the release signing keys in order to successfully compromise the bootstrap compiler. This would be fine to add now, as the burden of installing the tool to verify signatures would only be placed on whoever updates the bootstrap compiler, instead of everyone compiling Rust.
* Add a check on CI that ensures the checksums in `src/stage0.json` are the expected ones. If a PR changes the stage0 file CI should also run the `bump-stage0` tool and fail if the output in CI doesn't match the committed file. This prevents the PR author from tweaking the output of the tool manually, which would otherwise be close to impossible for a human to detect.
* Automate creating the PRs bumping the bootstrap compiler, by setting up a scheduled job in GitHub Actions that runs the tool and opens a PR.
* Investigate whether a similar mitigation can be done for "download from CI" components like the prebuilt LLVM.
r? `@Mark-Simulacrum`
|
|
x.py clippy: don't run with --all-targets by default
this caused a lot of noise because benchmarks and tests were also checked
before:
`./x.py clippy |& wc -l`
`74026`
with change:
`./x.py clippy |& wc -l`
`43269`
Zulip thread: https://rust-lang.zulipchat.com/#narrow/stream/122651-general/topic/x.2Epy.20check.20default.20targets
r? `@jyn514` or `@Mark-Simulacrum`
|
|
Make sure FileCheck is copied in the LLVM output directory
The tool, which is needed by parts of our test suite, is built as part of LLVM but is *not* copied to the directory containing the output LLVM binaries. This adds a flag to ensure the binary is copied. This shouldn't add any extra built time, as the flag just installs extra binaries that were already compiled.
This is not strictly needed for the test suite to work (as it also checks `build/$target/llvm/build/bin` for the binary), but it allows deleting the `build/$TARGET/llvm/build` directory (which also contains the intermediary build artifacts) without affecting the test suite, saving disk space.
|
|
sunos systems add sanitizer supported.
|
|
this caused a lot of noise because benchmarks and tests were also checked
|
|
|
|
Broken in #85600
|
|
Make `-Z gcc-ld=lld` work for Apple targets
`-Z gcc-ld=lld` was introduced in #85961. It does not work on Macos because lld needs be either named `ld64` or passed `-flavor darwin` as the first two arguments in order to select the Mach-O flavor. Rust invokes cc (=clang) on Macos for linking which calls `ld` as linker binary and not `ld64`, so just creating an `ld64` binary and modifying the search path with `-B` does not work.
In order to solve this patch does:
* Set the `lld_flavor` for all Apple-derived targets to `LldFlavor::Ld64`. As far as I can see this actually works towards fixing `-Xlinker=rust-lld` as all those targets use the Mach-O object format.
* Copy/hardlink rust-lld to the gcc-ld subdirectory as ld64 next to ld.
* If `-Z gcc-ld=lld` is used and the target lld flavor is Ld64 add `-fuse-ld=/path/to/ld64` to the linker invocation.
Fixes #86945.
|
|
|
|
The tool, which is needed by parts of our test suite, is built as part
of LLVM but is *not* copied to the directory containing the output LLVM
binaries. This adds a flag to ensure the binary is copied.
This shouldn't add any extra built time, as the flag just installs extra
binaries that were already compiled.
|
|
|
|
|
|
PGO for LLVM builds on x86_64-unknown-linux-gnu in CI
This shows up to 6% less instruction counts with larger - up to 18% - wins on cycles
on multiple benchmarks, and up to 19% wins on the -j1 wall times for rustc self-compilation.
We can afford to spend the extra cycles building LLVM essentially once more for
the x86_64-unknown-linux-gnu CI build today. The builder finishes in around 50
minutes on average, and this adds just 10 more minutes. Given the sizeable
improvements in compiler performance, this is definitely worth it.
|
|
This will ensure the optimized LLVM is used for local builds after rebasing
immediately, rather than needing to wait for a future LLVM bump.
|
|
bootstrap.py: recognize riscv64 when auto-detect
The architecture auto-detect table has no entry for riscv64 (which rustc
uses riscv64gc for the first part of triplet, assuming it's a generic
Linux distro).
Add it to the table to allow riscv64 systems to bootstrap Rust.
Signed-off-by: Icenowy Zheng <icenowy@aosc.io>
|
|
This shows up to 5% less instruction counts on multiple benchmarks, and up to
19% wins on the -j1 wall times for rustc self-compilation.
We can afford to spend the extra cycles building LLVM essentially once more for
the x86_64-unknown-linux-gnu CI build today. The builder finishes in around 50
minutes on average, and this adds just 10 more minutes. Given the sizeable
improvements in compiler performance, this is definitely worth it.
|
|
|
|
r=Mark-Simulacrum
Allow the iOS toolchain to be built on Linux
The iOS toolchain can be built on Linux with minor changes. The compilation will invoke `xcrun` to find the path to the iPhone SDK but a fake `xcrun` executable can be used.
```
#!/bin/sh
echo "/path/to/sdk"
```
The iOS toolchain can then be built and linked with rustup.
```
$ ./x.py build --stage 2 --host x86_64-unknown-linux-gnu \
--target aarch64-apple-ios
$ rustup toolchain link stage1 build/x86_64-unknown-linux-gnu/stage1
```
It's possible to take this toolchain and compile an iOS executable with it. This requires the ld64 linker and an iOS SDK. The ld64 linker can be taken from [cctools](https://github.com/tpoechtrager/cctools-port). A project's .cargo/config can then be edited to use the linker for this target.
```
[target.aarch64-apple-ios]
linker = "/path/to/cctools/bin/arm-apple-darwin-ld"
rustflags = [
"-C",
"""
link-args=
-F/path/to/sdk/System/Library/Frameworks
-L/path/to/sdk/usr/lib
-L/path/to/sdk/usr/lib/system/
-adhoc_codesign
""",
]
```
|
|
Upgrade to LLVM 13
Work in progress update to LLVM 13. Main changes:
* InlineAsm diagnostics reported using SrcMgr diagnostic kind are now handled. Previously these used a separate diag handler.
* Codegen tests are updated for additional attributes.
* Some data layouts have changed.
* Switch `#[used]` attribute from `llvm.used` to `llvm.compiler.used` to avoid SHF_GNU_RETAIN flag introduced in https://reviews.llvm.org/D97448, which appears to trigger a bug in older versions of gold.
* Set `LLVM_INCLUDE_TESTS=OFF` to avoid Python 3.6 requirement.
Upstream issues:
* ~~https://bugs.llvm.org/show_bug.cgi?id=51210 (InlineAsm diagnostic reporting for module asm)~~ Fixed by https://github.com/llvm/llvm-project/commit/1558bb80c01b695ce12642527cbfccf16cf54ece.
* ~~https://bugs.llvm.org/show_bug.cgi?id=51476 (Miscompile on AArch64 due to incorrect comparison elimination)~~ Fixed by https://github.com/llvm/llvm-project/commit/81b106584f2baf33e09be2362c35c1bf2f6bfe94.
* https://bugs.llvm.org/show_bug.cgi?id=51207 (Can't set custom section flags anymore). Problematic change reverted in our fork, https://reviews.llvm.org/D107216 posted for upstream revert.
* https://bugs.llvm.org/show_bug.cgi?id=51211 (Regression in codegen for #83623). This is an optimization regression that we may likely have to eat for this release. The fix for #83623 was based on an incorrect premise, and this needs to be properly addressed in the MergeICmps pass.
The [compile-time impact](https://perf.rust-lang.org/compare.html?start=ef9549b6c0efb7525c9b012148689c8d070f9bc0&end=0983094463497eec22d550dad25576a894687002) is mixed, but quite positive as LLVM upgrades go.
The LLVM 13 final release is scheduled for Sep 21st. The current nightly is scheduled for stable release on Oct 21st.
r? `@ghost`
|
|
Fix compiling other codegen backends when llvm is enabled
Extracted from #81746
Without this change rustbuild will not pass the required linker argument to find libllvm. While other backends likely don't use libllvm, it is necessary to be able to link against rustc_driver as the llvm backend is linked into it.
|
|
The architecture auto-detect table has no entry for riscv64 (which rustc
uses riscv64gc for the first part of triplet, assuming it's a generic
Linux distro).
Add it to the table to allow riscv64 systems to bootstrap Rust.
Signed-off-by: Icenowy Zheng <icenowy@aosc.io>
|
|
Take into account jobs number for rustdoc GUI tests
Fixes #88054.
r? `@Mark-Simulacrum`
|
|
Document private items for rustfmt
This is possible now that https://github.com/rust-lang/rust/pull/73936 has been merged.
|
|
|
|
When LLVM_INCLUDE_TESTS is enabled (by default), LLVM requires
Python 3.6 for the lit test runner, otherwise only Python 3.0 is
required.
As we have many docker images using Ubuntu 16.04, which only has
Python 3.5, this avoids the need to install a newer Python version
for them.
|
|
The iOS toolchain can be built on Linux with minor changes. The
compilation will invoke `xcrun` to find the path to the iPhone SDK but
a fake `xcrun` executable can be used.
```
#!/bin/sh
echo "/path/to/sdk"
```
The iOS toolchain can then be built and linked with rustup.
```
$ ./x.py build --stage 2 --host x86_64-unknown-linux-gnu \
--target aarch64-apple-ios
$ rustup toolchain link stage1 build/x86_64-unknown-linux-gnu/stage1
```
It's possible to take this toolchain and compile an iOS executable
with it. This requires the ld64 linker and an iOS SDK. The ld64 linker
can be taken from
[cctools](https://github.com/tpoechtrager/cctools-port). A project's
.cargo/config can then be edited to use the linker for this target.
```
[target.aarch64-apple-ios]
linker = "/path/to/cctools/bin/arm-apple-darwin-ld"
rustflags = [
"-C",
"""
link-args=
-F/path/to/sdk/System/Library/Frameworks
-L/path/to/sdk/usr/lib
-L/path/to/sdk/usr/lib/system/
-adhoc_codesign
""",
]
```
|
|
This is possible now that rustdoc allows passing
`--document-private-items` more than once.
|
|
Now that Cargo deduplicates diagnostics from different targets, this doesn't flood the console with
duplicate errors.
Note that this doesn't add `--all-targets` in `Builder::cargo` directly because `impl Step for Std`
actually wants to omit `--all-targets` the first time while it's still building libtest.
When passed `--all-targets`, this warns that the option isn't needed, but still continues to compile.
|
