| Age | Commit message (Collapse) | Author | Lines |
|---|
|
Usually, we do want to use the static C++ library when building rustc_llvm, but do not want to have that dependency at compiler runtime. Change the defaults to Make It So.
|
|
|
|
|
|
On stable, our artifacts are uploaded with the raw version number (e.g.,
1.65.0), not the channel. This adjusts our detection logic to use the
version number from src/version when we detect the stable channel.
|
|
This reverts commit 3acb505ee560770c62bad5362f6caf7567d467b9
(PR #101833).
The changes in this commit caused several bugs or at least
incompatibilies. For now we're reverting this commit and will re-land it
alongside fixes for those bugs.
|
|
|
|
|
|
Use BOLT in CI to optimize LLVM
This PR adds an optimization step in the Linux `dist` CI pipeline that uses [BOLT](https://github.com/llvm/llvm-project/tree/main/bolt) to optimize the `libLLVM.so` library built by boostrap.
Steps:
- [x] Use LLVM 15 as a bootstrap compiler and use it to build BOLT
- [x] Compile LLVM with support for relocations (`-DCMAKE_SHARED_LINKER_FLAGS="-Wl,-q"`)
- [x] Gather profile data using instrumented LLVM
- [x] Apply profile to LLVM that has already been PGOfied
- [x] Run with BOLT profiling on more benchmarks
- [x] Decide on the order of optimization (PGO -> BOLT?)
- [x] Decide how we should get `bolt` (currently we use the host `bolt`)
- [x] Clean up
The latest perf results can be found [here](https://github.com/rust-lang/rust/pull/94381#issuecomment-1258269440). The current CI build time with BOLT applied is around 1h 55 minutes.
|
|
|
|
Make the `config.src` handling for downloadable bootstrap more conservative
In particular, this supports build directories within an unrelated git repository. Fixes https://github.com/rust-lang/rust/issues/102562.
As a side effect, it will fall back to the old logic when the source directory is being built from a tarball within an unrelated git repository. However, that second case is unsupported and untested; we reserve the right to break it in the future.
`@cr1901` can you confirm this fixes your problem?
cc `@kleisauke,` I believe this will also fix your issue (although your use case still isn't supported).
r? `@Mark-Simulacrum`
|
|
fix: use git-commit-info for version information
Fixes #33286.
Fixes #86587.
This PR changes the current `git-commit-hash` file that `./x.py` dist puts in the `rustc-{version}-src.tar.{x,g}z` to contain the hash, the short hash, and the commit date from which the tarball was created, assuming git was available when it was. It uses this for reading the version so that rustc has all the appropriate metadata.
# Testing
Testing this is kind of a pain. I did it with something like
```sh
./x.py dist # ensure that `ignore-git` is `false` in config.toml
cp ./build/dist/rustc-1.65.0-dev-src.tar.gz ../rustc-1.65.0-dev-src.tar.gz
cd .. && tar -xzf rustc-1.65.0-dev-src && cd rustc-1.65.0-dev-src
./x.py build
```
Then, the output of `rustc -vV` with the stage1 compiler should have the `commit-hash` and `commit-date` fields filled, rather than be `unknown`. To be completely sure, you can use `rustc --sysroot` with the stdlib that the original `./x.py dist` made, which will require that the metadata matches.
|
|
conservative
In particular, this supports build directories within an unrelated git repository.
As a side effect, it will fall back to the old logic when the source directory is being built from a tarball within an unrelated git repository.
However, that second case is unsupported and untested; we reserve the right to break it in the future.
|
|
This PR adds support for fetching version information from the
`git-commit-info` file when building the compiler from a source tarball.
|
|
r=Mark-Simulacrum
Make the `c` feature for `compiler-builtins` an explicit opt-in
Its build script doesn't support cross-compilation. I tried fixing it, but the cc crate itself doesn't appear to support cross-compiling to windows either unless you use the -gnu toolchain:
```
error occurred: Failed to find tool. Is `lib.exe` installed?
```
Fixes https://github.com/rust-lang/rust/issues/101172.
|
|
The build script for `compiler_builtins` doesn't support cross-compilation. I tried fixing it, but the cc crate itself
doesn't appear to support cross-compiling to windows either unless you use the -gnu toolchain:
```
error occurred: Failed to find tool. Is `lib.exe` installed?
```
Rather than trying to fix it or special-case the platforms without bugs,
make it opt-in instead of automatic.
|
|
This restores functionality broken by #98483. Unfortunately, it doesn't
add a test to verify this works, but in this case we notice pretty
quickly as perf uses this functionality and so reports breakage
immediately after merging.
|
|
Co-authored-by: Joshua Nelson <github@jyn.dev>
|
|
Co-authored-by: Bruno Kolenbrander <59372212+mejrs@users.noreply.github.com>
|
|
built
- Default to trying git rev-parse for the root directory
CARGO_MANIFEST_DIR is a path on the build machine, not the running machine.
Don't require this to succeed, to allow building from a tarball; in that case fall back to CARGO_MANIFEST_DIR.
- Set `initial_rustc` to a path based on the path of the running executable, not CARGO_MANIFEST_DIR.
We only reset `initial_rustc` if we're sure this isn't the working tree bootstrap was originally built in,
since I'm paranoid that setting this in other cases will cause things to break;
it's not clear to me when $RUSTC differs from `build/$TARGET/stage0/bin/rustc` (maybe never? but better to be sure).
Instead, only set this when
a) We are not using a custom rustc. If someone has specified a custom rustc we should respect their wishes.
b) We are in a checkout of rust-lang/rust other than the one bootstrap was built in.
|
|
bootstrap: Add llvm-has-rust-patches target option
This is so you can check out an upstream commit in src/llvm-project and
have everything just work.
This simplifies the logic in `is_rust_llvm` a bit; it doesn't need to
check for download-ci-llvm because we would have already errored if both
that and llvm-config were specified on the host platform.
|
|
This is so you can check out an upstream commit in src/llvm-project and
have everything just work.
|
|
See comment added for details on the test builder restriction. This is primarily
intended for macOS CI, but is likely to be a slight win on other builders too.
|
|
This verifies if the HEAD sha matches with the detected LLVM SHA, and if not,
permits usage of the detected LLVM. Otherwise, we fallback on regular
non-downloaded LLVM (currently still cached with sccache, though that's still
10+ minutes on macOS).
|
|
|
|
and rustc_session""
This reverts commit 1ae4b258267462da0b1aae1badcf83578153c799.
|
|
Removing libunwind from Fuchsia target docs
|
|
When we're downloading based on a CI commit, that can still be -beta- or even
-stable-, so we should lookup the channel it was built with.
|
|
https://github.com/rust-lang/rust/issues/97322
|
|
|
|
|
|
Allow using `download-ci-llvm = true` outside the git checkout
`@bjorn3` noticed that this is already allowed today when download-llvm is disabled, but breaks with it enabled:
```
$ ./rust2/x.py build
fatal: not a git repository (or any of the parent directories): .git
thread 'main' panicked at 'command did not execute successfully: "git" "rev-list" "--author=bors@rust-lang.org" "-n1" "--first-parent" "HEAD" "--" "/home/jnelson/rust-lang/rust2/src/llvm-project" "/home/jnelson/rust-lang/rust2/src/bootstrap/download-ci-llvm-stamp" "/home/jnelson/rust-lang/rust2/src/version"
expected success, got: exit status: 128', src/bootstrap/native.rs:134:20
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
Support it too for consistency. It's unclear to me when anyone would need to use this, but `@bjorn3`
feels we should support it, and it's not much additional effort to get it working.
|
|
@bjorn3 noticed that this is already allowed today when download-llvm is disabled, but breaks with it enabled:
```
$ ./rust2/x.py build
fatal: not a git repository (or any of the parent directories): .git
thread 'main' panicked at 'command did not execute successfully: "git" "rev-list" "--author=bors@rust-lang.org" "-n1" "--first-parent" "HEAD" "--" "/home/jnelson/rust-lang/rust2/src/llvm-project" "/home/jnelson/rust-lang/rust2/src/bootstrap/download-ci-llvm-stamp" "/home/jnelson/rust-lang/rust2/src/version"
expected success, got: exit status: 128', src/bootstrap/native.rs:134:20
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
Support it too for consistency. It's unclear to me when anyone would need to use this, but @bjorn3
feels we should support it, and it's not much additional effort to get it working.
This also updates a bunch of other git commands that were similarly depending on the current directory.
|
|
Fix caching bug in `download-rustc = true`
When moving this to rustbuild, I introduced a bug: if you had the file already downloaded, but
deleted the sysroot for whatever reason, rustbuil would fail to unpack the cached tarball.
This only affects people if they have a cached tarball, which is probably why we haven't seen an issue yet -
wiping `build/cache` would work around the issue, or just not deleting `build/$TARGET/stage2`.
Fixes the following error:
```
thread 'main' panicked at 'fs::read_dir(&lib_dir) failed with No such file or directory (os error 2) ("/home/jnelson/rust-lang/rust2/build/x86_64-unknown-linux-gnu/ci-rustc/lib")', config.rs:1563:20
```
r? ``@Mark-Simulacrum``
|
|
implement detail_exit but I'm not sure it is right.
not create new file and write detail exit in lib.rs
replace std::process::exit to detail_exit
that is not related to code runnning.
remove pub
|
|
When moving this to rustbuild, I introduced a bug: if you had the file already downloaded, but
deleted the sysroot for whatever reason, rustbuil would fail to unpack the cached tarball.
This only affects people if they have a cached tarball, which is probably why we haven't seen an issue yet -
wiping `build/cache` would work around the issue, or just not deleting `build/$TARGET/stage2`.
|
|
|
|
Allow configuring where artifacts are downloaded from
Bootstrap has support for downloading prebuilt LLVM and rustc artifacts to speed up local builds, but that currently works only for users working on `rust-lang/rust`. Forks of the repository (for example Ferrocene) might have different URLs to download artifacts from, or might use a different email address on merge commits, breaking both LLVM and rustc artifact downloads.
This PR refactors bootstrap to load the download URLs and other constants from `src/stage0.json`, allowing downstream forks to tweak those values. It also future-proofs the download code to easily allow forks to add their own custom protocols (like `s3://`).
This PR is best reviewed commit-by-commit.
|
|
|
|
|
|
|
|
rustc_session"
This reverts commit 2d854f9c340df887e30896f49270ae81feb3e227.
|
|
This simplifies the arguments to `download_component` in config.rs.
It also moves stage0.json metadata handling to `Build::new`, making it easier to download the stage0
compiler in rustbuild later if necessary.
|
|
|
|
|
|
|
|
It was confusing to have two functions with the same name but different behavior.
|
|
Add build metrics to rustbuild
This PR adds a new module of rustbuild, `ci_profiler`, whose job is to gather as much information as possible about the CI build as possible and store it in a JSON file uploaded to `ci-artifacts`. Right now for each step it collects:
* Type name and debug representation of the `Step` object.
* Duration of the step (excluding child steps).
* Systemwide CPU stats for the duration of the step (both single core and all cores).
* Which child steps were executed.
This is capable of replacing both the scripts to collect CPU stats and the `[TIMING]` lines in build logs (not yet removed, until we port our tooling to use the CI profiler). The format is also extensible to be able in the future to collect more information.
r? `@Mark-Simulacrum`
|
|
Remove dead code
Missing }
./x.py fmt
Remove duplicate check
Recursively remove all usage of help_on_error
|
|
Move download-rustc from python to rustbuild
- Remove download-rustc handling from bootstrap.py
- Allow a custom `pattern` in `builder.unpack()`
- Only download rustc once another part of bootstrap depends on it.
This is somewhat necessary since the download functions rely on having a full
`Builder`, which isn't available until after config parsing finishes.
Helps with https://github.com/rust-lang/rust/issues/94829.
|
|
r=Mark-Simulacrum
Print stderr consistently
Solves https://github.com/rust-lang/rust/issues/96712
I tried to follow what I perceived as the general consensus for error messages in boostrap i.e messages that were ..
* resulting from an Err(...) =>
* literally called as "Error: ...."
* by the end of the block scope forced to run a panic! or process::exit with a guaranteed non-zero error code.
|
