| Age | Commit message (Collapse) | Author | Lines |
|---|
|
|
|
various download-rustc fixes
separated out from https://github.com/rust-lang/rust/pull/112143 because it keeps getting stuck in limbo.
best reviewed commit-by-commit
|
|
|
|
|
|
It might happen that a synthetic target name does not match one of the
hardcoded ones in std's build script, causing std to fail to build. This
commit changes the std build script avoid including the restricted-std
feature unconditionally when a synthetic target is being built.
|
|
Some targets are added to these hashmaps at runtime, and are not present
during dry runs. To avoid errors, this commit changes all the related
functions to always return empty strings/paths during dry runs.
|
|
This will be needed to create synthetic targets in future commits.
|
|
Give more helpful progress messages in `Assemble`
Before (download-rustc):
```
# no output
```
After (download-rustc):
```
Creating a sysroot for stage2 compiler (use `rustup toolchain link 'name' build/host/stage2`)
```
Before (compiling from source):
```
Building compiler artifacts (stage0 -> stage1, x86_64-unknown-linux-gnu)
Assembling stage1 compiler
Build stage1 library artifacts (x86_64-unknown-linux-gnu -> i686-unknown-linux-gnu)
Building compiler artifacts (stage0:x86_64-unknown-linux-gnu -> stage1:i686-unknown-linux-gnu)
Assembling stage1 compiler (i686-unknown-linux-gnu)
```
After (compiling from source):
```
Building compiler artifacts (stage0 -> stage1, x86_64-unknown-linux-gnu)
Creating a sysroot for stage1 compiler (use `rustup toolchain link 'name' build/host/stage1`)
Build stage1 library artifacts (x86_64-unknown-linux-gnu)
Building compiler artifacts (stage0:x86_64-unknown-linux-gnu -> stage1:i686-unknown-linux-gnu)
Creating a sysroot for stage1 compiler (i686-unknown-linux-gnu) (use `rustup toolchain link 'name' build/i686-unknown-linux-gnu/stage1`)
```
cc https://rust-lang.zulipchat.com/#narrow/stream/122651-general/topic/Unable.20to.20compile.20rustc.20MSVC, https://discord.com/channels/273534239310479360/957720175619215380/1116867245499498506
|
|
Before (download-rustc):
```
# no output
```
After (download-rustc):
```
Creating a sysroot for stage2 compiler (use `rustup toolchain link 'name' build/host/stage2`)
```
Before (compiling from source):
```
Building compiler artifacts (stage0 -> stage1, x86_64-unknown-linux-gnu)
Assembling stage1 compiler
Build stage1 library artifacts (x86_64-unknown-linux-gnu -> i686-unknown-linux-gnu)
Building compiler artifacts (stage0:x86_64-unknown-linux-gnu -> stage1:i686-unknown-linux-gnu)
Assembling stage1 compiler (i686-unknown-linux-gnu)
```
After (compiling from source):
```
Building compiler artifacts (stage0 -> stage1, x86_64-unknown-linux-gnu)
Creating a sysroot for stage1 compiler (use `rustup toolchain link 'name' build/host/stage1`)
Build stage1 library artifacts (x86_64-unknown-linux-gnu)
Building compiler artifacts (stage0:x86_64-unknown-linux-gnu -> stage1:i686-unknown-linux-gnu)
Creating a sysroot for stage1 compiler (i686-unknown-linux-gnu) (use `rustup toolchain link 'name' build/i686-unknown-linux-gnu/stage1`)
```
|
|
Avoid one `rustc` rebuild in the optimized build pipeline
This PR changes the optimized build pipeline to avoid one `rustc` rebuild, inspired by [this comment](https://github.com/rust-lang/rust/issues/112011#issuecomment-1564991175). This speeds up the pipeline by 5-10 minutes. After this change, we **no longer gather LLVM PGO profiles from compiling stage 2 of `rustc`**.
Now we build `rustc` two times (1x PGO instrumented, 1x PGO optimized) and LLVM three times (1x normal, 1x PGO instrumented, 1x PGO optimized). It should be possible to cache the normal LLVM build, but I'll leave that for another PR.
|
|
before, this hardcoded `.so` as the extension for dynamically linked objects, which is incorrect
everywhere except linux
|
|
|
|
Fix `x test core` when download-rustc is enabled
Fix `x test --stage 2 core` when download-rustc is enabled
This works by building std from source instead of downloading it, for library tests only.
This was somewhat complicated because of the following requirements:
1. Unconditionally downloading libstd breaks `x test core`, because `coretests` requires the std loaded from the sysroot to match the std that's currently being tested.
2. Unconditionally rebuilding libstd breaks `x test ui-fulldeps librustdoc`, because anything loading `rustc_private` needs to use the same libstd that rustc was built with.
Break the knot by introducing a new `stage2-test-sysroot`, used only for testing `std` itself. This
holds a freshly compiled std, while `stage2` and `ci-rustc-sysroot` still hold the downloaded std.
This also extends the existing `cp_filtered` in Sysroot to apply to the `rust-std` component, not just the `rustc-dev` component, to avoid having both versions of std in `stage2-test-sysroot`.
Fixes #110352.
|
|
enhancements on build_helper utilization and rustdoc-gui-test
This change provides codebase improvements, resolves `FIXME` in `rustdoc-gui-test` and makes `rustdoc-gui` test able to find local `node_modules` directory outside of the source root.
|
|
This works by building std from source unconditionally instead of downloading it, for library tests only.
This was somewhat complicated because of the following requirements:
1. Unconditionally downloading libstd breaks `x test std`, because `coretests` requires the std loaded from the sysroot to match the std that's currently being tested.
2. Unconditionally rebuilding libstd breaks `x test ui-fulldeps librustdoc`, because anything loading `rustc_private` needs to use the same libstd that rustc was built with.
Break the knot by introducing a new `stage2-test-sysroot`, used only for testing `std` itself. This
holds a freshly compiled std, while `stage2` and `ci-rustc-sysroot` still hold the downloaded std.
This also extends the existing `cp_filtered` in Sysroot to apply to the `rust-std` component, not just the `rustc-dev` component.
|
|
Signed-off-by: ozkanonur <work@onurozkan.dev>
|
|
and document why the single remaining place can't switch
|
|
This fixes the panic from the previous commit.
|
|
r=Mark-Simulacrum
enable `rust_2018_idioms` lint group for doctests
With this change, `rust_2018_idioms` lint group will be enabled for compiler/libstd doctests.
Resolves #106086
Resolves #99144
Signed-off-by: ozkanonur <work@onurozkan.dev>
|
|
Signed-off-by: ozkanonur <work@onurozkan.dev>
|
|
|
|
Group some sections of our logs in github actions
This makes logs a little bit more readable as you can now collapse all the parts that don't interest you (and they get collapsed automatically)
Obviously there's a lot more sites where we can/need to do this, too, but this is already helpful imo
r? ```@jyn514```
|
|
|
|
Signed-off-by: ozkanonur <work@onurozkan.dev>
|
|
Fix no_std tests that load libc from the sysroot when download-rustc is enabled
There were a series of unfortunate interactions here. Here's an MCVE of the test this fixes (committed as `tests/ui/meta/no_std-extern-libc.rs`):
```rust
#![crate_type = "lib"]
#![no_std]
#![feature(rustc_private)]
extern crate libc;
```
Before, this would give an error about duplicate versions of libc:
```
error[E0464]: multiple candidates for `rlib` dependency `libc` found
--> fake-test-src-base/allocator/no_std-alloc-error-handler-default.rs:15:1
|
LL | extern crate libc;
| ^^^^^^^^^^^^^^^^^^
|
= note: candidate #1: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-358db1024b7d9957.rlib
= note: candidate #2: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-ebc478710122a279.rmeta
```
Both these versions were downloaded from CI, but one came from the `rust-std` component and one came from `rustc-dev`:
```
; tar -tf build/cache/f2d9a3d0771504f1ae776226a5799dcb4408a91a/rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz | grep liblibc
rust-std-nightly-x86_64-unknown-linux-gnu/rust-std-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68a2d9e195dd6ed2.rlib
; tar -tf build/cache/f2d9a3d0771504f1ae776226a5799dcb4408a91a/rustc-dev-nightly-x86_64-unknown-linux-gnu.tar.xz | grep liblibc
rustc-dev-nightly-x86_64-unknown-linux-gnu/rustc-dev/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-f226c9fbdd92a0fd.rmeta
```
The fix was to only copy files from `rust-std` unless a Step explicitly requests for the `rustc-dev` components to be available by calling `builder.ensure(compile::Rustc)`.
To avoid having to re-parse the `rustc-dev.tar.xz` tarball every time, which is quite slow, this adds a new `build/host/ci-rustc/.rustc-dev-contents` cache file which stores only the names of files we need to copy into the sysroot.
This also allows reverting the hack in https://github.com/rust-lang/rust/pull/110121; now that we only copy rustc-dev on-demand, we can correctly add the `Rustc` check artifacts into the sysroot, so that this works correctly even when `download-rustc` is forced to `true` and some tool depends on a local change to `compiler`.
---
See https://github.com/rust-lang/rust/issues/108767#issuecomment-1501217657 for why `no_std` is required for the MCVE test to fail; it's complicated and not particularly important.
Fixes https://github.com/rust-lang/rust/issues/108767.
|
|
There were a series of unfortunate interactions here. Here's an MCVE of the test this fixes (committed as `tests/ui/meta/no_std-extern-libc.rs`):
```rust
#![crate_type = "lib"]
#![no_std]
#![feature(rustc_private)]
extern crate libc;
```
Before, this would give an error about duplicate versions of libc:
```
error[E0464]: multiple candidates for `rlib` dependency `libc` found
--> fake-test-src-base/allocator/no_std-alloc-error-handler-default.rs:15:1
|
LL | extern crate libc;
| ^^^^^^^^^^^^^^^^^^
|
= note: candidate #1: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-358db1024b7d9957.rlib
= note: candidate #2: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-ebc478710122a279.rmeta
```
Both these versions were downloaded from CI, but one came from the `rust-std` component and one came from `rustc-dev`:
```
; tar -tf build/cache/f2d9a3d0771504f1ae776226a5799dcb4408a91a/rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz | grep liblibc
rust-std-nightly-x86_64-unknown-linux-gnu/rust-std-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68a2d9e195dd6ed2.rlib
; tar -tf build/cache/f2d9a3d0771504f1ae776226a5799dcb4408a91a/rustc-dev-nightly-x86_64-unknown-linux-gnu.tar.xz | grep liblibc
rustc-dev-nightly-x86_64-unknown-linux-gnu/rustc-dev/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-f226c9fbdd92a0fd.rmeta
```
The fix was to only copy files from `rust-std` unless a Step explicitly requests for the `rustc-dev` components to be available by calling `builder.ensure(compile::Rustc)`.
To avoid having to re-parse the `rustc-dev.tar.xz` tarball every time, which is quite slow, this adds a new `build/host/ci-rustc/.rustc-dev-contents` cache file which stores only the names of files we need to copy into the sysroot.
This also allows reverting the hack in
https://github.com/rust-lang/rust/pull/110121; now that we only copy
rustc-dev on-demand, we can correctly add the `Rustc` check artifacts
into the sysroot, so that this works correctly even when
`download-rustc` is forced to `true`.
---
See https://github.com/rust-lang/rust/issues/108767#issuecomment-1501217657 for why `no_std` is required for the MCVE test to fail; it's complicated and not particularly important.
Fixes https://github.com/rust-lang/rust/issues/108767.
|
|
Nils had an excellent idea the other day: the same way that rustdoc is
able to load `rustc_driver` from the sysroot, ui-fulldeps tests should
also be able to load it from the sysroot. That allows us to run fulldeps
tests with stage1, without having to fully rebuild the compiler twice.
It does unfortunately have the downside that we're running the tests on
the *bootstrap* compiler, not the in-tree sources, but since most of the
fulldeps tests are for the *API* of the compiler, that seems ok.
I think it's possible to extend this to `run-make-fulldeps`, but I've
run out of energy for tonight.
- Move `plugin` tests into a subdirectory.
Plugins are loaded at runtime with `dlopen` and so require the ABI of
the running compile to match the ABI of the compiler linked with
`rustc_driver`. As a result they can't be supported in stage 1 and have
to use `// ignore-stage1`.
- Remove `ignore-stage1` from most non-plugin tests
- Ignore diagnostic tests in stage 1. Even though this requires a stage
2 build to load rustc_driver, it's primarily testing the error message
that the *running* compiler emits when the diagnostic struct is malformed.
- Pass `-Zforce-unstable-if-unmarked` in stage1, not just stage2. That
allows running `hash-stable-is-unstable` in stage1, since it now
suggests adding `rustc_private` to enable loading the crates.
- Add libLLVM.so to the stage0 target sysroot, to allow fulldeps tests
that act as custom drivers to load it at runtime.
- Pass `--sysroot stage0-sysroot` in compiletest so that we use the
correct version of std.
|
|
Add `libLLVM.so` to the target libdir when download-rustc is enabled
Previously, we would only add it to the host libdir, which meant it couldn't be loaded by `ui-fulldeps` tests that used rustc_private.
Fixes https://github.com/rust-lang/rust/issues/110225, fixes https://github.com/rust-lang/rust/issues/110226.
|
|
Previously, we would only add it to the host libdir, which meant it
couldn't be loaded by `ui-fulldeps` tests that used rustc_private.
|
|
Fix x check --stage 1 when download-ci-llvm=false
Bootstrap tries to avoid building LLVM unless it needs to; in particular we only build it for `x build`, not `x check`. Unfortunately, the check forgot about existence of stages - it would break if you used `x check --stage 1`:
```
= note: /usr/bin/ld: cannot find -lPolly: No such file or directory
/usr/bin/ld: cannot find -lPollyISL: No such file or directory
```
Fix it to work for stage 1.
I recommend reading this commit-by-commit; the first one makes a bunch of whitespace changes but otherwise doesn't change the logic.
|
|
|
|
Previously, we would never build the target std, only the host std.
|
|
|
|
Rollup of 7 pull requests
Successful merges:
- #109526 (LIBPATH is used as dylib's path environment variable on AIX)
- #109642 (check for missing codegen backeng config)
- #109722 (Implement read_buf for RustHermit)
- #109856 (fix(middle): emit error rather than delay bug when reaching limit)
- #109868 (Improve PR job names in Github Actions preview)
- #109871 (Include invocation start times)
- #109873 (Move some UI tests into subdirectories)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
lenko-d:rust_codegen-backends_interacts_confusingly_with_paths, r=Mark-Simulacrum
check for missing codegen backeng config
Fixes [#109610](https://github.com/rust-lang/rust/issues/109610)
|
|
fix `build --stage 2 compiler/rustc` panic
Skip assembling(which causes panic due to not found `.librustc.stamp` file) process
for stage3(since it has problems with sysroot) if full-bootstrap isn't used.
Resolves #90244
|
|
Signed-off-by: ozkanonur <work@onurozkan.dev>
|
|
|
|
Renamed 'native.rs' to 'llvm.rs', also moved `TestHelpers` to `test.rs`.Replaced all the `native.rs` ocurrences at `src/bootstrap` files to `llvm.rs`
|
|
|
|
Allow building serde and serde_derive in parallel
This reduces build time of bootstrap by ~6s
|
|
This reduces build time of bootstrap by ~6s
|
|
This reverts commit 6990ab9ad2cde9b67073ffac29ffecc2be8e722f.
|
|
|
|
keith:ks/add-sanitizer-support-for-modern-ios-platforms, r=badboy
Add sanitizer support for modern iOS platforms
asan and tsan generally support iOS, but that previously wasn't configured in rust. This only adds support for the simulator architectures, and arm64 device architecture, not the older 32 bit architectures.
|
|
|
|
asan and tsan generally support iOS, but that previously wasn't
configured in rust. This only adds support for the simulator
architectures, and arm64 device architecture, not the older 32 bit
architectures.
|
|
|
|
|
|
This reverts commit 9dfe50440e6d48bd2fd40a4b7b3992998e55eace.
Fixes `x clippy`.
|
