| Age | Commit message (Collapse) | Author | Lines |
|---|
|
Signed-off-by: Jens Reidel <adrian@travitia.xyz>
|
|
bootstrap: add package.json and package-lock.json to dist tarball
this ensures that js-related tests can still be run from within such a dist tarball.
followup to rust-lang/rust#142924
r? ```````@Kobzol```````
|
|
Add powerpc64le-unknown-linux-musl to CI rustc targets
I missed this in the promotion to tier 2 with host tools.
|
|
Move `std_detect` into stdlib
This PR moves the `std_detect` crate from `stdarch` to be a part of rust-lang/rust instead.
The first commit actually moves the whole directory from the stdarch Josh subtree, so that git blame history is kept intact. Then I had to make a few changes to appease `tidy`.
The most complex thing here is porting the tests. We can't have `std_detect` both in r-l/r and stdarch, because they could get desynchronized, so we have to perform the move more or less "atomically", which means that we also have to port all the existing `std_detect` tests from the `stdarch` repository.
The stdarch repo runs the following `std_detect` tests:
### Build
The `build-std-detect.sh` script (https://github.com/rust-lang/stdarch/blob/e2b6512aed87df45294ae680181eeef7a802cd95/ci/build-std-detect.sh) builds `std_detect` using the nightly compiler for several targets. This will be subsumed by normal `x build library` on our Tier 1/2 targets. However, the stdarch repository also tests the following targets:
- aarch64-unknown-freebsd
- armv6-unknown-freebsd
- powerpc-unknown-freebsd
- powerpc64-unknown-freebsd
- aarch64-unknown-openbsd
Which we don't build/test on our CI currently. I think we have mostly two options here:
1) Ignore these targets
2) Create a special CI job that will build stage 1 rustc and then cross-compile std (or just the `std_detect` crate?) for these targets.
### Documentation
The `dox.sh` script (https://github.com/rust-lang/stdarch/blob/3fec5adcd52a815f227805d4959a25b6402c7fd5/ci/dox.sh) builds and documents `std_detect` for several targets. All of them are Tier 2/we have `dist-` jobs for them, so I think that we can just skip this and let our normal CI subsume it?
### Tests
The `run.sh` script (https://github.com/rust-lang/stdarch/blob/1b201cec2cca7465602a65ed6ae60517224b15f3/ci/run.sh) runs `cargo test` on `std_detect` with a bunch of variations of feature flags. This will be subsumed by `x test library` in our CI. The only problem is that `stdarch` runs these tests for a ludicrous number of targets:
```
- tuple: i686-unknown-linux-gnu
- tuple: x86_64-unknown-linux-gnu
- tuple: arm-unknown-linux-gnueabihf
- tuple: armv7-unknown-linux-gnueabihf
- tuple: aarch64-unknown-linux-gnu
- tuple: aarch64_be-unknown-linux-gnu
- tuple: riscv32gc-unknown-linux-gnu
- tuple: riscv64gc-unknown-linux-gnu
- tuple: powerpc-unknown-linux-gnu
- tuple: powerpc64-unknown-linux-gnu
- tuple: powerpc64le-unknown-linux-gnu
- tuple: s390x-unknown-linux-gnu
- tuple: i586-unknown-linux-gnu
- tuple: nvptx64-nvidia-cuda
- tuple: thumbv6m-none-eabi
- tuple: thumbv7m-none-eabi
- tuple: thumbv7em-none-eabi
- tuple: thumbv7em-none-eabihf
- tuple: loongarch64-unknown-linux-gnu
- tuple: wasm32-wasip1
- tuple: x86_64-apple-darwin
- tuple: x86_64-apple-ios-macabi
- tuple: aarch64-apple-darwin
- tuple: aarch64-apple-ios-macabi
- tuple: x86_64-pc-windows-msvc
- tuple: i686-pc-windows-msvc
- tuple: aarch64-pc-windows-msvc
- tuple: x86_64-pc-windows-gnu
- tuple: aarch64-unknown-linux-gnu
- tuple: aarch64_be-unknown-linux-gnu
- tuple: armv7-unknown-linux-gnueabihf
- tuple: loongarch64-unknown-linux-gnu
- tuple: powerpc-unknown-linux-gnu
- tuple: powerpc64-unknown-linux-gnu
- tuple: powerpc64le-unknown-linux-gnu
- tuple: riscv32gc-unknown-linux-gnu
- tuple: riscv64gc-unknown-linux-gnu
- tuple: s390x-unknown-linux-gnu
- tuple: x86_64-unknown-linux-gnu
- tuple: aarch64-apple-darwin
- tuple: aarch64-apple-ios-macabi
```
We definitely do not run *tests* for all of these targets on our CI.
# Outcome
We have decided to just subsume std_detect tests by our normal test suite for now, and not create a separate CI job. Therefore, this PR performs the following changes in target testing for `std_detect`:
The following T3 targets would go from "build" to "nothing":
```
aarch64-unknown-freebsd (T3)
armv6-unknown-freebsd (T3)
powerpc-unknown-freebsd (T3)
powerpc64-unknown-freebsd (T3)
aarch64-unknown-openbsd (T3)
```
The following T3 targets would go from "test" to "nothing":
```
aarch64_be-unknown-linux-gnu (T3)
riscv32gc-unknown-liux-gnu (T3)
```
The following T2 targets would go from "test" to "build":
```
arm-unknown-linux-gnueabihf (T2)
armv7-unknown-linux-gnueabihf (T2)
riscv64gc-unknown-linux-gnu (T2)
powerpc-unknown-linux-gnu (T2)
powerpc64-unknown-linux-gnu (T2)
powerpc64le-unknown-linux-gnu (T2)
s390x-unknown-linux-gnu (T2)
i586-unknown-linux-gnu (T2)
loongarch64-unknown-linux-gnu (T2)
wasm32-wasip1 (T2)
x86_64-apple-ios-macabi (T2)
aarch64-apple-ios-macabi (T2)
aarch64-pc-windows-msvc (T2)
armv7-unknown-linux-gnueabihf (T2)
loongarch64-unknown-linux-gnu (T2)
powerpc-unknown-linux-gnu (T2)
```
I have confirmed in https://github.com/rust-lang/stdarch/pull/1873 that the current version of this PR would pass stdarch's CI testsuite.
r? `@ghost`
try-job: armhf-gnu
try-job: arm-android
|
|
Rename `tests/{assembly,codegen}` into `tests/{assembly,codegen}-llvm` and ignore these testsuites if configured backend doesn't match
Follow-up of https://github.com/rust-lang/rust/pull/144125.
This PR changes `compiletest` so that `asm` tests are only run if they match the current codegen backend. To better reflect it, I renamed the `tests/ui/asm` folder into `tests/ui/asm-llvm`. Like that, we can add new asm tests for other backends if we want without needing to add extra code to `compiletest`.
Next step will be to use the new code annotations added in rust-lang/rust#144125 to ignore ui tests failing in cg_gcc until it's fixed on our side.
cc `@antoyo` `@oli-obk`
r? `@Kobzol`
|
|
|
|
|
|
Previously, the musl root would only be set to the fallback /usr by
the sanity check, which isn't ran for the bootstrap tests.
Signed-off-by: Jens Reidel <adrian@travitia.xyz>
|
|
this ensures that js-related tests can still be run from within
such a dist tarball.
|
|
I missed this in the promotion to tier 2 with host tools.
Signed-off-by: Jens Reidel <adrian@travitia.xyz>
|
|
|
|
|
|
r=Kobzol
Fix empty target_config in apply_rust_config bootstrap
This PR fixes the issue of an empty target_config in apply_rust_config, which was caused by the ordering of TOML config parsing. This was inadvertently introduced during the last config refactor. The test and the corresponding configuration order have been corrected in this PR.
r? ```@Kobzol```
|
|
tidy: move rustdoc js stuff into a tidy extra check
Most of these were factored out of CI scripts, but `eslint` in particular was previously implemented with its own special cased logic.
A new option has been added to bootstrap, `build.tidy-extra-checks`, which serves as a default value for the `--extra-checks` flag. This is mostly for the benefit of rustdoc js maintainers, but should also help bootstrap py maintainers.
Additionally, `--extra-checks=cpp` has been documented.
I'm not super happy with how long the extra check names are in comparison to the others (in particular `typecheck`), but I couldn't think of anything better (I didn't want to name it `tsc` on the off chance we want to switch to a different typechecking engine in the future).
It would be nice to convert the extra checks arg into a proper enum, both for warning on unknown values and to provide better shell completion.
r? ```@GuillaumeGomez```
Fixes: https://github.com/rust-lang/rust/issues/144093
|
|
clippy: make tests work in stage 1
This finally fixes https://github.com/rust-lang/rust/issues/78717 :)
Similar to what Miri already does, the clippy test step needs to carefully consider which compiler is used to build clippy and which compiler is linked into clippy (and thus must be used to build the test dependencies). On top of that we have some extra complications that Miri avoided by using `cargo-miri` for building its test dependencies: we need cargo to use the right rustc and the right sysroot, but https://github.com/rust-lang/cargo/issues/4423 makes this quite hard to do. See the long comment in `src/tools/clippy/tests/compile-test.rs` for details.
Some clippy tests tried to import rustc crates; that fundamentally requires a full bootstrap loop so it cannot work in stage 1. I had to kind of guess what those tests were doing so I don't know if my changes there make any sense.
Cc ```@flip1995``` ```@Kobzol```
|
|
Implement `check` for compiletest and RA using tool macro
Small cleanup to reduce the number of places that require custom check steps. Of course I had to include one mini hack because of Rust Analyzer.. but I think it's worth it here.
r? ````@jieyouxu````
|
|
|
|
add test for lld opt in and also add thread_local defined state to change opt in targets
make the config lld test parameter smoother to work, and have no_llvm_config set even when target_config is not present
|
|
|
|
|
|
Add new `ignore-backends` and `needs-backends` tests annotations
Part of https://github.com/rust-lang/compiler-team/issues/891.
Next step will be to add these annotations in the files where either the output is different based on the codegen (like `asm` tests) or that are known to fail in the GCC backend.
cc `@oli-obk` `@antoyo`
r? `@Kobzol`
|
|
bootstrap: Ignore `rust.debuginfo-level-tests` for codegen tests
As dicussed in https://github.com/rust-lang/rust/issues/61117#issuecomment-495587364, codegen tests typically depend on the raw LLVM IR output and are sensitive to debuginfo level. So do not apply `rust.debuginfo-level-tests` for codegen tests.
Before this commit:
$ ./x test --set rust.debuginfo-level-tests=2 tests/codegen --force-rerun
test result: FAILED. 654 passed; 136 failed; 75 ignored; 0 measured; 0 filtered out; finished in 3.22s
After this commit:
$ ./x test --set rust.debuginfo-level-tests=2 tests/codegen --force-rerun
NOTE: ignoring `rust.debuginfo-level-tests=2` for codegen tests
test result: ok. 790 passed; 0 failed; 75 ignored; 0 measured; 0 filtered out; finished in 3.21s
### Run this in CI?
Maybe it will make sense to add this to CI later but I think it is too early to do now before more non-codegen tests work with `rust.debuginfo-level-tests=2`.
|
|
bootstrap: Don't trigger an unnecessary LLVM build from check builds
Coming back to r-l/r development after a few weeks away, I found a major regression in my dev workflows: running `x check compiler` (either manually or via rust-analyzer) would have the side-effect of building LLVM, even though that shouldn't be necessary.
For my main build directory this would be a minor annoyance, but for my separate rust-analyzer build directory it's a huge problem because it causes a completely separate build of LLVM, which takes a long time and should be completely unnecessary.
---
After some investigation, I tracked down the problem to the `can_skip_build` check in this code:
https://github.com/rust-lang/rust/blob/3014e79f9c8d5510ea7b3a3b70d171d0948b1e96/src/bootstrap/src/core/build_steps/compile.rs#L1382-L1396
Historically, this would skip the LLVM build for stage 0 check builds. But after the recent stage 0 std redesign and some associated check stage renumbering (e.g. rust-lang/rust#143048), the condition `builder.top_stage == build_stage` is now false, because `top_stage` is 1 (due to the renumbering) but `build_stage` is 0 (because a “stage 1” non-library check build still uses the stage 0 compiler).
---
Because this is a critical contributor roadblock for me, I have tried to fix this in a relatively narrow way. It's possible that all of this surrounding logic could be greatly simplified (especially in light of the stage redesign changes), but I didn't want this fix to be held back by scope creep.
---
(Zulip thread: https://rust-lang.zulipchat.com/#narrow/channel/326414-t-infra.2Fbootstrap/topic/Bootstrap.20incorrectly.20building.20LLVM.20for.20check.20builds/near/528991035)
|
|
|
|
|
|
|
|
|
|
Add `ToolTarget` to bootstrap
Oh, you thought I'm done with refactoring bootstrap tools? Na-ah, think again! After the failure of https://github.com/rust-lang/rust/pull/143581, `ToolTarget` is back with a vengeance. This time, I implemented the test changes and tool cleanups without forcing these tools to be built with the stage0 compiler.
There are still some small wins though, `LlvmBitcodeLinker` now starts at stage 1, and not stage 2. Cargo should also be ported to this new mode, but I'm leaving that for a follow-up PR.
Hopefully X-th time's the charm :crossed_fingers:
r? `@jieyouxu`
|
|
|
|
bootstrap: Detect musl hosts
Currently, all non-Android Linux hosts are assumed to be using glibc. This obviously isn't very portable and will currently result in downloading a stage0 toolchain for glibc even on musl hosts.
There are multiple ways to detect musl somewhat reliably, but the easiest option is to check for the python target, which is exposed in `sys.implementation._multiarch` and has values like "x86_64-linux-gnu" or "powerpc64le-linux-musl".
|
|
|
|
Currently, all non-Android Linux hosts are assumed to be using glibc.
This obviously isn't very portable and will currently result in
downloading a stage0 toolchain for glibc even on musl hosts.
There are multiple ways to detect musl somewhat reliably, but the
easiest option is to check for the python SOABI config variable, which
has values like "cpython-313-x86_64-linux-gnu" or
"cpython-313-powerpc64-linux-musl".
Signed-off-by: Jens Reidel <adrian@travitia.xyz>
|
|
codegen tests typically depend on the raw LLVM IR output and are
sensitive to debuginfo level. So do not apply
`rust.debuginfo-level-tests` for codegen tests.
Before this commit:
$ ./x test --set rust.debuginfo-level-tests=2 tests/codegen --force-rerun
test result: FAILED. 654 passed; 136 failed; 75 ignored; 0 measured; 0 filtered out; finished in 3.22s
After this commit:
$ ./x test --set rust.debuginfo-level-tests=2 tests/codegen --force-rerun
NOTE: ignoring `rust.debuginfo-level-tests=2` for codegen tests
test result: ok. 790 passed; 0 failed; 75 ignored; 0 measured; 0 filtered out; finished in 3.21s
|
|
|
|
|
|
When a snapshot test fails, it only emits a `.pending-snap` file for the first
snapshot assertion that actually failed, because subsequent assertions aren't
executed. That makes it cumbersome to re-bless tests that contain multiple
snapshot assertions.
|
|
Copy GCC sources into the build directory even outside CI
It takes ~3.5s on my Linux notebook to perform the copy, but it should only be executed when we actually go build GCC, and that will almost certainly take much longer :) So I think it should be fine. At least we won't be polluting the source directory for local builds.
Fixes: https://github.com/rust-lang/rust/issues/143986
r? `````@nikic`````
|
|
Boostrap: add warning on `optimize = false`
I recently came across a bug that can be traced back to the use of `optimize = false` in `bootstrap.toml` in combination with other settings. Following [this](https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/Missing.20box-related.20symbols.20with.20panic.20.3D.20'abort'/with/528992909) conversation, this PR adds a warning from `bootstrap` when `optimize = false` is used.
I notice that in the same file I edited there are two different styles for warnings (`WARN`, `Warning`). I used `WARNING` because, by happenstance, when testing I got a `WARNING` that I didn't set a change id: let me know if I can unify the styles in the file I edited.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Don't panic if WASI_SDK_PATH not set when detecting compiler
The fedora packaging builds the wasm sysroot outside of the rust build system. Fedora applies a couple of patches related to wasm which I think make this possible. Not panicking seems consistent with the detection logic of other targets when they cannot find cc.
|
|
Drop `./x suggest`
This PR removes the current `./x suggest` implementation (rust-lang/rust#109933, rust-lang/rust#106249) and associated docs for several reasons:
1. Primarily, `./x suggest` is another "flow" in bootstrap that incurs extra complexity and more invariants that bootstrap has to maintain. This causes more friction when trying to investigate and fix staging problems. As far as I know, this flow has not been actively maintained in quite a while, and I'm not aware of interest in maintaining it. Bootstrap really could use less implementation complexity with a very limited maintenance bandwidth.
2. The current `./x suggest` implementation "bypasses" the usual stage defaults for the various check/build/test/etc. flows, and it's not really possible to have a stage default because `./x suggest --run` produces a *sequence* of suggestions like [`./x check`, `./x test library/std`, ..] and then tries to run all of them in sequence, based on which files are modified.
3. We've not seen a lot of interest both in using it or extending static/dynamic test suggestions. Last extensions were rust-lang/rust#117961 and rust-lang/rust#120763. I'm not convinced the extra implementation complexity is worth it. This was discussed in:
- [#t-infra/bootstrap > Dropping the current `./x suggest` flow implementation](https://rust-lang.zulipchat.com/#narrow/channel/326414-t-infra.2Fbootstrap/topic/Dropping.20the.20current.20.60.2E.2Fx.20suggest.60.20flow.20implementation/with/527456699)
- [#t-compiler > Dropping current `./x suggest` implementation](https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/Dropping.20current.20.60.2E.2Fx.20suggest.60.20implementation/with/527528696)
Closes rust-lang/rust#109933 (the current implementation is being removed).
Closes rust-lang/rust#143569 (by removing `./x suggest` altogether).
|
|
They are not always needed when building std, as is the case when
packaging on Fedora. Panic if building from CI, but warn otherwise.
|
|
This is quite a bit of implementation complexity, yet it is quite
broken, and we don't have the maintenance bandwidth to address.
Remove the current implementation if only to reduce bootstrap's
implementation complexity; the `suggest` flow comes with its own set of
hacks.
|
|
|
|
Rename `stable_mir` and `rustc_smir`
This PR only renames the two crate names.
There is no doubt that we want to rename `stable_mir` to `rustc_public`, while it hasn't been discussed yet that what the new name for `rustc_smir` should be.
This PR proposes a new name for `rustc_smir`, that is `rustc_public_shim`, since `rustc_smir` now is mostly a proxy to do calls to rustc queries and the public API of rustc that is invoked by the `rustc_public` crate.
However, I don't think that name is good enough. I hope there would be a way better name.
r? `@oli-obk`
|
