| Age | Commit message (Collapse) | Author | Lines |
|---|
|
Fixes: #99208
|
|
Build the Clippy book as part of x.py doc
r? ``@ehuss`` since you said you would be interested in helping moving this forward.
cc ``@jyn514`` as part of the bootstrap team.
|
|
Fix caching bug in `download-rustc = true`
When moving this to rustbuild, I introduced a bug: if you had the file already downloaded, but
deleted the sysroot for whatever reason, rustbuil would fail to unpack the cached tarball.
This only affects people if they have a cached tarball, which is probably why we haven't seen an issue yet -
wiping `build/cache` would work around the issue, or just not deleting `build/$TARGET/stage2`.
Fixes the following error:
```
thread 'main' panicked at 'fs::read_dir(&lib_dir) failed with No such file or directory (os error 2) ("/home/jnelson/rust-lang/rust2/build/x86_64-unknown-linux-gnu/ci-rustc/lib")', config.rs:1563:20
```
r? ``@Mark-Simulacrum``
|
|
replace process exit with more detailed exit in src/bootstrap/*.rs
Fixes [#98830](https://github.com/rust-lang/rust/issues/98830)
I implemeted "detail_exit.rs" in lib.rs, and replace all of std::process::exit.
So, error code should panic in test code.
```
// lib.rs
pub fn detail_exit(code: i32) -> ! {
// Successful exit
if code == 0 {
std::process::exit(0);
}
if cfg!(test) {
panic!("status code: {}", code);
} else {
std::panic::resume_unwind(Box::new(code));
}
}
```
<details>
<summary>% rg "exit\(" src/bootstrap/*.rs</summary>
```
builder.rs
351: crate::detail_exit(1);
1000: crate::detail_exit(1);
1429: crate::detail_exit(1);
compile.rs
1331: crate::detail_exit(1);
config.rs
818: crate::detail_exit(2);
1488: crate::detail_exit(1);
flags.rs
263: crate::detail_exit(exit_code);
349: crate::detail_exit(exit_code);
381: crate::detail_exit(1);
602: crate::detail_exit(1);
616: crate::detail_exit(1);
807: crate::detail_exit(1);
format.rs
35: crate::detail_exit(1);
117: crate::detail_exit(1);
lib.rs
714: detail_exit(1);
1620: detail_exit(1);
1651:pub fn detail_exit(code: i32) -> ! {
1654: std::process::exit(0);
sanity.rs
107: crate::detail_exit(1);
setup.rs
97: crate::detail_exit(1);
290: crate::detail_exit(1);
test.rs
676: crate::detail_exit(1);
1024: crate::detail_exit(1);
1254: crate::detail_exit(1);
tool.rs
207: crate::detail_exit(1);
toolstate.rs
96: crate::detail_exit(3);
111: crate::detail_exit(1);
182: crate::detail_exit(1);
228: crate::detail_exit(1);
util.rs
339: crate::detail_exit(1);
378: crate::detail_exit(1);
468: crate::detail_exit(1);
```
</details>
|
|
implement detail_exit but I'm not sure it is right.
not create new file and write detail exit in lib.rs
replace std::process::exit to detail_exit
that is not related to code runnning.
remove pub
|
|
|
|
bootstrap: Allow building individual crates
This aims to be as unintrusive as possible, but did still require adding a new `tail_args` field to all `Rustc` and `Std` steps.
New library and compiler crates are added to the sysroot as they are built, since it's useful to have e.g. just alloc and not std.
Fixes https://github.com/rust-lang/rust/issues/44293.
|
|
This adds miri to https://doc.rust-lang.org/nightly/nightly-rustc/
Signed-off-by: InfRandomness <infrandomness@gmail.com>
|
|
|
|
- Add `Interned<Vec<String>>` and use it for tail args
- Refactor `cache.rs` not to need a separate impl for each internable type
|
|
When moving this to rustbuild, I introduced a bug: if you had the file already downloaded, but
deleted the sysroot for whatever reason, rustbuil would fail to unpack the cached tarball.
This only affects people if they have a cached tarball, which is probably why we haven't seen an issue yet -
wiping `build/cache` would work around the issue, or just not deleting `build/$TARGET/stage2`.
|
|
Bootstrap currently provides `-Zunstable-options` for platforms
when using split debuginfo - this commit adds it for the illumos
target too.
|
|
Running steps multiple times defeats the whole point of #96501,
since lint messages will be duplicated.
|
|
Pass all paths to `Step::run` at once when using `ShouldRun::krate`
Helps with https://github.com/rust-lang/rust/pull/95503. The goal is to run `cargo test -p rustc_data_structures -p rustc_lint_defs` instead of `cargo test -p rustc_data_structures; cargo test -p rustc_lint_defs`, which should both recompile less and avoid replaying cached warnings.
This was surprisingly complicated. The main changes are:
1. Invert the order of iteration in `StepDescription::run`.
Previously, it did something like:
```python
for path in paths:
for (step, should_run) in should_runs:
if let Some(set) = should_run.pathset_for_path(path):
step.run(builder, set)
```
That worked ok for individual paths, but didn't allow passing more than one path at a time to `Step::run`
(since `pathset_for_paths` only had one path available to it).
Change it to instead look at the intersection of `paths` and `should_run.paths`:
```python
for (step, should_run) in should_runs:
if let Some(set) = should_run.pathset_for_paths(paths):
step.run(builder, set)
```
2. Change `pathset_for_path` to take multiple pathsets.
The goal is to avoid `x test library/alloc` testing *all* library crates, instead of just alloc.
The changes here are similarly subtle, to use the intersection between the paths rather than all
paths in `should_run.paths`. I added a test for the behavior to try and make it more clear.
Note that we use pathsets instead of just paths to allow for sets with multiple aliases (*cough* `all_krates` *cough*).
See the documentation added in the next commit for more detail.
3. Change `StepDescription::run` to explicitly handle 0 paths.
Before this was implicitly handled by the `for` loop, which just didn't excute when there were no paths.
Now it needs a check, to avoid trying to run all steps (this is a problem for steps that use `default_condition`).
4. Change `RunDescription` to have a list of pathsets, rather than a single path.
5. Remove paths as they're matched
This allows checking at the end that no invalid paths are left over.
Note that if two steps matched the same path, this will no longer run both;
but that's a bug anyway.
6. Handle suite paths separately from regular sets.
Running multiple suite paths at once instead of in separate `make_run` invocations is both tricky and not particularly useful.
The respective test Steps already handle this by introspecting the original paths.
Avoid having to deal with it by moving suite handling into a seperate loop than `PathSet::Set` checks.
`@rustbot` label +A-rustbuild
|
|
|
|
This was surprisingly complicated. The main changes are:
1. Invert the order of iteration in `StepDescription::run`.
Previously, it did something like:
```python
for path in paths:
for (step, should_run) in should_runs:
if let Some(set) = should_run.pathset_for_path(path):
step.run(builder, set)
```
That worked ok for individual paths, but didn't allow passing more than one path at a time to `Step::run`
(since `pathset_for_paths` only had one path available to it).
Change it to instead look at the intersection of `paths` and `should_run.paths`:
```python
for (step, should_run) in should_runs:
if let Some(set) = should_run.pathset_for_paths(paths):
step.run(builder, set)
```
2. Change `pathset_for_path` to take multiple pathsets.
The goal is to avoid `x test library/alloc` testing *all* library crates, instead of just alloc.
The changes here are similarly subtle, to use the intersection between the paths rather than all
paths in `should_run.paths`. I added a test for the behavior to try and make it more clear.
Note that we use pathsets instead of just paths to allow for sets with multiple aliases (*cough* `all_krates` *cough*).
See the documentation added in the next commit for more detail.
3. Change `StepDescription::run` to explicitly handle 0 paths.
Before this was implicitly handled by the `for` loop, which just didn't excute when there were no paths.
Now it needs a check, to avoid trying to run all steps (this is a problem for steps that use `default_condition`).
4. Change `RunDescription` to have a list of pathsets, rather than a single path.
5. Remove paths as they're matched
This allows checking at the end that no invalid paths are left over.
Note that if two steps matched the same path, this will no longer run both;
but that's a bug anyway.
6. Handle suite paths separately from regular sets.
Running multiple suite paths at once instead of in separate `make_run` invocations is both tricky and not particularly useful.
The respective test Steps already handle this by introspecting the original paths.
Avoid having to deal with it by moving suite handling into a seperate loop than `PathSet::Set` checks.
|
|
|
|
|
|
|
|
|
|
Add build metrics to rustbuild
This PR adds a new module of rustbuild, `ci_profiler`, whose job is to gather as much information as possible about the CI build as possible and store it in a JSON file uploaded to `ci-artifacts`. Right now for each step it collects:
* Type name and debug representation of the `Step` object.
* Duration of the step (excluding child steps).
* Systemwide CPU stats for the duration of the step (both single core and all cores).
* Which child steps were executed.
This is capable of replacing both the scripts to collect CPU stats and the `[TIMING]` lines in build logs (not yet removed, until we port our tooling to use the CI profiler). The format is also extensible to be able in the future to collect more information.
r? `@Mark-Simulacrum`
|
|
Use new cargo argument in bootstrap for cfg checking
This PR use new cargo argument in bootstrap for doing cfg checking.
Follow-up to https://github.com/rust-lang/rust/pull/97044 and https://github.com/rust-lang/rust/pull/97214.
r? `@Mark-Simulacrum`
|
|
|
|
Re-add help_on_error for download-ci-llvm
Closes #97503
- Re-added `help_on_error` for `download_component()` and the downstream functions
- Removed dead code in `bootstrap.py`
Thanks `@jyn514` for the helpful tips!
(first contribution here, please let me know if I missed anything out!)
|
|
Finish bumping stage0
It looks like the last time had left some remaining cfg's -- which made me think
that the stage0 bump was actually successful. This brings us to a released 1.62
beta though.
This now brings us to cfg-clean, with the exception of check-cfg-features in bootstrap;
I'd prefer to leave that for a separate PR at this time since it's likely to be more tricky.
cc https://github.com/rust-lang/rust/pull/97147#issuecomment-1132845061
r? `@pietroalbini`
|
|
Remove dead code
Missing }
./x.py fmt
Remove duplicate check
Recursively remove all usage of help_on_error
|
|
Move download-rustc from python to rustbuild
- Remove download-rustc handling from bootstrap.py
- Allow a custom `pattern` in `builder.unpack()`
- Only download rustc once another part of bootstrap depends on it.
This is somewhat necessary since the download functions rely on having a full
`Builder`, which isn't available until after config parsing finishes.
Helps with https://github.com/rust-lang/rust/issues/94829.
|
|
r=Mark-Simulacrum
Print stderr consistently
Solves https://github.com/rust-lang/rust/issues/96712
I tried to follow what I perceived as the general consensus for error messages in boostrap i.e messages that were ..
* resulting from an Err(...) =>
* literally called as "Error: ...."
* by the end of the block scope forced to run a panic! or process::exit with a guaranteed non-zero error code.
|
|
It looks like the last time had left some remaining cfg's -- which made me think
that the stage0 bump was actually successful. This brings us to a released 1.62
beta though.
|
|
|
|
Using language features occasionally causes issues when using nightly to bootstrap, rather than beta.
See #59264 for additional context.
|
|
This is for a pre-existing FIXME, but it was easy enough to do.
|
|
|
|
- Remove download-rustc handling from bootstrap.py
- Allow a custom `pattern` in `builder.unpack()`
- Only download rustc once another part of bootstrap depends on it.
This is somewhat necessary since the download functions rely on having a full
`Builder`, which isn't available until after config parsing finishes.
|
|
This has no logic changes, just a move.
|
|
Avoid accidentally enabling unstable features in compilers (take 2)
This allows rustbuild to control whether crates can use nightly features or not.
It also prevents rustbuild from using nightly features itself.
This is #92261, but I fixed the CI error.
|
|
This statically prevents issues like https://github.com/rust-lang/rust/issues/59264,
where tools can only be built with the in-tree compiler and not beta.
|
|
|
|
|
|
Enable cfg checking of cargo features for everything but std
This PR enable `cfg` checking of cargo features for everything but std, it also adds a `FIXME` to myself.
> Note: `std`, `alloc` and `core` imports some dependencies by #[path] (like
> backtrace, core_simd, std_float, ...), those dependencies have their own features
> but cargo isn't involved in the #[path] and so cannot pass the complete list of
> features, so for that reason we don't enable checking of features for std.
r? `@Mark-Simulacrum`
|
|
r=Mark-Simulacrum
bootstrap: bsd platform flags for split debuginfo
Addresses https://github.com/rust-lang/rust/pull/96597#issuecomment-1118905025.
Bootstrap currently provides `-Zunstable-options` for OpenBSD when using split debuginfo - this commit provides it for all BSD targets.
We should probably work out a better way of handling the stability of the split debuginfo flag - all options for the flag are unstable but one of them is the default for each platform already.
cc `@m-ou-se`
r? `@Mark-Simulacrum`
|
|
[bootstrap] Give a better error when trying to run a path with no registered step
Before:
```
thread 'main' panicked at 'error: no rules matched invalid', src/bootstrap/builder.rs:287:17
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
After:
```
error: no `check` rules matched 'invalid'
help: run `x.py check --help --verbose` to show a list of available paths
note: if you are adding a new Step to bootstrap itself, make sure you register it with `describe!`
```
|
|
Bootstrap currently provides `-Zunstable-options` for OpenBSD when using
split debuginfo - this commit provides it for all BSD targets.
Signed-off-by: David Wood <david.wood@huawei.com>
|
|
r=davidtwco,Mark-Simulacrum
openbsd: unbreak build on native platform
after #95612, only linux and windows target are build with `-Zunstable-options`, but others platforms might use `-Csplit-debuginfo`
currently, without this PR, the build of rustc on OpenBSD fails with:
```
Building stage0 tool unstable-book-gen (x86_64-unknown-openbsd)
running: "/data/semarie/build-rust/install_dir/beta/bin/cargo" "build" "--target" "x86_64-unknown-openbsd" "-Zbinary-dep-depinfo" "-j" "4" "-v" "--release" "--frozen" "--manifest-path"
"/data/semarie/build-rust/build_dir/rustc-nightly-src/src/tools/unstable-book-gen/Cargo.toml" "--message-format" "json-render-diagnostics"
error: failed to run `rustc` to learn about target-specific information
Caused by:
process didn't exit successfully: `/data/semarie/build-rust/build_dir/build/bootstrap/debug/rustc - --crate-name ___ --print=file-names --cfg=bootstrap -Csymbol-mangling-version=v0 -Zmacro-backtrace -Clink-args=-Wl,-z,origin
'-Clink-args=-Wl,-rpath,$ORIGIN/../lib' -Csplit-debuginfo=off -Ztls-model=initial-exec --target x86_64-unknown-openbsd --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro
--print=sysroot --print=cfg` (exit status: 1)
--- stdout
Did not run successfully: exit status: 1
"/data/semarie/build-rust/install_dir/beta/bin/rustc" "-" "--crate-name" "___" "--print=file-names" "--cfg=bootstrap" "-Csymbol-mangling-version=v0" "-Zmacro-backtrace" "-Clink-args=-Wl,-z,origin" "-Clink-args=-Wl,-rpath,$ORIGIN/../lib"
"-Csplit-debuginfo=off" "-Ztls-model=initial-exec" "--target" "x86_64-unknown-openbsd" "--crate-type" "bin" "--crate-type" "rlib" "--crate-type" "dylib" "--crate-type" "cdylib" "--crate-type" "staticlib" "--crate-type" "proc-macro"
"--print=sysroot" "--print=cfg" "-Wrust_2018_idioms" "-Wunused_lifetimes" "-Wsemicolon_in_expressions_from_macros" "-Dwarnings" "--sysroot" "/data/semarie/build-rust/install_dir/beta"
-------------
--- stderr
error: `-Csplit-debuginfo` is unstable on this platform
command did not execute successfully: "/data/semarie/build-rust/install_dir/beta/bin/cargo.bin" "build" "--target" "x86_64-unknown-openbsd" "-Zbinary-dep-depinfo" "-j" "4" "-v" "--release" "--frozen" "--manifest-path"
```
I am suspecting that all unix might be affected, but I am unsure about the right conditional to use. so I only added "openbsd" target inside it as I am able to test it.
rustc nightly built correctly with this PR on openbsd.
|
|
|
|
step
Before:
```
thread 'main' panicked at 'error: no rules matched invalid', src/bootstrap/builder.rs:287:17
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
After:
```
error: no `check` rules matched 'invalid'
help: run `x.py check --help --verbose` to show a list of available paths
note: if you are adding a new Step to bootstrap itself, make sure you register it with `describe!`
```
|
|
This tool will generate a JSON file with statistics about each
individual step to disk. It will be used in rust-lang/rust's CI to
replace the mix of scripts and log scraping we currently have to gather
this data.
|
|
Remove DebbugerScripts from bootstrap CLI
This PR implements #95992 and removes the debugger scripts from the bootstrap CLI.
I could not find a lot of documentation on the bootstrap binary so perhaps there's still some documentation to be updated.
|
|
|
|
after #95612, only linux and windows target are build with -Zunstable-options,
but others platforms might use -Csplit-debuginfo
add "openbsd" target in the list of platforms using it.
|
