| Age | Commit message (Collapse) | Author | Lines |
|---|
|
r=Mark-Simulacrum
Fix linking stage1 toolchain in `./x.py setup`
Closes [92319](https://github.com/rust-lang/rust/issues/92319)
Fix linking stage1 toolchain in `./x.py setup`. I guess this can be considered a follow up to https://github.com/rust-lang/rust/pull/89212 by `````@Sl1mb0.`````
We create 2 directories and 1 file that are required by rustup to [link a custom toolchain from path](https://github.com/rust-lang/rustup/blob/5225e87a5d974ab5f1626bcb2a7b43f76ab883f0/src/toolchain.rs#L479-L497).
cc `````@jyn514````` and `````@Mark-Simulacrum````` as they were active in https://github.com/rust-lang/rust/issues/89206
|
|
Make the pre-commit script pre-push instead
This should make it substantially less annoying, and hopefully more
people will find it useful. In particular, it will no longer run tidy
each time you run `git commit --amend` or rebase a branch.
This also warns if you have the old script in pre-commit; see the HACK
comment for details.
r? ````@Mark-Simulacrum```` cc ````@caass````
|
|
Signed-off-by: Muhammad Falak R Wani <falakreyaz@gmail.com>
|
|
Previously, rustbuild would give strange errors if you tried to reuse the same build directory under two names:
```
$ mkdir tmp && cd tmp
$ ../x.py check
Building rustbuild
Finished dev [unoptimized] target(s) in 35.27s
Checking stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
^C
$ cd ..
$ mv tmp/build build
$ ./x.py check
Building rustbuild
Compiling bootstrap v0.0.0 (/home/jnelson/rust-lang/rust/src/bootstrap)
Finished dev [unoptimized] target(s) in 11.18s
failed to execute command: "/home/jnelson/rust-lang/rust/tmp/build/x86_64-unknown-linux-gnu/stage0/bin/rustc" "--target" "x86_64-unknown-linux-gnu" "--print" "target-libdir"
error: No such file or directory (os error 2)
```
This fixes the error. Reusing the same build directory is useful if you want to test path-things in
bootstrap itself, without having to recompile it each time.
For good measure, this also reruns the build script when PATH changes.
|
|
I missed this in https://github.com/rust-lang/rust/pull/89757; it made
`x.py test src/bootstrap` very slow.
|
|
`semver-parser` versions
pretty_assertions 0.6 -> 0.7, to drop some `ansi_term` version
futures 0.1.29 -> 0.1.31, backported some [fixes](https://github.com/rust-lang/futures-rs/compare/0.1.29...0.1.31) to old verions
futures-* 0.3.12 -> 0.3.19, to remove `proc-macro-hack`, `proc-macro-nested` and fix some [issues](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md#0319---2021-12-18). There exist 0.3.21, but it's quite new (06.02.22), so not updated to.
itertools 0.9 -> 0.10 for rustdoc, will be droppped when rustfmt will bump `itertools` version
linked-hash-map 0.5.3 -> 0.5.4, fix [UB](https://github.com/contain-rs/linked-hash-map/pull/106)
markup5ever 0.10.0 -> 0.10.1, internally drops `serde`, reducing [build time](https://github.com/servo/html5ever/commit/3afd8d63853627e530b3063b0185eea3732cc29f#diff-4c20e8293515259c0aa26932413a55a334aa5f2b37de5a5adc92a2186f632606) for some usecases
mio 0.7.13 -> 0.7.14 fix [unsoundness](https://github.com/tokio-rs/mio/compare/v0.7.13...v0.7.14)
num_cpus 1.13.0 -> 1.13.1 fix parsing mountinfo and other [fixes](https://github.com/seanmonstar/num_cpus/compare/v1.13.0...v1.13.1)
openssl-src 111.16.0+1.1.1l -> 111.17.0+1.1.1m fix CVE-2021-4160
|
|
|
|
pipeline without causing rebuilds
Useful for -Ztreat-err-as-bug
|
|
Hide failed command unless in verbose mode
This is particularly intended for invoking compiletest; the command line there
is long (3,350 characters on my system) and takes up a lot of screen real estate
for little benefit to the majority of those running bootstrap. This moves
printing it to verbose mode (-v must be passed) which means that it's still
possible to access when needed for debugging.
The main downside is that CI logs will by-default become less usable for
debugging (particularly) spurious failures, but it is pretty rare for us to
really need the information there -- it's usually fairly obvious what is being
run with a little investigation.
r? `@ehuss` as you've done some of the spurious failure investigations, so can
(hopefully) confirm my intuition that this won't seriously hinder them.
|
|
This is particularly intended for invoking compiletest; the command line there
is long (3,350 characters on my system) and takes up a lot of screen real estate
for little benefit to the majority of those running bootstrap. This moves
printing it to verbose mode (-v must be passed) which means that it's still
possible to access when needed for debugging.
The main downside is that CI logs will by-default become less usable for
debugging (particularly) spurious failures, but it is pretty rare for us to
really need the information there -- it's usually fairly obvious what is being
run with a little investigation.
|
|
Bootstrap compiler update
r? ``@Mark-Simulacrum``
|
|
|
|
|
|
r=Mark-Simulacrum
build: dist: defer PlainSourceTarball
Apparently it changes some tool sources and invalidates their fingerprints, forcing us to build them several times (before and after vendoring sources).
I have not dug into why vendoring actually invalidates the figreprints, but moving the vendoring lower in the pipeline seems to avoid the issue.
I could imagine that we somehow write a .cargo/config somewhere which somehow makes subsequent builds use the vendored deps but I was not able to find anything.
I checked the sizes of generated archives pre and post patch and their are the same, so I hope there is no functional change.
Fixes #93033
|
|
Rollup of 10 pull requests
Successful merges:
- #91965 (Add more granular `--exclude` in `x.py`)
- #92467 (Ensure that early-bound function lifetimes are always 'local')
- #92586 (Set the allocation MIN_ALIGN for espidf to 4.)
- #92835 (Improve error message for key="value" cfg arguments.)
- #92843 (Improve string concatenation suggestion)
- #92963 (Implement tuple array diagnostic)
- #93046 (Use let_else in even more places)
- #93109 (Improve `Arc` and `Rc` documentation)
- #93134 (delete `Stdin::split` forwarder)
- #93139 (rustdoc: fix overflow-wrap for table layouts)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
x.py has support for excluding some steps from the invocation, but
unfortunately that's not granular enough: some steps have the same name
in different modules, and that prevents excluding only *some* of them.
As a practical example, let's say you need to run everything in `./x.py
test` except for the standard library tests, as those tests require IPv6
and need to be executed on a separate machine. Before this commit, if
you were to just run this:
./x.py test --exclude library/std
...the execution would fail, as that would not only exclude running the
tests for the standard library, it would also exclude generating its
documentation (breaking linkchecker).
This commit adds support for an optional module annotation in --exclude
paths, allowing the user to choose which module to exclude from:
./x.py test --exclude test::library/std
This maintains backward compatibility, but also allows for more ganular
exclusion. More examples on how this works:
| `--exclude` | Docs | Tests |
| ------------------- | ------- | ------- |
| `library/std` | Skipped | Skipped |
| `doc::library/std` | Skipped | Run |
| `test::library/std` | Run | Skipped |
Note that the new behavior only works in the `--exclude` flag, and not
in other x.py arguments or flags yet.
|
|
|
|
Building a dozen separate regexps for each test in compiletest consumes significant amounts of CPU cycles.
Using `RUSTC_FORCE_INCR_COMP_ARTIFACT_HEADER` stabilizes hashes calcuated for the individual tests so
no test-dependent normalization is needed. Hashes for the standard library still change so some
normalizations are still needed.
|
|
Add manifest docs fallback.
This adds a fallback so that the rustup manifest will contain the rust-docs component for all hosts. There is a mapping so that the docs that get downloaded are roughly close to the actual host. There inevitably will be things that don't match. Ideally the standard library docs would be the same for every platform (`cfg(doc)` goes a long way towards this), but there are still lots of minor differences.
Closes #69525
|
|
Apparently it changes some tool sources and invalidates their fingerprints, forcing us to build them several times (before and after vendoring sources).
I have not dug into why vendoring actually invalidates the figreprints, but the moving the vendoring lower in the pipeline seems to avoid the issue.
I could imagine that we somehow write a .cargo/config somewhere which somehow makes subsequent builds use the vendored deps but I was not able to find anything.
I checked the sizes of generated archives pre and post patch and their are the same, so I hope there is not functional change.
Fixes #93033
|
|
This builder is the slowest in the fleet. This should cut a considerable
amount of time. The manifest should now include the docs from
x86_64-apple-darwin. Although those docs are slightly different, it
should be close enough. When aarch64-apple-darwin heads towards tier 1,
we can revisit whether or not to re-enable the docs.
|
|
Exclude llvm-libunwind from the self-contained set on s390x-musl targets
llvm-libunwind does not support s390x targets at present, so we cannot build it
for s390x targets. Accordingly, remove it from the self-contained set.
|
|
llvm-libunwind does not support s390x targets at present, so we cannot build it
for s390x targets. Accordingly, remove it from the self-contained set.
|
|
|
|
Break the loop
A missing break statement lead to an infinite loop in bootstrap.py.
I also added a short sleep so it's not constantly running at 100%. But I can remove that if it's not wanted.
Fixes #76661
|
|
This reverts commit 241160de72b5b55187ca54243e2a6e82e336d07c.
|
|
A missing break statement lead to an infinite loop in bootstrap.py.
|
|
Label more build steps
Some small improvements.
r? ```@Mark-Simulacrum```
|
|
|
|
They're a bit out of date, and overly complicated.
|
|
Currently the output of a command like `./x.py build --stage 0 library/std` is this:
```
Updating only changed submodules
Submodules updated in 0.02 seconds
extracting [...]
Compiling [...]
Finished dev [unoptimized] target(s) in 17.53s
Building stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Compling [...]
Finished release [optimized + debuginfo] target(s) in 21.99s
Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Build completed successfully in 0:00:51
```
I find the part before the "Building stage0 std artifacts" a bit confusing.
After this commit, it looks like this:
```
Updating only changed submodules
Submodules updated in 0.02 seconds
extracting [...]
Building rustbuild
Compiling [...]
Finished dev [unoptimized] target(s) in 17.53s
Building stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Compling [...]
Finished release [optimized + debuginfo] target(s) in 21.99s
Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Build completed successfully in 0:00:51
```
The "Building rustbuild" label makes it clear what the first cargo build
invocation is for. The indentation of the "Submodules updated" line
indicates it is a sub-step of a parent task.
|
|
|
|
|
|
This slightly improves compilation time by reducing linking time
(saving about a 1/10 of the the total compilation time after
changing rustbuild) and slightly reduces disk usage (from 16MB for
the rustc wrapper to 4MB).
|
|
The task of the macro is simple enough that a decl macro is almost ten
times shorter than the original proc macro. The proc macro is 159 lines
while the decl macro is just 18 lines.
This reduces the amount of dependencies of rustbuild from 45 to 37. It
also slight reduces compilation time from 47s to 44s for debug builds.
|
|
Rustbuild already depends on once_cell which in the future can be
replaced with std::lazy::Lazy.
|
|
Lock bootstrap (x.py) build directory
Closes #76661, closes #80849,
`x.py` creates a lock file at `project_root/lock.db`
r? `@jyn514` , because he was one that told me about this~
|
|
|
|
Prevent spurious build failures and other bugs caused by parallel runs of
x.py. We back the lock with sqlite, so that we have a cross-platform locking
strategy, and which can be done nearly first in the build process (from Python),
which helps move the lock as early as possible.
|
|
DF_ORIGIN flag signifies that the object being loaded may make reference to the $ORIGIN substitution string.
Some implementations are just ignoring DF_ORIGIN and do substitution for $ORIGIN if present (whatever DF_ORIGIN pr
Set the flag inconditionally if rpath is wanted.
|
|
On Windows, `x.py dist` command panics without proper error message if 'WIX' environment variable is not set. This patch changes `Option::unwrap()` to `Option::expect()`.
|
|
Don't copy llvm tools to sysroot when using download-ci-llvm
Fixes #91710
|
|
Improve x.py logging and defaults a bit more
r? ```@Mark-Simulacrum```
|
|
Remove the match on `ErrorKind::Other`
It's a) superfluous and b) doesn't work any more.
|
|
Fixes #91710
|
|
This already enables `download-rustc`, so it's quick to build rustdoc,
and this makes it less confusing when changes to rustdoc aren't reflected in the docs.
Note that this uses 2 and not 1 because `download-rustc` only affects stage 2 runs.
|
|
Previously, passing `-v` would emit an overwhelming amount of logging:
```
> Std { stage: 1, target: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }
> Assemble { target_compiler: Compiler { stage: 1, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } } }
> Assemble { target_compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } } }
< Assemble { target_compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } } }
> Rustc { target: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None }, compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } } }
> Std { target: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None }, compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } } }
> StartupObjects { compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }, target: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }
< StartupObjects { compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }, target: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }
c Assemble { target_compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } } }
> Libdir { compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }, target: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }
> Sysroot { compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } } }
< Sysroot { compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } } }
< Libdir { compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }, target: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }
c Libdir { compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }, target: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }
c Sysroot { compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } } }
c Assemble { target_compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } } }
> StdLink { compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }, target_compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }, target: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }
c Libdir { compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }, target: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }
c Libdir { compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }, target: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }
< StdLink { compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }, target_compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }, target: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } }
< Std { target: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None }, compiler: Compiler { stage: 0, host: TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None } } }
... continues for another 150 lines ...
```
This info is occasionally useful when debugging bootstrap itself, but not very useful for figuring
out why a config option was ignored or command wasn't run. Demote it to `-vv` logging so that `-v`
is more useful.
|
|
download-rustc is ignored
|
|
|
|
|
