| Age | Commit message (Collapse) | Author | Lines |
|---|
|
|
|
Fix NixOS patching
Moving the `.nix-deps` has resulted in rpath links being broken and
therefore bootstrap on NixOS broken entirely.
This PR still produces a `.nix-deps` but only for the purposes of
producing a gc root. We rpath a symlink-resolved result instead.
For purposes of simplicity we also use joinSymlink to produce a single
merged output directory so that we don't need to update multiple
locations every time we add a library or something.
Fixes a regression from https://github.com/rust-lang/rust/pull/82739.
|
|
Moving the `.nix-deps` has resulted in rpath links being broken and
therefore bootstrap on NixOS broken entirely.
This PR still produces a `.nix-deps` but only for the purposes of
producing a gc root. We rpath a symlink-resolved result instead.
For purposes of simplicity we also use joinSymlink to produce a single
merged output directory so that we don't need to update multiple
locations every time we add a library or something.
|
|
reduce threads spawned by ui-tests
The test harness already spawns enough tests to keep all cores busy.
Individual tests should keep their own threading to a minimum to avoid context switch overhead.
When running ui tests with lld enabled this shaves about 10% off that testsuite on my machine.
Resolves #81946
|
|
|
|
the test harness already spawns enough tests for all cores, individual
tests should keep their own threading to a minimum to avoid context switch
overhead
some tests fail with 1 CGU, so explicit compile flags have been added
to keep their old behavior
|
|
Cleanup option parsing and config.toml.example
- Add an assertion that `link-shared = true` when `thin-lto = true`.
Previously, link-shared would be silently overwritten.
- Get rid of `Option<bool>` in bootstrap/config.rs. Set defaults
immediately instead of delaying until later in bootstrap. This makes
it easier to find what the default value is.
- Remove redundant `config.x = false` when the default was already false
- Set defaults for `bindir` in `default_opts()` instead of `parse()`
- Update `download-ci-llvm = if-supported` option to match bootstrap.py
- Remove redundant check for link_shared. Previously, it was checked twice.
- Update various options in config.toml.example to their defaults.
Previously, some options showed an example value instead of the
default value.
- Fix incorrect defaults in config.toml.example
+ `use-libcxx` defaults to false
+ Add missing `check-stage = 0`
+ Update several defaults to be conditional (e.g. `if incremental { 10 } else { 100 }`)
- Remove redundant defaults in prose
- Use the same comment for the default and target-dependent `musl-root`
- Fix typos
- Link to `cc_detect` for `cc` and `cxx`, since the logic is ... complicated.
- Update more defaults to better reflect how they actually get set
- Remove ignored `gpg-password-file` option
This stopped being used in
7704d35,
but was never removed from config.toml.
- Remove unused flags from `config.toml`
+ Disallow `infodir` and `localstatedir` in `config.toml`
+ Allow the flags in `./configure`, but give a warning that they will be
ignored.
+ Fix incorrect comment that `datadir` will be ignored.
Example output:
```
$ ./configure --set install.infodir=xxx
configure: processing command line
configure:
configure: install.infodir := xxx
configure: build.configure-args := ['--set', 'install.infodir=xxx']
warning: infodir will be ignored
configure:
configure: writing `config.toml` in current directory
configure:
configure: run `python /home/joshua/rustc3/x.py --help`
configure:
```
- Update CHANGELOG
cc https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/bootstrap.20defaults
|
|
- Add an assertion that `link-shared = true` when `thin-lto = true`.
Previously, link-shared would be silently overwritten.
- Get rid of `Option<bool>` in bootstrap/config.rs. Set defaults
immediately instead of delaying until later in bootstrap. This makes
it easier to find what the default value is.
- Remove redundant `config.x = false` when the default was already false
- Set defaults for `bindir` in `default_opts()` instead of `parse()`
- Update `download-ci-llvm = if-supported` option to match bootstrap.py
- Remove redundant check for link_shared. Previously, it was checked twice.
- Update various options in config.toml.example to their defaults.
Previously, some options showed an example value instead of the
default value.
- Fix incorrect defaults in config.toml.example
+ `use-libcxx` defaults to false
+ Add missing `check-stage = 0`
+ Update several defaults to be conditional (e.g. `if incremental { 10 } else { 100 }`)
- Remove redundant defaults in prose
- Use the same comment for the default and target-dependent `musl-root`
- Fix typos
- Link to `cc_detect` for `cc` and `cxx`, since the logic is ... complicated.
- Update more defaults to better reflect how they actually get set
- Remove ignored `gpg-password-file` option
This stopped being used in
https://github.com/rust-lang/rust/commit/7704d35accfe1b587ce41ea09ca3bf6a47aca117,
but was never removed from config.toml.
- Remove unused flags from `config.toml`
+ Disallow `infodir` and `localstatedir` in `config.toml`
+ Allow the flags in `./configure`, but give a warning that they will be
ignored.
+ Fix incorrect comment that `datadir` will be ignored.
Example output:
```
$ ./configure --set install.infodir=xxx
configure: processing command line
configure:
configure: install.infodir := xxx
configure: build.configure-args := ['--set', 'install.infodir=xxx']
warning: infodir will be ignored
configure:
configure: writing `config.toml` in current directory
configure:
configure: run `python /home/joshua/rustc3/x.py --help`
configure:
```
- Update CHANGELOG
- Add "as an example" where appropriate
- Link to an issue instead of to ephemeral chats
|
|
|
|
Rollup of 8 pull requests
Successful merges:
- #83370 (Add `x.py setup tools` which enables `download-rustc` by default)
- #83489 (Properly suggest deref in else block)
- #83734 (Catch a bad placeholder type error for statics in `extern`s)
- #83814 (expand: Do not ICE when a legacy AST-based macro attribute produces and empty expression)
- #83835 (rustdoc: sort search index items for compression)
- #83849 (rustdoc: Cleanup handling of associated items for intra-doc links)
- #83881 (:arrow_up: rust-analyzer)
- #83885 (Document compiler/ with -Aprivate-intra-doc-links)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
Document compiler/ with -Aprivate-intra-doc-links
Since compiler/ always passes --document-private-items, it's ok to link to items that are private.
|
|
Use tikv-jemallocator in rustc/rustdoc in addition to jemalloc-sys when enabled.
In https://github.com/rust-lang/rust/pull/81782 it was mentioned that one reason rustc may benefit from minimalloc is it doesn't use the `sdallocx` api from jemalloc.
Currently, on unix, rust uses jemalloc by importing its symbols to use them with the default, System (libc) global allocator.
This PR switches its global alloc to `tikv-jemallocator`, which correctly uses sized deallocation (https://docs.rs/tikv-jemallocator/0.4.1/src/tikv_jemallocator/lib.rs.html#121-126). `tikv-jemallocator`, as far as I can tell, is a more up-to-date set of bindings to jemalloc than `jemallocator`
The perf results of this pr are in large part due to the version upgrade of jemalloc, but sized deallocation has a non-trivial improvement, particularly to rustdoc.
This pr also includes changes to bootstrap to correctly pass the jemalloc feature through to the rustdoc build
|
|
Otherwise no one will be able to find the setting.
|
|
Add `download-rustc = "if-unchanged"`
This allows keeping the setting to a fixed value without having to
toggle it when you want to work on the compiler instead of on tools.
This sets `BOOTSTRAP_DOWNLOAD_RUSTC` in bootstrap.py so rustbuild doesn't have to try and replicate its logic.
Helps with https://github.com/rust-lang/rust/issues/81930.
r? `@Mark-Simulacrum` cc `@camelid`
|
|
This allows keeping the setting to a fixed value without having to
toggle it when you want to work on the compiler instead of on tools.
|
|
Since compiler/ always passes --document-private-items, it's ok to link
to items that are private.
|
|
Use the beta compiler for building bootstrap tools when `download-rustc` is set
## Motivation
This avoids having to rebuild bootstrap and tidy each time you rebase
over master. In particular, it makes rebasing and running `x.py fmt` on
each commit in a branch significantly faster. It also avoids having to
rebuild bootstrap after setting `download-rustc = true`.
## Implementation
Instead of extracting the CI artifacts directly to `stage0/`, extract
them to `ci-rustc/` instead. Continue to copy them to the proper
sysroots as necessary for all stages except stage 0.
This also requires `bootstrap.py` to download both stage0 and CI
artifacts and distinguish between the two when checking stamp files.
Note that since tools have to be built by the same compiler that built
`rustc-dev` and the standard library, the downloaded artifacts can't be
reused when building with the beta compiler. To make sure this is still
a good user experience, warn when building with the beta compiler, and
default to building with stage 2.
I tested this by rebasing this PR from edeee915b1c52f97411e57ef6b1a8bd46548a37a over 1c77a1fa3ca574f2a40056f64d498db8efe0d8a8 and confirming that only the bootstrap library itself had to be rebuilt, not any dependencies and not `tidy`. I also tested that a clean build with `x.py build` builds rustdoc exactly once and does no other work, and that `touch src/librustdoc/lib.rs && x.py build` works. `x.py check` still behaves as before (checks using the beta compiler, even if there are changes to `compiler/`).
Helps with https://github.com/rust-lang/rust/issues/81930.
r? `@Mark-Simulacrum`
|
|
|
|
## Motivation
This avoids having to rebuild bootstrap and tidy each time you rebase
over master. In particular, it makes rebasing and running `x.py fmt` on
each commit in a branch significantly faster. It also avoids having to
rebuild bootstrap after setting `download-rustc = true`.
## Implementation
Instead of extracting the CI artifacts directly to `stage0/`, extract
them to `ci-rustc/` instead. Continue to copy them to the proper
sysroots as necessary for all stages except stage 0.
This also requires `bootstrap.py` to download both stage0 and CI
artifacts and distinguish between the two when checking stamp files.
Note that since tools have to be built by the same compiler that built
`rustc-dev` and the standard library, the downloaded artifacts can't be
reused when building with the beta compiler. To make sure this is still
a good user experience, warn when building with the beta compiler, and
default to building with stage 2.
|
|
1.52 Cargo adds rust-lang/cargo#8640 which means that cargo will try to purge
the doc directory caches for us. In theory this may mean that we can jettison
the clear_if_dirty for rustdoc versioning entirely, but for now just workaround
the effects of this change in a less principled but more local way.
|
|
|
|
Parallelize tidy
Split off from #81833
While that PR brings wall time of `x.py test tidy` down to 0m2.847s adding this one on top should bring it down to 0m1.673s.
r? `@Mark-Simulacrum`
Previous concerns can be found at https://github.com/rust-lang/rust/pull/81833#issuecomment-782754685 and https://github.com/rust-lang/rust/pull/81833#discussion_r575194633
|
|
|
|
Adds bootstrap rules to support installing rust-demangler.
When compiling with `-Z instrument-coverage`, the coverage reports are
generated by `llvm-cov`. `llvm-cov` includes a built-in demangler for
C++, and an option to supply an alternate demangler. For Rust, we have
`rust-demangler`, currently used in `rustc` coverage tests.
Fuchsia's toolchain for Rust is built via `./x.py install`. Fuchsia is
adding support for Rust coverage, and we need to include the
`rust-demangler` in the installed `bin` directory.
Configured rust-demangler as an in-tree extended tool.
Added tests to support `./x.py test rust-demangler`.
Install with extended tools by default only if `profiler = true`.
|
|
We don't need to complain to the user about linkcheck having different
hosts and targets when it is already excluded. This can be achieved by
moving the check to when the step is run instead of in should_run.
|
|
Riscv64linux Test fixes
Get tests passing again using the riscv64gc-unknown-linux-gnu docker image.
Test with
```
src/ci/docker/run.sh riscv64gc-linux
```
## linkcheck
Linkcheck tests that interdocument links in the documentation are correct. Some interdocument links go between rustc and tools (such as rustdoc and cargo). When cross compiling, rustc is built for the host while some tools are built for the target. This goes for the documentation too. Because of this, links in the rustc documentation reffering to cargo or rustdoc documentation look broken.
This issue is worked around by disabling linkcheck for cross compilation builds.
## run-make tests
#78911 seems to happen because `--target` was not passed to `rustc`, but the target linker was specified, causing the target linker to be called with options intended for the host.
Resolves #78911
In a separate issue, `issue-36710` was trying to run a binary built for the target on the host system. This will not work for any platform using `remote-test-server`/`client` (such as riscv64). I don't know of a way of skipping those platforms specifically, so I set this test to skip only on riscv64 for now.
|
|
When we cross compile, some things (and their documentation) are built
for the host (e.g. rustc), while others (and their documentation) are built
for the target. This generated documentation will have broken links
between documentation for different platforms e.g. between rustc and
cargo.
|
|
|
|
Fix bootstrap tests on beta
Forward-porting a fix from the beta branch, which led to test failure on beta.
r? ``@pietroalbini``
|
|
|
|
|
|
When bumping the bootstrap version, the name of the generated LLVM
shared object file is changed, even though it's the same contents as
before. If bootstrap tries to use an older version, it will get linking
errors:
```
Building rustdoc for stage1 (x86_64-unknown-linux-gnu)
Compiling rustdoc-tool v0.0.0 (/home/joshua/rustc/src/tools/rustdoc)
error: linking with `cc` failed: exit code: 1
|
= note: "cc" "-Wl,--as-needed" ... lots of args ...
= note: /usr/bin/ld: cannot find -lLLVM-12-rust-1.53.0-nightly
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: could not compile `rustdoc-tool`
```
|
|
Attempt to gather similar stats as rusage on Windows
A follow up to #82532. This is a bit hacked in because I think we need to discuss this before merging, but this is an attempt to gather similar metrics as `libc::rusage` on Windows.
Some comments on differences:
* Currently, we're passing `RUSAGE_CHILDREN` to `rusage` which collects statistics on all children that have been waited on and terminated. I believe this is currently just the invocation of the real `rustc` that the shim is wrapping. Does `rustc` itself spawn children processes? The windows version gets the child processes handle when spawning it, and uses that to collect the statistics. For maxrss, `rusage` will return "the resident set size of the largest child, not the maximum resident set size of the process tree.", the Windows version will only collect statistics on the wrapped `rustc` child process directly even if some theoretical sub process has a larger memory footprint.
* There might be subtle differences between `rusage`'s "resident set" and Window's "working set". The "working set" and "resident set" should both be the number of pages that are in memory and which would not cause a page fault when accessed.
* I'm not yet sure how best to get the same information that `ru_minflt`, `ru_inblock`, `ru_oublock`, `ru_nivcsw ` and `ru_nvcsw` provide.
r? `@pnkfelix`
|
|
checkout without other mods.
To be clear, by default running `x.py test` on a checkout of the beta branch
currently fails, and with this change will continue to fail, because `x.py
tests` runs `x.py test src/tools/tidy` which tries to run `rustfmt` and that
will fail because the `rustfmt` binary is pinned to the current nighlty and we
do not attempt to distribute one for the beta builds.
This change gives a better error message than the current message, which is just
"./x.py fmt is not supported on this channel" without providing any hint about
what one might do about that problem.
(update: placated tidy.)
|
|
|
|
This way, you can debug rustdoc's JavaScript and CSS file
with normal F12 Dev Tools and you'll have useful line numbers
to work with.
|
|
|
|
WASI: Switch to crt1-command.o to enable support for new-style commands
This switches Rust's WASI target to use crt1-command.o instead of
crt1.o, which enables support for new-style commands. By default,
new-style commands work the same way as old-style commands, so nothing
immediately changes here, but this will be needed by later changes to
enable support for typed arguments.
See here for more information on new-style commands:
- https://github.com/WebAssembly/wasi-libc/pull/203
- https://reviews.llvm.org/D81689
r? ```@alexcrichton```
|
|
Build rustdoc for run-make tests, not just run-make-fulldeps
Rustdoc almost never needs a full stage 2 compiler, and requiring
rustdoc tests to be in run-make-fulldeps adds a lot of compile time for
no reason.
This is the same change from https://github.com/rust-lang/rust/pull/81197, but separated into its own PR. I ran into this again today while working on https://github.com/rust-lang/docs.rs/issues/1302.
r? ```@Mark-Simulacrum```
|
|
This switches Rust's WASI target to use crt1-command.o instead of
crt1.o, which enables support for new-style commands. By default,
new-style commands work the same way as old-style commands, so nothing
immediately changes here, but this will be needed by later changes to
enable support for typed arguments.
See here for more information on new-style commands:
- https://github.com/WebAssembly/wasi-libc/pull/203
- https://reviews.llvm.org/D81689
|
|
|
|
unconditionally.
1. I added `--test` based on review feedback from simulacrum: I decided I would
rather include such extra context than get confused later on by its absence.
(However, I chose to encode it differently than how `[RUSTC-TIMING]` does... I
don't have much basis for doing so, other than `--test` to me more directly
reflects what it came from.)
2. I also decided to include `[RUSTC-SHIM]` at start of all of these lines
driven by the verbosity level, to make to clear where these lines of text
originate from. (Basically, I skimmed over the output and realized that a casual
observer might not be able to tell where this huge set of new lines were coming
from.)
|
|
Rustdoc almost never needs a full stage 2 compiler, and requiring
rustdoc tests to be in run-make-fulldeps adds a lot of compile time for
no reason.
|
|
|
|
This change is mainly motivated by an issue with the environment
printing I added in PR 82403: multiple rustc invocations progress
in parallel, and the environment output, spanning multiple lines,
gets interleaved in ways make it difficult to extra the enviroment settings.
(This aforementioned difficulty is more of a hiccup than an outright
show-stopper, because the environment variables tend to be the same for all of
the rustc invocations, so it doesn't matter too much if one mixes up which lines
one is looking at. But still: Better to fix it.)
|
|
Upgrade to LLVM 12
This implements the necessary adjustments to make rustc work with LLVM 12. I didn't encounter any major issues so far.
r? `@cuviper`
|
|
Fix commit detected when using `download-rustc`
On reflection on the issue in https://github.com/rust-lang/rust/pull/79540#discussion_r572572280, I think the bug was actually using the `compiler/` filter, not using `--author=bors`. https://github.com/rust-lang/rust/commit/9a1d6174c925f54c923599e29b09d6855e6b3a78 has no CI artifacts because it was merged as part of a rollup:
```
$ curl -I https://ci-artifacts.rust-lang.org/rustc-builds/96e843ce6ae42e0aa519ba45e148269de347fd84/rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz
HTTP/2 404
```
So 9a1d6174c925f54c923599e29b09d6855e6b3a78 is the correct commit to download, and that's what `--author=bors` does:
$ git log --author=bors 4aec8a5da5547d6e1c24e99dad0003b7cac107f5
commit 9a1d6174c925f54c923599e29b09d6855e6b3a78
Ideally it would look for "the most recent bors commit not followed by a change to `compiler/`", which would exclude things like documentation changes and avoid redownloading more than necessary, but
- Redownloading isn't the end of the world,
- That metric is hard to implement, and
- Documentation-only or library-only changes are very rare anyway since they're usually rolled up with changes to the compiler.
Helps with https://github.com/rust-lang/rust/issues/81930.
r? `@Mark-Simulacrum`
|
|
|
|
On reflection on the issue in https://github.com/rust-lang/rust/pull/79540#discussion_r572572280, I think the bug was actually using the `compiler/` filter, not using `--author=bors`. https://github.com/rust-lang/rust/commit/9a1d6174c925f54c923599e29b09d6855e6b3a78 has no CI artifacts because it was merged as part of a rollup:
```
$ curl -I https://ci-artifacts.rust-lang.org/rustc-builds/96e843ce6ae42e0aa519ba45e148269de347fd84/rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz
HTTP/2 404
```
So 9a1d6174c925f54c923599e29b09d6855e6b3a78 is the correct commit to download, and that's what `--author=bors` does:
$ git log --author=bors 4aec8a5da5547d6e1c24e99dad0003b7cac107f5
commit 9a1d6174c925f54c923599e29b09d6855e6b3a78
Ideally it would look for "the most recent bors commit not followed by a change to `compiler/`", which would exclude things like documentation changes and avoid redownloading more than necessary, but
- Redownloading isn't the end of the world,
- That metric is hard to implement, and
- Documentation-only or library-only changes are very rare anyway since they're usually rolled up with changes to the compiler.
|
|
- Rename `broken_intra_doc_links` to `rustdoc::broken_intra_doc_links`
- Ensure that the old lint names still work and give deprecation errors
- Register lints even when running doctests
Otherwise, all `rustdoc::` lints would be ignored.
- Register all existing lints as removed
This unfortunately doesn't work with `register_renamed` because tool
lints have not yet been registered when rustc is running. For similar
reasons, `check_backwards_compat` doesn't work either. Call
`register_removed` directly instead.
- Fix fallout
+ Rustdoc lints for compiler/
+ Rustdoc lints for library/
Note that this does *not* suggest `rustdoc::broken_intra_doc_links` for
`rustdoc::intra_doc_link_resolution_failure`, since there was no time
when the latter was valid.
|
