| Age | Commit message (Collapse) | Author | Lines |
|---|
|
|
|
counts
|
|
support non-defining uses in closures
|
|
rustc_data_structures.
|
|
|
|
|
|
Remove some unnecessary `unsafe` in VecCache
I'm pretty sure, but until perf confirms,
r? ghost
|
|
|
|
setup typos check in CI
This allows to check typos in CI, currently for compiler only (to reduce commit size with fixes). With current setup, exclude list is quite short, so it worth trying?
Also includes commits with actual typo fixes.
MCP: https://github.com/rust-lang/compiler-team/issues/817
typos check currently turned for:
* ./compiler
* ./library
* ./src/bootstrap
* ./src/librustdoc
After merging, PRs which enables checks for other crates (tools) can be implemented too.
Found typos will **not break** other jobs immediately: (tests, building compiler for perf run). Job will be marked as red on completion in ~ 20 secs, so you will not forget to fix it whenever you want, before merging pr.
Check typos: `python x.py test tidy --extra-checks=spellcheck`
Apply typo fixes: `python x.py test tidy --extra-checks=spellcheck:fix` (in case if there only 1 suggestion of each typo)
Current fail in this pr is expected and shows how typo errors emitted. Commit with error will be removed after r+.
|
|
|
|
Add new self-profiling event to cheaply aggregate query cache hit counts
Self-profile can record various types of things, some of them are not enabled, like query cache hits. Rustc currently records cache hits as "instant" measureme events, which records the thread ID, current timestamp, and constructs an individual event for each such cache hit. This is incredibly expensive, in a small hello world benchmark that just depends on serde, it makes compilation with nightly go from ~3s (with `-Zself-profile`) to ~15s (with `-Zself-profile -Zself-profile-events=default,query-cache-hit`).
We'd like to add query cache hits to rustc-perf (https://github.com/rust-lang/rustc-perf/pull/2168), but there we only need the actualy cache hit counts, not the timestamp/thread ID metadata associated with it.
This PR adds a new `query-cache-hit-count` event. Instead of generating individual instant events, it simply aggregates cache hit counts per *query invocation* (so a combination of a query and its arguments, if I understand it correctly) using an atomic counter. At the end of the compilation session, these counts are then dumped to the self-profile log using integer events (in a similar fashion as how we record artifact sizes). I suppose that we could dedup the query invocations in rustc directly, but I don't think it's really required. In local experiments with the hello world + serde case, the query invocation records generated ~30 KiB more data in the self-profile, which was ~10% increase in this case.
With this PR, the overhead of `-Zself-profile` seems to be the same as before, at least on my machine, so I also enabled query cache hit counts by default when self profiling is enabled.
We should also modify `analyzeme`, specifically [this](https://github.com/rust-lang/measureme/blob/master/analyzeme/src/analysis.rs#L139), and make it load the integer events with query cache hit counts. I can do that as a follow-up, it's not required to be done in sync with this PR, and it doesn't require changes in rustc.
CC `@cjgillot`
r? `@oli-obk`
|
|
|
|
|
|
|
|
|
|
remove few allow(dead_code)
Few from serial/parallel compiler leftovers and few from bootstrap.
|
|
|
|
Bringing `rustc_rayon_core` in tree as `rustc_thread_pool`
This PR moves [`rustc_rayon_core`](https://github.com/rust-lang/rustc-rayon/tree/5fadf44/rayon-core) from commit `5fadf44` as suggested in [this zulip thread](https://rust-lang.zulipchat.com/#narrow/channel/187679-t-compiler.2Fparallel-rustc/topic/Bringing.20.60rustc_rayon_core.60.20in.20tree). I tried to split the work into separate commits so it is easy to review. The first commit is a simple copy and paste from the fork, and subsequent changes were made to use the new crate and to ensure the new crate complies with different format and lint expectations.
**Call-out:** I was also wondering if I need to make any further changes to accommodate licensing requirements.
r? oli-obk
|
|
As core uses an extern type (`ptr::VTable`), the default `?Sized` to
`MetaSized` migration isn't sufficient, and some code that previously
accepted `VTable` needs relaxed to continue to accept extern types.
Similarly, the compiler uses many extern types in `rustc_codegen_llvm`
and in the `rustc_middle::ty::List` implementation (`OpaqueListContents`)
some bounds must be relaxed to continue to accept these types.
Unfortunately, due to the current inability to relax `Deref::Target`,
some of the bounds in the standard library are forced to be stricter than
they ideally would be.
|
|
It currently only inserts separators into `usize`s, because that's all
that has been needed so far. `-Zmacro-stats` will need `isize` and `f64`
handling, so this commit adds that.
|
|
|
|
|
|
Remove all unused feature gates from the compiler
|
|
|
|
|
|
`slot_index_exhaustive` has additional complexity in its loop that only
applies for index 0. Pull that case out of the loop.
|
|
Break out bucket 0 (containing `idx < 4096`) as an early return, which
simplifies the remainder of the function, and allows optimizing the
`checked_ilog2` since it can no longer return `None`.
This reduces the runtime of `vec_cache::tests::slot_index_exhaustive`
(which calls `SlotIndex::from_index` for every `u32`, twice) from ~15.5s
to ~13.3s.
|
|
Signed-off-by: onur-ozkan <work@onurozkan.dev>
|
|
Handle regions equivalent to 'static in non_local_bounds
`non_local_bounds` would only find non local bounds that strictly bound a given region, but it's possible that a local region is equated to 'static when showing a type referencing a locally bound lifetime, such as `dyn Any + 'a` in the tests added, is well-formed. In this case we should return 'static.
closes #122704
closes #139004
|
|
At [1] it was pointed out that `cfg_match!` syntax does not actually
align well with match syntax, which is a possible source of confusion.
The comment points out that usage is instead more similar to ecosystem
`select!` macros. Rename `cfg_match!` to `cfg_select!` to match this.
Tracking issue: https://github.com/rust-lang/rust/issues/115585
[1]: https://github.com/rust-lang/rust/issues/115585#issuecomment-2346307605
|
|
|
|
Use thread local dep graph encoding
This adds thread local encoding of dep graph nodes. Each thread has a `MemEncoder` that gets flushed to the global `FileEncoder` when it exceeds 64 kB. Each thread also has a local cache of dep indices. This means there can now be empty gaps in `SerializedDepGraph`.
Indices are marked green and also allocated by the new atomic operation `DepNodeColorMap::try_mark_green` as the encoder lock is removed.
|
|
|
|
r=lcnr
Decouple SCC annotations from SCCs
This rewires SCC annotations to have them be a separate, visitor-type data structure. It was broken out of #130227, which needed them to be able to remove unused annotations after computation without recomputing the SCCs themselves.
As a drive-by it also removes some redundant code from the hot loop in SCC construction for a performance improvement.
r? lcnr
|
|
|
|
Add a jobserver proxy to ensure at least one token is always held
This adds a jobserver proxy to ensure at least one token is always held by `rustc`. Currently with `-Z threads` `rustc` can temporarily give up all its tokens, causing `cargo` to spawn additional `rustc` instances beyond the job limit.
The current behavior causes an issue with `cargo fix` which has a global lock preventing concurrent `rustc` instances, but it also holds a jobserver token, causing a deadlock when `rustc` gives up its token. That is fixed by this PR.
Fixes https://github.com/rust-lang/rust/issues/67385.
Fixes https://github.com/rust-lang/rust/issues/133873.
Fixes https://github.com/rust-lang/rust/issues/140093.
|
|
|
|
|
|
|
|
This rewires SCC annotations to have them be a separate,
visitor-type data structure. It was broken out of #130227,
which needed them to be able to remove unused annotations
after computation without recomputing the SCCs themselves.
As a drive-by it also removes some redundant code from
the hot loop in SCC construction for a performance improvement.
|
|
These have grown out of sync with the platforms.
|
|
|
|
`non_local_bounds` would only find non local bounds that strictly bound a given region,
but it's possible that a local region is equated to 'static when showing a type referencing
a locally bound lifetime, such as `dyn Any + 'a` in the tests added, is well-formed. In
this case we should return 'static.
|
|
|
|
Avoid a reverse map that is only used in diagnostics paths
r? `@petrochenkov`
iterating a map until a value matches and returning the key is bad obviously, but it happens very rarely and only on diagnostics paths. It would also be a lot cheaper with https://github.com/rust-lang/rust/pull/138995. Which is actually why I'm trying this out, that PR adds a new entry in `create_def`, which makes `create_def` show up in cachegrind. So I'm trying out if removing adding an entry in `create_def` is a perf improvement
|
|
|
|
Remove the use of Rayon iterators
This removes the use of Rayon iterators and the use of the `rustc-rayon` crate. `rustc-rayon-core` is still used however.
In parallel loops, instead of a Rayon iterator a serial iterator are used to collect items into a `Vec` and we use a parallel loop over its elements using the new `par_slice` function which is built on `rustc-rayon-core`'s `join`.
This change makes it easier to bring `rustc-rayon-core` in-tree.
Tests using 7 threads:
<table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th><td align="right">Physical Memory</td><td align="right">Physical Memory</td><td align="right">%</th><td align="right">Committed Memory</td><td align="right">Committed Memory</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check</td><td align="right">0.4827s</td><td align="right">0.4828s</td><td align="right"> 0.02%</td><td align="right">201.23 MiB</td><td align="right">201.31 MiB</td><td align="right"> 0.04%</td><td align="right">279.03 MiB</td><td align="right">279.46 MiB</td><td align="right"> 0.15%</td></tr><tr><td>🟣 <b>hyper</b>:check</td><td align="right">0.1443s</td><td align="right">0.1401s</td><td align="right">💚 -2.91%</td><td align="right">126.42 MiB</td><td align="right">126.70 MiB</td><td align="right"> 0.22%</td><td align="right">199.79 MiB</td><td align="right">199.99 MiB</td><td align="right"> 0.10%</td></tr><tr><td>🟣 <b>regex</b>:check</td><td align="right">0.3252s</td><td align="right">0.3065s</td><td align="right">💚 -5.78%</td><td align="right">161.87 MiB</td><td align="right">161.78 MiB</td><td align="right"> -0.05%</td><td align="right">229.59 MiB</td><td align="right">230.23 MiB</td><td align="right"> 0.28%</td></tr><tr><td>🟣 <b>syn</b>:check</td><td align="right">0.5845s</td><td align="right">0.5876s</td><td align="right"> 0.53%</td><td align="right">197.01 MiB</td><td align="right">196.89 MiB</td><td align="right"> -0.06%</td><td align="right">267.62 MiB</td><td align="right">267.47 MiB</td><td align="right"> -0.06%</td></tr><tr><td>Total</td><td align="right">1.5367s</td><td align="right">1.5169s</td><td align="right">💚 -1.29%</td><td align="right">686.53 MiB</td><td align="right">686.68 MiB</td><td align="right"> 0.02%</td><td align="right">976.04 MiB</td><td align="right">977.14 MiB</td><td align="right"> 0.11%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9796s</td><td align="right">💚 -2.04%</td><td align="right">1 byte</td><td align="right">1.00 bytes</td><td align="right"> 0.04%</td><td align="right">1 byte</td><td align="right">1.00 bytes</td><td align="right"> 0.12%</td></tr></table>
<table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th><td align="right">Physical Memory</td><td align="right">Physical Memory</td><td align="right">%</th><td align="right">Committed Memory</td><td align="right">Committed Memory</td><td align="right">%</th></tr><tr><td>🟠 <b>clap</b>:debug</td><td align="right">1.6371s</td><td align="right">1.6529s</td><td align="right"> 0.96%</td><td align="right">395.58 MiB</td><td align="right">396.21 MiB</td><td align="right"> 0.16%</td><td align="right">460.98 MiB</td><td align="right">461.52 MiB</td><td align="right"> 0.12%</td></tr><tr><td>🟠 <b>hyper</b>:debug</td><td align="right">0.3248s</td><td align="right">0.3210s</td><td align="right">💚 -1.16%</td><td align="right">155.16 MiB</td><td align="right">155.19 MiB</td><td align="right"> 0.02%</td><td align="right">219.21 MiB</td><td align="right">219.30 MiB</td><td align="right"> 0.04%</td></tr><tr><td>🟠 <b>regex</b>:debug</td><td align="right">1.0148s</td><td align="right">0.9929s</td><td align="right">💚 -2.16%</td><td align="right">297.96 MiB</td><td align="right">295.07 MiB</td><td align="right"> -0.97%</td><td align="right">354.53 MiB</td><td align="right">351.58 MiB</td><td align="right"> -0.83%</td></tr><tr><td>🟠 <b>syn</b>:debug</td><td align="right">1.3614s</td><td align="right">1.3717s</td><td align="right"> 0.76%</td><td align="right">319.10 MiB</td><td align="right">321.19 MiB</td><td align="right"> 0.65%</td><td align="right">378.90 MiB</td><td align="right">381.27 MiB</td><td align="right"> 0.62%</td></tr><tr><td>Total</td><td align="right">4.3381s</td><td align="right">4.3386s</td><td align="right"> 0.01%</td><td align="right">1.14 GiB</td><td align="right">1.14 GiB</td><td align="right"> -0.01%</td><td align="right">1.38 GiB</td><td align="right">1.38 GiB</td><td align="right"> 0.00%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9960s</td><td align="right"> -0.40%</td><td align="right">1 byte</td><td align="right">1.00 bytes</td><td align="right"> -0.03%</td><td align="right">1 byte</td><td align="right">1.00 bytes</td><td align="right"> -0.01%</td></tr></table>
|
|
|
|
fix "still mutable" ice while metrics are enabled
Resolves "still mutable" ICE discovered by `@matthiaskrgr` here: [#t-docs-rs > metrics intitiative @ 💬](https://rust-lang.zulipchat.com/#narrow/channel/356853-t-docs-rs/topic/metrics.20intitiative/near/510490790)
This was caused by invoking `crate_hash` before the `definitions` struct was frozen here: https://github.com/rust-lang/rust/blob/e643f59f6da3a84f43e75dea99afaa5b041ea6bf/compiler/rustc_interface/src/passes.rs#L951
resolved by moving metrics dumping to occur after `analysis` freezes the definitions
I'm guessing we didn't discover this in CI because the problem only occurs when you try to calculate the crash hash with incremental compilation enabled when it tries to freeze the definitions here: https://github.com/rust-lang/rust/blob/e643f59f6da3a84f43e75dea99afaa5b041ea6bf/compiler/rustc_middle/src/hir/map.rs#L1172
my understanding is that this causes us to freeze the definitions too early in compilation, then we subsequently try to mutate them, likely during `analysis`, and this causes the ICE.
r? `@bjorn3`
|
|
|
