about summary refs log tree commit diff
path: root/compiler/rustc_query_system
AgeCommit message (Collapse)AuthorLines
2023-09-07Auto merge of #110050 - saethlin:better-u32-encoding, r=nnethercotebors-43/+381
Use a specialized varint + bitpacking scheme for DepGraph encoding The previous scheme here uses leb128 to encode the edge tables that represent the incr comp dependency graph. The problem with that scheme is that leb128 has overhead for larger values, and generally relies on the distribution of encoded values being heavily skewed towards smaller values. That is definitely not the case for a dep node index, since they are handed out sequentially and the whole range is covered, the distribution is actually biased in the opposite direction: Most dep nodes are large. This PR implements a different varint encoding scheme. Instead of applying varint encoding to individual dep node indices (which is extremely branchy) we now apply it per node. While being built, each node now stores its edges in a `SmallVec` with a bit of extra logic to track the max value of each edge. Then we varint encode the whole batch. This is a gamble: We save on space by only claiming 2 bits per node instead of ~3 bits per edge which is a nice savings but needs to balance out with the space overhead that a single large index in a node with a lot of edges will encode unnecessary bytes in each of that node's edge indices. Then, to keep the runtime overhead of this encoding scheme down we deserialize our indices by loading 4 bytes for each then masking off the bytes that are't ours. This is much less code and branches than leb128, but relies on having some readable bytes past the end of each edge list. We explicitly add such padding to the in-memory data during decoding. And we also do this decoding lazily, turning a dense on-disk encoding into a peak memory reduction. Then we apply a bit-packing scheme; since in https://github.com/rust-lang/rust/pull/115391 we now have unused bits on `DepKind`, we use those unused bits (currently there are 7!) to store the 2 bits that we need for the byte width of the edges in each node, then use the remaining bits to store the length of the edge list, if it fits. r? `@nnethercote`
2023-09-06Add comments with the same level of detail as the PR descriptionBen Kimock-12/+54
2023-09-04Use a specialized varint + bitpacking scheme for DepGraph encodingBen Kimock-44/+340
2023-09-03Use relative positions inside a SourceFile.Camille GILLOT-33/+7
2023-09-01Use `OnceLock` for `SingleCache`John Kåre Alsaker-6/+6
2023-08-30Don't use `wait_for_query` without the Rayon thread poolJohn Kåre Alsaker-13/+13
2023-08-29Auto merge of #114894 - Zoxc:sharded-cfg-cleanup2, r=cjgillotbors-47/+10
Remove conditional use of `Sharded` from query state `Sharded` is already a zero cost abstraction, so it shouldn't affect the performance of the single thread compiler if LLVM does its job. r? `@cjgillot`
2023-08-27Pass ErrorGuaranteed to cycle errorMichael Goulet-6/+10
2023-08-25Fix waiting on a query that panickedJohn Kåre Alsaker-1/+12
2023-08-24Optimize `lock_shards`John Kåre Alsaker-9/+5
2023-08-24Remove conditional use of `Sharded` from query stateJohn Kåre Alsaker-43/+10
2023-08-24Auto merge of #114860 - Zoxc:sharded-layout, r=SparrowLiibors-1/+1
Make `Sharded` an enum and specialize it for the single thread case This changes `Sharded` to use a single shard by an enum, reducing the size of `Sharded` for greater cache efficiency. Performance improvement with 1 thread and `cfg(parallel_compiler)`: <table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check</td><td align="right">1.7009s</td><td align="right">1.6748s</td><td align="right">💚 -1.53%</td></tr><tr><td>🟣 <b>hyper</b>:check</td><td align="right">0.2525s</td><td align="right">0.2451s</td><td align="right">💚 -2.90%</td></tr><tr><td>🟣 <b>regex</b>:check</td><td align="right">0.9519s</td><td align="right">0.9353s</td><td align="right">💚 -1.74%</td></tr><tr><td>🟣 <b>syn</b>:check</td><td align="right">1.5504s</td><td align="right">1.5280s</td><td align="right">💚 -1.45%</td></tr><tr><td>🟣 <b>syntex_syntax</b>:check</td><td align="right">5.9536s</td><td align="right">5.8873s</td><td align="right">💚 -1.11%</td></tr><tr><td>Total</td><td align="right">10.4092s</td><td align="right">10.2706s</td><td align="right">💚 -1.33%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9825s</td><td align="right">💚 -1.75%</td></tr></table> I did see an unexpected 0.23% change for the serial compiler, so this could use a perf run to see if that reproduces. cc `@SparrowLii`
2023-08-16Remove conditional use of `Sharded` from query cachesJohn Kåre Alsaker-49/+7
2023-08-16Keep SHARDS fixed instead of a function of `cfg!(parallel_compiler)`John Kåre Alsaker-1/+1
2023-08-13Pass WorkProductMap to build_dep_graph instead of FxIndexMapbjorn3-5/+7
Constructing an FxIndexMap is useless work as the iteration order never matters.
2023-08-08Rollup merge of #114566 - fmease:type-alias-laziness-is-crate-specific, ↵Matthias Krüger-1/+4
r=oli-obk Store the laziness of type aliases in their `DefKind` Previously, we would treat paths referring to type aliases as *lazy* type aliases if the current crate had lazy type aliases enabled independently of whether the crate which the alias was defined in had the feature enabled or not. With this PR, the laziness of a type alias depends on the crate it is defined in. This generally makes more sense to me especially if / once lazy type aliases become the default in a new edition and we need to think about *edition interoperability*: Consider the hypothetical case where the dependency crate has an older edition (and thus eager type aliases), it exports a type alias with bounds & a where-clause (which are void but technically valid), the dependent crate has the latest edition (and thus lazy type aliases) and it uses that type alias. Arguably, the bounds should *not* be checked since at any time, the dependency crate should be allowed to change the bounds at will with a *non*-major version bump & without negatively affecting downstream crates. As for the reverse case (dependency: lazy type aliases, dependent: eager type aliases), I guess it rules out anything from slight confusion to mild annoyance from upstream crate authors that would be caused by the compiler ignoring the bounds of their type aliases in downstream crates with older editions. --- This fixes #114468 since before, my assumption that the type alias associated with a given weak projection was lazy (and therefore had its variances computed) did not necessarily hold in cross-crate scenarios (which [I kinda had a hunch about](https://github.com/rust-lang/rust/pull/114253#discussion_r1278608099)) as outlined above. Now it does hold. `@rustbot` label F-lazy_type_alias r? `@oli-obk`
2023-08-07Store the laziness of type aliases in the DefKindLeón Orell Valerian Liehr-1/+4
2023-08-04replace few explicit use of parking_lot with rustc_data_structures::sync oncesklensy-14/+12
2023-08-03bump parking_lot 0.11 to 0.12klensy-1/+1
2023-07-30inline format!() args up to and including rustc_middleMatthias Krüger-2/+2
2023-07-21Revert "Auto merge of #113166 - moulins:ref-niches-initial, r=oli-obk"David Tolnay-2/+1
This reverts commit 557359f92512ca88b62a602ebda291f17a953002, reversing changes made to 1e6c09a803fd543a98bfbe1624d697a55300a786.
2023-07-21add `naive_layout_of` queryMoulins-1/+2
2023-07-20Auto merge of #113622 - RickleAndMortimer:issue-113184-fix, r=oli-obkbors-0/+4
add links to query documentation for E0391 This PR adds links to https://rustc-dev-guide.rust-lang.org/overview.html#queries and https://rustc-dev-guide.rust-lang.org/query.html for the rustc --explain E0391 and within the compiler error itself. Fixes: #113184
2023-07-19On nightly, dump ICE backtraces to diskEsteban Küber-12/+33
Implement rust-lang/compiler-team#578. When an ICE is encountered on nightly releases, the new rustc panic handler will also write the contents of the backtrace to disk. If any `delay_span_bug`s are encountered, their backtrace is also added to the file. The platform and rustc version will also be collected.
2023-07-18moved note as unspanned note, moved note to the bottom of the msgnxya-1/+3
2023-07-18added links as a notenxya-1/+3
2023-07-18add links to query documentation for E0391nxya-1/+1
2023-07-05Show which type was not specialized on query cycle misuseOli Scherer-2/+5
2023-07-05Remove a redundant argumentOli Scherer-11/+5
2023-06-08Rollup merge of #112333 - Zoxc:try_collect_active_jobs-deadlock, r=cjgillotMatthias Krüger-4/+11
Don't hold the active queries lock while calling `make_query` This moves the call to `make_query` outside the parts that holds the active queries lock in `try_collect_active_jobs`. This should help removed the deadlock and borrow panic that has been observed when printing the query stack during an ICE. cc `@SparrowLii` r? `@cjgillot`
2023-06-08Removed stable/unstable sort arg from into_sorted_stable_ord, fixed a few ↵Andrew Xie-1/+5
misc issues, added collect to UnordItems
2023-06-06Don't hold the active queries lock while calling `make_query`John Kåre Alsaker-4/+11
2023-06-04Fixed failing test + minor cleanupAndrew Xie-1/+2
2023-06-04Switched some uses to UnordMapAndrew Xie-1/+8
2023-06-04Removed use of iteration through a HashMap/HashSet in rustc_incremental and ↵Andrew Xie-5/+5
replaced with IndexMap/IndexSet
2023-05-25Ensure Fluent messages are in alphabetical orderclubby789-15/+15
2023-05-25Rollup merge of #111875 - WaffleLapkin:defer_on_drop, r=NilstriebMatthias Krüger-2/+2
Don't leak the function that is called on drop It probably wasn't causing problems anyway, but still, a `// this leaks, please don't pass anything that owns memory` is not sustainable. I could implement a version which does not require `Option`, but it would require `unsafe`, at which point it's probably not worth it.
2023-05-24Use `Option::is_some_and` and `Result::is_ok_and` in the compilerMaybe Waffle-5/+5
2023-05-23Don't leak the function that is called on dropMaybe Waffle-2/+2
2023-05-17Delay a bug when overwriting fed value.Camille GILLOT-10/+16
2023-05-14Specialize query execution for incremental and non-incrementalJohn Kåre Alsaker-10/+36
2023-05-14Auto merge of #108638 - Zoxc:erase-query-values-map, r=cjgillotbors-52/+42
Use dynamic dispatch for queries This replaces most concrete query values `V` with `MaybeUninit<[u8; { size_of::<V>() }]>` reducing the code instantiated by queries. The compile time of `rustc_query_impl` is reduced by 27%. It is an alternative to https://github.com/rust-lang/rust/pull/107937 which uses unstable const generics while this uses a `EraseType` trait which maps query values to their erased variant. This is achieved by introducing an `Erased` type which does sanity check with `cfg(debug_assertions)`. The query caches gets instantiated with these erased types leaving the code in `rustc_query_system` unaware of them. `rustc_query_system` is changed to use instances of `QueryConfig` so that `rustc_query_impl` can pass in `DynamicConfig` which holds a pointer to a virtual table. <table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check</td><td align="right">1.7055s</td><td align="right">1.6949s</td><td align="right"> -0.62%</td></tr><tr><td>🟣 <b>hyper</b>:check</td><td align="right">0.2547s</td><td align="right">0.2528s</td><td align="right"> -0.73%</td></tr><tr><td>🟣 <b>regex</b>:check</td><td align="right">0.9590s</td><td align="right">0.9553s</td><td align="right"> -0.39%</td></tr><tr><td>🟣 <b>syn</b>:check</td><td align="right">1.5457s</td><td align="right">1.5440s</td><td align="right"> -0.11%</td></tr><tr><td>🟣 <b>syntex_syntax</b>:check</td><td align="right">5.9092s</td><td align="right">5.9009s</td><td align="right"> -0.14%</td></tr><tr><td>Total</td><td align="right">10.3741s</td><td align="right">10.3479s</td><td align="right"> -0.25%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9960s</td><td align="right"> -0.40%</td></tr></table> <table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check:initial</td><td align="right">2.0605s</td><td align="right">2.0575s</td><td align="right"> -0.15%</td></tr><tr><td>🟣 <b>hyper</b>:check:initial</td><td align="right">0.3218s</td><td align="right">0.3216s</td><td align="right"> -0.07%</td></tr><tr><td>🟣 <b>regex</b>:check:initial</td><td align="right">1.1848s</td><td align="right">1.1839s</td><td align="right"> -0.07%</td></tr><tr><td>🟣 <b>syn</b>:check:initial</td><td align="right">1.9409s</td><td align="right">1.9376s</td><td align="right"> -0.17%</td></tr><tr><td>🟣 <b>syntex_syntax</b>:check:initial</td><td align="right">7.3105s</td><td align="right">7.2928s</td><td align="right"> -0.24%</td></tr><tr><td>Total</td><td align="right">12.8185s</td><td align="right">12.7935s</td><td align="right"> -0.20%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9986s</td><td align="right"> -0.14%</td></tr></table> <table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check:unchanged</td><td align="right">0.4606s</td><td align="right">0.4617s</td><td align="right"> 0.24%</td></tr><tr><td>🟣 <b>hyper</b>:check:unchanged</td><td align="right">0.1335s</td><td align="right">0.1336s</td><td align="right"> 0.08%</td></tr><tr><td>🟣 <b>regex</b>:check:unchanged</td><td align="right">0.3324s</td><td align="right">0.3346s</td><td align="right"> 0.65%</td></tr><tr><td>🟣 <b>syn</b>:check:unchanged</td><td align="right">0.6268s</td><td align="right">0.6307s</td><td align="right"> 0.64%</td></tr><tr><td>🟣 <b>syntex_syntax</b>:check:unchanged</td><td align="right">1.8248s</td><td align="right">1.8508s</td><td align="right">💔 1.43%</td></tr><tr><td>Total</td><td align="right">3.3779s</td><td align="right">3.4113s</td><td align="right"> 0.99%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">1.0061s</td><td align="right"> 0.61%</td></tr></table> It's based on https://github.com/rust-lang/rust/pull/108167. r? `@cjgillot`
2023-05-03Restrict `From<S>` for `{D,Subd}iagnosticMessage`.Nicholas Nethercote-1/+1
Currently a `{D,Subd}iagnosticMessage` can be created from any type that impls `Into<String>`. That includes `&str`, `String`, and `Cow<'static, str>`, which are reasonable. It also includes `&String`, which is pretty weird, and results in many places making unnecessary allocations for patterns like this: ``` self.fatal(&format!(...)) ``` This creates a string with `format!`, takes a reference, passes the reference to `fatal`, which does an `into()`, which clones the reference, doing a second allocation. Two allocations for a single string, bleh. This commit changes the `From` impls so that you can only create a `{D,Subd}iagnosticMessage` from `&str`, `String`, or `Cow<'static, str>`. This requires changing all the places that currently create one from a `&String`. Most of these are of the `&format!(...)` form described above; each one removes an unnecessary static `&`, plus an allocation when executed. There are also a few places where the existing use of `&String` was more reasonable; these now just use `clone()` at the call site. As well as making the code nicer and more efficient, this is a step towards possibly using `Cow<'static, str>` in `{D,Subd}iagnosticMessage::{Str,Eager}`. That would require changing the `From<&'a str>` impls to `From<&'static str>`, which is doable, but I'm not yet sure if it's worthwhile.
2023-04-30Use dynamic dispatch for queriesJohn Kåre Alsaker-52/+42
2023-04-29Auto merge of #109611 - Zoxc:query-engine-rem, r=cjgillotbors-1/+1
Remove `QueryEngine` trait This removes the `QueryEngine` trait and `Queries` from `rustc_query_impl` and replaced them with function pointers and fields in `QuerySystem`. As a side effect `OnDiskCache` is moved back into `rustc_middle` and the `OnDiskCache` trait is also removed. This has a couple of benefits. - `TyCtxt` is used in the query system instead of the removed `QueryCtxt` which is larger. - Function pointers are more flexible to work with. A variant of https://github.com/rust-lang/rust/pull/107802 is included which avoids the double indirection. For https://github.com/rust-lang/rust/pull/108938 we can name entry point `__rust_end_short_backtrace` to avoid some overhead. For https://github.com/rust-lang/rust/pull/108062 it avoids the duplicate `QueryEngine` structs. - `QueryContext` now implements `DepContext` which avoids many `dep_context()` calls in `rustc_query_system`. - The `rustc_driver` size is reduced by 0.33%, hopefully that means some bootstrap improvements. - This avoids the unsafe code around the `QueryEngine` trait. r? `@cjgillot`
2023-04-27Rollup merge of #110886 - nnethercote:dep-graph-cleanups, r=cjgillotMatthias Krüger-64/+32
`DepGraph` cleanups r? `@cjgillot`
2023-04-27Factor out common code in `intern_node`.Nicholas Nethercote-54/+26
There are three very similar blocks in this function.
2023-04-27Clean up `with_task`.Nicholas Nethercote-10/+6
Currently it creates an `Option` and then does `map`/`unwrap_or` and `map_or_else` on it, which is hard to read. This commit simplifies things by moving more code into the two arms of the if/else.
2023-04-26Add query accessor functionsJohn Kåre Alsaker-1/+1
2023-04-26Auto merge of #110634 - saethlin:pointy-decoder, r=cjgillotbors-9/+7
Rewrite MemDecoder around pointers not a slice This is basically https://github.com/rust-lang/rust/pull/109910 but I'm being a lot more aggressive. The pointer-based structure means that it makes a lot more sense to absorb more complexity into `MemDecoder`, most of the diff is just complexity moving from one place to another. The primary argument for this structure is that we only incur a single bounds check when doing multi-byte reads from a `MemDecoder`. With the slice-based implementation we need to do those with `data[position..position + len]` , which needs to account for `position + len` wrapping. It would be possible to dodge the first bounds check if we stored a slice that starts at `position`, but that would require updating the pointer and length on every read. This PR also embeds the failure path in a separate function, which means that this PR should subsume all the perf wins observed in https://github.com/rust-lang/rust/pull/109867.
generated by cgit-pink 1.4.1-3-g733a5 (git 2.37.1) at 2026-02-26 01:41:14 +0000