| Age | Commit message (Collapse) | Author | Lines |
|---|
|
|
|
|
|
Avoid specialization in the metadata serialization code
With the exception of a perf-only specialization for byte slices and byte vectors.
This uses the same trick of introducing a new trait and having the Encodable and Decodable derives add a bound to it as used for TyEncoder/TyDecoder. The new code is clearer about which encoder/decoder uses which impl and it reduces the dependency of rustc on specialization, making it easier to remove support for specialization entirely or turn it into a construct that is only allowed for perf optimizations if we decide to do this.
|
|
|
|
|
|
Report I/O errors from rmeta encoding with emit_fatal
https://github.com/rust-lang/rust/issues/119456 reminded me that I never did systematic testing to provoke the out-of-disk ICEs so I grepped through a recent crater run (https://github.com/rust-lang/rust/pull/119440#issuecomment-1873393963) for more out-of-disk ICEs on current master and yep there's 2 in there.
So I finally cooked up a way to provoke for these crashes. I wrote a little `cdylib` crate that has a `#[no_mangle] pub extern "C" fn write` which occasionally reports `ENOSPC`, and prints a backtrace when it does.
<details><summary><strong>code for the dylib</strong></summary>
<p>
```rust
// cargo add libc rand backtrace
use rand::Rng;
#[no_mangle]
pub extern "C" fn write(
fd: libc::c_int,
buf: *const libc::c_void,
count: libc::size_t,
) -> libc::ssize_t {
if fd > 2 && rand::thread_rng().gen::<u8>() == 0 {
let mut count = 0;
backtrace::trace(|frame| {
backtrace::resolve_frame(frame, |symbol| {
if let Some(name) = symbol.name() {
if count > 3 {
eprintln!("{}", name);
}
}
count += 1;
});
true
});
unsafe {
*libc::__errno_location() = libc::ENOSPC;
}
return -1;
} else {
unsafe {
let res =
libc::syscall(libc::SYS_write, fd as usize, buf as usize, count as usize) as isize;
if res < 0 {
*libc::__errno_location() = -res as i32;
-1
} else {
res
}
}
}
}
```
</p>
</details>
Then `LD_PRELOAD` that dylib and repeatedly build a big project until it ICEs, such as with this:
```bash
while true; do
cargo clean
LD_PRELOAD=/home/ben/evil/target/release/libevil.so cargo +stage1 check 2> errors
if grep "thread 'rustc' panicked" errors; then
break
fi
done
```
My "big project" for testing was an otherwise-empty project with `cargo add axum`.
Before this PR, the above procedure finds a crash in between 1 and 15 minutes. With this PR, I have not found a crash in 30 minutes, and I'll be leaving this to run overnight (starting now). (A night has now passed, no crashes were found)
I believe the problem is that even though since https://github.com/rust-lang/rust/pull/117301 we correctly check `FileEncoder` for errors on all paths, we use `emit_err`, so there is a window of time between the call to `emit_err` and the full error reporting where rustc believes it has emitted a valid rmeta file and will permit Cargo to launch a build for a dependent crate. Changing these calls to `emit_fatal` closes that window.
I think there are a number of other cases where `emit_err` has been used instead of the more-correct `emit_fatal` such as https://github.com/rust-lang/rust/blob/e51e98dde6a60637b6a71b8105245b629ac3fe77/compiler/rustc_codegen_ssa/src/back/write.rs#L542 but unlike rmeta encoding I am not aware of those cases of those causing problems.
r? ``@WaffleLapkin``
|
|
|
|
|
|
|
|
Spans are now stored in a more compact form which cuts down on at least
1 byte per span (indirect/direct encoding) and at most 3 bytes per span
(indirect/direct encoding, context byte, length byte). As a result,
libcore metadata shrinks by 1.5MB.
|
|
|
|
|
|
Support encoding spans with relative offsets
The relative offset is often smaller than the absolute offset, and with
the LEB128 encoding, this ends up cutting the overall metadata size
considerably (~1.5 megabytes on libcore). We can support both relative
and absolute encodings essentially for free since we already take a full
byte to differentiate between direct and indirect encodings (so an extra
variant is quite cheap).
|
|
The relative offset is often smaller than the absolute offset, and with
the LEB128 encoding, this ends up cutting the overall metadata size
considerably (~1.5 megabytes on libcore). We can support both relative
and absolute encodings essentially for free since we already take a full
byte to differentiate between direct and indirect encodings (so an extra
variant is quite cheap).
|
|
r=compiler-errors
Remove `DiagCtxt` API duplication
`DiagCtxt` defines the internal API for creating and emitting diagnostics: methods like `struct_err`, `struct_span_warn`, `note`, `create_fatal`, `emit_bug`. There are over 50 methods.
Some of these methods are then duplicated across several other types: `Session`, `ParseSess`, `Parser`, `ExtCtxt`, and `MirBorrowckCtxt`. `Session` duplicates the most, though half the ones it does are unused. Each duplicated method just calls forward to the corresponding method in `DiagCtxt`. So this duplication exists to (in the best case) shorten chains like `ecx.tcx.sess.parse_sess.dcx.emit_err()` to `ecx.emit_err()`.
This API duplication is ugly and has been bugging me for a while. And it's inconsistent: there's no real logic about which methods are duplicated, and the use of `#[rustc_lint_diagnostic]` and `#[track_caller]` attributes vary across the duplicates.
This PR removes the duplicated API methods and makes all diagnostic creation and emission go through `DiagCtxt`. It also adds `dcx` getter methods to several types to shorten chains. This approach scales *much* better than API duplication; indeed, the PR adds `dcx()` to numerous types that didn't have API duplication: `TyCtxt`, `LoweringCtxt`, `ConstCx`, `FnCtxt`, `TypeErrCtxt`, `InferCtxt`, `CrateLoader`, `CheckAttrVisitor`, and `Resolver`. These result in a lot of changes from `foo.tcx.sess.emit_err()` to `foo.dcx().emit_err()`. (You could do this with more types, but it gets into diminishing returns territory for types that don't emit many diagnostics.)
After all these changes, some call sites are more verbose, some are less verbose, and many are the same. The total number of lines is reduced, mostly because of the removed API duplication. And consistency is increased, because calls to `emit_err` and friends are always preceded with `.dcx()` or `.dcx`.
r? `@compiler-errors`
|
|
r=cjgillot
Unify SourceFile::name_hash and StableSourceFileId
This PR adapts the existing `StableSourceFileId` type so that it can be used instead of the `name_hash` field of `SourceFile`. This simplifies a few things that were kind of duplicated before.
The PR should also fix issues https://github.com/rust-lang/rust/issues/112700 and https://github.com/rust-lang/rust/issues/115835, but I was not able to reproduce these issues in a regression test. As far as I can tell, the root cause of these issues is that the id of the originating crate is not hashed in the `HashStable` impl of `Span` and thus cache entries that should have been considered invalidated were loaded. After this PR, the `stable_id` field of `SourceFile` includes information about the originating crate, so that ICE should not occur anymore.
|
|
Skip duplicate stable crate ID encoding into metadata
Instead, we store just the local crate hash as a bare u64. On decoding,
we recombine it with the crate's stable crate ID stored separately in
metadata. The end result is that we save ~8 bytes/DefIndex in metadata
size.
One key detail here is that we no longer distinguish in encoded metadata
between present and non-present DefPathHashes. It used to be highly
likely we could distinguish as we used DefPathHash::default(), an
all-zero representation. However in theory even that is fallible as
nothing strictly prevents the StableCrateId from being zero. In review it
was pointed out that we should never have a missing hash for a DefIndex anyway,
so this shouldn't matter.
|
|
Also add some `dcx` methods to types that wrap `TyCtxt`, for easier
access.
|
|
Instead, we store just the local crate hash as a bare u64. On decoding,
we recombine it with the crate's stable crate ID stored separately in
metadata. The end result is that we save ~8 bytes/DefIndex in metadata
size.
One key detail here is that we no longer distinguish in encoded metadata
between present and non-present DefPathHashes. It used to be highly
likely we could distinguish as we used DefPathHash::default(), an
all-zero representation. However in theory even that is fallible as
nothing strictly prevents the StableCrateId from being zero.
|
|
Avoid redundant Option for cross_crate_inlinable
|
|
|
|
|
|
|
|
Renamings:
- find -> opt_hir_node
- get -> hir_node
- find_by_def_id -> opt_hir_node_by_def_id
- get_by_def_id -> hir_node_by_def_id
Fix rebase changes using removed methods
Use `tcx.hir_node_by_def_id()` whenever possible in compiler
Fix clippy errors
Fix compiler
Apply suggestions from code review
Co-authored-by: Vadim Petrochenkov <vadim.petrochenkov@gmail.com>
Add FIXME for `tcx.hir()` returned type about its removal
Simplify with with `tcx.hir_node_by_def_id`
|
|
Use a u64 for the rmeta root position
Waffle noticed this in https://github.com/rust-lang/rust/pull/117301#discussion_r1405410174
We've upgraded the other file offsets to u64, and this one only costs 4 bytes per file. Also the way the truncation was being done before was extremely easy to miss, I sure missed it! It's not clear to me if not having this change effectively made the other upgrades from u32 to u64 ineffective, but we can have it now.
r? `@WaffleLapkin`
|
|
detects redundant imports that can be eliminated.
for #117772 :
In order to facilitate review and modification, split the checking code and
removing redundant imports code into two PR.
|
|
|
|
|
|
Rollup of 4 pull requests
Successful merges:
- #118296 (rustdoc: replace `elemIsInParent` with `Node.contains`)
- #118302 (Clean dead codes)
- #118311 (merge `DefKind::Coroutine` into `Defkind::Closure`)
- #118318 (Remove myself from users on vacation)
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
merge `DefKind::Coroutine` into `Defkind::Closure`
Related to #118188
We no longer need to be concerned about the precise type whether it's `DefKind::Closure` or `DefKind::Coroutine`.
Furthermore, thanks for the great work done by `@petrochenkov` on investigating https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/Why.20does.20it.20hang.20when.20querying.20.EF.BB.BF.60opt_def_kind.60.3F
r? `@petrochenkov`
|
|
Call FileEncoder::finish in rmeta encoding
Fixes https://github.com/rust-lang/rust/issues/117254
The bug here was that rmeta encoding never called FileEncoder::finish. Now it does. Most of the changes here are needed to support that, since rmeta encoding wants to finish _then_ access the File in the encoder, so finish can't move out.
I tried adding a `cfg(debug_assertions)` exploding Drop impl to FileEncoder that checked for finish being called before dropping, but fatal errors cause unwinding so this isn't really possible. If we encounter a fatal error with a dirty FileEncoder, the Drop impl ICEs even though the implementation is correct. If we try to paper over that by wrapping FileEncoder in ManuallyDrop then that just erases the fact that Drop automatically checks that we call finish on all paths.
I also changed the name of DepGraph::encode to DepGraph::finish_encoding, because that's what it does and it makes the fact that it is the path to FileEncoder::finish less confusing.
r? `@WaffleLapkin`
|
|
|
|
cleanup
|
|
|
|
|
|
Fix some unnecessary casts
`x clippy compiler -Aclippy::all -Wclippy::unnecessary_cast --fix` with some manual review to ensure every fix is correct.
|
|
`x clippy compiler -Aclippy::all -Wclippy::unnecessary_cast --fix`
with some manual review to ensure every fix is correct.
|
|
`x clippy compiler -Aclippy::all -Wclippy::needless_borrow --fix`.
Then I had to remove a few unnecessary parens and muts that were exposed
now.
|
|
|
|
|
|
Format macro const literals with pretty printer
Fixes #115295
|
|
|
|
|
|
Implement rustc part of RFC 3127 trim-paths
This PR implements (or at least tries to) [RFC 3127 trim-paths](https://github.com/rust-lang/rust/issues/111540), the rustc part. That is `-Zremap-path-scope` with all of it's components/scopes.
`@rustbot` label: +F-trim-paths
|
|
|
|
|
|
|
|
|
|
|
|
|
