| Age | Commit message (Collapse) | Author | Lines |
|---|
|
|
|
Add regression test for #86106
Closes #86106
r? `@nikic`
Signed-off-by: Yuki Okushi <jtitor@2k36.org>
|
|
Perform SimplifyLocals before ConstProp.
MIR before `ConstProp` may have a lot of dead writes, this makes `ConstProp` do unnecessary work.
r? `@ghost`
|
|
Signed-off-by: Yuki Okushi <jtitor@2k36.org>
|
|
Signed-off-by: Yuki Okushi <jtitor@2k36.org>
|
|
Signed-off-by: Yuki Okushi <jtitor@2k36.org>
|
|
cleanup: handle -Zmutable-noalias like -Zbox-noalias
r? `@bjorn3`
cc `@RalfJung` this will conflict with #106180
|
|
|
|
Add codegen test for `Box::new(uninit)` of big arrays
Closes #58201
r? `@RalfJung`
|
|
do not add noalias in return position
`noalias` as a return attribute in LLVM indicates that the returned pointer does not alias anything else that is reachable from the caller, *including things reachable before this function call*. This is clearly not the case with a function like `fn id(Box<T>) -> Box<T>`, so we cannot use this attribute.
Fixes https://github.com/rust-lang/unsafe-code-guidelines/issues/385 (including an actual miscompilation that `@comex` managed to produce).
|
|
Add codegen test for issue 103840
Closes #103840
|
|
|
|
|
|
Monomorphise `#[repr(transparent)]` parameterized ADTs before turning
them into an Itanium mangled String.
`#[repr(transparent)]` ADTs currently use the single field to represent
them in their CFI type ID to ensure that they are compatible. However,
if that type involves a type parameter instantiated at the ADT level, as
in `ManuallyDrop`, this will currently ICE as the `Parameter` type
cannot be mangled. Since this happens at lowering time, it should always
be concrete after substitution.
Fixes #106230
|
|
|
|
abort immediately on bad mem::zeroed/uninit
Now that we have non-unwinding panics, let's use them for these assertions. This re-establishes the property that `mem::uninitialized` and `mem::zeroed` will never unwind -- the earlier approach of causing panics here sometimes led to hard-to-debug segfaults when the surrounding code was not able to cope with the unexpected unwinding.
Cc `@bjorn3` I did not touch cranelift but I assume it needs a similar patch. However it has a `codegen_panic` abstraction that I did not want to touch since I didn't know how else it is used.
|
|
|
|
|
|
Codegen test for derived `<` on trivial newtype [TEST ONLY]
I originally wrote this for #106065, but the libcore changes there aren't necessarily a win.
So I pulled out this test to be its own PR since it's important (see https://github.com/rust-lang/rust/pull/105840#discussion_r1056030324) and well-intentioned changes to core or the derive could accidentally break it without that being obvious (other than by massive unexplained perf changes).
|
|
|
|
KCFI test: Also support LLVM 16 output
With a regex we can support the LLVM 16 output now already (and there are some third-party build bots that test Rust with LLVM head, like https://buildkite.com/llvm-project/rust-llvm-integrate-prototype/builds?branch=master).
cc ```@rcvalle```
|
|
|
|
codegen tests: adapt patterns to also work with v0 symbol mangling
No functional changes intended.
These tests were failing under `new-symbol-mangling = true`, cf. https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/a.20few.20panic-abort.20tests.20fail.20under.20.60new-symbol-mangling.60.
This adapts the patterns to work in this case.
|
|
Add `-Zno-jump-tables`
This flag mimics GCC/Clang's `-fno-jump-tables` [1][2], which makes the codegen backend avoid generating jump tables when lowering switches.
In the case of LLVM, the `"no-jump-tables"="true"` function attribute is added to every function.
The kernel currently needs it for x86 when enabling IBT [3], as well as for Alpha (plus VDSO objects in MIPS/LoongArch).
[1] https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html#index-fno-jump-tables
[2] https://clang.llvm.org/docs/ClangCommandLineReference.html#cmdoption-clang-fjump-tables
[3] https://github.com/torvalds/linux/blob/v6.1/arch/x86/Makefile#L75-L83
|
|
No functional changes intended.
These tests were failing under `new-symbol-mangling = true`.
This adapts the patterns to work in this case.
|
|
This flag mimics GCC/Clang's `-fno-jump-tables` [1][2], which makes
the codegen backend avoid generating jump tables when lowering switches.
In the case of LLVM, the `"no-jump-tables"="true"` function attribute is
added to every function.
The kernel currently needs it for x86 when enabling IBT [3], as well
as for Alpha (plus VDSO objects in MIPS/LoongArch).
[1] https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html#index-fno-jump-tables
[2] https://clang.llvm.org/docs/ClangCommandLineReference.html#cmdoption-clang-fjump-tables
[3] https://github.com/torvalds/linux/blob/v6.1/arch/x86/Makefile#L75-L83
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
|
|
Add 0..=isize::MAX range metadata to size loads from vtables
This is the (much belated) size counterpart to #91569.
Inspired by https://rust-lang.zulipchat.com/#narrow/stream/187780-t-compiler.2Fwg-llvm/topic/Range.20metadata.20for.20.60size_of_val.60.20and.20other.20isize.3A.3AMAX.20limits. This could help optimize layout computations based on the size of a dyn trait. Though, admittedly, adding this to vtables wouldn't be as beneficial as adding it to slice len, which is used much more often.
Miri detects this UB already: https://github.com/rust-lang/rust/blob/b7cc99142ad0cfe47e2fe9f7a82eaf5b672c0573/compiler/rustc_const_eval/src/interpret/traits.rs#L119-L121
(In fact Miri goes further, [assuming a 48-bit address space on 64-bit platforms](https://github.com/rust-lang/rust/blob/9db224fc908059986c179fc6ec433944e9cfce50/compiler/rustc_abi/src/lib.rs#L312-L331), but I don't think we can assume that in an optimization.)
|
|
https://reviews.llvm.org/D140192 caused the LLVM variable generated
for enum discriminant checks to be named differently (%narrow vs %1).
This adjusts the test CHECK directives to match any name.
|
|
Fix transmutes between pointers in different address spaces (e.g. fn ptrs on AVR)
Currently, this causes a verifier error (https://godbolt.org/z/YYohed4bj), since it uses `bitcast`, which can't convert between address spaces.
Uncovered due to https://github.com/rust-lang/rust/pull/105545#discussion_r1045269309
r? `@bjorn3`
|
|
|
|
|
|
Don't internalize __llvm_profile_counter_bias
Currently, LLVM profiling runtime counter relocation cannot be used by rust during LTO because symbols are being internalized before all symbol information is known.
This mode makes LLVM emit a __llvm_profile_counter_bias symbol which is referenced by the profiling initialization, which itself is pulled in by the rust driver here [1].
It is enabled with -Cllvm-args=-runtime-counter-relocation for platforms which are opt-in to this mode like Linux. On these platforms there will be no link error, rather just surprising behavior for a user which request runtime counter relocation. The profiling runtime will not see that symbol go on as if it were never there. On Fuchsia, the profiling runtime must have this symbol which will cause a hard link error.
As an aside, I don't have enough context as to why rust's LTO model is how it is. AFAICT, the internalize pass is only safe to run at link time when all symbol information is actually known, this being an example as to why. I think special casing this symbol as a known one that LLVM can emit which should not have it's visbility de-escalated should be fine given how seldom this pattern of defining an undefined symbol to get initilization code pulled in is. From a quick grep, __llvm_profile_runtime is the only symbol that rustc does this for.
[1] https://github.com/rust-lang/rust/blob/0265a3e93bf1b89d97cae113ed214954d5c35e22/compiler/rustc_codegen_ssa/src/back/linker.rs#L598
|
|
Rollup of 6 pull requests
Successful merges:
- #104460 (Migrate parts of `rustc_expand` to session diagnostics)
- #105192 (Point at LHS on binop type err if relevant)
- #105234 (Remove unneeded field from `SwitchTargets`)
- #105239 (Avoid heap allocation when truncating thread names)
- #105410 (Consider `parent_count` for const param defaults)
- #105482 (Fix invalid codegen during debuginfo lowering)
Failed merges:
- #105411 (Introduce `with_forced_trimmed_paths`)
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
Fix invalid codegen during debuginfo lowering
In order for LLVM to correctly generate debuginfo for msvc, we sometimes need to spill arguments to the stack and perform some direct & indirect offsets into the value. Previously, this code always performed those actions, even when not required as LLVM would clean it up during optimization.
However, when MIR inlining is enabled, this can cause problems as the operations occur prior to the spilled value being initialized. To solve this, we first calculate the necessary offsets using just the type which is side-effect free and does not alter the LLVM IR. Then, if we are in a situation which requires us to generate the LLVM IR (and this situation only occurs for arguments, not local variables) then we perform the same calculation again, this time generating the appropriate LLVM IR as we go.
r? `@tmiasko` but feel free to reassign if you want 🙂
Fixes #105386
|
|
Fix failing codegen tests on s390x
Several codegen tests are currently failing due to making assumptions that are not valid for the s390x architecture:
- catch-unwind.rs: fails due to inlining differences. Already ignored on another platform for the same reason. Solution: Ignore on s390x.
- remap_path_prefix/main.rs: fails due to different alignment requirement for string constants. Solution: Do not test for the alignment requirement.
- repr-transparent-aggregates-1.rs: many ABI assumptions. Already ignored on many platforms for the same reason. Solution: Ignore on s390x.
- repr-transparent.rs: no vector ABI by default on s390x. Already ignored on another platform for a similar reason. Solution: Ignore on s390x.
- uninit-consts.rs: hard-coded little-endian constant. Solution: Match both little- and big-endian versions.
Fixes part of https://github.com/rust-lang/rust/issues/105383.
|
|
Add LLVM KCFI support to the Rust compiler
This PR adds LLVM Kernel Control Flow Integrity (KCFI) support to the Rust compiler. It initially provides forward-edge control flow protection for operating systems kernels for Rust-compiled code only by aggregating function pointers in groups identified by their return and parameter types. (See llvm/llvm-project@cff5bef.)
Forward-edge control flow protection for C or C++ and Rust -compiled code "mixed binaries" (i.e., for when C or C++ and Rust -compiled code share the same virtual address space) will be provided in later work as part of this project by identifying C char and integer type uses at the time types are encoded (see Type metadata in the design document in the tracking issue #89653).
LLVM KCFI can be enabled with -Zsanitizer=kcfi.
Thank you again, `@bjorn3,` `@eddyb,` `@nagisa,` and `@ojeda,` for all the help!
|
|
In order for LLVM to correctly generate debuginfo for msvc, we sometimes
need to spill arguments to the stack and perform some direct & indirect
offsets into the value. Previously, this code always performed those
actions, even when not required as LLVM would clean it up during
optimization.
However, when MIR inlining is enabled, this can cause problems as the
operations occur prior to the spilled value being initialized. To solve
this, we first calculate the necessary offsets using just the type which
is side-effect free and does not alter the LLVM IR. Then, if we are in a
situation which requires us to generate the LLVM IR (and this situation
only occurs for arguments, not local variables) then we perform the same
calculation again, this time generating the appropriate LLVM IR as we
go.
|
|
|
|
This commit adds LLVM Kernel Control Flow Integrity (KCFI) support to
the Rust compiler. It initially provides forward-edge control flow
protection for operating systems kernels for Rust-compiled code only by
aggregating function pointers in groups identified by their return and
parameter types. (See llvm/llvm-project@cff5bef.)
Forward-edge control flow protection for C or C++ and Rust -compiled
code "mixed binaries" (i.e., for when C or C++ and Rust -compiled code
share the same virtual address space) will be provided in later work as
part of this project by identifying C char and integer type uses at the
time types are encoded (see Type metadata in the design document in the
tracking issue #89653).
LLVM KCFI can be enabled with -Zsanitizer=kcfi.
Co-authored-by: bjorn3 <17426603+bjorn3@users.noreply.github.com>
|
|
|
|
|
|
Currently, LLVM profiling runtime counter relocation cannot be
used by rust during LTO because symbols are being internalized
before all symbol information is known.
This mode makes LLVM emit a __llvm_profile_counter_bias symbol
which is referenced by the profiling initialization, which itself
is pulled in by the rust driver here [1].
It is enabled with -Cllvm-args=-runtime-counter-relocation for
platforms which are opt-in to this mode like Linux. On these
platforms there will be no link error, rather just surprising
behavior for a user which request runtime counter relocation.
The profiling runtime will not see that symbol go on as if it
were never there. On Fuchsia, the profiling runtime must have
this symbol which will cause a hard link error.
As an aside, I don't have enough context as to why rust's LTO
model is how it is. AFAICT, the internalize pass is only safe
to run at link time when all symbol information is actually
known, this being an example as to why. I think special casing
this symbol as a known one that LLVM can emit which should not
have it's visbility de-escalated should be fine given how
seldom this pattern of defining an undefined symbol to get
initilization code pulled in is. From a quick grep,
__llvm_profile_runtime is the only symbol that rustc does this
for.
[1] https://github.com/rust-lang/rust/blob/0265a3e93bf1b89d97cae113ed214954d5c35e22/compiler/rustc_codegen_ssa/src/back/linker.rs#L598
|
|
Several codegen tests are currently failing due to making
assumptions that are not valid for the s390x architecture:
- catch-unwind.rs: fails due to inlining differences.
Already ignored on another platform for the same reason.
Solution: Ignore on s390x.
- remap_path_prefix/main.rs: fails due to different alignment
requirement for string constants.
Solution: Do not test for the alignment requirement.
- repr-transparent-aggregates-1.rs: many ABI assumptions.
Already ignored on many platforms for the same reason.
Solution: Ignore on s390x.
- repr-transparent.rs: no vector ABI by default on s390x.
Already ignored on another platform for a similar reason.
Solution: Ignore on s390x.
- uninit-consts.rs: hard-coded little-endian constant.
Solution: Match both little- and big-endian versions.
Fixes part of https://github.com/rust-lang/rust/issues/105383.
|
|
rustc_codegen_ssa: Fix for codegen_get_discr
When doing the optimized implementation of getting the discriminant, the arithmetic needs to be done in the tag type so wrapping behavior works correctly.
Fixes #104519
|
|
|
|
Run Windows-only tests only on Windows
This removes the need to maintain an ignore-list of all other OSs.
See https://github.com/rust-lang/rust/pull/102305 for a similar change.
|
|
|
|
Add a codegen-test that verifies inter-crate linkage with the static
relocation model. We expect all symbols that are part of a rust
compilation to end up in the same DSO, thus we expect `dso_local`
annotations.
|
|
This removes the need to maintain a list of all other
OSs which ignore the tests.
|
|
Stop peeling the last iteration of the loop in `Vec::resize_with`
`resize_with` uses the `ExtendWith` code that peels the last iteration:
https://github.com/rust-lang/rust/blob/341d8b8a2c290b4535e965867e876b095461ff6e/library/alloc/src/vec/mod.rs#L2525-L2529
But that's kinda weird for `ExtendFunc` because it does the same thing on the last iteration anyway:
https://github.com/rust-lang/rust/blob/341d8b8a2c290b4535e965867e876b095461ff6e/library/alloc/src/vec/mod.rs#L2494-L2502
So this just has it use the normal `extend`-from-`TrustedLen` code instead.
r? `@ghost`
|
