| Age | Commit message (Collapse) | Author | Lines |
|---|
|
Add LLVM15-specific codegen test for `try`/`?`s that now optimize away
These still generated a bunch of code back in Rust 1.63 (<https://rust.godbolt.org/z/z31P8h6rz>), but with LLVM 15 merged they no longer do 🎉
|
|
These still generated a bunch of code back in Rust 1.63 (<https://rust.godbolt.org/z/z31P8h6rz>), but with LLVM 15 merged they no longer do :tada:
|
|
Update the minimum external LLVM to 13
With this change, we'll have stable support for LLVM 13 through 15 (pending release).
For reference, the previous increase to LLVM 12 was #90175.
r? `@nagisa`
|
|
Add support for generating unique profraw files by default when using `-C instrument-coverage`
Currently, enabling the rustc flag `-C instrument-coverage` instruments the given crate and by default uses the naming scheme `default.profraw` for any instrumented profile files generated during the execution of a binary linked against this crate. This leads to multiple binaries being executed overwriting one another and causing only the last executable run to contain actual coverage results.
This can be overridden by manually setting the environment variable `LLVM_PROFILE_FILE` to use a unique naming scheme.
This PR adds a change to add support for a reasonable default for rustc to use when enabling coverage instrumentation similar to how the Rust compiler treats generating these same `profraw` files when PGO is enabled.
The new naming scheme is set to `default_%m_%p.profraw` to ensure the uniqueness of each file being generated using [LLVMs special pattern strings](https://clang.llvm.org/docs/SourceBasedCodeCoverage.html#running-the-instrumented-program).
Today the compiler sets the default for PGO `profraw` files to `default_%m.profraw` to ensure a unique file for each run. The same can be done for the instrumented profile files generated via the `-C instrument-coverage` flag as well which LLVM has API support for.
Linked Issue: https://github.com/rust-lang/rust/issues/100381
r? `@wesleywiser`
|
|
|
|
|
|
`-C instrument-coverage` flag.
Respond to PR comments.
|
|
Reoptimize layout array
This way it's one check instead of two, so hopefully (cc #99117) it'll be simpler for rustc perf too 🤞
Quick demonstration:
```rust
pub fn demo(n: usize) -> Option<Layout> {
Layout::array::<i32>(n).ok()
}
```
Nightly: <https://play.rust-lang.org/?version=nightly&mode=release&edition=2021&gist=e97bf33508aa03f38968101cdeb5322d>
```nasm
mov rax, rdi
mov ecx, 4
mul rcx
seto cl
movabs rdx, 9223372036854775805
xor esi, esi
cmp rax, rdx
setb sil
shl rsi, 2
xor edx, edx
test cl, cl
cmove rdx, rsi
ret
```
This PR (note no `mul`, in addition to being much shorter):
```nasm
xor edx, edx
lea rax, [4*rcx]
shr rcx, 61
sete dl
shl rdx, 2
ret
```
This is built atop `@CAD97` 's #99136; the new changes are cb8aba66ef6a0e17f08a0574e4820653e31b45a0.
I added a bunch more tests for `Layout::from_size_align` and `Layout::array` too.
|
|
Enable function merging when opt is for size
It is, of course, natural to want to merge aliasing functions when
optimizing for code size, since that can eliminate several bytes.
And an exhaustive match helps make the code less brittle.
Closes #98215.
|
|
It is, of course, natural to want to merge aliasing functions when
optimizing for code size, since that can eliminate several bytes.
And an exhaustive match helps make the code less brittle.
|
|
|
|
This attribute now does more than just place noalias on the return,
and has specific requirements for the signature.
Drop the test entirely, as we already check __rust_alloc attributes
in other codegen tests.
|
|
|
|
This obviates the patch that teaches LLVM internals about
_rust_{re,de}alloc functions by putting annotations directly in the IR
for the optimizer.
The sole test change is required to anchor FileCheck to the body of the
`box_uninitialized` method, so it doesn't see the `allocalign` on
`__rust_alloc` and get mad about the string `alloca` showing up. Since I
was there anyway, I added some checks on the attributes to prove the
right attributes got set.
While we're here, we also emit allocator attributes on
__rust_alloc_zeroed. This should allow LLVM to perform more
optimizations for zeroed blocks, and probably fixes #90032. [This
comment](https://github.com/rust-lang/rust/issues/24194#issuecomment-308791157)
mentions "weird UB-like behaviour with bitvec iterators in
rustc_data_structures" so we may need to back this change out if things
go wrong.
The new test cases require LLVM 15, so we copy them into LLVM
14-supporting versions, which we can delete when we drop LLVM 14.
|
|
AngelicosPhosphoros:improve_calloc_check_in_vec_macro_for_tuples, r=Mark-Simulacrum
Support vec zero-alloc optimization for tuples and byte arrays
* Implement IsZero trait for tuples up to 8 IsZero elements;
* Implement IsZero for u8/i8, leading to implementation of it for arrays of them too;
* Add more codegen tests for this optimization.
* Lower size of array for IsZero trait because it fails to inline checks
|
|
* Implement IsZero trait for tuples up to 8 IsZero elements;
* Implement IsZero for u8/i8, leading to implementation of it for arrays of them too;
* Add more codegen tests for this optimization.
* Lower size of array for IsZero trait because it fails to inline checks
|
|
Test codegen of atomic compare-exchange with additional memory orderings
* Add a test for atomic operations introduced in #97423 & #98383.
* Add a test for fallback code generation strategy used on LLVM 12 introduced in #98385. Use a separate test case instead of a revision system since test will be gone once LLVM 12 is no longer supported.
|
|
Add fine-grained LLVM CFI support to the Rust compiler
This PR improves the LLVM Control Flow Integrity (CFI) support in the Rust compiler by providing forward-edge control flow protection for Rust-compiled code only by aggregating function pointers in groups identified by their return and parameter types.
Forward-edge control flow protection for C or C++ and Rust -compiled code "mixed binaries" (i.e., for when C or C++ and Rust -compiled code share the same virtual address space) will be provided in later work as part of this project by identifying C char and integer type uses at the time types are encoded (see Type metadata in the design document in the tracking issue https://github.com/rust-lang/rust/issues/89653).
LLVM CFI can be enabled with -Zsanitizer=cfi and requires LTO (i.e., -Clto).
Thank you again, `@eddyb,` `@nagisa,` `@pcc,` and `@tmiasko` for all the help!
|
|
Add support for LLVM ShadowCallStack.
LLVMs ShadowCallStack provides backward edge control flow integrity protection by using a separate shadow stack to store and retrieve a function's return address.
LLVM currently only supports this for AArch64 targets. The x18 register is used to hold the pointer to the shadow stack, and therefore this only works on ABIs which reserve x18. Further details are available in the [LLVM ShadowCallStack](https://clang.llvm.org/docs/ShadowCallStack.html) docs.
# Usage
`-Zsanitizer=shadow-call-stack`
# Comments/Caveats
* Currently only enabled for the aarch64-linux-android target
* Requires the platform to define a runtime to initialize the shadow stack, see the [LLVM docs](https://clang.llvm.org/docs/ShadowCallStack.html) for more detail.
|
|
This commit improves the LLVM Control Flow Integrity (CFI) support in
the Rust compiler by providing forward-edge control flow protection for
Rust-compiled code only by aggregating function pointers in groups
identified by their return and parameter types.
Forward-edge control flow protection for C or C++ and Rust -compiled
code "mixed binaries" (i.e., for when C or C++ and Rust -compiled code
share the same virtual address space) will be provided in later work as
part of this project by identifying C char and integer type uses at the
time types are encoded (see Type metadata in the design document in the
tracking issue #89653).
LLVM CFI can be enabled with -Zsanitizer=cfi and requires LTO (i.e.,
-Clto).
|
|
|
|
Sync in portable-simd subtree
r? `@ghost`
|
|
|
|
* Add a test for atomic operations introduced in #97423 & #98383.
* Add a test for fallback code generation strategy used on LLVM 12
introduced in #98385. Use a separate test case instead of a revision
system since test will be gone once LLVM 12 is no longer supported.
|
|
Adds support for the LLVM ShadowCallStack sanitizer.
|
|
Add flag to configure `noalias` on `Box<T>`
The aliasing rules of `Box<T>` are still not decided, but currently, `Box<T>` is unique and gets `noalias`. To aid making an informed decision about the future of `Box<T>`, this PR adds a flag `-Zbox-noalias` to configure `noalias` for `Box<T>` (for example, for benchmarking). The same flag already exists for `&mut T` `noalias`, where it was added because it was the problem of various miscompilations in LLVM.
For more information, see rust-lang/unsafe-code-guidelines#326
|
|
To aid making an informed decision about the aliasing
rules of box, give users an option to remove `noalias`
from box.
|
|
workingjubilee:naked-means-no-clothes-enforcement-technology, r=Amanieu
Remove branch target prologues from `#[naked] fn`
This patch hacks around rust-lang/rust#98768 for now via injecting appropriate attributes into the LLVMIR we emit for naked functions. I intend to pursue this upstream so that these attributes can be removed in general, but it's slow going wading through C++ for me.
|
|
This way it's one check instead of two, so hopefully it'll be better
Nightly:
```
layout_array_i32:
movq %rdi, %rax
movl $4, %ecx
mulq %rcx
jo .LBB1_2
movabsq $9223372036854775805, %rcx
cmpq %rcx, %rax
jae .LBB1_2
movl $4, %edx
retq
.LBB1_2:
…
```
This PR:
```
movq %rcx, %rax
shrq $61, %rax
jne .LBB2_1
shlq $2, %rcx
movl $4, %edx
movq %rcx, %rax
retq
.LBB2_1:
…
```
|
|
|
|
r=Mark-Simulacrum
More derive output improvements
This PR includes:
- Some test improvements.
- Some cosmetic changes to derive output that make the code look more like what a human would write.
- Some more fundamental improvements to `cmp` and `partial_cmp` generation.
r? `@Mark-Simulacrum`
|
|
|
|
|
|
We can apply nocf_check as a hack for now.
|
|
adapt issue-37945 codegen test to accept any order of ops
Adapt this test to accept `icmp` operands in any order as a follow-up to https://github.com/rust-lang/rust/commit/cbbf06b0cd39dc93033568f1e65f5363cbbdebcd#commitcomment-77670922.
|
|
|
|
Change enum->int casts to not go through MIR casts.
follow-up to https://github.com/rust-lang/rust/pull/96814
this simplifies all backends and even gives LLVM more information about the return value of `Rvalue::Discriminant`, enabling optimizations in more cases.
|
|
|
|
|
|
r=oli-obk
Added llvm lifetime annotations to function call argument temporaries.
The goal of this change is to ensure that llvm will do stack slot
optimization on these temporaries. This ensures that in code like:
```rust
const A: [u8; 1024] = [0; 1024];
fn copy_const() {
f(A);
f(A);
}
```
we only use 1024 bytes of stack space, instead of 2048 bytes.
I am new to developing for the rust compiler, and as such not entirely sure, but I believe this should be sufficient to close #98156.
Also, this does not contain a test case to ensure this keeps working, primarily because I am not sure how to go about testing this. I would love some suggestions as to how that could be approached.
|
|
Instead we generate a discriminant rvalue and cast the result of that.
|
|
|
|
Make RwLockReadGuard covariant
Hi, first time contributor here, if anything is not as expected, please let me know.
`RwLockReadGoard`'s type constructor is invariant. Since it behaves like a smart pointer to an immutable reference, there is no reason that it should not be covariant. Take e.g.
```
fn test_read_guard_covariance() {
fn do_stuff<'a>(_: RwLockReadGuard<'_, &'a i32>, _: &'a i32) {}
let j: i32 = 5;
let lock = RwLock::new(&j);
{
let i = 6;
do_stuff(lock.read().unwrap(), &i);
}
drop(lock);
}
```
where the compiler complains that &i doesn't live long enough. If `RwLockReadGuard` is covariant, then the above code is accepted because the lifetime can be shorter than `'a`.
In order for `RwLockReadGuard` to be covariant, it can't contain a full reference to the `RwLock`, which can never be covariant (because it exposes a mutable reference to the underlying data structure). By reducing the data structure to the required pieces of `RwLock`, the rest falls in place.
If there is a better way to do a test that tests successful compilation, please let me know.
Fixes #80392
|
|
Remove dereferencing of Box from codegen
Through #94043, #94414, #94873, and #95328, I've been fixing issues caused by Box being treated like a pointer when it is not a pointer. However, these PRs just introduced special cases for Box. This PR removes those special cases and instead transforms a deref of Box into a deref of the pointer it contains.
Hopefully, this is the end of the Box<T, A> ICEs.
|
|
|
|
|
|
|
|
|
|
Use unchecked mul to compute slice sizes
This allows LLVM to realize that `slice.len() > 0` iff `slice.len() * size_of::<T>() > 0`, allowing a branch on the latter to be folded into the former when dropping vecs and boxed slices, in some cases.
Fixes (partially) #96497
|
|
...since slice sizes can't signed wrap
see https://doc.rust-lang.org/std/slice/fn.from_raw_parts.html
> The total size len * mem::size_of::<T>() of the slice must be no larger than isize::MAX.
|
