about summary refs log tree commit diff
path: root/src/test/codegen
AgeCommit message (Collapse)AuthorLines
2022-03-01tests: avoid problems on 32 bit machinesAugie Fackler-2/+2
2022-03-01tests: accept llvm intrinsic in align-checking testAugie Fackler-2/+2
This changed in upstream change https://reviews.llvm.org/D98152 (aka https://github.com/llvm/llvm-project/commit/a266af721153fab6452094207b09ed265ab0be7b) wherein LLVM got smarter about using intrinsics. As best I can tell the change I've made here preserves the intent of the test on LLVM 14 and before while also passing on LLVM 15 and later.
2022-03-01Auto merge of #94402 - erikdesjardins:revert-coldland, r=nagisabors-52/+19
Revert "Auto merge of #92419 - erikdesjardins:coldland, r=nagisa" Should fix (untested) #94390 Reopens #46515, #87055 r? `@ehuss`
2022-02-28Auto merge of #94216 - psumbera:sparc64-abi-fix2, r=nagisabors-0/+20
more complete sparc64 ABI fix for aggregates with floating point members Previous fix didn't handle nested structures at all.
2022-02-28Auto merge of #94158 - erikdesjardins:more-more-noundef, r=nikicbors-4/+83
Apply noundef metadata to loads of types that do not permit raw init This matches the noundef attributes we apply on arguments/return types. Fixes (partially) #74378.
2022-02-27update vec-shrink-panik test to allow panic_no_unwind in landingpadsErik Desjardins-0/+19
2022-02-27Revert "Auto merge of #92419 - erikdesjardins:coldland, r=nagisa"Erik Desjardins-52/+0
This reverts commit 4f49627c6fe2a32d1fed6310466bb0e1c535c0c0, reversing changes made to 028c6f1454787c068ff5117e9000a1de4fd98374.
2022-02-27Auto merge of #94157 - erikdesjardins:more-noundef, r=nikicbors-6/+56
Apply noundef attribute to all scalar types which do not permit raw init Beyond `&`/`&mut`/`Box`, this covers `char`, enum discriminants, `NonZero*`, etc. All such types currently cause a Miri error if left uninitialized, and an `invalid_value` lint in cases like `mem::uninitialized::<char>()`. Note that this _does not_ change whether or not it is UB for `u64` (or other integer types with no invalid values) to be undef. Fixes (partially) #74378. r? `@ghost` (blocked on #94127) `@rustbot` label S-blocked
2022-02-27Auto merge of #94412 - scottmcm:cfg-out-miri-from-swap, r=oli-obkbors-0/+27
For MIRI, cfg out the swap vectorization logic from 94212 Because of #69488 the swap logic from #94212 doesn't currently work in MIRI. Copying in smaller pieces is probably much worse for its performance anyway, so it'd probably rather just use the simple path regardless. Part of #94371, though another PR will be needed for the CTFE aspect. r? `@oli-obk` cc `@RalfJung`
2022-02-27Apply noundef metadata to loads of types that do not permit raw initErik Desjardins-4/+83
This matches the noundef attributes we apply on arguments/return types.
2022-02-26For MIRI, cfg out the swap logic from 94212Scott McMurray-0/+27
2022-02-26Auto merge of #93516 - nagisa:branch-protection, r=cjgillotbors-24/+29
No branch protection metadata unless enabled Even if we emit metadata disabling branch protection, this metadata may conflict with other modules (e.g. during LTO) that have different branch protection metadata set. This is an unstable flag and feature, so ideally the flag not being specified should act as if the feature wasn't implemented in the first place. Additionally this PR also ensures we emit an error if `-Zbranch-protection` is set on targets other than the supported aarch64. For now the error is being output from codegen, but ideally it should be moved to earlier in the pipeline before stabilization.
2022-02-26Apply noundef attribute to all scalar types which do not permit raw initErik Desjardins-6/+56
Beyond `&`/`&mut`/`Box`, this covers `char`, discriminants, `NonZero*`, etc. All such types currently cause a Miri error if left uninitialized, and an `invalid_value` lint in cases like `mem::uninitialized::<char>()` Note that this _does not_ change whether or not it is UB for `u64` (or other integer types with no invalid values) to be undef.
2022-02-26Auto merge of #94127 - erikdesjardins:debugattr, r=nikicbors-4/+67
At opt-level=0, apply only ABI-affecting attributes to functions This should provide a small perf improvement for debug builds, and should more than cancel out the perf regression from adding noundef (https://github.com/rust-lang/rust/pull/93670#issuecomment-1038347581, #94106). r? `@nikic`
2022-02-25avoid test failure on targets where all functions are dso_local (e.g. wasm)Erik Desjardins-4/+4
2022-02-25make tests work on noopt builderErik Desjardins-4/+4
2022-02-25Auto merge of #93644 - michaelwoerister:simpler-debuginfo-typemap, r=wesleywiserbors-1/+1
debuginfo: Simplify TypeMap used during LLVM debuginfo generation. This PR simplifies the TypeMap that is used in `rustc_codegen_llvm::debuginfo::metadata`. It was unnecessarily complicated because it was originally implemented when types were not yet normalized before codegen. So it did it's own normalization and kept track of multiple unnormalized types being mapped to a single unique id. This PR is based on https://github.com/rust-lang/rust/pull/93503, which is not merged yet. The PR also removes the arena used for allocating string ids and instead uses `InlinableString` from the [inlinable_string](https://crates.io/crates/inlinable_string) crate. That might not be the best choice, since that crate does not seem to be very actively maintained. The [flexible-string](https://crates.io/crates/flexible-string) crate would be an alternative. r? `@ghost`
2022-02-25Auto merge of #94130 - erikdesjardins:partially, r=nikicbors-40/+20
Use undef for (some) partially-uninit constants There needs to be some limit to avoid perf regressions on large arrays with undef in each element (see comment in the code). Fixes: #84565 Original PR: #83698 Depends on LLVM 14: #93577
2022-02-24Rollup merge of #94212 - scottmcm:swapper, r=dtolnayDylan DPC-0/+140
Stop manually SIMDing in `swap_nonoverlapping` Like I previously did for `reverse` (#90821), this leaves it to LLVM to pick how to vectorize it, since it can know better the chunk size to use, compared to the "32 bytes always" approach we currently have. A variety of codegen tests are included to confirm that the various cases are still being vectorized. It does still need logic to type-erase in some cases, though, as while LLVM is now smart enough to vectorize over slices of things like `[u8; 4]`, it fails to do so over slices of `[u8; 3]`. As a bonus, this change also means one no longer gets the spurious `memcpy`(s?) at the end up swapping a slice of `__m256`s: <https://rust.godbolt.org/z/joofr4v8Y> <details> <summary>ASM for this example</summary> ## Before (from godbolt) note the `push`/`pop`s and `memcpy` ```x86 swap_m256_slice: push r15 push r14 push r13 push r12 push rbx sub rsp, 32 cmp rsi, rcx jne .LBB0_6 mov r14, rsi shl r14, 5 je .LBB0_6 mov r15, rdx mov rbx, rdi xor eax, eax .LBB0_3: mov rcx, rax vmovaps ymm0, ymmword ptr [rbx + rax] vmovaps ymm1, ymmword ptr [r15 + rax] vmovaps ymmword ptr [rbx + rax], ymm1 vmovaps ymmword ptr [r15 + rax], ymm0 add rax, 32 add rcx, 64 cmp rcx, r14 jbe .LBB0_3 sub r14, rax jbe .LBB0_6 add rbx, rax add r15, rax mov r12, rsp mov r13, qword ptr [rip + memcpy@GOTPCREL] mov rdi, r12 mov rsi, rbx mov rdx, r14 vzeroupper call r13 mov rdi, rbx mov rsi, r15 mov rdx, r14 call r13 mov rdi, r15 mov rsi, r12 mov rdx, r14 call r13 .LBB0_6: add rsp, 32 pop rbx pop r12 pop r13 pop r14 pop r15 vzeroupper ret ``` ## After (from my machine) Note no `rsp` manipulation, sorry for different ASM syntax ```x86 swap_m256_slice: cmpq %r9, %rdx jne .LBB1_6 testq %rdx, %rdx je .LBB1_6 cmpq $1, %rdx jne .LBB1_7 xorl %r10d, %r10d jmp .LBB1_4 .LBB1_7: movq %rdx, %r9 andq $-2, %r9 movl $32, %eax xorl %r10d, %r10d .p2align 4, 0x90 .LBB1_8: vmovaps -32(%rcx,%rax), %ymm0 vmovaps -32(%r8,%rax), %ymm1 vmovaps %ymm1, -32(%rcx,%rax) vmovaps %ymm0, -32(%r8,%rax) vmovaps (%rcx,%rax), %ymm0 vmovaps (%r8,%rax), %ymm1 vmovaps %ymm1, (%rcx,%rax) vmovaps %ymm0, (%r8,%rax) addq $2, %r10 addq $64, %rax cmpq %r10, %r9 jne .LBB1_8 .LBB1_4: testb $1, %dl je .LBB1_6 shlq $5, %r10 vmovaps (%rcx,%r10), %ymm0 vmovaps (%r8,%r10), %ymm1 vmovaps %ymm1, (%rcx,%r10) vmovaps %ymm0, (%r8,%r10) .LBB1_6: vzeroupper retq ``` </details> This does all its copying operations as either the original type or as `MaybeUninit`s, so as far as I know there should be no potential abstract machine issues with reading padding bytes as integers. <details> <summary>Perf is essentially unchanged</summary> Though perhaps with more target features this would help more, if it could pick bigger chunks ## Before ``` running 10 tests test slice::swap_with_slice_4x_usize_30 ... bench: 894 ns/iter (+/- 11) test slice::swap_with_slice_4x_usize_3000 ... bench: 99,476 ns/iter (+/- 2,784) test slice::swap_with_slice_5x_usize_30 ... bench: 1,257 ns/iter (+/- 7) test slice::swap_with_slice_5x_usize_3000 ... bench: 139,922 ns/iter (+/- 959) test slice::swap_with_slice_rgb_30 ... bench: 328 ns/iter (+/- 27) test slice::swap_with_slice_rgb_3000 ... bench: 16,215 ns/iter (+/- 176) test slice::swap_with_slice_u8_30 ... bench: 312 ns/iter (+/- 9) test slice::swap_with_slice_u8_3000 ... bench: 5,401 ns/iter (+/- 123) test slice::swap_with_slice_usize_30 ... bench: 368 ns/iter (+/- 3) test slice::swap_with_slice_usize_3000 ... bench: 28,472 ns/iter (+/- 3,913) ``` ## After ``` running 10 tests test slice::swap_with_slice_4x_usize_30 ... bench: 868 ns/iter (+/- 36) test slice::swap_with_slice_4x_usize_3000 ... bench: 99,642 ns/iter (+/- 1,507) test slice::swap_with_slice_5x_usize_30 ... bench: 1,194 ns/iter (+/- 11) test slice::swap_with_slice_5x_usize_3000 ... bench: 139,761 ns/iter (+/- 5,018) test slice::swap_with_slice_rgb_30 ... bench: 324 ns/iter (+/- 6) test slice::swap_with_slice_rgb_3000 ... bench: 15,962 ns/iter (+/- 287) test slice::swap_with_slice_u8_30 ... bench: 281 ns/iter (+/- 5) test slice::swap_with_slice_u8_3000 ... bench: 5,324 ns/iter (+/- 40) test slice::swap_with_slice_usize_30 ... bench: 275 ns/iter (+/- 5) test slice::swap_with_slice_usize_3000 ... bench: 28,277 ns/iter (+/- 277) ``` </detail>
2022-02-24Add test for nested structures.Petr Sumbera-0/+20
2022-02-21debuginfo: Simplify TypeMap used during LLVM debuginfo generation.Michael Woerister-1/+1
The previous implementation was written before types were properly normalized for code generation and had to assume a more complicated relationship between types and their debuginfo -- generating separate identifiers for debuginfo nodes that were based on normalized types. Since types are now already normalized, we can use them as identifiers for debuginfo nodes.
2022-02-21Stop manually SIMDing in swap_nonoverlappingScott McMurray-0/+140
Like I previously did for `reverse`, this leaves it to LLVM to pick how to vectorize it, since it can know better the chunk size to use, compared to the "32 bytes always" approach we currently have. It does still need logic to type-erase where appropriate, though, as while LLVM is now smart enough to vectorize over slices of things like `[u8; 4]`, it fails to do so over slices of `[u8; 3]`. As a bonus, this also means one no longer gets the spurious `memcpy`(s?) at the end up swapping a slice of `__m256`s: <https://rust.godbolt.org/z/joofr4v8Y>
2022-02-20limit tests to llvm 14+Erik Desjardins-1/+2
2022-02-19Auto merge of #92911 - nbdd0121:unwind, r=Amanieubors-12/+2
Guard against unwinding in cleanup code Currently the only safe guard we have against double unwind is the panic count (which is local to Rust). When double unwinds indeed happen (e.g. C++ exception + Rust panic, or two C++ exceptions), then the second unwind actually goes through and the first unwind is leaked. This can cause UB. cc rust-lang/project-ffi-unwind#6 E.g. given the following C++ code: ```c++ extern "C" void foo() { throw "A"; } extern "C" void execute(void (*fn)()) { try { fn(); } catch(...) { } } ``` This program is well-defined to terminate: ```c++ struct dtor { ~dtor() noexcept(false) { foo(); } }; void a() { dtor a; dtor b; } int main() { execute(a); return 0; } ``` But this Rust code doesn't catch the double unwind: ```rust extern "C-unwind" { fn foo(); fn execute(f: unsafe extern "C-unwind" fn()); } struct Dtor; impl Drop for Dtor { fn drop(&mut self) { unsafe { foo(); } } } extern "C-unwind" fn a() { let _a = Dtor; let _b = Dtor; } fn main() { unsafe { execute(a) }; } ``` To address this issue, this PR adds an unwind edge to an abort block, so that the Rust example aborts. This is similar to how clang guards against double unwind (except clang calls terminate per C++ spec and we abort). The cost should be very small; it's an additional trap instruction (well, two for now, since we use TrapUnreachable, but that's a different issue) for each function with landing pads; if LLVM gains support to encode "abort/terminate" info directly in LSDA like GCC does, then it'll be free. It's an additional basic block though so compile time may be worse, so I'd like a perf run. r? `@ghost` `@rustbot` label: F-c_unwind
2022-02-19Fix codegen test for MSVCGary Guo-13/+2
2022-02-19reduce default uninit_const_chunk_threshold to 16 (from 256)Erik Desjardins-2/+2
2022-02-19No branch protection metadata unless enabledSimonas Kazlauskas-24/+29
Even if we emit metadata disabling branch protection, this metadata may conflict with other modules (e.g. during LTO) that have different branch protection metadata set. This is an unstable flag and feature, so ideally the flag not being specified should act as if the feature wasn't implemented in the first place. Additionally this PR also ensures we emit an error if `-Zbranch-protection` is set on targets other than the supported aarch64. For now the error is being output from codegen, but ideally it should be moved to earlier in the pipeline before stabilization.
2022-02-19always add align attributesErik Desjardins-2/+2
2022-02-19switch to limiting the number of init/uninit chunksErik Desjardins-1/+2
2022-02-18Rollup merge of #91675 - ivanloz:memtagsan, r=nagisaMatthias Krüger-0/+12
Add MemTagSanitizer Support Add support for the LLVM [MemTagSanitizer](https://llvm.org/docs/MemTagSanitizer.html). On hardware which supports it (see caveats below), the MemTagSanitizer can catch bugs similar to AddressSanitizer and HardwareAddressSanitizer, but with lower overhead. On a tag mismatch, a SIGSEGV is signaled with code SEGV_MTESERR / SEGV_MTEAERR. # Usage `-Zsanitizer=memtag -C target-feature="+mte"` # Comments/Caveats * MemTagSanitizer is only supported on AArch64 targets with hardware support * Requires `-C target-feature="+mte"` * LLVM MemTagSanitizer currently only performs stack tagging. # TODO * Tests * Example
2022-02-18Use undef for partially-uninit constants up to 1024 bytesErik Desjardins-39/+17
There needs to be some limit to avoid perf regressions on large arrays with undef in each element (see comment in the code).
2022-02-18At opt-level=0, apply only ABI-affecting attributes to functionsErik Desjardins-0/+63
This should provide a small perf improvement for debug builds, and should more than cancel out the regression from adding noundef, which was only significant in debug builds.
2022-02-16MemTagSanitizer SupportIvan Lozano-0/+12
Adds support for the LLVM MemTagSanitizer.
2022-02-14Add support for control-flow protectionAndrew Brown-0/+38
This change adds a flag for configuring control-flow protection in the LLVM backend. In Clang, this flag is exposed as `-fcf-protection` with options `none|branch|return|full`. This convention is followed for `rustc`, though as a codegen option: `rustc -Z cf-protection=<none|branch|return|full>`. Co-authored-by: BlackHoleFox <blackholefoxdev@gmail.com>
2022-02-13Fix codegen testsGary Guo-6/+7
2022-02-13Auto merge of #93670 - erikdesjardins:noundef, r=nikicbors-88/+76
Apply noundef attribute to &T, &mut T, Box<T>, bool This doesn't handle `char` because it's a bit awkward to distinguish it from `u32` at this point in codegen. Note that this _does not_ change whether or not it is UB for `&`, `&mut`, or `Box` to point to undef. It only applies to the pointer itself, not the pointed-to memory. Fixes (partially) #74378. r? `@nikic` cc `@RalfJung`
2022-02-12make fastcall-inreg and riscv64-lp64-lp64f-lp64d-abi tests able to run on ↵Erik Desjardins-56/+21
any host platform (with the right llvm components)
2022-02-12fix non-x64 testsErik Desjardins-2/+2
2022-02-09Rollup merge of #93503 - ↵Matthias Krüger-26/+49
michaelwoerister:fix-vtable-holder-debuginfo-regression, r=wesleywiser debuginfo: Fix DW_AT_containing_type vtable debuginfo regression This PR brings back the `DW_AT_containing_type` attribute for vtables after it has accidentally been removed in #89597. It also implements a more accurate description of vtables. Instead of describing them as an array of void pointers, the compiler will now emit a struct type description with a field for each entry of the vtable. r? ``@wesleywiser`` This PR should fix issue https://github.com/rust-lang/rust/issues/93164. ~~The PR is blocked on https://github.com/rust-lang/rust/pull/93154 because both of them modify the `codegen/debug-vtable.rs` test case.~~
2022-02-09Rollup merge of #91504 - cynecx:used_retain, r=nikicMatthias Krüger-0/+10
`#[used(linker)]` attribute See https://github.com/dtolnay/linkme/issues/41#issuecomment-927255631.
2022-02-08add more tests and make used(linker/compiler) mutually exclusivecynecx-4/+2
2022-02-08Auto merge of #93561 - Amanieu:more-unwind-abi, r=nagisabors-0/+184
Add more *-unwind ABI variants The following *-unwind ABIs are now supported: - "C-unwind" - "cdecl-unwind" - "stdcall-unwind" - "fastcall-unwind" - "vectorcall-unwind" - "thiscall-unwind" - "aapcs-unwind" - "win64-unwind" - "sysv64-unwind" - "system-unwind" cc `@rust-lang/wg-ffi-unwind`
2022-02-06apply noundef explicitly in all cases instead of relying on dereferenceable ↵Erik Desjardins-14/+14
implying it
2022-02-06test that MaybeUninit<bool> is not noundefErik Desjardins-0/+8
2022-02-07add tests and fix commentscynecx-0/+12
2022-02-05Apply noundef attribute to &T, &mut T, Box<T>, boolErik Desjardins-19/+34
This doesn't handle `char` because it's a bit awkward to distinguish it from u32 at this point in codegen. Note that for some types (like `&Struct` and `&mut Struct`), we already apply `dereferenceable`, which implies `noundef`, so the IR does not change.
2022-02-03debuginfo: Bring back DW_AT_containing_type for vtables after it has ↵Michael Woerister-26/+49
accidentally been removed in https://github.com/rust-lang/rust/pull/89597. Also describe vtables as structs with a field for each entry.
2022-02-02Add more *-unwind ABI variantsAmanieu d'Antras-0/+184
The following *-unwind ABIs are now supported: - "C-unwind" - "cdecl-unwind" - "stdcall-unwind" - "fastcall-unwind" - "vectorcall-unwind" - "thiscall-unwind" - "aapcs-unwind" - "win64-unwind" - "sysv64-unwind" - "system-unwind"
2022-02-02Auto merge of #93154 - ↵bors-18/+134
michaelwoerister:fix-generic-closure-and-generator-debuginfo, r=wesleywiser debuginfo: Make sure that type names for closure and generator environments are unique in debuginfo. Before this change, closure/generator environments coming from different instantiations of the same generic function were all assigned the same name even though they were distinct types with potentially different data layout. Now we append the generic arguments of the originating function to the type name. This commit also emits `{closure_env#0}` as the name of these types in order to disambiguate them from the accompanying closure function (which keeps being called `{closure#0}`). Previously both were assigned the same name. NOTE: Changing debuginfo names like this can break pretty printers and other debugger plugins. I think it's OK in this particular case because the names we are changing were ambiguous anyway. In general though it would be great to have a process for doing changes like these.
2022-02-01debuginfo: Make sure that type names for closure and generator environments ↵Michael Woerister-18/+134
are unique in debuginfo. Before this change, closure/generator environments coming from different instantiations of the same generic function were all assigned the same name even though they were distinct types with potentially different data layout. Now we append the generic arguments of the originating function to the type name. This commit also emits '{closure_env#0}' as the name of these types in order to disambiguate them from the accompanying closure function '{closure#0}'. Previously both were assigned the same name.
generated by cgit-pink 1.4.1-3-g733a5 (git 2.37.1) at 2026-02-23 10:07:33 +0000