diff options
| author | Stuart Cook <Zalathar@users.noreply.github.com> | 2025-09-12 20:02:10 +1000 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2025-09-12 20:02:10 +1000 |
| commit | 48d684111e462b1ff600728b9a816520a4f3276a (patch) | |
| tree | c76b3154a008882915152eb791c2d3fd127d4d56 /compiler/rustc_codegen_ssa | |
| parent | 8e2ed71effd5f81bff319c0c7adaca42084e2a71 (diff) | |
| parent | 94fbb2178ae5960894aaae199233b0d3ac020d79 (diff) | |
| download | rust-48d684111e462b1ff600728b9a816520a4f3276a.tar.gz rust-48d684111e462b1ff600728b9a816520a4f3276a.zip | |
Rollup merge of #144549 - folkertdev:va-arg-arm, r=saethlin
match clang's `va_arg` assembly on arm targets
tracking issue: https://github.com/rust-lang/rust/issues/44930
For this example
```rust
#![feature(c_variadic)]
#[unsafe(no_mangle)]
unsafe extern "C" fn variadic(a: f64, mut args: ...) -> f64 {
let b = args.arg::<f64>();
let c = args.arg::<f64>();
a + b + c
}
```
We currently generate (via llvm):
```asm
variadic:
sub sp, sp, #12
stmib sp, {r2, r3}
vmov d0, r0, r1
add r0, sp, #4
vldr d1, [sp, #4]
add r0, r0, #15
bic r0, r0, #7
vadd.f64 d0, d0, d1
add r1, r0, #8
str r1, [sp]
vldr d1, [r0]
vadd.f64 d0, d0, d1
vmov r0, r1, d0
add sp, sp, #12
bx lr
```
LLVM is not doing a good job. In fact, it's well-known that LLVM's implementation of `va_arg` is kind of bad, and we implement it ourselves (based on clang) for many targets already. For arm, our own `emit_ptr_va_arg` saves 3 instructions.
Next, it turns out it's important for LLVM to explicitly start and end the lifetime of the `va_list`. In https://github.com/rust-lang/rust/pull/146059 I already end the lifetime, but when looking at this again, I noticed that it is important to also start it, see https://godbolt.org/z/EGqvKTTsK: failing to explicitly start the lifetime uses an extra register.
So, the combination of `emit_ptr_va_arg` with starting/ending the lifetime makes rustc emit exactly the instructions that clang generates::
```asm
variadic:
sub sp, sp, #12
stmib sp, {r2, r3}
vmov d16, r0, r1
vldr d17, [sp, #4]
vadd.f64 d16, d16, d17
vldr d17, [sp, #12]
vadd.f64 d16, d16, d17
vmov r0, r1, d16
add sp, sp, #12
bx lr
```
The arguments to `emit_ptr_va_arg` are based on [the clang implementation](https://github.com/llvm/llvm-project/blob/03dc2a41f3d9a500e47b513de5c5008c06860d65/clang/lib/CodeGen/Targets/ARM.cpp#L798-L844).
r? ``@workingjubilee`` (I can re-roll if your queue is too full, but you do seem like the right person here)
try-job: armhf-gnu
Diffstat (limited to 'compiler/rustc_codegen_ssa')
| -rw-r--r-- | compiler/rustc_codegen_ssa/src/mir/block.rs | 2 | ||||
| -rw-r--r-- | compiler/rustc_codegen_ssa/src/mir/mod.rs | 4 |
2 files changed, 5 insertions, 1 deletions
diff --git a/compiler/rustc_codegen_ssa/src/mir/block.rs b/compiler/rustc_codegen_ssa/src/mir/block.rs index 5f6976f5d00..6492ef73956 100644 --- a/compiler/rustc_codegen_ssa/src/mir/block.rs +++ b/compiler/rustc_codegen_ssa/src/mir/block.rs @@ -520,7 +520,7 @@ impl<'a, 'tcx, Bx: BuilderMethods<'a, 'tcx>> FunctionCx<'a, 'tcx, Bx> { LocalRef::Place(va_list) => { bx.va_end(va_list.val.llval); - // Explicitly end the lifetime of the `va_list`, this matters for LLVM. + // Explicitly end the lifetime of the `va_list`, improves LLVM codegen. bx.lifetime_end(va_list.val.llval, va_list.layout.size); } _ => bug!("C-variadic function must have a `VaList` place"), diff --git a/compiler/rustc_codegen_ssa/src/mir/mod.rs b/compiler/rustc_codegen_ssa/src/mir/mod.rs index 06873313e2e..6b109e8b8e2 100644 --- a/compiler/rustc_codegen_ssa/src/mir/mod.rs +++ b/compiler/rustc_codegen_ssa/src/mir/mod.rs @@ -438,6 +438,10 @@ fn arg_local_refs<'a, 'tcx, Bx: BuilderMethods<'a, 'tcx>>( if fx.fn_abi.c_variadic && arg_index == fx.fn_abi.args.len() { let va_list = PlaceRef::alloca(bx, bx.layout_of(arg_ty)); + + // Explicitly start the lifetime of the `va_list`, improves LLVM codegen. + bx.lifetime_start(va_list.val.llval, va_list.layout.size); + bx.va_start(va_list.val.llval); return LocalRef::Place(va_list); |
