| Age | Commit message (Collapse) | Author | Lines |
|---|
|
Respect verify-llvm-ir option in the backend
We are currently unconditionally verifying the LLVM IR in the backend (twice), ignoring the value of the verify-llvm-ir option. This has substantial compile-time impact for debug builds.
|
|
Support input/output in vector registers of PowerPC inline assembly
This extends currently clobber-only vector registers (`vreg`) support to allow passing `#[repr(simd)]` types as input/output.
| Architecture | Register class | Target feature | Allowed types |
| ------------ | -------------- | -------------- | -------------- |
| PowerPC | `vreg` | `altivec` | `i8x16`, `i16x8`, `i32x4`, `f32x4` |
| PowerPC | `vreg` | `vsx` | `f32`, `f64`, `i64x2`, `f64x2` |
In addition to floats and `core::simd` types listed above, `core::arch` types and custom `#[repr(simd)]` types of the same size and type are also allowed. All allowed types and relevant target features are currently unstable.
r? `@Amanieu`
`@rustbot` label +O-PowerPC +A-inline-assembly
|
|
This reverts commit adf9b5fcd1de43eaf0a779e10612caee8b47bede, reversing
changes made to af1ca153d4aed5ffe22445273aa388a8d3f8f4ae.
Reverting due to <https://github.com/rust-lang/rust/issues/133606>.
|
|
Enable -Zshare-generics for inline(never) functions
This avoids inlining cross-crate generic items when possible that are
already marked inline(never), implying that the author is not intending
for the function to be inlined by callers. As such, having a local copy
may make it easier for LLVM to optimize but mostly just adds to binary
bloat and codegen time. In practice our benchmarks indicate this is
indeed a win for larger compilations, where the extra cost in dynamic
linking to these symbols is diminished compared to the advantages in
fewer copies that need optimizing in each binary.
It might also make sense it expand this with other heuristics (e.g.,
`#[cold]`) in the future, but this seems like a good starting point.
FWIW, I expect that doing cleanup in where we make the decision
what should/shouldn't be shared is also a good idea. Way too
much code needed to be tweaked to check this. But I'm hoping
to leave that for a follow-up PR rather than blocking this on it.
|
|
This reduces code sizes and better respects programmer intent when
marking inline(never). Previously such a marking was essentially ignored
for generic functions, as we'd still inline them in remote crates.
|
|
|
|
|
|
Support predicate registers (clobber-only) in Hexagon inline assembly
The result of the Hexagon instructions such as comparison, store conditional, etc. is stored in predicate registers (`p[0-3]`), but currently there is no way to mark it as clobbered in `asm!`.
This is also needed for `clobber_abi` (although implementing `clobber_abi` will require the addition of support for [several more register classes](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/Hexagon/HexagonRegisterInfo.cpp#L71-L90). see also https://github.com/rust-lang/rust/issues/93335#issuecomment-2395210055).
Refs:
- [Section 6 "Conditional Execution" in Qualcomm Hexagon V73 Programmer’s Reference Manual](https://docs.qualcomm.com/bundle/publicresource/80-N2040-53_REV_AB_Qualcomm_Hexagon_V73_Programmers_Reference_Manual.pdf#page=90)
- [Register definition in LLVM](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/Hexagon/HexagonRegisterInfo.td#L155)
cc `@androm3da` (target maintainer of hexagon-unknown-{[none-elf](https://doc.rust-lang.org/nightly/rustc/platform-support/hexagon-unknown-none-elf.html#target-maintainers),[linux-musl](https://doc.rust-lang.org/nightly/rustc/platform-support/hexagon-unknown-linux-musl.html#target-maintainers)})
r? `@Amanieu`
`@rustbot` label +A-inline-assembly
(Currently there is no O-hexagon label...)
|
|
coverage: Store coverage source regions as `Span` until codegen
Historically, coverage spans were converted into line/column coordinates during the MIR instrumentation pass.
This PR moves that conversion step into codegen, so that coverage spans spend most of their time stored as `Span` instead.
In addition to being conceptually nicer, this also reduces the size of coverage mappings in MIR, because `Span` is smaller than 4x u32.
---
There should be no changes to coverage output.
|
|
We are currently unconditionally verifying the LLVM IR in the
backend (twice), ignoring the value of the verify-llvm-ir option.
|
|
|
|
Support input/output in vector registers of s390x inline assembly (under asm_experimental_reg feature)
This extends currently clobber-only vector registers (`vreg`) support to allow passing `#[repr(simd)]` types, floats (f32/f64/f128), and integers (i32/i64/i128) as input/output.
This is unstable and gated under new `#![feature(asm_experimental_reg)]` (tracking issue: https://github.com/rust-lang/rust/issues/133416). If the feature is not enabled, only clober is supported as before.
| Architecture | Register class | Target feature | Allowed types |
| ------------ | -------------- | -------------- | -------------- |
| s390x | `vreg` | `vector` | `i32`, `f32`, `i64`, `f64`, `i128`, `f128`, `i8x16`, `i16x8`, `i32x4`, `i64x2`, `f32x4`, `f64x2` |
This matches the list of types that are supported by the vector registers in LLVM:
https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.td#L301-L313
In addition to `core::simd` types and floats listed above, custom `#[repr(simd)]` types of the same size and type are also allowed. All allowed types other than i32/f32/i64/f64/i128, and relevant target features are currently unstable.
Currently there is no SIMD type for s390x in `core::arch`, but this is tracked in https://github.com/rust-lang/rust/issues/130869.
cc https://github.com/rust-lang/rust/issues/130869 about vector facility support in s390x
cc https://github.com/rust-lang/rust/issues/125398 & https://github.com/rust-lang/rust/issues/116909 about f128 support in asm
`@rustbot` label +O-SystemZ +A-inline-assembly
|
|
Fix asm goto with outputs and move it to a separate feature gate
Tracking issue: #119364
This PR addresses 3 aspects of asm goto with outputs:
* Codegen is fixed. My initial implementation has an oversight which cause the output to be only stored in fallthrough path, but not in label blocks.
* Outputs can now be used with `options(noreturn)` if a label block is given.
* All of this is moved to a new feature gate, because we likely want to stabilise `asm_goto` before asm goto with outputs.
`@rustbot` labels: +A-inline-assembly +F-asm
|
|
Remove legacy bitcode for iOS
Follow #117364.
|
|
When labels are present, the `noreturn` option really means that asm block
won't fallthrough -- if labels are present, then outputs can still be
meaningfully used.
|
|
When outputs are used together with labels, they are considered
to be written for all destinations, not only when falling through.
|
|
|
|
|
|
|
|
This will avoid confusion with actual `Span` spans.
|
|
A used function with no mappings has historically indicated a bug, but that
will no longer be the case after moving some fallible span-processing steps
into codegen.
|
|
|
|
Allow disabling ASan instrumentation for globals
AddressSanitizer adds instrumentation to global variables unless the [`no_sanitize_address`](https://llvm.org/docs/LangRef.html#global-attributes) attribute is set on them.
This commit extends the existing `#[no_sanitize(address)]` attribute to set this; previously it only had the desired effect on functions.
(cc https://github.com/rust-lang/rust/issues/39699)
|
|
|
|
info for the function instead of panicking.
The maximum discriminator value LLVM can currently encode is 2^12. If macro use
results in more than 2^12 calls to the same function attributed to the same
callsite, and those calls are MIR-inlined, we will require more than the maximum
discriminator value to completely represent the debug information. Once we reach
that point drop the debug info instead.
|
|
The test relies on the fact that inlining more than 2^12 calls at the same
callsite will trigger a panic (and after the following commit, a warning) due to
LLVM limitations but with collapse_debuginfo the callsites should not be the
same.
|
|
the behavior of the type system not only depends on the current
assumptions, but also the currentnphase of the compiler. This is
mostly necessary as we need to decide whether and how to reveal
opaque types. We track this via the `TypingMode`.
|
|
|
|
allow CFGuard on windows-gnullvm
No unit tests because of https://github.com/rust-lang/rust/issues/132278
|
|
CFI: Append debug location to CFI blocks
Currently we're not appending debug locations to the inserted CFI blocks. This shows up in #132615 and #100783. This change fixes that by passing down the debug location to the CFI type-test generation and appending it to the blocks.
Credits also belong to `@jakos-sec` who worked with me on this.
|
|
|
|
Add a default implementation for CodegenBackend::link
As a side effect this should add raw-dylib support to cg_gcc as the default ArchiveBuilderBuilder that is used implements create_dll_import_lib. I haven't tested if the raw-dylib support actually works however.
|
|
|
|
target_features: explain what exacty 'implied' means here
|
|
|
|
coverage: Restrict empty-span expansion to only cover `{` and `}`
Coverage instrumentation has some tricky code for converting a coverage-relevant `Span` into a set of start/end line/byte-column coordinates that will be embedded in the CGU's coverage metadata.
A big part of this complexity is special code for handling empty spans, which are expanded into non-empty spans (if possible) because LLVM's coverage reporter does not handle empty spans well.
This PR simplifies that code by restricting it to only apply in two specific situations: when the character after the empty span is `{`, or the character before the empty span is `}`.
(As an added benefit, this means that the expanded spans no longer extend awkwardly beyond the end of a physical line, which was common under the previous implementation.)
Along the way, this PR also removes some unhelpful code for dealing with function source code spread across multiple files. Functions currently can't have coverage spans in multiple files, and if that ever changes (e.g. to properly support expansion regions) then this code will need to be completely overhauled anyway.
|
|
|
|
After link_binary the temporary files referenced by CodegenResults are
deleted, so calling link_binary again with the same CodegenResults
should not be allowed.
|
|
single point.
LLVM does not expect to ever see multiple dbg_declares for the same variable at the same
location with different values. proc-macros make it possible for arbitrary code,
including multiple calls that get inlined, to happen at any given location in the source
code. Add discriminators when that happens so these locations are different to LLVM.
This may interfere with the AddDiscriminators pass in LLVM, which is added by the
unstable flag -Zdebug-info-for-profiling.
Fixes #131944
|
|
Rollup of 5 pull requests
Successful merges:
- #132552 (Add v9, v8plus, and leoncasa target feature to sparc and use v8plus in create_object_file)
- #132745 (pointee_info_at: fix logic for recursing into enums)
- #132777 (try_question_mark_nop: update test for LLVM 20)
- #132785 (rustc_target: more target string fixes for LLVM 20)
- #132794 (Use a separate dir for r-a builds consistently in helix config)
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
Add v9, v8plus, and leoncasa target feature to sparc and use v8plus in create_object_file
This adds the following three unstable target features:
- `v9`: SPARC-V9 instructions ([LLVM definition][sparc-v9])
- Relevant to https://github.com/rust-lang/rust/pull/131222#issuecomment-2453310963
- Relevant to https://github.com/rust-lang/rust/pull/132472#discussion_r1832606081
- This is also needed to implement https://github.com/taiki-e/atomic-maybe-uninit/pull/31 (depends on inline assembly support) more robustly.
- `v8plus`: SPARC-V8+ ABI ([LLVM definition][sparc-v8plus])
- This is added in LLVM 20. In LLVM 19 and older, it is emulated to work the same way as LLVM in each LLVM version.
- See https://github.com/rust-lang/rust/issues/132585#issuecomment-2453926257 for more.
- `leoncasa`: CASA instruction[^1] of LEON3 and LEON4 processors ([LLVM definition][sparc-leoncasa], LLVM feature name: `hasleoncasa`)
- This is needed to implement https://github.com/taiki-e/atomic-maybe-uninit/pull/31 (depends on inline assembly support) more robustly.
[^1]: Atomic CAS instruction
[sparc-v9]: https://github.com/llvm/llvm-project/blob/f5e4ffaa49254706ad6fa209de8aec28e20f0041/llvm/lib/Target/Sparc/Sparc.td#L37-L39
[sparc-v8plus]: https://github.com/llvm/llvm-project/blob/f5e4ffaa49254706ad6fa209de8aec28e20f0041/llvm/lib/Target/Sparc/Sparc.td#L37-L39
[sparc-leoncasa]: https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/Sparc/LeonFeatures.td#L32-L37
|
|
Trim and tidy includes in `rustc_llvm`
These includes tend to accumulate over time, and are usually only removed when something breaks in a new LLVM version, so it's nice to clean them up manually once in a while.
General strategy used for this PR:
- Remove all includes from `LLVMWrapper.h` that aren't needed by the header itself, transplanting them to individual source files as necessary.
- For each source file, temporarily remove each include if doing so doesn't cause a compile error.
- If a “required” include looks like it shouldn't be needed, try replacing it with its sub-includes, then trim that list.
- After doing all of the above, go back and re-add any removed include if the file does actually use things defined in that header, even if the header happens to also be included by something else.
|
|
|
|
|
|
|
|
Functions currently can't have mappings in multiple files, and if that ever
changes (e.g. to properly support expansion regions), this code will need to be
completely overhauled anyway.
|
|
We already had a dedicated `LocalFileId` index type, but previously we used a
raw `u32` for global file IDs, because index types were harder to pass through
FFI.
|
|
Simplify FFI calls for `-Ztime-llvm-passes` and `-Zprint-codegen-stats`
The existing code for these unstable LLVM-infodump flags was jumping through hoops to pass an allocated C string across the FFI boundary, when it's much simpler to just write to a `&RustString` instead.
|
|
coverage: Extract safe FFI wrapper functions to `llvm_cov`
This PR takes all of the inline `unsafe` calls in coverage codegen, and all the safe wrapper functions in `coverageinfo/mod.rs`, and moves them to a new `llvm_cov` submodule that is dedicated to safe FFI wrapper functions. This reduces the mixing of abstraction levels in the rest of coverage codegen.
As a follow-up, this PR also tidies up the names and signatures of several of the coverage FFI functions.
|
|
Update mips64 data layout to match LLVM 20 change
LLVM changed the data layout in https://github.com/llvm/llvm-project/pull/112084
|
