| Age | Commit message (Collapse) | Author | Lines |
|---|
|
Introduce debuginfo to statements in MIR
The PR introduces support for debug information within dead statements. Currently, only the reference statement is supported, which is sufficient to fix rust-lang/rust#128081.
I don't modify Stable MIR, as I don't think we need debug information when using it.
This PR represents the debug information for the dead reference statement via `#dbg_value`. For example, `let _foo_b = &foo.b` becomes `#dbg_value(ptr %foo, !22, !DIExpression(DW_OP_plus_uconst, 4, DW_OP_stack_value), !26)`. You can see this here: https://rust.godbolt.org/z/d43js6adv.
The general principle for handling debug information is to never provide less debug information than the optimized LLVM IR.
The current rules for dropping debug information in this PR are:
- If the LLVM IR cannot represent a reference address, it's replaced with poison or simply dropped. For example, see: https://rust.godbolt.org/z/shGqPec8W. I'm using poison in all such cases now.
- All debuginfos is dropped when merging multiple successor BBs. An example is available here: https://rust.godbolt.org/z/TE1q3Wq6M.
I doesn't drop debuginfos in `MatchBranchSimplification`, because LLVM also pick one branch for it.
|
|
|
|
|
|
|
|
|
|
|
|
Fix autodiff empty ret regression
closes https://github.com/rust-lang/rust/issues/147144
The two gsoc summer projects caused a bit of churn, which was to be expected, especially since we don't run autodiff in CI yet.
This adds a void return testcase that we should have had anyway, and fixes the regression.
r? `@Zalathar` (Just guessing since I've seen you in a few LLVM PRs and Oli is probably still busy. Feel free to reroll!)
|
|
Rollup of 6 pull requests
Successful merges:
- rust-lang/rust#143069 (Add fast-path for accessing the current thread id)
- rust-lang/rust#146518 (Improve the documentation around `ZERO_AR_DATE`)
- rust-lang/rust#146596 (Add a dummy codegen backend)
- rust-lang/rust#146617 (Don’t suggest foreign `doc(hidden)` types in "the following other types implement trait" diagnostics)
- rust-lang/rust#146635 (cg_llvm: Stop using `as_c_char_ptr` for coverage-related bindings)
- rust-lang/rust#147184 (Fix the bevy implied bounds hack for the next solver)
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
|
|
cg_llvm: Stop using `as_c_char_ptr` for coverage-related bindings
[As explained by a note in `ffi.rs`](https://github.com/rust-lang/rust/blob/8a1b39995e5b630c5872f5de5079f1f569bd5ac2/compiler/rustc_codegen_llvm/src/llvm/ffi.rs#L4-L11), passing strings and byte slices through FFI is more convenient if we take advantage of the fact that `*const c_uchar` and `*const c_char` have the same ABI.
Doing so avoids having to rely on a special helper function, since we can just call `as_ptr` instead.
(The same logic applies to every other binding that currently uses the `as_c_char_ptr` helper; I just haven't adjusted all of them yet.)
---
As a drive-by change, this PR also marks some coverage-related FFI bindings as `safe`.
|
|
|
|
|
|
|
|
Emit allocator attributes for allocator shim
This emits the same attributes we place on allocator declarations on the definitions in the allocator shim as well. This complements https://github.com/rust-lang/rust/pull/146766, which added the attribute for `#[global_allocator]` definitions. Emitting the attributes on the definitions ensures that they cannot be lost of the allocator shim participates in LTO.
See https://github.com/rust-lang/rust/issues/145995 for context, though that one was about `#[global_allocator]`. I'm not sure whether this can occur with the allocator shim as well or not, but better safe than sorry.
I'm not sure whether there is any good way to test this, as the allocator shim is not part of `--emit=llvm-ir`. I've verified this locally by inspecting the bitcode produced by `-C save-temps`.
r? ``@bjorn3``
|
|
This emits the same attributes we place on allocator declarations
(and allocator definitions using `#[global_allocator]`) on the
definitions in the allocator shim as well, making sure that the
attributes are not lost if the allocator shim participates in LTO.
|
|
Rollup of 3 pull requests
Successful merges:
- rust-lang/rust#147100 (tests: Remove ignore-android directive for fixed issue)
- rust-lang/rust#147116 (compiler: remove AbiAlign inside TargetDataLayout)
- rust-lang/rust#147134 (remove explicit deref of AbiAlign for most methods)
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
r=Zalathar
remove explicit deref of AbiAlign for most methods
Much of the compiler calls functions on Align projected from AbiAlign. AbiAlign impls Deref to its inner Align, so we can simplify these away. Also, it will minimize disruption when AbiAlign is removed.
For now, preserve usages that might resolve to PartialOrd or PartialEq, as those have odd inference.
|
|
compiler: remove AbiAlign inside TargetDataLayout
AbiAlign is a thin wrapper around Align, extant mostly because we used to track a separate quasi-notion of alignment that was never a real notion of alignment and removing all of it at once was too churny. This PR maintains AbiAlign usage in public API and most of the compiler, but direct access of these fields for TargetDataLayout is now in terms of Align only.
|
|
Add a leading dash to linker plugin arguments in the gcc codegen
Fix rust-lang/rust#130583
r? ``@bjorn3``
|
|
Much of the compiler calls functions on Align projected from AbiAlign.
AbiAlign impls Deref to its inner Align, so we can simplify these away.
Also, it will minimize disruption when AbiAlign is removed.
For now, preserve usages that might resolve to PartialOrd or PartialEq,
as those have odd inference.
|
|
|
|
TypeTree support in autodiff
# TypeTrees for Autodiff
## What are TypeTrees?
Memory layout descriptors for Enzyme. Tell Enzyme exactly how types are structured in memory so it can compute derivatives efficiently.
## Structure
```rust
TypeTree(Vec<Type>)
Type {
offset: isize, // byte offset (-1 = everywhere)
size: usize, // size in bytes
kind: Kind, // Float, Integer, Pointer, etc.
child: TypeTree // nested structure
}
```
## Example: `fn compute(x: &f32, data: &[f32]) -> f32`
**Input 0: `x: &f32`**
```rust
TypeTree(vec![Type {
offset: -1, size: 8, kind: Pointer,
child: TypeTree(vec![Type {
offset: -1, size: 4, kind: Float,
child: TypeTree::new()
}])
}])
```
**Input 1: `data: &[f32]`**
```rust
TypeTree(vec![Type {
offset: -1, size: 8, kind: Pointer,
child: TypeTree(vec![Type {
offset: -1, size: 4, kind: Float, // -1 = all elements
child: TypeTree::new()
}])
}])
```
**Output: `f32`**
```rust
TypeTree(vec![Type {
offset: -1, size: 4, kind: Float,
child: TypeTree::new()
}])
```
## Why Needed?
- Enzyme can't deduce complex type layouts from LLVM IR
- Prevents slow memory pattern analysis
- Enables correct derivative computation for nested structures
- Tells Enzyme which bytes are differentiable vs metadata
## What Enzyme Does With This Information:
Without TypeTrees (current state):
```llvm
; Enzyme sees generic LLVM IR:
define float ``@distance(ptr*`` %p1, ptr* %p2) {
; Has to guess what these pointers point to
; Slow analysis of all memory operations
; May miss optimization opportunities
}
```
With TypeTrees (our implementation):
```llvm
define "enzyme_type"="{[]:Float@float}" float ``@distance(``
ptr "enzyme_type"="{[]:Pointer}" %p1,
ptr "enzyme_type"="{[]:Pointer}" %p2
) {
; Enzyme knows exact type layout
; Can generate efficient derivative code directly
}
```
# TypeTrees - Offset and -1 Explained
## Type Structure
```rust
Type {
offset: isize, // WHERE this type starts
size: usize, // HOW BIG this type is
kind: Kind, // WHAT KIND of data (Float, Int, Pointer)
child: TypeTree // WHAT'S INSIDE (for pointers/containers)
}
```
## Offset Values
### Regular Offset (0, 4, 8, etc.)
**Specific byte position within a structure**
```rust
struct Point {
x: f32, // offset 0, size 4
y: f32, // offset 4, size 4
id: i32, // offset 8, size 4
}
```
TypeTree for `&Point` (internal representation):
```rust
TypeTree(vec![
Type { offset: 0, size: 4, kind: Float }, // x at byte 0
Type { offset: 4, size: 4, kind: Float }, // y at byte 4
Type { offset: 8, size: 4, kind: Integer } // id at byte 8
])
```
Generates LLVM:
```llvm
"enzyme_type"="{[]:Float@float}"
```
### Offset -1 (Special: "Everywhere")
**Means "this pattern repeats for ALL elements"**
#### Example 1: Array `[f32; 100]`
```rust
TypeTree(vec![Type {
offset: -1, // ALL positions
size: 4, // each f32 is 4 bytes
kind: Float, // every element is float
}])
```
Instead of listing 100 separate Types with offsets `0,4,8,12...396`
#### Example 2: Slice `&[i32]`
```rust
// Pointer to slice data
TypeTree(vec![Type {
offset: -1, size: 8, kind: Pointer,
child: TypeTree(vec![Type {
offset: -1, // ALL slice elements
size: 4, // each i32 is 4 bytes
kind: Integer
}])
}])
```
#### Example 3: Mixed Structure
```rust
struct Container {
header: i64, // offset 0
data: [f32; 1000], // offset 8, but elements use -1
}
```
```rust
TypeTree(vec![
Type { offset: 0, size: 8, kind: Integer }, // header
Type { offset: 8, size: 4000, kind: Pointer,
child: TypeTree(vec![Type {
offset: -1, size: 4, kind: Float // ALL array elements
}])
}
])
```
|
|
cg_llvm: Replace some DIBuilder wrappers with LLVM-C API bindings (part 5)
- Part of rust-lang/rust#134001
- Follow-up to rust-lang/rust#146673
---
This is another batch of LLVMDIBuilder binding migrations, replacing some our own LLVMRust bindings with bindings to upstream LLVM-C APIs.
Some of these are a little more complex than most of the previous migrations, because they split one LLVMRust binding into multiple LLVM bindings, but nothing too fancy.
This appears to be the last of the low-hanging fruit. As noted in https://github.com/rust-lang/rust/issues/134001#issuecomment-2524979268, the remaining bindings are difficult or impossible to migrate at present.
|
|
This maintains AbiAlign usage in public API and most of the compiler,
but direct access of these fields is now in terms of Align only.
|
|
Use standard attribute logic for allocator shim
Use llfn_attrs_from_instance() to generate the attributes for the allocator shim. This ensures that we generate all the usual attributes (and don't get to find out one-by-one that a certain attribute is important for a certain target). Additionally this will enable emitting the allocator-specific attributes (not included here).
This change is quite awkward because the allocator shim uses SimpleCx, while llfn_attrs_from_instance uses CodegenCx. I've switched it to use SimpleCx plus tcx/sess arguments where necessary. If there's a simpler way to do this, I'd love to know about it...
|
|
port `#[debugger_visualizer]` to the new attribute system
|
|
Use `LLVMDisposeTargetMachine`
After bumping the minimum LLVM version to 20 (rust-lang/rust#145071), we no longer need to run any custom code when disposing of a TargetMachine, so we can just use the upstream LLVM-C function.
|
|
Add an attribute to check the number of lanes in a SIMD vector after monomorphization
Allows std::simd to drop the `LaneCount<N>: SupportedLaneCount` trait and maintain good error messages.
Also, extends rust-lang/rust#145967 by including spans in layout errors for all ADTs.
r? ``@RalfJung``
cc ``@workingjubilee`` ``@programmerjake``
|
|
|
|
Use llfn_attrs_from_instance() to generate the attributes for the
allocator shim. This ensures that we generate all the usual
attributes (and don't get to find out one-by-one that a certain
attribute is important for a certain target). Additionally this
will enable emitting the allocator-specific attributes (not
included here).
This change is quite awkward because the allocator shim uses
SimpleCx, while llfn_attrs_from_instance uses CodegenCx. I've
switched it to use SimpleCx plus tcx/sess arguments where necessary.
If there's a simpler way to do this, I'd love to know about it...
|
|
r=Urgau,davidtwco
Extends AArch64 branch protection support to include GCS
Extends existing support for AArch64 branch protection to include support for [Guarded Control Stacks](https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/arm-a-profile-architecture-2022#guarded-control-stack-gcs:~:text=Extraction%20or%20tracking.-,Guarded%20Control%20Stack%20(GCS),-With%20the%202022).
|
|
|
|
monomorphization
Unify zero-length and oversized SIMD errors
|
|
Add panic=immediate-abort
MCP: https://github.com/rust-lang/compiler-team/issues/909
This adds a new panic strategy, `-Cpanic=immediate-abort`. This panic strategy essentially just codifies use of `-Zbuild-std-features=panic_immediate_abort`. This PR is intended to just set up infrastructure, and while it will change how the compiler is invoked for users of the feature, there should be no other impacts.
In many parts of the compiler, `PanicStrategy::ImmediateAbort` behaves just like `PanicStrategy::Abort`, because actually most parts of the compiler just mean to ask "can this unwind?" so I've added a helper function so we can say `sess.panic_strategy().unwinds()`.
The panic and unwind strategies have some level of compatibility, which mostly means that we can pre-compile the sysroot with unwinding panics then the sysroot can be linked with aborting panics later. The immediate-abort strategy is all-or-nothing, enforced by `compiler/rustc_metadata/src/dependency_format.rs` and this is tested for in `tests/ui/panic-runtime/`. We could _technically_ be more compatible with the other panic strategies, but immediately-aborting panics primarily exist for users who want to eliminate all the code size responsible for the panic runtime. I'm open to other use cases if people want to present them, but not right now. This PR is already large.
`-Cpanic=immediate-abort` sets both `cfg(panic = "immediate-abort")` _and_ `cfg(panic = "abort")`. bjorn3 pointed out that people may be checking for the abort cfg to ask if panics will unwind, and also the sysroot feature this is replacing used to require `-Cpanic=abort` so this seems like a good back-compat step. At least for the moment. Unclear if this is a good idea indefinitely. I can imagine this being confusing.
The changes to the standard library attributes are purely mechanical. Apart from that, I removed an `unsafe` we haven't needed for a while since the `abort` intrinsic became safe, and I've added a helpful diagnostic for people trying to use the old feature.
To test that `-Cpanic=immediate-abort` conflicts with other panic strategies, I've beefed up the core-stubs infrastructure a bit. There is now a separate attribute to set flags on it.
I've added a test that this produces the desired codegen, called `tests/run-make-cargo/panic-immediate-abort-codegen/` and also a separate run-make-cargo test that checks that we can build a binary.
|
|
Add self-profile events for target-machine creation
These code paths are surprisingly hot in the `large-workspace` benchmark (e.g. see perf changes from rust-lang/rust#146700), suggesting room for more improvement. It would be handy to see some detailed timings and execution counts.
|
|
Support ctr and lr as clobber-only registers in PowerPC inline assembly
Follow-up to rust-lang/rust#131341.
CTR and LR are marked as volatile in all ABIs, but I skipped them in rust-lang/rust#131341 due to they are currently marked as reserved.
https://github.com/rust-lang/rust/blob/dd7fda570040e8a736f7d8bc28ddd1b444aabc82/compiler/rustc_target/src/asm/powerpc.rs#L209-L212
However, they are actually only unusable as input/output of inline assembly, and should be fine to support as clobber-only registers as discussed in [#t-compiler > ppc/ppc64 inline asm support](https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/ppc.2Fppc64.20inline.20asm.20support/with/540413845).
r? ````@Amanieu```` or ````@workingjubilee````
cc ````@programmerjake````
````@rustbot```` label +O-PowerPC +A-inline-assembly
|
|
|
|
|
|
|
|
These code paths are surprisingly hot in the `large-workspace` benchmark; it
would be handy to see some detailed timings and execution counts.
|
|
|
|
This helps us avoid the hardcoded lists elsewhere.
|
|
cg_llvm: Move target machine command-line quoting from C++ to Rust
When this code was introduced in rust-lang/rust#130446 and rust-lang/rust#131805, it was complicated by the need to maintain compatibility with earlier versions of LLVM.
Now that LLVM 20 is the baseline (rust-lang/rust#145071), we can do all of the quoting in pure Rust code, and pass two flat strings to LLVM to be used as-is.
---
In this PR, my priority has been to preserve the existing behaviour as much as possible, without worrying too much about what the behaviour *should* be. (Though I did avoid a leading space before the first argument.)
|
|
r=workingjubilee
rustc_codegen_llvm: Feature Conversion Tidying
The author thinks we can improve `to_llvm_features`, a function to convert a Rust target feature name into an LLVM feature (or nothing, to ignore features unsupported by LLVM) for better maintainability.
1. We can simplify some clauses and some expressions.
2. There are some readability issues.
This PR attempts to resolve some of them by tidying many cases.
|
|
In each of these casts, the LHS is already `u64`.
|
|
|
|
|
|
Signed-off-by: Karan Janthe <karanjanthe@gmail.com>
|
|
|
|
|
