| Age | Commit message (Collapse) | Author | Lines |
|---|
|
TypeTree support in autodiff
# TypeTrees for Autodiff
## What are TypeTrees?
Memory layout descriptors for Enzyme. Tell Enzyme exactly how types are structured in memory so it can compute derivatives efficiently.
## Structure
```rust
TypeTree(Vec<Type>)
Type {
offset: isize, // byte offset (-1 = everywhere)
size: usize, // size in bytes
kind: Kind, // Float, Integer, Pointer, etc.
child: TypeTree // nested structure
}
```
## Example: `fn compute(x: &f32, data: &[f32]) -> f32`
**Input 0: `x: &f32`**
```rust
TypeTree(vec![Type {
offset: -1, size: 8, kind: Pointer,
child: TypeTree(vec![Type {
offset: -1, size: 4, kind: Float,
child: TypeTree::new()
}])
}])
```
**Input 1: `data: &[f32]`**
```rust
TypeTree(vec![Type {
offset: -1, size: 8, kind: Pointer,
child: TypeTree(vec![Type {
offset: -1, size: 4, kind: Float, // -1 = all elements
child: TypeTree::new()
}])
}])
```
**Output: `f32`**
```rust
TypeTree(vec![Type {
offset: -1, size: 4, kind: Float,
child: TypeTree::new()
}])
```
## Why Needed?
- Enzyme can't deduce complex type layouts from LLVM IR
- Prevents slow memory pattern analysis
- Enables correct derivative computation for nested structures
- Tells Enzyme which bytes are differentiable vs metadata
## What Enzyme Does With This Information:
Without TypeTrees (current state):
```llvm
; Enzyme sees generic LLVM IR:
define float ``@distance(ptr*`` %p1, ptr* %p2) {
; Has to guess what these pointers point to
; Slow analysis of all memory operations
; May miss optimization opportunities
}
```
With TypeTrees (our implementation):
```llvm
define "enzyme_type"="{[]:Float@float}" float ``@distance(``
ptr "enzyme_type"="{[]:Pointer}" %p1,
ptr "enzyme_type"="{[]:Pointer}" %p2
) {
; Enzyme knows exact type layout
; Can generate efficient derivative code directly
}
```
# TypeTrees - Offset and -1 Explained
## Type Structure
```rust
Type {
offset: isize, // WHERE this type starts
size: usize, // HOW BIG this type is
kind: Kind, // WHAT KIND of data (Float, Int, Pointer)
child: TypeTree // WHAT'S INSIDE (for pointers/containers)
}
```
## Offset Values
### Regular Offset (0, 4, 8, etc.)
**Specific byte position within a structure**
```rust
struct Point {
x: f32, // offset 0, size 4
y: f32, // offset 4, size 4
id: i32, // offset 8, size 4
}
```
TypeTree for `&Point` (internal representation):
```rust
TypeTree(vec![
Type { offset: 0, size: 4, kind: Float }, // x at byte 0
Type { offset: 4, size: 4, kind: Float }, // y at byte 4
Type { offset: 8, size: 4, kind: Integer } // id at byte 8
])
```
Generates LLVM:
```llvm
"enzyme_type"="{[]:Float@float}"
```
### Offset -1 (Special: "Everywhere")
**Means "this pattern repeats for ALL elements"**
#### Example 1: Array `[f32; 100]`
```rust
TypeTree(vec![Type {
offset: -1, // ALL positions
size: 4, // each f32 is 4 bytes
kind: Float, // every element is float
}])
```
Instead of listing 100 separate Types with offsets `0,4,8,12...396`
#### Example 2: Slice `&[i32]`
```rust
// Pointer to slice data
TypeTree(vec![Type {
offset: -1, size: 8, kind: Pointer,
child: TypeTree(vec![Type {
offset: -1, // ALL slice elements
size: 4, // each i32 is 4 bytes
kind: Integer
}])
}])
```
#### Example 3: Mixed Structure
```rust
struct Container {
header: i64, // offset 0
data: [f32; 1000], // offset 8, but elements use -1
}
```
```rust
TypeTree(vec![
Type { offset: 0, size: 8, kind: Integer }, // header
Type { offset: 8, size: 4000, kind: Pointer,
child: TypeTree(vec![Type {
offset: -1, size: 4, kind: Float // ALL array elements
}])
}
])
```
|
|
cg_llvm: Replace some DIBuilder wrappers with LLVM-C API bindings (part 5)
- Part of rust-lang/rust#134001
- Follow-up to rust-lang/rust#146673
---
This is another batch of LLVMDIBuilder binding migrations, replacing some our own LLVMRust bindings with bindings to upstream LLVM-C APIs.
Some of these are a little more complex than most of the previous migrations, because they split one LLVMRust binding into multiple LLVM bindings, but nothing too fancy.
This appears to be the last of the low-hanging fruit. As noted in https://github.com/rust-lang/rust/issues/134001#issuecomment-2524979268, the remaining bindings are difficult or impossible to migrate at present.
|
|
|
|
LLVM change dfbd76bda01e removed separate remark support entirely, but
it turns out we can just drop the parameter and everything appears to
work fine.
Fixes 146912 as far as I can tell (the test passes.)
@rustbot label llvm-main
|
|
|
|
These should have been removed earlier, when we switched to the corresponding
LLVM-C bindings.
|
|
|
|
|
|
Signed-off-by: Karan Janthe <karanjanthe@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Use captures(address) instead of captures(none) for indirect args
While provenance cannot be captured through these arguments, the address / object identity can.
Fixes https://github.com/rust-lang/rust/issues/137668.
r? `@ghost`
|
|
While provenance cannot be captured through these arguments, the
address / object identity can.
|
|
The underlying implementation of `LLVMCreateConstantRangeAttribute` assumes
that each of `LowerWords` and `UpperWords` points to enough u64 values to
define an integer of the specified bit-length, and will encounter UB if that is
not the case.
Our safe wrapper function always passes pointers to `[u64; 2]` arrays,
regardless of the bit-length specified. That's fine in practice, because scalar
primitives never exceed 128 bits, but it is technically a soundness hole in a
safe function.
We can close the soundness hole by explicitly asserting `size_bits <= 128`.
This is effectively just a stricter version of the existing check that the
value must be small enough to fit in `c_uint`.
|
|
`&Freeze` parameters are not only `readonly` within the function,
but any captures of the pointer can also only be used for reads.
This can now be encoded using the `captures(address, read_provenance)`
attribute.
|
|
|
|
|
|
Set the dead_on_return attribute (added in LLVM 21) for arguments
that are passed indirectly, but not byval.
This indicates that the value of the argument on return does not
matter, enabling additional dead store elimination.
|
|
Implement support for `become` and explicit tail call codegen for the LLVM backend
This PR implements codegen of explicit tail calls via `become` in `rustc_codegen_ssa` and support within the LLVM backend. Completes a task on (https://github.com/rust-lang/rust/issues/112788). This PR implements all the necessary bits to make explicit tail calls usable, other backends have received stubs for now and will ICE if you use `become` on them. I suspect there is some bikeshedding to be done on how we should go about implementing this for other backends, but it should be relatively straightforward for GCC after this is merged.
During development I also put together a POC bytecode VM based on tail call dispatch to test these changes out and analyze the codegen to make sure it generates expected assembly. That is available [here](https://github.com/xacrimon/tcvm).
|
|
LLVM codegen backend.
|
|
Link: https://github.com/llvm/llvm-project/pull/144383
|
|
|
|
This LLVM-C binding replaces the existing `LLVMRustInlineAsm` function.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This API allows us to set the nuw flag as well.
|
|
|
|
The LLVM-C binding takes an explicit context, whereas our binding obtained the
context from the scope argument.
|
|
|
|
|
|
|
|
|
|
Rollup of 9 pull requests
Successful merges:
- #134531 ([rustdoc] Add `--extract-doctests` command-line flag)
- #135860 (Compiler: Finalize dyn compatibility renaming)
- #135992 (Improve documentation when adding a new target)
- #136194 (Support clobber_abi in BPF inline assembly)
- #136325 (Delay a bug when indexing unsized slices)
- #136326 (Replace our `LLVMRustDIBuilderRef` with LLVM-C's `LLVMDIBuilderRef`)
- #136330 (Remove unnecessary hooks)
- #136336 (Overhaul `rustc_middle::util`)
- #136341 (Remove myself from vacation)
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
Replace our `LLVMRustDIBuilderRef` with LLVM-C's `LLVMDIBuilderRef`
Inspired by trying to split #134009 into smaller steps that are easier to review individually.
This makes it possible to start incrementally replacing our debuginfo bindings with the ones in the LLVM-C API, all of which operate on `LLVMDIBuilderRef`.
There should be no change to compiler behaviour.
|
|
Rollup of 9 pull requests
Successful merges:
- #132156 (When encountering unexpected closure return type, point at return type/expression)
- #133429 (Autodiff Upstreaming - rustc_codegen_ssa, rustc_middle)
- #136281 (`rustc_hir_analysis` cleanups)
- #136297 (Fix a typo in profile-guided-optimization.md)
- #136300 (atomic: extend compare_and_swap migration docs)
- #136310 (normalize `*.long-type.txt` paths for compare-mode tests)
- #136312 (Disable `overflow_delimited_expr` in edition 2024)
- #136313 (Filter out RPITITs when suggesting unconstrained assoc type on too many generics)
- #136323 (Fix a typo in conventions.md)
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
If we're already churning all of the debuginfo bindings, we might as well fix
this at the same time.
|
