| Age | Commit message (Collapse) | Author | Lines |
|---|
|
code generation
|
|
Emit warning when there is no space between `-o` and arg
Closes rust-lang/rust#142812
`getopt` doesn't seem to have an API to check this, so we have to check the args manually.
r? compiler
|
|
Signed-off-by: xizheyin <xizheyin@smail.nju.edu.cn>
|
|
This stabilizes a subset of the `-Clink-self-contained` components on x64 linux:
the rust-lld opt-out.
The opt-in is not stabilized, as interactions with other stable flags require
more internal work, but are not needed for stabilizing using rust-lld by default.
Similarly, since we only switch to rust-lld on x64 linux, the opt-out is
only stabilized there. Other targets still require `-Zunstable-options`
to use it.
|
|
This stabilizes a subset of the `-Clinker-features` components on x64 linux:
the lld opt-out.
The opt-in is not stabilized, as interactions with other stable flags require
more internal work, but are not needed for stabilizing using rust-lld by default.
Similarly, since we only switch to rust-lld on x64 linux, the opt-out is
only stabilized there. Other targets still require `-Zunstable-options`
to use it.
|
|
Make __rust_alloc_error_handler_should_panic a function
Fixes rust-lang/rust#143253
`__rust_alloc_error_handler_should_panic` is a static but was being exported as a function.
For most targets this doesn't matter, but Arm64EC Windows uses different decorations for exported variables vs functions, hence it fails to link when `-Z oom=abort` is enabled.
We've had issues in the past with statics like this (see rust-lang/rust#141061) but the tldr; is that Arm64EC needs symbols correctly exported as either a function or data, and data MUST and MUST ONLY be marked `dllimport` when the symbol is being imported from another binary, which is non-trivial to calculate for these compiler-generated statics.
So, instead, the easiest thing to do is to make `__rust_alloc_error_handler_should_panic` a function instead.
Since `__rust_alloc_error_handler_should_panic` isn't involved in any linking shenanigans, I've marked it as `AlwaysInline` with the hopes that the various backends will see that it is just returning a constant and perform the same optimizations as the previous implementation.
r? `@bjorn3`
|
|
|
|
|
|
Add PrintTAFn flag for targeted type analysis printing
## Summary
This PR adds a new `PrintTAFn` flag to the `-Z autodiff` option that allows printing type analysis information for a specific function, rather than all functions.
## Changes
### New Flag
- Added `PrintTAFn=<function_name>` option to `-Z autodiff`
- Usage: `-Z autodiff=Enable,PrintTAFn=my_function_name`
### Implementation Details
- **Rust side**: Added `PrintTAFn(String)` variant to `AutoDiff` enum
- **Parser**: Updated `parse_autodiff` to handle `PrintTAFn=<function_name>` syntax with proper error handling
- **FFI**: Added `set_print_type_fun` function to interface with Enzyme's `FunctionToAnalyze` command line option
- **Documentation**: Updated help text and documentation for the new flag
### Files Modified
- `compiler/rustc_session/src/config.rs`: Added `PrintTAFn(String)` variant
- `compiler/rustc_session/src/options.rs`: Updated parser and help text (now shows `PrintTAFn` in the list)
- `compiler/rustc_codegen_llvm/src/llvm/enzyme_ffi.rs`: Added FFI function and static variable
- `compiler/rustc_codegen_llvm/src/back/lto.rs`: Added handling for new flag
- `src/doc/rustc-dev-guide/src/autodiff/flags.md`: Updated documentation
- `src/doc/unstable-book/src/compiler-flags/autodiff.md`: Updated documentation
## Testing
The flag can be tested with:
```bash
rustc +enzyme -Z autodiff=Enable,PrintTAFn=square test.rs
```
This will print type analysis information only for the function named "square" instead of all functions.
## Error Handling
The parser includes proper error handling:
- Missing argument: `PrintTAFn` without `=<function_name>` will show an error
- Unknown options: Invalid autodiff options will be reported
r? ```@ZuseZ4```
|
|
Signed-off-by: Karan Janthe <karanjanthe@gmail.com>
|
|
Also avoid creating and cloning sysroot unnecessarily.
|
|
Try unremapping compiler sources
See [#t-compiler/help > Span pointing to wrong file location (`rustc-dev` component)](https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/Span.20pointing.20to.20wrong.20file.20location.20.28.60rustc-dev.60.20component.29/with/521087083).
This PR is a follow-up to rust-lang/rust#141751 regarding the compiler side.
Specifically we now take into account the `CFG_VIRTUAL_RUSTC_DEV_SOURCE_BASE_DIR` env from rust-lang/rust#141751 when trying to unremap sources from `$sysroot/lib/rustlib/rustc-src/rust` (the `rustc-dev` component install directory).
Best reviewed commit by commit.
cc ``@samueltardieu``
r? ``@jieyouxu``
|
|
Implement initial support for timing sections (`--json=timings`)
This PR implements initial support for emitting high-level compilation section timings. The idea is to provide a very lightweight way of emitting durations of various compilation sections (frontend, backend, linker, or on a more granular level macro expansion, typeck, borrowck, etc.). The ultimate goal is to stabilize this output (in some form), make Cargo pass `--json=timings` and then display this information in the HTML output of `cargo build --timings`, to make it easier to quickly profile "what takes so long" during the compilation of a Cargo project. I would personally also like if Cargo printed some of this information in the interactive `cargo build` output, but the `build --timings` use-case is the main one.
Now, this information is already available with several other sources, but I don't think that we can just use them as they are, which is why I proposed a new way of outputting this data (`--json=timings`):
- This data is available under `-Zself-profile`, but that is very expensive and forever unstable. It's just a too big of a hammer to tell us the duration it took to run the linker.
- It could also be extracted with `-Ztime-passes`. That is pretty much "for free" in terms of performance, and it can be emitted in a structured form to JSON via `-Ztime-passes-format=json`. I guess that one alternative might be to stabilize this flag in some form, but that form might just be `--json=timings`? I guess what we could do in theory is take the already emitted time passes and reuse them for `--json=timings`. Happy to hear suggestions!
I'm sending this PR mostly for a vibeck, to see if the way I implemented it is passable. There are some things to figure out:
- How do we represent the sections? Originally I wanted to output `{ section, duration }`, but then I realized that it might be more useful to actually emit `start` and `end` events. Both because it enables to see the output incrementally (in case compilation takes a long time and you read the outputs directly, or Cargo decides to show this data in `cargo build` some day in the future), and because it makes it simpler to represent hierarchy (see below). The timestamps currently emit microseconds elapsed from a predetermined point in time (~start of rustc), but otherwise they are fully opaque, and should be only ever used to calculate the duration using `end - start`. We could also precompute the duration for the user in the `end` event, but that would require doing more work in rustc, which I would ideally like to avoid :P
- Do we want to have some form of hierarchy? I think that it would be nice to show some more granular sections rather than just frontend/backend/linker (e.g. macro expansion, typeck and borrowck as a part of the frontend). But for that we would need some way of representing hierarchy. A simple way would be something like `{ parent: "frontend" }`, but I realized that with start/end timestamps we get the hierarchy "for free", only the client will need to reconstruct it from the order of start/end events (e.g. `start A`, `start B` means that `B` is a child of `A`).
- What exactly do we want to stabilize? This is probably a question for later. I think that we should definitely stabilize the format of the emitted JSON objects, and *maybe* some specific section names (but we should also make it clear that they can be missing, e.g. you don't link everytime you invoke `rustc`).
The PR be tested e.g. with `rustc +stage1 src/main.rs --json=timings --error-format=json -Zunstable-options` on a crate without dependencies (it is not easy to use `--json` with stock Cargo, because it also passes this flag to `rustc`, so this will later need Cargo integration to be usable with it).
Zulip discussions: [#t-compiler > Outputting time spent in various compiler sections](https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/Outputting.20time.20spent.20in.20various.20compiler.20sections/with/518850162)
MCP: https://github.com/rust-lang/compiler-team/issues/873
r? ``@nnethercote``
|
|
This allows a more gradual transition path for projects that need to use
use the spec-complaint C ABI both with older and newer rustc versions.
|
|
|
|
|
|
|
|
|
|
|
|
enable retpoline-related target features
|
|
This reverts commit 3b22c21dd8c30f499051fe7a758ca0e5d81eb638, reversing
changes made to 5f292eea6d63abbd26f1e6e00a0b8cf21d828d7d.
|
|
This case can't actually happen yet (other than via a testing flag), because
currently all of a function's spans must belong to the same file and expansion.
But this will be an important edge case when adding expansion region support.
|
|
Fix `-Zremap-path-scope` rmeta handling
This PR fixes the conditional remapping (`-Zremap-path-scope`) of rmeta file paths ~~by using the `debuginfo` scope~~ by conditionally embedding the local path in addition to the remapped path.
Fixes https://github.com/rust-lang/rust/issues/139217
|
|
|
|
|
|
It wants an owned path, so pass an owned path
|
|
Unify the format of rustc cli flags
As mentioned in #140102, I unified the format of rustc CLI flags.
I use the following rules:
1. `<param>`: Indicates a required parameter
2. `[param]`: Indicates an optional parameter
3. `|`: Indicates a mutually exclusive option
4. `*`: a list element with description
Current output:
```bash
Usage: rustc [OPTIONS] INPUT
Options:
-h, --help Display this message
--cfg <SPEC> Configure the compilation environment.
SPEC supports the syntax `<NAME>[="<VALUE>"]`.
--check-cfg <SPEC>
Provide list of expected cfgs for checking
-L [<KIND>=]<PATH> Add a directory to the library search path. The
optional KIND can be one of
<dependency|crate|native|framework|all> (default:
all).
-l [<KIND>[:<MODIFIERS>]=]<NAME>[:<RENAME>]
Link the generated crate(s) to the specified native
library NAME. The optional KIND can be one of
<static|framework|dylib> (default: dylib).
Optional comma separated MODIFIERS
<bundle|verbatim|whole-archive|as-needed>
may be specified each with a prefix of either '+' to
enable or '-' to disable.
--crate-type <bin|lib|rlib|dylib|cdylib|staticlib|proc-macro>
Comma separated list of types of crates
for the compiler to emit
--crate-name <NAME>
Specify the name of the crate being built
--edition <2015|2018|2021|2024|future>
Specify which edition of the compiler to use when
compiling code. The default is 2015 and the latest
stable edition is 2024.
--emit <TYPE>[=<FILE>]
Comma separated list of types of output for the
compiler to emit.
Each TYPE has the default FILE name:
* asm - CRATE_NAME.s
* llvm-bc - CRATE_NAME.bc
* dep-info - CRATE_NAME.d
* link - (platform and crate-type dependent)
* llvm-ir - CRATE_NAME.ll
* metadata - libCRATE_NAME.rmeta
* mir - CRATE_NAME.mir
* obj - CRATE_NAME.o
* thin-link-bitcode - CRATE_NAME.indexing.o
--print <INFO>[=<FILE>]
Compiler information to print on stdout (or to a file)
INFO may be one of
<all-target-specs-json|calling-conventions|cfg|check-cfg|code-models|crate-name|crate-root-lint-levels|deployment-target|file-names|host-tuple|link-args|native-static-libs|relocation-models|split-debuginfo|stack-protector-strategies|supported-crate-types|sysroot|target-cpus|target-features|target-libdir|target-list|target-spec-json|tls-models>.
-g Equivalent to -C debuginfo=2
-O Equivalent to -C opt-level=3
-o <FILENAME> Write output to FILENAME
--out-dir <DIR> Write output to compiler-chosen filename in DIR
--explain <OPT> Provide a detailed explanation of an error message
--test Build a test harness
--target <TARGET>
Target triple for which the code is compiled
-A, --allow <LINT> Set lint allowed
-W, --warn <LINT> Set lint warnings
--force-warn <LINT>
Set lint force-warn
-D, --deny <LINT> Set lint denied
-F, --forbid <LINT> Set lint forbidden
--cap-lints <LEVEL>
Set the most restrictive lint level. More restrictive
lints are capped at this level
-C, --codegen <OPT>[=<VALUE>]
Set a codegen option
-V, --version Print version info and exit
-v, --verbose Use verbose output
Additional help:
-C help Print codegen options
-W help Print 'lint' options and default settings
-Z help Print unstable compiler options
--help -v Print the full set of options rustc accepts
```
|
|
Signed-off-by: xizheyin <xizheyin@smail.nju.edu.cn>
|
|
Autodiff flags
Interestingly, it seems that some other projects have conflicts with exactly the same LLVM optimization passes as autodiff.
At least `LLVMRustOptimize` has exactly the flags that we need to disable problematic opt passes.
This PR enables us to compile code where users differentiate two identical functions in the same module. This has been especially common in test cases, but it's not impossible to encounter in the wild.
It also enables two new flags for testing/debugging. I consider writing an MCP to upgrade PrintPasses to be a standalone -Z flag, since it is *not* the same as `-Z print-llvm-passes`, which IMHO gives less useful output. A discussion can be found here: [#t-compiler/llvm > Print llvm passes. @ 💬](https://rust-lang.zulipchat.com/#narrow/channel/187780-t-compiler.2Fllvm/topic/Print.20llvm.20passes.2E/near/511533038)
Finally, it improves `PrintModBefore` and `PrintModAfter`. They used to work reliable, but now we just schedule enzyme as part of an existing ModulePassManager (MPM). Since Enzyme is last in the MPM scheduling, PrintModBefore became very inaccurate. It used to print the input module, which we gave to the Enzyme and was great to create llvm-ir reproducer. However, lately the MPM would run the whole `default<O3>` pipeline, which heavily modifies the llvm module, before we pass it to Enzyme. That made it impossible to use the flag to create llvm-ir reproducers for Enzyme bugs. We now schedule a PrintModule pass just before Enzyme, solving this problem.
Based on the PrintPass output, it also _seems_ like changing `registerEnzymeAndPassPipeline(PB, true);` to `registerEnzymeAndPassPipeline(PB, false);` has no effect. In theory, the bool should tell Enzyme to schedule some helpful passes in the PassBuilder. However, since it doesn't do anything and I'm not 100% sure anymore on whether we really need it, I'll just disable it for now and postpone investigations.
r? ``@oli-obk``
closes #139471
Tracking:
- https://github.com/rust-lang/rust/issues/124509
|
|
Signed-off-by: xizheyin <xizheyin@smail.nju.edu.cn>
|
|
Signed-off-by: xizheyin <xizheyin@smail.nju.edu.cn>
|
|
Hide unstable print kinds within emit_unknown_print_request_help in stable channel
Fixes #138698
We need to get the channel from `matches`. However, since `matches`(Line 1169) is constructed after `rustc_optgroups` (Line1165, where `RustcOptGroup::value_hint` is generated, i.e. what `rustc --print print` prints), I've left it unchanged here for now.
https://github.com/rust-lang/rust/blob/2da29dbe8fe23df1c7c4ab1d8740ca3c32b15526/compiler/rustc_driver_impl/src/lib.rs#L1161-L1169
There is actually a way to manually parse the `--crate-name` parameter, but I'm afraid that's an unorthodox practice. So I conservatively just modified `emit_unknown_print_request_help` to print different parameters depending on whether they are nightly or not when passing the error parameter.
r? ```@jieyouxu```
|
|
channel
Signed-off-by: xizheyin <xizheyin@smail.nju.edu.cn>
|
|
Also refactors some of the crate name parsing code and adds unit tests
Issue #122349
Co-authored-by: León Orell Valerian Liehr <me@fmease.dev>
|
|
|
|
|
|
|
|
|
|
use lower case to match other error messages
|
|
Autodiff batching
Enzyme supports batching, which is especially known from the ML side when training neural networks.
There we would normally have a training loop, where in each iteration we would pass in some data (e.g. an image), and a target vector. Based on how close we are with our prediction we compute our loss, and then use backpropagation to compute the gradients and update our weights.
That's quite inefficient, so what you normally do is passing in a batch of 8/16/.. images and targets, and compute the gradients for those all at once, allowing better optimizations.
Enzyme supports batching in two ways, the first one (which I implemented here) just accepts a Batch size,
and then each Dual/Duplicated argument has not one, but N shadow arguments. So instead of
```rs
for i in 0..100 {
df(x[i], y[i], 1234);
}
```
You can now do
```rs
for i in 0..100.step_by(4) {
df(x[i+0],x[i+1],x[i+2],x[i+3], y[i+0], y[i+1], y[i+2], y[i+3], 1234);
}
```
which will give the same results, but allows better compiler optimizations. See the testcase for details.
There is a second variant, where we can mark certain arguments and instead of having to pass in N shadow arguments, Enzyme assumes that the argument is N times longer. I.e. instead of accepting 4 slices with 12 floats each, we would accept one slice with 48 floats. I'll implement this over the next days.
I will also add more tests for both modes.
For any one preferring some more interactive explanation, here's a video of Tim's llvm dev talk, where he presents his work. https://www.youtube.com/watch?v=edvaLAL5RqU
I'll also add some other docs to the dev guide and user docs in another PR.
r? ghost
Tracking:
- https://github.com/rust-lang/rust/issues/124509
- https://github.com/rust-lang/rust/issues/135283
|
|
|
|
|
|
|
|
|
|
|
|
As an unstable print request.
|
|
Signed-off-by: xizheyin <xizheyin@smail.nju.edu.cn>
|
|
`{AllTargetSpecsJson,TargetSpecJson}`
To correspond to their actual print request names, `target-spec-json`
and `all-target-specs-json`, and for consistency with other print name
<-> print kind mappings.
|
|
helpers
To avoid duplicating stability check logic and make the print request
collection logic more straightforward.
|
|
|
