about summary refs log tree commit diff
path: root/compiler/rustc_codegen_llvm/src
AgeCommit message (Collapse)AuthorLines
2025-03-12Rollup merge of #138331 - nnethercote:use-RUSTC_LINT_FLAGS-more, ↵Matthias Krüger-1/+0
r=onur-ozkan,jieyouxu Use `RUSTC_LINT_FLAGS` more An alternative to the failed #138084. Fixes #138106. r? ````@jieyouxu````
2025-03-11Auto merge of #137586 - nnethercote:SetImpliedBits, r=bjorn3bors-59/+60
Speed up target feature computation The LLVM backend calls `LLVMRustHasFeature` twice for every feature. In short-running rustc invocations, this accounts for a surprising amount of work. r? `@bjorn3`
2025-03-11Remove `#![warn(unreachable_pub)]` from all `compiler/` crates.Nicholas Nethercote-1/+0
It's no longer necessary now that `-Wunreachable_pub` is being passed.
2025-03-10Revert "Use workspace lints for crates in `compiler/` #138084"许杰友 Jieyou Xu (Joe)-0/+1
Revert <https://github.com/rust-lang/rust/pull/138084> to buy time to consider options that avoids breaking downstream usages of cargo on distributed `rustc-src` artifacts, where such cargo invocations fail due to inability to inherit `lints` from workspace root manifest's `workspace.lints` (this is only valid for the source rust-lang/rust workspace, but not really the distributed `rustc-src` artifacts). This breakage was reported in <https://github.com/rust-lang/rust/issues/138304>. This reverts commit 48caf81484b50dca5a5cebb614899a3df81ca898, reversing changes made to c6662879b27f5161e95f39395e3c9513a7b97028.
2025-03-09Rollup merge of #122790 - Zoxc:dllimp-rev, r=ChrisDentonMatthias Krüger-6/+5
Apply dllimport in ThinLTO This partially reverts https://github.com/rust-lang/rust/pull/103353 by properly applying `dllimport` if `-Z dylib-lto` is passed. That PR should probably fully be reverted as it looks quite sketchy. We don't know locally if the entire crate graph would be statically linked. This should hopefully be sufficient to make ThinLTO work for rustc on Windows. r? ``@wesleywiser`` --- Edit: This PR is changed to just generally revert https://github.com/rust-lang/rust/pull/103353.
2025-03-09Rollup merge of #138084 - nnethercote:workspace-lints, r=jieyouxuMatthias Krüger-1/+0
Use workspace lints for crates in `compiler/` This is nicer and hopefully less error prone than specifying lints via bootstrap. r? ``@jieyouxu``
2025-03-08Remove `#![warn(unreachable_pub)]` from all `compiler/` crates.Nicholas Nethercote-1/+0
(Except for `rustc_codegen_cranelift`.) It's no longer necessary now that `unreachable_pub` is in the workspace lints.
2025-03-07Rollup merge of #137549 - oli-obk:llvm-ffi, r=davidtwcoMatthias Krüger-348/+316
Clean up various LLVM FFI things in codegen_llvm cc ```@ZuseZ4``` I touched some autodiff parts The major change of this PR is [bfd88ce](https://github.com/rust-lang/rust/pull/137549/commits/bfd88cead0dd79717f123ad7e9a26ecad88653cb) which makes `CodegenCx` generic just like `GenericBuilder` The other commits mostly took advantage of the new feature of making extern functions safe, but also just used some wrappers that were already there and shrunk unsafe blocks. best reviewed commit-by-commit
2025-03-05Remove out of date comment.Nicholas Nethercote-1/+0
No smallvecs here.
2025-03-05Use `collect` to initialize `features`.Nicholas Nethercote-28/+25
2025-03-05Change signature of `target_features_cfg`.Nicholas Nethercote-26/+32
Currently it is called twice, once with `allow_unstable` set to true and once with it set to false. This results in some duplicated work. Most notably, for the LLVM backend, `LLVMRustHasFeature` is called twice for every feature, and it's moderately slow. For very short running compilations on platforms with many features (e.g. a `check` build of hello-world on x86) this is a significant fraction of runtime. This commit changes `target_features_cfg` so it is only called once, and it now returns a pair of feature sets. This halves the number of `LLVMRustHasFeature` calls.
2025-03-05Simplify `implied_target_features`.Nicholas Nethercote-3/+3
Currently its argument is an iterator, but in practice it's always a singleton.
2025-03-05Avoid double interning of feature names.Nicholas Nethercote-8/+7
Also improve some comments.
2025-03-04Auto merge of #137959 - matthiaskrgr:rollup-62vjvwr, r=matthiaskrgrbors-5/+0
Rollup of 12 pull requests Successful merges: - #135767 (Future incompatibility warning `unsupported_fn_ptr_calling_conventions`: Also warn in dependencies) - #137852 (Remove layouting dead code for non-array SIMD types.) - #137863 (Fix pretty printing of unsafe binders) - #137882 (do not build additional stage on compiler paths) - #137894 (Revert "store ScalarPair via memset when one side is undef and the other side can be memset") - #137902 (Make `ast::TokenKind` more like `lexer::TokenKind`) - #137921 (Subtree update of `rust-analyzer`) - #137922 (A few cleanups after the removal of `cfg(not(parallel))`) - #137939 (fix order on shl impl) - #137946 (Fix docker run-local docs) - #137955 (Always allow rustdoc-json tests to contain long lines) - #137958 (triagebot.toml: Don't label `test/rustdoc-json` as A-rustdoc-search) r? `@ghost` `@rustbot` modify labels: rollup
2025-03-03Rollup merge of #137894 - compiler-errors:no-scalar-pair-opt, r=oli-obkMatthias Krüger-5/+0
Revert "store ScalarPair via memset when one side is undef and the other side can be memset" cc #137892 reverts #135335 r? oli-obk
2025-03-03Apply dllimport in ThinLTOJohn Kåre Alsaker-6/+5
2025-03-03Rollup merge of #137741 - cuviper:const_str-raw_entry, r=Mark-SimulacrumMatthias Krüger-23/+18
Stop using `hash_raw_entry` in `CodegenCx::const_str` That unstable feature (#56167) completed fcp-close, so the compiler needs to be migrated away to allow its removal. In this case, `cg_llvm` and `cg_gcc` were using raw entries to optimize their `const_str_cache` lookup and insertion. We can change that to separate `get` and (on miss) `insert` calls, so we still have the fast path avoiding string allocation when the cache hits.
2025-03-02Revert "Auto merge of #135335 - oli-obk:push-zxwssomxxtnq, r=saethlin"Michael Goulet-5/+0
This reverts commit a7a6c64a657f68113301c2ffe0745b49a16442d1, reversing changes made to ebbe63891f1fae21734cb97f2f863b08b1d44bf8.
2025-03-01Rollup merge of #137804 - RalfJung:backend-repr-simd-vector, r=workingjubileeMatthias Krüger-24/+27
rename BackendRepr::Vector → SimdVector For many Rustaceans, "vector" does not imply "SIMD", so let's be more clear in this type that is used pervasively in the compiler. r? `@workingjubilee`
2025-03-01Auto merge of #133250 - DianQK:embed-bitcode-pgo, r=nikicbors-63/+104
The embedded bitcode should always be prepared for LTO/ThinLTO Fixes #115344. Fixes #117220. There are currently two methods for generating bitcode that used for LTO. One method involves using `-C linker-plugin-lto` to emit object files as bitcode, which is the typical setting used by cargo. The other method is through `-C embed-bitcode=yes`. When using with `-C embed-bitcode=yes -C lto=no`, we run a complete non-LTO LLVM pipeline to obtain bitcode, then the bitcode is used for LTO. We run the Call Graph Profile Pass twice on the same module. This PR is doing something similar to LLVM's `buildFatLTODefaultPipeline`, obtaining the bitcode for embedding after running `buildThinLTOPreLinkDefaultPipeline`. r? nikic
2025-03-01Auto merge of #137796 - jieyouxu:rollup-qt9yr1g, r=jieyouxubors-3/+22
Rollup of 10 pull requests Successful merges: - #134943 (Add FileCheck annotations to mir-opt/issues) - #137017 (Don't error when adding a staticlib with bitcode files compiled by newer LLVM) - #137197 (Update some comparison codegen tests now that they pass in LLVM20) - #137540 (Fix (more) test directives that were accidentally ignored) - #137551 (import `simd_` intrinsics) - #137599 (tests: use minicore more) - #137673 (Fix Windows `Command` search path bug) - #137676 (linker: Fix escaping style for response files on Windows) - #137693 (Re-enable `--generate-link-to-defintion` for tools internal rustdoc) - #137770 (Fix sized constraint for unsafe binder) r? `@ghost` `@rustbot` modify labels: rollup
2025-02-28rename BackendRepr::Vector → SimdVectorRalf Jung-24/+27
2025-02-28Rollup merge of #137017 - bjorn3:ignore_invalid_bitcode, r=oli-obk许杰友 Jieyou Xu (Joe)-3/+22
Don't error when adding a staticlib with bitcode files compiled by newer LLVM cc https://github.com/rust-lang/rust/issues/128955#issuecomment-2657811196
2025-02-28Rollup merge of #137713 - vayunbiyani:fix-enzyme-build-errors, r=oli-obk许杰友 Jieyou Xu (Joe)-2/+2
Fix enzyme build errors After [this PR](https://github.com/rust-lang/rust/pull/136428) was merged, I switched to master and attempted building `./x.py build --stage 1 library` with the config mentioned in the enzyme rustbook but it resulted in some errors tho the config.example.toml build succeeded The errors were re: ### 1. Use of ref in match patterns The errors were related to match ergonomics in Rust 2024, where ref is no longer needed when matching on references. Examples: ``` error: binding modifiers may only be written when the default binding mode is `move` --> compiler/rustc_builtin_macros/src/autodiff.rs:136:31 | 136 | Annotatable::Item(ref iitem) => { | ^^^ binding modifier not allowed under `ref` default binding mode | = note: for more information, see <https://doc.rust-lang.org/nightly/edition-guide/rust-2024/match-ergonomics.html> note: matching on a reference type with a non-reference pattern changes the default binding mode --> compiler/rustc_builtin_macros/src/autodiff.rs:136:13 | 136 | Annotatable::Item(ref iitem) => { | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this matches on type `&_` help: remove the unnecessary binding modifier | 136 - Annotatable::Item(ref iitem) => { 136 + Annotatable::Item(iitem) => { | error: binding modifiers may only be written when the default binding mode is `move` --> compiler/rustc_builtin_macros/src/autodiff.rs:146:36 | 146 | Annotatable::AssocItem(ref assoc_item, _) => { | ^^^ binding modifier not allowed under `ref` default binding mode | = note: for more information, see <https://doc.rust-lang.org/nightly/edition-guide/rust-2024/match-ergonomics.html> note: matching on a reference type with a non-reference pattern changes the default binding mode --> compiler/rustc_builtin_macros/src/autodiff.rs:146:13 | 146 | Annotatable::AssocItem(ref assoc_item, _) => { | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this matches on type `&_` help: remove the unnecessary binding modifier | 146 - Annotatable::AssocItem(ref assoc_item, _) => { 146 + Annotatable::AssocItem(assoc_item, _) => { | error: binding modifiers may only be written when the default binding mode is `move` --> compiler/rustc_builtin_macros/src/autodiff.rs:174:31 | 174 | ... Annotatable::Item(ref iitem) => (iitem.vis.clone(), iitem.ide... | ^^^ binding modifier not allowed under `ref` default binding mode | = note: for more information, see <https://doc.rust-lang.org/nightly/edition-guide/rust-2024/match-ergonomics.html> note: matching on a reference type with a non-reference pattern changes the default binding mode --> compiler/rustc_builtin_macros/src/autodiff.rs:174:13 | 174 | ... Annotatable::Item(ref iitem) => (iitem.vis.clone(), iitem.ident.c... | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this matches on type `&_` help: remove the unnecessary binding modifier | 174 - Annotatable::Item(ref iitem) => (iitem.vis.clone(), iitem.ident.clone()), 174 + Annotatable::Item(iitem) => (iitem.vis.clone(), iitem.ident.clone()), | error: binding modifiers may only be written when the default binding mode is `move` --> compiler/rustc_builtin_macros/src/autodiff.rs:175:36 | 175 | Annotatable::AssocItem(ref assoc_item, _) => { | ^^^ binding modifier not allowed under `ref` default binding mode | = note: for more information, see <https://doc.rust-lang.org/nightly/edition-guide/rust-2024/match-ergonomics.html> note: matching on a reference type with a non-reference pattern changes the default binding mode --> compiler/rustc_builtin_macros/src/autodiff.rs:175:13 | 175 | Annotatable::AssocItem(ref assoc_item, _) => { | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this matches on type `&_` help: remove the unnecessary binding modifier | 175 - Annotatable::AssocItem(ref assoc_item, _) => { 175 + Annotatable::AssocItem(assoc_item, _) => { | error: could not compile `rustc_builtin_macros` (lib) due to 4 previous errors warning: build failed, waiting for other jobs to finish... Build completed unsuccessfully in 0:19:39 ``` ### 2. the use of external C blocks without unsafe in compiler/rustc_codegen_llvm/src/llvm/enzyme_ffi.rs (I don't have the error message handy) The first commit fixes the errors above --- ## Additional Improvement: `@ZuseZ4` suggested we consolidate the variants under `#[cfg(llvm_enzyme)]` and `#[cfg(not(llvm_enzyme))]` by conditionally checking for `cfg!(llvm_enzyme)` instead. This way, the autodiff code is compiled but not executed avoiding such regressions r? `@ZuseZ4` cc: `@oli-obk`
2025-02-27Stop using `hash_raw_entry` in `CodegenCx::const_str`Josh Stone-23/+18
That unstable feature completed fcp-close, so the compiler needs to be migrated away to allow its removal. In this case, `cg_llvm` and `cg_gcc` were using raw entries to optimize their `const_str_cache` lookup and insertion. We can change that to separate `get` and (on miss) `insert` calls, so we still have the fast path avoiding string allocation when the cache hits.
2025-02-26Restore usage of io::Errorbjorn3-4/+7
2025-02-26Rollup merge of #137603 - davidtwco:extern-types-no-deref, r=lcnrLeón Orell Valerian Liehr-30/+31
codegen_llvm: avoid `Deref` impls w/ extern type `rustc_codegen_llvm` relied on `Deref` impls where `Deref::Target` was or contained an extern type - in my experimental implementation of rust-lang/rfcs#3729, this isn't possible as the `Target` associated type's `?Sized` bound cannot be relaxed backwards compatibly (unless we come up with some way of doing this). In later pull requests with the rust-lang/rfcs#3729 implementation, breakage like this could only occur for nightly users relying on the `extern_types` feature. Upstreaming this to avoid needing to keep carrying this patch locally, and I think it'll necessarily need to change eventually.
2025-02-25Rollup merge of #137595 - folkertdev:remove-simd-pow-powi, r=RalfJungLeón Orell Valerian Liehr-4/+0
remove `simd_fpow` and `simd_fpowi` Discussed in https://github.com/rust-lang/rust/issues/137555 These functions are not exposed from `std::intrinsics::simd`, and not used anywhere outside of the compiler. They also don't lower to particularly good code at least on the major ISAs (I checked x86_64, aarch64, s390x, powerpc), where the vector is just spilled to the stack and scalar functions are used for the actual logic. r? `@RalfJung`
2025-02-25Fix enzyme build errorsVayun Biyani-2/+2
2025-02-25remove `simd_fpow` and `simd_fpowi`Folkert de Vries-4/+0
2025-02-24Rollup merge of #137556 - RalfJung:simd_shuffle_const_generic, r=oli-obkMichael Goulet-1/+1
rename simd_shuffle_generic → simd_shuffle_const_generic I've been confused by this name one time too often. ;) r? `@oli-obk`
2025-02-24Rollup merge of #137417 - taiki-e:riscv-atomic, r=AmanieuMichael Goulet-0/+2
rustc_target: Add more RISC-V atomic-related features This is a continuation of https://github.com/rust-lang/rust/pull/130877 and adds a few target features, including `zacas`, which was experimental in LLVM 19 and marked non-experimental in LLVM 20. This adds the following target features to unstable riscv_target_feature: - `za64rs` (Za64rs Extension 1.0): Reservation Set Size of at Most 64 Bytes ([definition in LLVM](https://github.com/llvm/llvm-project/blob/llvmorg-20.1.0-rc2/llvm/lib/Target/RISCV/RISCVFeatures.td#L227-L228), [available since LLVM 18](https://github.com/llvm/llvm-project/commit/8649328060b4e748502d1d859f9c9c1bd3c2bccc)) - `za128rs` (Za128rs Extension 1.0): Reservation Set Size of at Most 128 Bytes ([definition in LLVM](https://github.com/llvm/llvm-project/blob/llvmorg-20.1.0-rc2/llvm/lib/Target/RISCV/RISCVFeatures.td#L230-L231), [available since LLVM 18](https://github.com/llvm/llvm-project/commit/8649328060b4e748502d1d859f9c9c1bd3c2bccc)) - IIUC, `za*rs` can be referenced when implementing helpers to reduce contention in synchronization primitives, like [`crossbeam_utils::CachePadded`](https://docs.rs/crossbeam-utils/latest/crossbeam_utils/struct.CachePadded.html). (relevant discussion: https://github.com/riscv/riscv-profiles/issues/79) - `zacas` (Zacas Extension 1.0): Atomic Compare-And-Swap Instructions (`amocas.{w,d,q}{,.aq,.rl,.aqrl}` and `amocas.{b,h}{,.aq,.rl,.aqrl}` when `zabha` is also enabled) ([definition in LLVM](https://github.com/llvm/llvm-project/blob/llvmorg-20.1.0-rc2/llvm/lib/Target/RISCV/RISCVFeatures.td#L240-L243), [available as non-experimental since LLVM 20](https://github.com/llvm/llvm-project/commit/614aeda93b2225c6eb42b00ba189ba7ca2585c60)) - This implies `zaamo`. - This is used to optimize CAS in existing atomics and/or implement 64-bit/128-bit atomics on riscv32/riscv64 (e.g., https://github.com/taiki-e/portable-atomic/pull/173). - Note that [LLVM does not automatically use this instruction for 64-bit/128-bit atomics on riscv32/riscv64 even if this feature is enabled, because doing it changes the ABI](https://github.com/llvm/llvm-project/blob/876174ffd7533dc220f94721173bb767b659fa7f/llvm/docs/RISCVUsage.rst#riscv-zacas-note). (If the ability to do that is provided by LLVM in the future, it should probably be controlled by another ABI feature similar to `forced-atomics`.) - `zama16b` (Zama16b Extension 1.0): Atomic 16-byte misaligned loads, stores and AMOs ([definition in LLVM](https://github.com/llvm/llvm-project/blob/llvmorg-20.1.0-rc2/llvm/lib/Target/RISCV/RISCVFeatures.td#L255-L256), [available since LLVM 19](https://github.com/llvm/llvm-project/commit/b090569685699abe4a8031ad442a0f81e373146b)) - IIUC, unlike AArch64 FEAT_LSE2 which also makes 16-byte aligned ldp ({i,u}128 load) atomic, this extension only affects instructions that already considered atomic if they were naturally aligned. i.e., fld (f64 load) on riscv32 would not be atomic with or without this extension ([relevant QEMU code](https://github.com/qemu/qemu/blob/b69801dd6b1eb4d107f7c2f643adf0a4e3ec9124/target/riscv/insn_trans/trans_rvd.c.inc#L50-L62)). - `zawrs` (Zawrs Extension 1.0): Wait on Reservation Set (`wrs.nto` and `wrs.sto`) ([definition in LLVM](https://github.com/llvm/llvm-project/blob/llvmorg-20.1.0-rc2/llvm/lib/Target/RISCV/RISCVFeatures.td#L258), [available as non-experimental since LLVM 17](https://github.com/llvm/llvm-project/commit/d41a73aa94cb8945dcd0f2906992c2fcea6ed001)) - This is used to optimize synchronization primitives (e.g., Linux uses this for spinlocks (https://github.com/torvalds/linux/commit/b8ddb0df30f9f6e70422f1e705b7416da115bd24)). Btw, the question of whether `zaamo` is implied by `zabha` or not, which was discussed in https://github.com/rust-lang/rust/pull/130877, has been resolved in LLVM 20, since LLVM now treats `zaamo` as implied by `zabha`/`zacas` (https://github.com/llvm/llvm-project/pull/115694), just like GCC and rustc. r? `@Amanieu` `@rustbot` label +O-riscv +A-target-feature
2025-02-24rename simd_shuffle_generic → simd_shuffle_const_genericRalf Jung-1/+1
2025-02-24Mark more LLVM FFI as safeOli Scherer-58/+68
2025-02-24Use a safe wrapper around an LLVM FFI functionOli Scherer-8/+13
2025-02-24Remove inherent function that has a trait method duplicate of a commonly ↵Oli Scherer-16/+10
imported trait
2025-02-24Deduplicate more functions between `SimpleCx` and `CodegenCx`Oli Scherer-54/+4
2025-02-24Remove an unused lifetime paramOli Scherer-1/+1
2025-02-24Make allocator shim creation mostly use safe codeOli Scherer-96/+94
2025-02-24Generalize BaseTypeCodegenMethodsOli Scherer-22/+28
2025-02-24Generalize `BackendTypes` over `GenericCx`Oli Scherer-12/+15
2025-02-24Avoid some duplication between SimpleCx and CodegenCxOli Scherer-88/+94
2025-02-24Remove an unnecessary lifetimeOli Scherer-2/+2
2025-02-24Use safe FFI for various functions in codegen_llvmOli Scherer-7/+3
2025-02-24codegen_llvm: avoid `Deref` impls w/ extern typeDavid Wood-30/+31
`rustc_codegen_llvm` relied on `Deref` impls where `Deref::Target` was or contained an extern type - in my experimental implementation of rust-lang/rfcs#3729, this isn't possible as the `Target` associated type's `?Sized` bound cannot be relaxed backwards compatibly (unless we come up with some way of doing this). In later pull requests with the rust-lang/rfcs#3729 implementation, breakage like this could only occur for nightly users relying on the `extern_types` feature. Upstreaming this to avoid needing to keep carrying this patch locally, and I think it'll necessarily need to change eventually.
2025-02-24Auto merge of #137271 - nikic:gep-nuw-2, r=scottmcmbors-12/+39
Emit getelementptr inbounds nuw for pointer::add() Lower pointer::add (via intrinsic::offset with unsigned offset) to getelementptr inbounds nuw on LLVM versions that support it. This lets LLVM make use of the pre-condition that the offset addition does not wrap in an unsigned sense. Together with inbounds, this also implies that the offset is non-negative. Fixes https://github.com/rust-lang/rust/issues/137217.
2025-02-23Rollup merge of #136543 - RalfJung:round-ties-even, r=tgross35Trevor Gross-14/+8
intrinsics: unify rint, roundeven, nearbyint in a single round_ties_even intrinsic LLVM has three intrinsics here that all do the same thing (when used in the default FP environment). There's no reason Rust needs to copy that historically-grown mess -- let's just have one intrinsic and leave it up to the LLVM backend to decide how to lower that. Suggested by `@hanna-kruppe` in https://github.com/rust-lang/rust/issues/136459; Cc `@tgross35` try-job: test-various
2025-02-23Save pre-link bitcode to `ModuleCodegen`DianQK-16/+11
2025-02-23Add `new_regular` and `new_allocator` to `ModuleCodegen`DianQK-16/+5
2025-02-23The embedded bitcode should always be prepared for LTO/ThinLTODianQK-45/+102