about summary refs log tree commit diff
path: root/compiler/rustc_codegen_llvm
AgeCommit message (Collapse)AuthorLines
2021-04-06Auto merge of #83592 - nagisa:nagisa/dso_local, r=davidtwcobors-8/+54
Set dso_local for hidden, private and local items This should probably have no real effect in most cases, as e.g. `hidden` visibility already implies `dso_local` (or at least LLVM IR does not preserve the `dso_local` setting if the item is already `hidden`), but it should fix `-Crelocation-model=static` and improve codegen in executables. Note that this PR does not exhaustively port the logic in [clang], only the portion that is necessary to fix a regression from LLVM 12 that relates to `-Crelocation_model=static`. Fixes #83335 [clang]: https://github.com/llvm/llvm-project/blob/3001d080c813da20b329303bf8f45451480e5905/clang/lib/CodeGen/CodeGenModule.cpp#L945-L1039
2021-04-05Allow specifying alignment for functionsWesley Norris-0/+3
2021-04-05Rollup merge of #82483 - tmiasko:option-from-str, r=matthewjasperDylan DPC-1/+1
Use FromStr trait for number option parsing Replace `parse_uint` with generic `parse_number` based on `FromStr`. Use it for parsing inlining threshold to avoid casting later.
2021-04-05Rollup merge of #83841 - Amanieu:asm_clobber_feature, r=nagisaDylan DPC-1/+26
Allow clobbering unsupported registers in asm! Previously registers could only be marked as clobbered if the target feature for that register was enabled. This restriction is now removed. cc #81092 r? ``@nagisa``
2021-04-05Rollup merge of #80525 - devsnek:wasm64, r=nagisaDylan DPC-7/+4
wasm64 support There is still some upstream llvm work needed before this can land.
2021-04-04wasm64Gus Caplan-7/+4
2021-04-04Allow clobbering unsupported registers in asm!Amanieu d'Antras-1/+26
Previously registers could only be marked as clobbered if the target feature for that register was enabled. This restriction is now removed.
2021-04-03Auto merge of #83774 - richkadel:zero-based-counters, r=tmandrybors-6/+2
Translate counters from Rust 1-based to LLVM 0-based counter ids A colleague contacted me and asked why Rust's counters start at 1, when Clangs appear to start at 0. There is a reason why Rust's internal counters start at 1 (see the docs), and I tried to keep them consistent when codegenned to LLVM's coverage mapping format. LLVM should be tolerant of missing counters, but as my colleague pointed out, `llvm-cov` will silently fail to generate a coverage report for a function based on LLVM's assumption that the counters are 0-based. See: https://github.com/llvm/llvm-project/blob/main/llvm/lib/ProfileData/Coverage/CoverageMapping.cpp#L170 Apparently, if, for example, a function has no branches, it would have exactly 1 counter. `CounterValues.size()` would be 1, and (with the 1-based index), the counter ID would be 1. This would fail the check and abort reporting coverage for the function. It turns out that by correcting for this during coverage map generation, by subtracting 1 from the Rust Counter ID (both when generating the counter increment intrinsic call, and when adding counters to the map), some uncovered functions (including in tests) now appear covered! This corrects the coverage for a few tests! r? `@tmandry` FYI: `@wesleywiser`
2021-04-02Translate counters from Rust 1-based to LLVM 0-based counter idsRich Kadel-6/+2
A colleague contacted me and asked why Rust's counters start at 1, when Clangs appear to start at 0. There is a reason why Rust's internal counters start at 1 (see the docs), and I tried to keep them consistent when codegenned to LLVM's coverage mapping format. LLVM should be tolerant of missing counters, but as my colleague pointed out, `llvm-cov` will silently fail to generate a coverage report for a function based on LLVM's assumption that the counters are 0-based. See: https://github.com/llvm/llvm-project/blob/main/llvm/lib/ProfileData/Coverage/CoverageMapping.cpp#L170 Apparently, if, for example, a function has no branches, it would have exactly 1 counter. `CounterValues.size()` would be 1, and (with the 1-based index), the counter ID would be 1. This would fail the check and abort reporting coverage for the function. It turns out that by correcting for this during coverage map generation, by subtracting 1 from the Rust Counter ID (both when generating the counter increment intrinsic call, and when adding counters to the map), some uncovered functions (including in tests) now appear covered! This corrects the coverage for a few tests!
2021-04-03Move SanitizerSet to rustc_targetSimonas Kazlauskas-5/+6
2021-04-03Manually set dso_local when its valid to do soSimonas Kazlauskas-8/+54
This should have no real effect in most cases, as e.g. `hidden` visibility already implies `dso_local` (or at least LLVM IR does not preserve the `dso_local` setting if the item is already `hidden`), but it should fix `-Crelocation-model=static` and improve codegen in executables. Note that this PR does not exhaustively port the logic in [clang]. Only the obviously correct portion and what is necessary to fix a regression from LLVM 12 that relates to `-Crelocation_model=static`. Fixes #83335 [clang]: https://github.com/llvm/llvm-project/blob/3001d080c813da20b329303bf8f45451480e5905/clang/lib/CodeGen/CodeGenModule.cpp#L945-L1039
2021-03-31Auto merge of #83666 - Amanieu:instrprof-order, r=tmandrybors-0/+9
Run LLVM coverage instrumentation passes before optimization passes This matches the behavior of Clang and allows us to remove several hacks which were needed to ensure functions weren't optimized away before reaching the instrumentation pass. Fixes #83429 cc `@richkadel` r? `@tmandry`
2021-03-30Apply review feedbackAmanieu d'Antras-1/+5
2021-03-30Run LLVM coverage instrumentation passes before optimization passesAmanieu d'Antras-0/+5
This matches the behavior of Clang and allows us to remove several hacks which were needed to ensure functions weren't optimized away before reaching the instrumentation pass.
2021-03-27Address review commentsJoshua Nelson-0/+68
- Add back `HirIdVec`, with a comment that it will soon be used. - Add back `*_region` functions, with a comment they may soon be used. - Remove `-Z borrowck_stats` completely. It didn't do anything. - Remove `make_nop` completely. - Add back `current_loc`, which is used by an out-of-tree tool. - Fix style nits - Remove `AtomicCell` with `cfg(parallel_compiler)` for consistency.
2021-03-27Remove (lots of) dead codeJoshua Nelson-87/+1
Found with https://github.com/est31/warnalyzer. Dubious changes: - Is anyone else using rustc_apfloat? I feel weird completely deleting x87 support. - Maybe some of the dead code in rustc_data_structures, in case someone wants to use it in the future? - Don't change rustc_serialize I plan to scrap most of the json module in the near future (see https://github.com/rust-lang/compiler-team/issues/418) and fixing the tests needed more work than I expected. TODO: check if any of the comments on the deleted code should be kept.
2021-03-26Use iter::zip in compiler/Josh Stone-18/+10
2021-03-26Auto merge of #82980 - tmiasko:import-cold-multiplier, r=michaelwoeristerbors-0/+3
Import small cold functions The Rust code is often written under an assumption that for generic methods inline attribute is mostly unnecessary, since for optimized builds using ThinLTO, a method will be code generated in at least one CGU and available for import. For example, deref implementations for Box, Vec, MutexGuard, and MutexGuard are not currently marked as inline, neither is identity implementation of From trait. In PGO builds, when functions are determined to be cold, the default multiplier of zero will stop the import, no matter how trivial the implementation. Increase slightly the default multiplier from 0 to 0.1. r? `@ghost`
2021-03-25Auto merge of #83387 - cuviper:min-llvm-10, r=nagisabors-9/+0
Update the minimum external LLVM to 10 r? `@nikic`
2021-03-25Auto merge of #83307 - richkadel:cov-unused-functions-1.1, r=tmandrybors-151/+255
coverage bug fixes and optimization support Adjusted LLVM codegen for code compiled with `-Zinstrument-coverage` to address multiple, somewhat related issues. Fixed a significant flaw in prior coverage solution: Every counter generated a new counter variable, but there should have only been one counter variable per function. This appears to have bloated .profraw files significantly. (For a small program, it increased the size by about 40%. I have not tested large programs, but there is anecdotal evidence that profraw files were way too large. This is a good fix, regardless, but hopefully it also addresses related issues. Fixes: #82144 Invalid LLVM coverage data produced when compiled with -C opt-level=1 Existing tests now work up to at least `opt-level=3`. This required a detailed analysis of the LLVM IR, comparisons with Clang C++ LLVM IR when compiled with coverage, and a lot of trial and error with codegen adjustments. The biggest hurdle was figuring out how to continue to support coverage results for unused functions and generics. Rust's coverage results have three advantages over Clang's coverage results: 1. Rust's coverage map does not include any overlapping code regions, making coverage counting unambiguous. 2. Rust generates coverage results (showing zero counts) for all unused functions, including generics. (Clang does not generate coverage for uninstantiated template functions.) 3. Rust's unused functions produce minimal stubbed functions in LLVM IR, sufficient for including in the coverage results; while Clang must generate the complete LLVM IR for each unused function, even though it will never be called. This PR removes the previous hack of attempting to inject coverage into some other existing function instance, and generates dedicated instances for each unused function. This change, and a few other adjustments (similar to what is required for `-C link-dead-code`, but with lower impact), makes it possible to support LLVM optimizations. Fixes: #79651 Coverage report: "Unexecuted instantiation:..." for a generic function from multiple crates Fixed by removing the aforementioned hack. Some "Unexecuted instantiation" notices are unavoidable, as explained in the `used_crate.rs` test, but `-Zinstrument-coverage` has new options to back off support for either unused generics, or all unused functions, which avoids the notice, at the cost of less coverage of unused functions. Fixes: #82875 Invalid LLVM coverage data produced with crate brotli_decompressor Fixed by disabling the LLVM function attribute that forces inlining, if `-Z instrument-coverage` is enabled. This attribute is applied to Rust functions with `#[inline(always)], and in some cases, the forced inlining breaks coverage instrumentation and reports. FYI: `@wesleywiser` r? `@tmandry`
2021-03-23Changes from review commentsRich Kadel-3/+23
2021-03-23Change def_id filter to use requires_monomorphization()Rich Kadel-1/+1
Per @wesleywiser's comment: https://github.com/rust-lang/rust/pull/83307#discussion_r599223342
2021-03-22Auto merge of #79278 - mark-i-m:stabilize-or-pattern, r=nikomatsakisbors-1/+1
Stabilize or_patterns (RFC 2535, 2530, 2175) closes #54883 This PR stabilizes the or_patterns feature in Rust 1.53. This is blocked on the following (in order): - [x] The crater run in https://github.com/rust-lang/rust/pull/78935#issuecomment-731564021 - [x] The resolution of the unresolved questions and a second crater run (https://github.com/rust-lang/rust/pull/78935#issuecomment-735412705) - It looks like we will need to pursue some sort of edition-based transition for `:pat`. - [x] Nomination and discussion by T-lang - [x] Implement new behavior for `:pat` based on consensus (https://github.com/rust-lang/rust/pull/80100). - [ ] An FCP on stabilization EDIT: Stabilization report is in https://github.com/rust-lang/rust/pull/79278#issuecomment-772815177
2021-03-22Update the minimum external LLVM to 10Josh Stone-9/+0
2021-03-22Rollup merge of #83329 - camelid:debuginfo-doc-cleanup, r=davidtwcoDylan DPC-181/+182
Cleanup LLVM debuginfo module docs - Move debuginfo docs from `doc.rs` module to `doc.md` file - Cleanup LLVM debuginfo module docs
2021-03-21Enable mutable noalias by default for LLVM 12Nikita Popov-12/+10
We don't have any known noalias bugs for LLVM 12 ... yet.
2021-03-21Convert -Z mutable-noalias to Optional<bool>Nikita Popov-1/+1
The default value will dependend on the LLVM version in the future, so don't specify one to start with.
2021-03-21Move decision aboute noalias into codegen_llvmNikita Popov-18/+49
The frontend shouldn't be deciding whether or not to use mutable noalias attributes, as this is a pure LLVM concern. Only provide the necessary information and do the actual decision in codegen_llvm.
2021-03-20Cleanup LLVM debuginfo module docsCamelid-14/+15
* Use Markdown list syntax and unindent a bit to prevent Markdown interpreting the nested lists as code blocks * A few more small typographical cleanups
2021-03-20Move debuginfo docs from `doc.rs` module to `doc.md` fileCamelid-181/+181
And use `#[doc = include_str!("doc.md")]` in `mod.rs` so the docs are rendered as if they were inline in the root module.
2021-03-19gave unused_fn WeakAnyLinkage; moved create_pgo_func_name_varRich Kadel-18/+19
The sample json5format tests produce coverage results again (and work with opt-level 3!)
2021-03-19stabilize or_patternsmark-1/+1
2021-03-19coverage bug fixes and optimization supportRich Kadel-148/+231
Adjusted LLVM codegen for code compiled with `-Zinstrument-coverage` to address multiple, somewhat related issues. Fixed a significant flaw in prior coverage solution: Every counter generated a new counter variable, but there should have only been one counter variable per function. This appears to have bloated .profraw files significantly. (For a small program, it increased the size by about 40%. I have not tested large programs, but there is anecdotal evidence that profraw files were way too large. This is a good fix, regardless, but hopefully it also addresses related issues. Fixes: #82144 Invalid LLVM coverage data produced when compiled with -C opt-level=1 Existing tests now work up to at least `opt-level=3`. This required a detailed analysis of the LLVM IR, comparisons with Clang C++ LLVM IR when compiled with coverage, and a lot of trial and error with codegen adjustments. The biggest hurdle was figuring out how to continue to support coverage results for unused functions and generics. Rust's coverage results have three advantages over Clang's coverage results: 1. Rust's coverage map does not include any overlapping code regions, making coverage counting unambiguous. 2. Rust generates coverage results (showing zero counts) for all unused functions, including generics. (Clang does not generate coverage for uninstantiated template functions.) 3. Rust's unused functions produce minimal stubbed functions in LLVM IR, sufficient for including in the coverage results; while Clang must generate the complete LLVM IR for each unused function, even though it will never be called. This PR removes the previous hack of attempting to inject coverage into some other existing function instance, and generates dedicated instances for each unused function. This change, and a few other adjustments (similar to what is required for `-C link-dead-code`, but with lower impact), makes it possible to support LLVM optimizations. Fixes: #79651 Coverage report: "Unexecuted instantiation:..." for a generic function from multiple crates Fixed by removing the aforementioned hack. Some "Unexecuted instantiation" notices are unavoidable, as explained in the `used_crate.rs` test, but `-Zinstrument-coverage` has new options to back off support for either unused generics, or all unused functions, which avoids the notice, at the cost of less coverage of unused functions. Fixes: #82875 Invalid LLVM coverage data produced with crate brotli_decompressor Fixed by disabling the LLVM function attribute that forces inlining, if `-Z instrument-coverage` is enabled. This attribute is applied to Rust functions with `#[inline(always)], and in some cases, the forced inlining breaks coverage instrumentation and reports.
2021-03-19correct macro namesSparrowLii-4/+4
2021-03-19Add simd_neg platform intrinsicSparrowLii-2/+21
2021-03-18Rollup merge of #83080 - tmiasko:inline-coverage, r=wesleywiserDylan DPC-2/+2
Make source-based code coverage compatible with MIR inlining When codegenning code coverage use the instance that coverage data was originally generated for, to ensure basic level of compatibility with MIR inlining. Fixes #83061
2021-03-17Auto merge of #83084 - nagisa:nagisa/features-native, r=petrochenkovbors-34/+67
Adjust `-Ctarget-cpu=native` handling in cg_llvm When cg_llvm encounters the `-Ctarget-cpu=native` it computes an explciit set of features that applies to the target in order to correctly compile code for the host CPU (because e.g. `skylake` alone is not sufficient to tell if some of the instructions are available or not). However there were a couple of issues with how we did this. Firstly, the order in which features were overriden wasn't quite right – conceptually you'd expect `-Ctarget-cpu=native` option to override the features that are implicitly set by the target definition. However due to how other `-Ctarget-cpu` values are handled we must adopt the following order of priority: * Features from -Ctarget-cpu=*; are overriden by * Features implied by --target; are overriden by * Features from -Ctarget-feature; are overriden by * function specific features. Another problem was in that the function level `target-features` attribute would overwrite the entire set of the globally enabled features, rather than just the features the `#[target_feature(enable/disable)]` specified. With something like `-Ctarget-cpu=native` we'd end up in a situation wherein a function without `#[target_feature(enable)]` annotation would have a broader set of features compared to a function with one such attribute. This turned out to be a cause of heavy run-time regressions in some code using these function-level attributes in conjunction with `-Ctarget-cpu=native`, for example. With this PR rustc is more careful about specifying the entire set of features for functions that use `#[target_feature(enable/disable)]` or `#[instruction_set]` attributes. Sadly testing the original reproducer for this behaviour is quite impossible – we cannot rely on `-Ctarget-cpu=native` to be anything in particular on developer or CI machines. cc https://github.com/rust-lang/rust/issues/83027 `@BurntSushi`
2021-03-16Adjust `-Ctarget-cpu=native` handling in cg_llvmSimonas Kazlauskas-34/+67
When cg_llvm encounters the `-Ctarget-cpu=native` it computes an explciit set of features that applies to the target in order to correctly compile code for the host CPU (because e.g. `skylake` alone is not sufficient to tell if some of the instructions are available or not). However there were a couple of issues with how we did this. Firstly, the order in which features were overriden wasn't quite right – conceptually you'd expect `-Ctarget-cpu=native` option to override the features that are implicitly set by the target definition. However due to how other `-Ctarget-cpu` values are handled we must adopt the following order of priority: * Features from -Ctarget-cpu=*; are overriden by * Features implied by --target; are overriden by * Features from -Ctarget-feature; are overriden by * function specific features. Another problem was in that the function level `target-features` attribute would overwrite the entire set of the globally enabled features, rather than just the features the `#[target_feature(enable/disable)]` specified. With something like `-Ctarget-cpu=native` we'd end up in a situation wherein a function without `#[target_feature(enable)]` annotation would have a broader set of features compared to a function with one such attribute. This turned out to be a cause of heavy run-time regressions in some code using these function-level attributes in conjunction with `-Ctarget-cpu=native`, for example. With this PR rustc is more careful about specifying the entire set of features for functions that use `#[target_feature(enable/disable)]` or `#[instruction_set]` attributes. Sadly testing the original reproducer for this behaviour is quite impossible – we cannot rely on `-Ctarget-cpu=native` to be anything in particular on developer or CI machines.
2021-03-16Auto merge of #82838 - Amanieu:rustdoc_asm, r=nagisabors-0/+3
Allow rustdoc to handle asm! of foreign architectures This allows rustdoc to process code containing `asm!` for architectures other than the current one. Since this never reaches codegen, we just replace target-specific registers and register classes with a dummy one. Fixes #82869
2021-03-15Functions inlined into reachable functions are reachableTomasz Miąsko-1/+1
Consider functions to be reachable for code coverage purposes, either when they reach the code generation directly, or indirectly as inlined part of another function.
2021-03-15Make source-based code coverage compatible with MIR inliningTomasz Miąsko-1/+1
When codegenning code coverage use the instance that coverage data was originally generated for, to ensure basic level of compatibility with MIR inlining.
2021-03-13Always lower asm! to valid HIRAmanieu d'Antras-0/+3
2021-03-12Add support for storing code model to LLVM module IRHiroki Noda-1/+10
This patch avoids undefined behavior by linking different object files. Also this would it could be propagated properly to LTO. See https://reviews.llvm.org/D52322 and https://reviews.llvm.org/D52323. This patch is based on https://github.com/rust-lang/rust/pull/74002
2021-03-11Import small cold functionsTomasz Miąsko-0/+3
The Rust code is often written under an assumption that for generic methods inline attribute is mostly unnecessary, since for optimized builds using ThinLTO, a method will be generated in at least one CGU and available for import. For example, deref implementations for Box, Vec, MutexGuard, and MutexGuard are not currently marked as inline, neither is identity implementation of From trait. In PGO builds, when functions are determined to be cold, the default multiplier of zero will stop the import, even for completely trivial functions. Increase slightly the default multiplier from 0 to 0.1 to import them regardless.
2021-03-10Remove the -Zinsert-sideeffectSimonas Kazlauskas-9/+5
This removes all of the code we had in place to work-around LLVM's handling of forward progress. From this removal excluded is a workaround where we'd insert a `sideeffect` into clearly infinite loops such as `loop {}`. This code remains conditionally effective when the LLVM version is earlier than 12.0, which fixed the forward progress related miscompilations at their root.
2021-03-09Use FromStr trait for number option parsingTomasz Miąsko-1/+1
Replace `parse_uint` with generic `parse_number` based on `FromStr`. Use it for parsing inlining threshold to avoid casting later.
2021-03-07Auto merge of #82285 - nhwn:nonzero-debug, r=nagisabors-25/+15
Use u32 over Option<u32> in DebugLoc ~~Changes `Option<u32>` fields in `DebugLoc` to `Option<NonZeroU32>`. Since the respective fields (`line` and `col`) are guaranteed to be 1-based, this layout optimization is a freebie.~~ EDIT: Changes `Option<u32>` fields in `DebugLoc` to `u32`. As `@bugadani` pointed out, an `Option<NonZeroU32>` is probably an unnecessary layer of abstraction since the `None` variant is always used as `UNKNOWN_LINE_NUMBER` (which is just `0`). Also, `SourceInfo` in `metadata.rs` already uses a `u32` instead of an `Option<u32>` to encode the same information, so I think this change is warranted. Since `@jyn514` raised some concerns over measuring performance in a similar PR (#82255), does this need a perf run?
2021-03-04Auto merge of #81451 - nikic:llvm-12, r=nagisabors-9/+31
Upgrade to LLVM 12 This implements the necessary adjustments to make rustc work with LLVM 12. I didn't encounter any major issues so far. r? `@cuviper`
2021-03-01Mark pure asm as willreturnNikita Popov-0/+2
2021-03-01Don't directly expose coverage::CounterMappingRegion via FFINikita Popov-6/+4
The definition of this struct changes in LLVM 12 due to the addition of branch coverage support. To avoid future mismatches, declare our own struct and then convert between them.
generated by cgit-pink 1.4.1-3-g733a5 (git 2.37.1) at 2026-03-02 12:44:03 +0000