| Age | Commit message (Collapse) | Author | Lines |
|---|
|
Scalar methods
|
|
Simply shift the bitcast from the store to the load, so that
we can use the destination type. I'm not sure the bitcast is
really necessary, but keeping it for now.
|
|
I'm not really sure what is wrong here, but I was getting load
type mismatches in the debuginfo code (which is the only place
using this function).
Replacing the project_deref() implementation with a generic
load_operand + deref did the trick.
|
|
This makes load generation compatible with opaque pointers.
The generation of nontemporal copies still accesses the pointer
element type, as fixing this requires more movement.
|
|
Instead of determining it from the pointer type, explicitly pass
the type to load.
|
|
Do not emit alloca for ZST locals with multiple assignments
This extends 35566bfd7dd2e316d190078703de54a4dadda062 to additionally stop emitting unnecessary allocas for zero sized locals that are assigned multiple times.
When rebuilding the standard library with `-Zbuild-std` this reduces the number of locals that require an allocation from 62315 to 61767.
|
|
representation.
|
|
When rebuilding the standard library with `-Zbuild-std` this reduces the
number of locals that require an allocation from 62315 to 61767.
|
|
When rebuilding the standard library, this change reduces the number of
locals that require an alloca from 62452 to 62348.
|
|
Remove dead code from `LocalAnalyzer`
|
|
The else branch is taken when projection slice is empty so everything
except for the call to the `visit_local` is a dead code.
|
|
This code has been dead since changes in 68961.
|
|
The special casing of `box_free` predates the use of dominators in
analyzer. It is no longer necessary now that analyzer verifies that
the first assignment dominates all uses.
|
|
|
|
|
|
rustc_codegen_ssa: generate MSVC cleanup pads on demand, like GNU landing pads.
This unblocks #84993 in terms of codegen tests, as it brings the MSVC-style (`cleanup_pad`) EH (LLVM) block order in line with the GNU-style (`landing_pad`) EH (LLVM) block order, by having both of them be on-demand (instead of MSVC-style being eager and GNU-style lazy/on-demand).
It also unifies the two implementations a bit, similar to #84699, but in the opposite direction (as that attempt made both kinds of EH pads eagerly built).
~~Opening as draft because I haven't done enough Windows testing just yet, of both this PR, and of #84993 rebased on it.~~ (**EDIT**: seems to be working as expected)
r? `@nagisa`
|
|
coverage bug fixes and some refactoring
This replaces the relevant commits (2 and 3) from PR #85082, and also corrects an error querying for coverageinfo.
1. `coverageinfo` query needs to use the same MIR as codegen
I ran into an error trying to fix dead block coverage and realized the
`coverageinfo` query is getting a different MIR compared to the
codegenned MIR, which can sometimes be a problem during mapgen.
I changed that query to use the `InstandeDef` (which includes the
generic parameter substitutions, prosibly specific to const params)
instead of the `DefId` (without unknown/default const substitutions).
2. Simplified body_span and filtered span code
Some code cleanup extracted from future (but unfinished) commit to fix
coverage in attr macro functions.
3. Spanview needs the relevant body_span used for coverage
The coverage body_span doesn't always match the function body_span.
r? ```@tmandry```
|
|
|
|
On x86, the default syntax is also switched to Intel to match asm!
|
|
I ran into an error trying to fix dead block coverage and realized the
`coverageinfo` query is getting a different MIR compared to the
codegenned MIR, which can sometimes be a problem during mapgen.
I changed that query to use the `InstandeDef` (which includes the
generic parameter substitutions, prosibly specific to const params)
instead of the `DefId` (without unknown/default const substitutions).
|
|
|
|
|
|
This commit updates rustc, with an applicable LLVM version, to use
LLVM's new `llvm.fpto{u,s}i.sat.*.*` intrinsics to implement saturating
floating-point-to-int conversions. This results in a little bit tighter
codegen for x86/x86_64, but the main purpose of this is to prepare for
upcoming changes to the WebAssembly backend in LLVM where wasm's
saturating float-to-int instructions will now be implemented with these
intrinsics.
This change allows simplifying a good deal of surrounding code, namely
removing a lot of wasm-specific behavior. WebAssembly no longer has any
special-casing of saturating arithmetic instructions and the need for
`fptoint_may_trap` is gone and all handling code for that is now
removed. This means that the only wasm-specific logic is in the
`fpto{s,u}i` instructions which only get used for "out of bounds is
undefined behavior". This does mean that for the WebAssembly target
specifically the Rust compiler will no longer be 100% compatible with
pre-LLVM 12 versions, but it seems like that's unlikely to be relied on
by too many folks.
Note that this change does immediately regress the codegen of saturating
float-to-int casts on WebAssembly due to the specialization of the LLVM
intrinsic not being present in our LLVM fork just yet. I'll be following
up with an LLVM update to pull in those patches, but affects a few other
SIMD things in flight for WebAssembly so I wanted to separate this change.
Eventually the entire `cast_float_to_int` function can be removed when
LLVM 12 is the minimum version, but that will require sinking the
complexity of it into other backends such as Cranelfit.
|
|
The issue was that the resulting debuginfo was too complex for LLVM to
translate into CodeView records correctly. As a result, it simply
ignored the debuginfo which meant Windows debuggers could not display
any closed over variables when stepping inside a closure.
This fixes that by spilling additional variables to the stack so that
the resulting debuginfo is simple (just `*my_variable.dbg.spill`) and
LLVM can generate the correct CV records.
|
|
Use AnonConst for asm! constants
This replaces the old system which used explicit promotion. See #83169 for more background.
The syntax for `const` operands is still the same as before: `const <expr>`.
Fixes #83169
Because the implementation is heavily based on inline consts, we suffer from the same issues:
- We lose the ability to use expressions derived from generics. See the deleted tests in `src/test/ui/asm/const.rs`.
- We are hitting the same ICEs as inline consts, for example #78174. It is unlikely that we will be able to stabilize this before inline consts are stabilized.
|
|
|
|
A colleague contacted me and asked why Rust's counters start at 1, when
Clangs appear to start at 0. There is a reason why Rust's internal
counters start at 1 (see the docs), and I tried to keep them consistent
when codegenned to LLVM's coverage mapping format. LLVM should be
tolerant of missing counters, but as my colleague pointed out,
`llvm-cov` will silently fail to generate a coverage report for a
function based on LLVM's assumption that the counters are 0-based.
See:
https://github.com/llvm/llvm-project/blob/main/llvm/lib/ProfileData/Coverage/CoverageMapping.cpp#L170
Apparently, if, for example, a function has no branches, it would have
exactly 1 counter. `CounterValues.size()` would be 1, and (with the
1-based index), the counter ID would be 1. This would fail the check
and abort reporting coverage for the function.
It turns out that by correcting for this during coverage map generation,
by subtracting 1 from the Rust Counter ID (both when generating the
counter increment intrinsic call, and when adding counters to the map),
some uncovered functions (including in tests) now appear covered! This
corrects the coverage for a few tests!
|
|
|
|
coverage bug fixes and optimization support
Adjusted LLVM codegen for code compiled with `-Zinstrument-coverage` to
address multiple, somewhat related issues.
Fixed a significant flaw in prior coverage solution: Every counter
generated a new counter variable, but there should have only been one
counter variable per function. This appears to have bloated .profraw
files significantly. (For a small program, it increased the size by
about 40%. I have not tested large programs, but there is anecdotal
evidence that profraw files were way too large. This is a good fix,
regardless, but hopefully it also addresses related issues.
Fixes: #82144
Invalid LLVM coverage data produced when compiled with -C opt-level=1
Existing tests now work up to at least `opt-level=3`. This required a
detailed analysis of the LLVM IR, comparisons with Clang C++ LLVM IR
when compiled with coverage, and a lot of trial and error with codegen
adjustments.
The biggest hurdle was figuring out how to continue to support coverage
results for unused functions and generics. Rust's coverage results have
three advantages over Clang's coverage results:
1. Rust's coverage map does not include any overlapping code regions,
making coverage counting unambiguous.
2. Rust generates coverage results (showing zero counts) for all unused
functions, including generics. (Clang does not generate coverage for
uninstantiated template functions.)
3. Rust's unused functions produce minimal stubbed functions in LLVM IR,
sufficient for including in the coverage results; while Clang must
generate the complete LLVM IR for each unused function, even though
it will never be called.
This PR removes the previous hack of attempting to inject coverage into
some other existing function instance, and generates dedicated instances
for each unused function. This change, and a few other adjustments
(similar to what is required for `-C link-dead-code`, but with lower
impact), makes it possible to support LLVM optimizations.
Fixes: #79651
Coverage report: "Unexecuted instantiation:..." for a generic function
from multiple crates
Fixed by removing the aforementioned hack. Some "Unexecuted
instantiation" notices are unavoidable, as explained in the
`used_crate.rs` test, but `-Zinstrument-coverage` has new options to
back off support for either unused generics, or all unused functions,
which avoids the notice, at the cost of less coverage of unused
functions.
Fixes: #82875
Invalid LLVM coverage data produced with crate brotli_decompressor
Fixed by disabling the LLVM function attribute that forces inlining, if
`-Z instrument-coverage` is enabled. This attribute is applied to
Rust functions with `#[inline(always)], and in some cases, the forced
inlining breaks coverage instrumentation and reports.
FYI: `@wesleywiser`
r? `@tmandry`
|
|
|
|
|
|
Adjusted LLVM codegen for code compiled with `-Zinstrument-coverage` to
address multiple, somewhat related issues.
Fixed a significant flaw in prior coverage solution: Every counter
generated a new counter variable, but there should have only been one
counter variable per function. This appears to have bloated .profraw
files significantly. (For a small program, it increased the size by
about 40%. I have not tested large programs, but there is anecdotal
evidence that profraw files were way too large. This is a good fix,
regardless, but hopefully it also addresses related issues.
Fixes: #82144
Invalid LLVM coverage data produced when compiled with -C opt-level=1
Existing tests now work up to at least `opt-level=3`. This required a
detailed analysis of the LLVM IR, comparisons with Clang C++ LLVM IR
when compiled with coverage, and a lot of trial and error with codegen
adjustments.
The biggest hurdle was figuring out how to continue to support coverage
results for unused functions and generics. Rust's coverage results have
three advantages over Clang's coverage results:
1. Rust's coverage map does not include any overlapping code regions,
making coverage counting unambiguous.
2. Rust generates coverage results (showing zero counts) for all unused
functions, including generics. (Clang does not generate coverage for
uninstantiated template functions.)
3. Rust's unused functions produce minimal stubbed functions in LLVM IR,
sufficient for including in the coverage results; while Clang must
generate the complete LLVM IR for each unused function, even though
it will never be called.
This PR removes the previous hack of attempting to inject coverage into
some other existing function instance, and generates dedicated instances
for each unused function. This change, and a few other adjustments
(similar to what is required for `-C link-dead-code`, but with lower
impact), makes it possible to support LLVM optimizations.
Fixes: #79651
Coverage report: "Unexecuted instantiation:..." for a generic function
from multiple crates
Fixed by removing the aforementioned hack. Some "Unexecuted
instantiation" notices are unavoidable, as explained in the
`used_crate.rs` test, but `-Zinstrument-coverage` has new options to
back off support for either unused generics, or all unused functions,
which avoids the notice, at the cost of less coverage of unused
functions.
Fixes: #82875
Invalid LLVM coverage data produced with crate brotli_decompressor
Fixed by disabling the LLVM function attribute that forces inlining, if
`-Z instrument-coverage` is enabled. This attribute is applied to
Rust functions with `#[inline(always)], and in some cases, the forced
inlining breaks coverage instrumentation and reports.
|
|
Make source-based code coverage compatible with MIR inlining
When codegenning code coverage use the instance that coverage data was
originally generated for, to ensure basic level of compatibility with
MIR inlining.
Fixes #83061
|
|
Implement (but don't use) valtree and refactor in preparation of use
This PR does not cause any functional change. It refactors various things that are needed to make valtrees possible. This refactoring got big enough that I decided I'd want it reviewed as a PR instead of trying to make one huge PR with all the changes.
cc `@rust-lang/wg-const-eval` on the following commits:
* 2027184 implement valtree
* eeecea9 fallible Scalar -> ScalarInt
* 042f663 ScalarInt convenience methods
cc `@eddyb` on ef04a6d
cc `@rust-lang/wg-mir-opt` for cf1700c (`mir::Constant` can now represent either a `ConstValue` or a `ty::Const`, and it is totally possible to have two different representations for the same value)
|
|
When codegenning code coverage use the instance that coverage data was
originally generated for, to ensure basic level of compatibility with
MIR inlining.
|
|
|
|
|
|
|
|
This is in preparation of the `literal` field becoming an enum that distinguishes between type level constants and runtime constants
|
|
test: add test case
make tidy happy
|
|
This removes all of the code we had in place to work-around LLVM's
handling of forward progress. From this removal excluded is a workaround
where we'd insert a `sideeffect` into clearly infinite loops such as
`loop {}`. This code remains conditionally effective when the LLVM
version is earlier than 12.0, which fixed the forward progress related
miscompilations at their root.
|
|
It was suggested to lower this in MIR instead of ssa, so do that instead.
|
|
This replaces where it was previously being constructed in intrinsics, with direct construction
of the Statement.
|
|
Fixes copy_non_overlapping codegen_ssa to properly handle pointees,
and use bytes instead of elem count
|
|
Changed to various implementations, copying the style of prior function calls in places I was
unsure of.
Also one minor style nit.
|
|
|
|
This updates all places where match branches check on StatementKind or UseContext.
This doesn't properly implement them, but adds TODOs where they are, and also adds some best
guesses to what they should be in some cases.
|
|
I'm still not totally sure if this is the right way to implement the memcpy, but that portion
compiles correctly now. Now to fix the compile errors everywhere else :).
|
|
|
|
|
