| Age | Commit message (Collapse) | Author | Lines |
|---|
|
|
|
r=wesleywiser
Stabilize `-Z instrument-coverage` as `-C instrument-coverage`
(Tracking issue for `instrument-coverage`: https://github.com/rust-lang/rust/issues/79121)
This PR stabilizes support for instrumentation-based code coverage, previously provided via the `-Z instrument-coverage` option. (Continue supporting `-Z instrument-coverage` for compatibility for now, but show a deprecation warning for it.)
Many, many people have tested this support, and there are numerous reports of it working as expected.
Move the documentation from the unstable book to stable rustc documentation. Update uses and documentation to use the `-C` option.
Addressing questions raised in the tracking issue:
> If/when stabilized, will the compiler flag be updated to -C instrument-coverage? (If so, the -Z variant could also be supported for some time, to ease migrations for existing users and scripts.)
This stabilization PR updates the option to `-C` and keeps the `-Z` variant to ease migration.
> The Rust coverage implementation depends on (and automatically turns on) -Z symbol-mangling-version=v0. Will stabilizing this feature depend on stabilizing v0 symbol-mangling first? If so, what is the current status and timeline?
This stabilization PR depends on https://github.com/rust-lang/rust/pull/90128 , which stabilizes `-C symbol-mangling-version=v0` (but does not change the default symbol-mangling-version).
> The Rust coverage implementation implements the latest version of LLVM's Coverage Mapping Format (version 4), which forces a dependency on LLVM 11 or later. A compiler error is generated if attempting to compile with coverage, and using an older version of LLVM.
Given that LLVM 13 has now been released, requiring LLVM 11 for coverage support seems like a reasonable requirement. If people don't have at least LLVM 11, nothing else breaks; they just can't use coverage support. Given that coverage support currently requires a nightly compiler and LLVM 11 or newer, allowing it on a stable compiler built with LLVM 11 or newer seems like an improvement.
The [tracking issue](https://github.com/rust-lang/rust/issues/79121) and the [issue label A-code-coverage](https://github.com/rust-lang/rust/labels/A-code-coverage) link to a few open issues related to `instrument-coverage`, but none of them seem like showstoppers. All of them seem like improvements and refinements we can make after stabilization.
The original `-Z instrument-coverage` support went through a compiler-team MCP at https://github.com/rust-lang/compiler-team/issues/278 . Based on that, `@pnkfelix` suggested that this needed a stabilization PR and a compiler-team FCP.
|
|
As discussed in
https://github.com/rust-lang/rust/pull/92142#issuecomment-1008239473,
tests that contain multiple files order their results differently on
Windows and Linux which the test runner currently can't handle
correctly. For now, ignore the "bin" part of the test and only include
the unused library dependency which is what the test really cares about
anyway.
|
|
Continue supporting -Z instrument-coverage for compatibility for now,
but show a deprecation warning for it.
Update uses and documentation to use the -C option.
Move the documentation from the unstable book to stable rustc
documentation.
|
|
This commit augments Swatinem's initial commit in uncommitted PR #90047,
which was a great starting point, but did not fully support LLVM
Coverage Mapping Format version 6.
Version 6 requires adding the compilation directory when file paths are
relative, and since Rustc coverage maps use relative paths, we should
add the expected compilation directory entry.
Note, however, that with the compilation directory, coverage reports
from `llvm-cov show` can now report file names (when the report includes
more than one file) with the full absolute path to the file.
This would be a problem for test results, but the workaround (for the
rust coverage tests) is to include an additional `llvm-cov show`
parameter: `--compilation-dir=.`
|
|
|
|
Previously, coverage tests were writing profiler data to files based on
their pid. As rustdoc spawns each doctest as its own process, it might
be possible in rare cases that a pid is being reused which would cause
a file to be overwritten, leading to incorrect coverage results.
|
|
This change reduces the risk of impacting coverage tests on unrelated
changes (such as MIR and Span changes), and reduces the burden when
blessing coverage changes in case it is necessary.
* Remove all spanview tests. The spanview tests were useful during
development, but they can be generated as needed, via compiler command
line flags. They aren't critical to confirming coverage results. (The
coverage report tests are sufficient.)
When spanview regeneration was necessary, the diffs were way too hard
to read to be useful anyway. So I'm removing them to reduce friction
from a feature that is no longer useful.
* Remove the requirement for `llvm-cov show --debug` when blessing
tests. The `--debug` flag is, unfortunately, only available if LLVM is
built with `optimize = false` (in Rust's config.toml). This adds
significant time and resource burdens to the contributor's build. As
it turns out, for other reasons in the past, I wasn't actually using
the debug output (counter info) to validate coverage anymore either,
so it was required for no reason, I now realize.
|
|
Adjusted LLVM codegen for code compiled with `-Zinstrument-coverage` to
address multiple, somewhat related issues.
Fixed a significant flaw in prior coverage solution: Every counter
generated a new counter variable, but there should have only been one
counter variable per function. This appears to have bloated .profraw
files significantly. (For a small program, it increased the size by
about 40%. I have not tested large programs, but there is anecdotal
evidence that profraw files were way too large. This is a good fix,
regardless, but hopefully it also addresses related issues.
Fixes: #82144
Invalid LLVM coverage data produced when compiled with -C opt-level=1
Existing tests now work up to at least `opt-level=3`. This required a
detailed analysis of the LLVM IR, comparisons with Clang C++ LLVM IR
when compiled with coverage, and a lot of trial and error with codegen
adjustments.
The biggest hurdle was figuring out how to continue to support coverage
results for unused functions and generics. Rust's coverage results have
three advantages over Clang's coverage results:
1. Rust's coverage map does not include any overlapping code regions,
making coverage counting unambiguous.
2. Rust generates coverage results (showing zero counts) for all unused
functions, including generics. (Clang does not generate coverage for
uninstantiated template functions.)
3. Rust's unused functions produce minimal stubbed functions in LLVM IR,
sufficient for including in the coverage results; while Clang must
generate the complete LLVM IR for each unused function, even though
it will never be called.
This PR removes the previous hack of attempting to inject coverage into
some other existing function instance, and generates dedicated instances
for each unused function. This change, and a few other adjustments
(similar to what is required for `-C link-dead-code`, but with lower
impact), makes it possible to support LLVM optimizations.
Fixes: #79651
Coverage report: "Unexecuted instantiation:..." for a generic function
from multiple crates
Fixed by removing the aforementioned hack. Some "Unexecuted
instantiation" notices are unavoidable, as explained in the
`used_crate.rs` test, but `-Zinstrument-coverage` has new options to
back off support for either unused generics, or all unused functions,
which avoids the notice, at the cost of less coverage of unused
functions.
Fixes: #82875
Invalid LLVM coverage data produced with crate brotli_decompressor
Fixed by disabling the LLVM function attribute that forces inlining, if
`-Z instrument-coverage` is enabled. This attribute is applied to
Rust functions with `#[inline(always)], and in some cases, the forced
inlining breaks coverage instrumentation and reports.
|
|
|
|
Updates `tools.mk` to explicitly require `SHELL := /bin/dash`, since CI
uses `dash` but other environments (including developer local machines)
may default to `bash`.
Replaces bash-specific shell command in one Makefile with a
dash-compatible alternative, and re-enables the affected Makefile test.
Removes apparently redundant definition of `UNAME`.
|
|
(Test was silently ignored on Linux CI prior to parent commit that switched to
using `# min-llvm-version`. But the switch made the ignoring stop, exposing
other brokenness in the form of bash-dependent syntax in the `$(shell ...)`
invocations.)
|
|
ad-hoc internal solution.
In particular: the specific code to define LLVM_VERSION_11_PLUS here was, for
some reason, using `$(shell ...)` with bash-specific variable replacement code.
On non-bash platforms like dash, that `shell` invocation would fail, and the
LLVM_VERSION_11_PLUS check would always fail, the test would always be ignored,
and thus be treated as a "success" (in the sense that `--bless` would never do
anything).
This was causing me a lot of pain.
|
|
make the generic function pattern more specific
|
|
This uses the `SourceMap::doctest_offset_line` method to re-map line
numbers from doctests. Remapping columns is not yet done.
Part of issue #79417.
|
|
The `coverage-reports` tests still generate counters and JSON reports
for inspection, but these files are no longer used in Makefile diffs, to
reduce complexity and confusion from unreliable or unexpected test
results, especially when maintaining them (i.e., generating `--bless`ed
results).
The associated `expected_` files for counters and JSON reports have been
removed, leaving only the files actually used for testing: the `llvm-cov
show` reports.
|
|
Fixes: #79725
Some macros can create a situation where `fn_sig_span` and `body_span`
map to different files.
New documentation on coverage tests incorrectly assumed multiple test
binaries could just be listed at the end of the `llvm-cov` command,
but it turns out each binary needs a `--object` prefix.
This PR fixes the bug and updates the documentation to correct that
issue. It also fixes a few other minor issues in internal implementation
comments, and adds documentation on getting coverage results for doc
tests.
|
|
|
|
The original test produced a single crate with two mods, which was not
the goal of the test.
|
|
|
|
Fixes multiple issue with counters, with simplification
Includes a change to the implicit else span in ast_lowering, so coverage
of the implicit else no longer spans the `then` block.
Adds coverage for unused closures and async function bodies.
Fixes: #78542
Adding unreachable regions for known MIR missing from coverage map
Cleaned up PR commits, and removed link-dead-code requirement and tests
Coverage no longer depends on Issue #76038 (`-C link-dead-code` is
no longer needed or enforced, so MSVC can use the same tests as
Linux and MacOS now)
Restrict adding unreachable regions to covered files
Improved the code that adds coverage for uncalled functions (with MIR
but not-codegenned) to avoid generating coverage in files not already
included in the files with covered functions.
Resolved last known issue requiring --emit llvm-ir workaround
Fixed bugs in how unreachable code spans were added.
|
|
In preparation for removing the -deadcode variants
|
