about summary refs log tree commit diff
path: root/src/test/run-make-fulldeps/coverage-reports
AgeCommit message (Collapse)AuthorLines
2021-04-18Fixes the issue with uncovered source in async function bodiesRich Kadel-78/+69
The body_span was assumed to be in the Span root context, but this was not the case for async function bodies.
2021-04-18DRAFT: coverage of async function bodies should match non-asyncRich Kadel-1/+125
The initial commit demonstrates the issue, but the fix is not yet implemented. Once corrected... Fixes: #83985
2021-04-18Add coverage to continue statementsRich Kadel-2/+77
`continue` statements were missing coverage. This was particularly noticeable in a match pattern that contained only a `continue` statement, leaving the branch appear uncounted. This PR addresses the problem and adds tests to prove it.
2021-04-02Translate counters from Rust 1-based to LLVM 0-based counter idsRich Kadel-42/+63
A colleague contacted me and asked why Rust's counters start at 1, when Clangs appear to start at 0. There is a reason why Rust's internal counters start at 1 (see the docs), and I tried to keep them consistent when codegenned to LLVM's coverage mapping format. LLVM should be tolerant of missing counters, but as my colleague pointed out, `llvm-cov` will silently fail to generate a coverage report for a function based on LLVM's assumption that the counters are 0-based. See: https://github.com/llvm/llvm-project/blob/main/llvm/lib/ProfileData/Coverage/CoverageMapping.cpp#L170 Apparently, if, for example, a function has no branches, it would have exactly 1 counter. `CounterValues.size()` would be 1, and (with the 1-based index), the counter ID would be 1. This would fail the check and abort reporting coverage for the function. It turns out that by correcting for this during coverage map generation, by subtracting 1 from the Rust Counter ID (both when generating the counter increment intrinsic call, and when adding counters to the map), some uncovered functions (including in tests) now appear covered! This corrects the coverage for a few tests!
2021-04-02Simplify coverage testsRich Kadel-80/+0
This change reduces the risk of impacting coverage tests on unrelated changes (such as MIR and Span changes), and reduces the burden when blessing coverage changes in case it is necessary. * Remove all spanview tests. The spanview tests were useful during development, but they can be generated as needed, via compiler command line flags. They aren't critical to confirming coverage results. (The coverage report tests are sufficient.) When spanview regeneration was necessary, the diffs were way too hard to read to be useful anyway. So I'm removing them to reduce friction from a feature that is no longer useful. * Remove the requirement for `llvm-cov show --debug` when blessing tests. The `--debug` flag is, unfortunately, only available if LLVM is built with `optimize = false` (in Rust's config.toml). This adds significant time and resource burdens to the contributor's build. As it turns out, for other reasons in the past, I wasn't actually using the debug output (counter info) to validate coverage anymore either, so it was required for no reason, I now realize.
2021-03-30Apply review feedbackAmanieu d'Antras-77/+83
2021-03-30Run LLVM coverage instrumentation passes before optimization passesAmanieu d'Antras-106/+83
This matches the behavior of Clang and allows us to remove several hacks which were needed to ensure functions weren't optimized away before reaching the instrumentation pass.
2021-03-19coverage bug fixes and optimization supportRich Kadel-51/+300
Adjusted LLVM codegen for code compiled with `-Zinstrument-coverage` to address multiple, somewhat related issues. Fixed a significant flaw in prior coverage solution: Every counter generated a new counter variable, but there should have only been one counter variable per function. This appears to have bloated .profraw files significantly. (For a small program, it increased the size by about 40%. I have not tested large programs, but there is anecdotal evidence that profraw files were way too large. This is a good fix, regardless, but hopefully it also addresses related issues. Fixes: #82144 Invalid LLVM coverage data produced when compiled with -C opt-level=1 Existing tests now work up to at least `opt-level=3`. This required a detailed analysis of the LLVM IR, comparisons with Clang C++ LLVM IR when compiled with coverage, and a lot of trial and error with codegen adjustments. The biggest hurdle was figuring out how to continue to support coverage results for unused functions and generics. Rust's coverage results have three advantages over Clang's coverage results: 1. Rust's coverage map does not include any overlapping code regions, making coverage counting unambiguous. 2. Rust generates coverage results (showing zero counts) for all unused functions, including generics. (Clang does not generate coverage for uninstantiated template functions.) 3. Rust's unused functions produce minimal stubbed functions in LLVM IR, sufficient for including in the coverage results; while Clang must generate the complete LLVM IR for each unused function, even though it will never be called. This PR removes the previous hack of attempting to inject coverage into some other existing function instance, and generates dedicated instances for each unused function. This change, and a few other adjustments (similar to what is required for `-C link-dead-code`, but with lower impact), makes it possible to support LLVM optimizations. Fixes: #79651 Coverage report: "Unexecuted instantiation:..." for a generic function from multiple crates Fixed by removing the aforementioned hack. Some "Unexecuted instantiation" notices are unavoidable, as explained in the `used_crate.rs` test, but `-Zinstrument-coverage` has new options to back off support for either unused generics, or all unused functions, which avoids the notice, at the cost of less coverage of unused functions. Fixes: #82875 Invalid LLVM coverage data produced with crate brotli_decompressor Fixed by disabling the LLVM function attribute that forces inlining, if `-Z instrument-coverage` is enabled. This attribute is applied to Rust functions with `#[inline(always)], and in some cases, the forced inlining breaks coverage instrumentation and reports.
2021-03-16Add test case for -Zinline-mir & -Zinstrument-coverageTomasz Miąsko-3/+56
2021-03-15Add support for compile-flags in coverage testsTomasz Miąsko-3/+3
2021-03-12Make def_key and HIR parenting consistent.Camille GILLOT-3/+0
2021-02-11Re-blessed the partial_eq.rs coverage testRich Kadel-18/+3
2021-02-11Ensures `make` tests run under /bin/dash, like CI, and fixes a MakefileRich Kadel-2/+1
Updates `tools.mk` to explicitly require `SHELL := /bin/dash`, since CI uses `dash` but other environments (including developer local machines) may default to `bash`. Replaces bash-specific shell command in one Makefile with a dash-compatible alternative, and re-enables the affected Makefile test. Removes apparently redundant definition of `UNAME`.
2021-02-05Auto merge of #81257 - ↵bors-2/+2
pnkfelix:issue-80949-short-term-resolution-via-revert-of-pr-78373, r=matthewjasper Revert 78373 ("dont leak return value after panic in drop") Short term resolution for issue #80949. Reopen #47949 after this lands. (We plan to fine-tune PR #78373 to not run into this problem.)
2021-02-04Revert "Avoid leaking block expression values"Felix S. Klock II-2/+2
This reverts commit 4fef39113a514bb270f5661a82fdba17d3e41dbb.
2021-02-03Ignore broken test.Felix S. Klock II-0/+1
(Test was silently ignored on Linux CI prior to parent commit that switched to using `# min-llvm-version`. But the switch made the ignoring stop, exposing other brokenness in the form of bash-dependent syntax in the `$(shell ...)` invocations.)
2021-02-02Use `# min-llvm-version: 11.0` to force a minimum LLVM version, rather than ↵Felix S. Klock II-5/+1
ad-hoc internal solution. In particular: the specific code to define LLVM_VERSION_11_PLUS here was, for some reason, using `$(shell ...)` with bash-specific variable replacement code. On non-bash platforms like dash, that `shell` invocation would fail, and the LLVM_VERSION_11_PLUS check would always fail, the test would always be ignored, and thus be treated as a "success" (in the sense that `--bless` would never do anything). This was causing me a lot of pain.
2021-01-08Rollup merge of #80761 - andjo403:generic_coverage, r=spastorinoYuki Okushi-1/+1
handle generic trait methods in coverage-report tests also make the generic function pattern more specific and remove the extra $ that fails the matching. r? `@wesleywiser` as this was failing the test of #76896
2021-01-07handle generic trait methods in coverage testsAndreas Jonson-1/+1
make the generic function pattern more specific
2021-01-04Inlining enabled by -mir-opt-level > 1 is incompatible with coverageRich Kadel-44/+80
Fixes: #80060 Also adds additional test cases for coverage of doctests.
2020-12-21make path normalization compatible with mac python2Arpad Borsos-0/+2
2020-12-19Remap instrument-coverage line numbers in doctestsArpad Borsos-6/+109
This uses the `SourceMap::doctest_offset_line` method to re-map line numbers from doctests. Remapping columns is not yet done. Part of issue #79417.
2020-12-16Remove redundant and unreliable coverage test resultsRich Kadel-3576/+86
The `coverage-reports` tests still generate counters and JSON reports for inspection, but these files are no longer used in Makefile diffs, to reduce complexity and confusion from unreliable or unexpected test results, especially when maintaining them (i.e., generating `--bless`ed results). The associated `expected_` files for counters and JSON reports have been removed, leaving only the files actually used for testing: the `llvm-cov show` reports.
2020-12-15Fixed conflict with drop elaboration and coverageRich Kadel-67/+67
See https://github.com/rust-lang/rust/issues/80045#issuecomment-745733339 Coverage statements are moved to the beginning of the BCB. This does also affect what's counted before a panic, changing some results, but I think these results may even be preferred? In any case, there are no guarantees about what's counted when a panic occurs (by design).
2020-12-11Fixes reported bugs in Rust CoverageRich Kadel-6/+213
Fixes: #79569 Fixes: #79566 Fixes: #79565 For the first issue (#79569), I got hit a `debug_assert!()` before encountering the reported error message (because I have `debug = true` enabled in my config.toml). The assertion showed me that some `SwitchInt`s can have more than one target pointing to the same `BasicBlock`. I had thought that was invalid, but since it seems to be possible, I'm allowing this now. I added a new test for this. ---- In the last two cases above, both tests (intentionally) fail to compile, but the `InstrumentCoverage` pass is invoked anyway. The MIR starts with an `Unreachable` `BasicBlock`, which I hadn't encountered before. (I had assumed the `InstrumentCoverage` pass would only be invoked with MIRs from successful compilations.) I don't have test infrastructure set up to test coverage on files that fail to compile, so I didn't add a new test.
2020-12-07Fixes to Rust coverageRich Kadel-24/+26
Fixes: #79725 Some macros can create a situation where `fn_sig_span` and `body_span` map to different files. New documentation on coverage tests incorrectly assumed multiple test binaries could just be listed at the end of the `llvm-cov` command, but it turns out each binary needs a `--object` prefix. This PR fixes the bug and updates the documentation to correct that issue. It also fixes a few other minor issues in internal implementation comments, and adds documentation on getting coverage results for doc tests.
2020-12-04Avoid leaking block expression valuesMatthew Jasper-15/+15
2020-12-03Tweak to Makefile to overcome MacOS make corruption bugRich Kadel-10/+4
2020-12-03Fixed cross-crate generic call test to compile lib and bin separatelyRich Kadel-93/+237
The original test produced a single crate with two mods, which was not the goal of the test.
2020-12-03Workaround for inconsistent order of llvm-cov results on WindowsRich Kadel-54/+49
2020-12-03Addressed feedback from 2020-12-01Rich Kadel-98/+358
Added one more test (two files) showing coverage of generics and unused functions across crates. Created and referenced new Issues, as requested. Added comments. Added a note about the possible effects of compiler options on LLVM coverage maps.
2020-12-03Combination of commitsRich Kadel-1132/+1592
Fixes multiple issue with counters, with simplification Includes a change to the implicit else span in ast_lowering, so coverage of the implicit else no longer spans the `then` block. Adds coverage for unused closures and async function bodies. Fixes: #78542 Adding unreachable regions for known MIR missing from coverage map Cleaned up PR commits, and removed link-dead-code requirement and tests Coverage no longer depends on Issue #76038 (`-C link-dead-code` is no longer needed or enforced, so MSVC can use the same tests as Linux and MacOS now) Restrict adding unreachable regions to covered files Improved the code that adds coverage for uncalled functions (with MIR but not-codegenned) to avoid generating coverage in files not already included in the files with covered functions. Resolved last known issue requiring --emit llvm-ir workaround Fixed bugs in how unreachable code spans were added.
2020-12-03Removed -base from run-make-fulldeps/coverage-*-baseRich Kadel-0/+4158
In preparation for removing the -deadcode variants