Tracking Issue for LLVM InstrProf code coverage (current flag: `-Zinstrument-coverage`)

[richkadel]

This is a tracking issue for the MCP "Implement LLVM-compatible source-based code coverage" (rust-lang/compiler-team#278).

Original feature request: Issue #34701.

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

Steps

• Implement the MCP
• Adjust documentation (see instructions on rustc-dev-guide)
  • The Rustc Dev Guide includes detailed documentation on the implementation in a new section, LLVM Source-Based Code Coverage
  • Usage is documented in the Rust Unstable Book at https://doc.rust-lang.org/nightly/unstable-book/compiler-flags/source-based-code-coverage.html
  • Also see the Inside Rust Blog article at https://blog.rust-lang.org/inside-rust/2020/11/12/source-based-code-coverage.html
• Stabilization PR (see instructions on rustc-dev-guide)

Unresolved Questions

There are no unresolved questions from the MCP.

Some implementation options were debated and, if deferred, were logged as FIXME comments in the compiler source code or tests. FIXME comments related to stabilization have been added as rust-lang/rust issues.

• If/when stabilized, will the compiler flag be updated to -C instrument-coverage? (If so, the -Z variant could also be supported for some time, to ease migrations for existing users and scripts.)

• The Rust coverage implementation depends on (and automatically turns on) -Z symbol-mangling-version=v0. Will stabilizing this feature depend on stabilizing v0 symbol-mangling first? If so, what is the current status and timeline?

• The Rust coverage implementation implements the latest version of LLVM's Coverage Mapping Format (version 4), which forces a dependency on LLVM 11 or later. A compiler error is generated if attempting to compile with coverage, and using an older version of LLVM.

• What other questions need to be answered before the feature can be stabilized?

Open Issues related to this Tracking Issue

Somewhat ordered by priority:

• "Rustdoc doctests should have their file/lines remapped when using -Zinstrument-coverage" (Rustdoc doctests should have their file/lines remapped when using -Zinstrument-coverage #79417)
  • Fixed to correctly offset the lines, but still need a fix to offset the column (character) number
• "MIR InstrumentCoverage: Improve coverage of #[should_panic] tests and catch_unwind() handlers" (MIR InstrumentCoverage: Improve coverage of #[should_panic] tests and catch_unwind() handlers #78544)
• "Improve coverage results on certain derived traits that show two different counters" (Improve coverage results on certain derived traits that show two different counters #79626)
  • This may have been resolved by another coincidental improvement, and just needs verification
• "Add a cargo option to generate coverage reports using -Z instrument-coverage" (#85902)
• "MIR InstrumentCoverage - Can the source_info.span for FakeRead statements be more consistent?" (MIR InstrumentCoverage - Can the source_info.span for FakeRead statements be more consistent? #78546)
• "std::process::exit() on MSVC bypasses writing LLVM InstrProf counters to profraw file" (std::process::exit() on MSVC bypasses writing LLVM InstrProf counters to profraw file #77553)

Also, note the issue tracking label: A-code-coverage

Implementation history

• This feature was functionally complete as of PR Working expression optimization, and some improvements to branch-level source coverage #78267 and available via the nightly channel as of November 8, 2020.
• There have been many subsequent PRs to expand and improve coverage, including support for coverage of and within macros, coverage of dead code within a function (based on const eval), coverage of unused functions, and coverage of unused templates: Summary of merged PRs contributing to the implementation

[richkadel]

@tmandry @wesleywiser

[richkadel]

Just a quick note to the compiler team and interested contributors:

As of today, I'm going to be much less active on Rust Coverage, due to other work priorities.

I believe the implementation is essentially "complete" (including detailed documentation in the Rust Dev Guide that should help future contributors).

I plan to support any serious bugs in the implementation, as they arise, but otherwise, I encourage any contributors interested in taking on any of the "Open Issues" listed above. I should be able to advise/answer questions as needed.

Thank you so much to @tmandry and @wesleywiser for partnering with me on this feature, exceptionally detailed code reviews, and incredibly helpful guidance along the way!

Thanks also to the many other members of the Rust community who contributed requirements, testing, bug reports, and helpful tips along the way!

[wesleywiser]

Thanks for your work on this @richkadel!