Summary: I think the code coverage metrics in sonarscan are lower than the real number. I ask Claude if it can see a problem and it finds that libdap4 is running tests in parallel and that is very likely leading to overwriting the data files used by gcov/gcovr (libdap4 uses gcovr). Fix that.
TL;DR
Here's what Claude says about test coverage reporting in libdap4:
My question: We use sonarscan (aka sonar qube) to perform a static analysis of our code. It has a feature that enables reporting on code coverage. Our code is C++ and we develop on Mac OS but run most of our CICD stack on Linux (ubuntu). In particular, the sonarscan run on Ubuntu using TravisCI. How can I configure those scans so the code coverage is accurate? Right now, one project has coverage, although I'm not confident it's correct and another project - with 1,000s of tests - has 'zero percent' coverage - clearly wrong. How do I configure code coverage for this C++ code?
Response: Sonar's C/C++ analyzer never generates coverage itself — it just imports a report from gcov/gcovr/lcov that your build produces. Nearly all "wrong coverage" problems trace back to a break somewhere in that chain: compile flags → test execution → report generation → path matching → scanner config. Since you build and scan entirely on Ubuntu in Travis, the Mac dev machine isn't a factor here — just make sure you never try to reuse coverage data generated on your Mac; always regenerate it fresh inside the same Linux CI job that runs the scan.
The pipeline you want, end to end:
- Compile with coverage instrumentation:
-fprofile-arcs -ftest-coverage (or just --coverage for both compile and link) on every translation unit you want measured. If any compiler/CMake flag is missing, no .gcno/.gcda files are produced for those files and they silently show as 0%.
- Run your tests.
.gcda files are written only when the test binary exits normally — a crashed or abort()'d test run leaves partial/missing data.
- Convert to a report Sonar understands, using the same toolchain that built the code (gcc version match matters —
gcov-9 for g++-9, etc.):
gcov --preserve-paths into a directory of .gcov files → feed via sonar.cfamily.gcov.reportsPath=<dir>, orgcovr --sonarqube > coverage.xml → feed via sonar.coverageReportPaths=coverage.xml (note: this is a different property than the one above — pointing gcov.reportsPath at a gcovr XML file is a common mistake and yields 0%). gcovr 5.0 is the known-good version; 5.1 has had compatibility issues with Sonar's parser.
- Make sure
sonar.cfamily.compile-commands points at the compile_commands.json produced by SonarSource's build-wrapper wrapping your actual build command. Without this, the C++ analyzer can't even reliably map compiled files to source, and coverage can't attach correctly regardless of the report.
Minimal working Travis example (gcov flavor, matches SonarSource's own reference repo for this exact combo):
script:
- autoreconf --install && ./configure
- build-wrapper-linux-x86-64 --out-dir build_wrapper_output_directory make clean all
- ./src/test1/coverage_test1
- ./src/test2/coverage_test2
- mkdir coverage-dir && cd coverage-dir
- find .. -name '*.o' | xargs gcov --preserve-paths
- cd ..
- sonar-scanner --define sonar.cfamily.gcov.reportsPath="coverage-dir"
with sonar-project.properties containing sonar.cfamily.compile-commands=build_wrapper_output_directory/compile_commands.json.
For the project showing 0%, work through these in order — they cover the vast majority of real cases:
- Confirm
--coverage/-fprofile-arcs -ftest-coverage is actually applied to the build used for CI (not just a local dev CMake preset) — check the actual compile lines in the Travis log.
- Confirm
.gcda files exist after the test run (find . -name '*.gcda' | wc -l right after tests, before gcov runs). If zero, tests aren't linking/running the instrumented binaries.
- If you build out-of-source (e.g. a separate
build/ dir with CMake), run gcov/gcovr from a location where relative paths resolve back to your actual source tree — mismatched paths cause Sonar to silently drop the coverage data for those files (looks like 0%, not an error).
- Verify you're using the correct property for the report type you produced (
gcov.reportsPath = directory of .gcov files only; coverageReportPaths = gcovr's --sonarqube XML). Mixing these up is the single most common cause of "coverage shows zero."
- Check that
sonar.sources in sonar-project.properties actually includes the directories where the instrumented/tested code lives — files outside sonar.sources never get coverage numbers even if the report has data for them.
- With thousands of tests, check whether tests run in parallel across multiple processes/build dirs — concurrent writes to the same
.gcda files, or tests run in a container with a different working directory than the build, both corrupt or orphan the coverage data.
If you want, I can look at the actual Travis config and sonar-project.properties for both projects (paste them or point me at the files in your workspace) and diagnose the specific misconfiguration rather than working from general causes.
And then I ask: Lets first look at the code in /Users/jhrg/src/hyrax/libdap4. That has coverage reporting that shows up in sonarscane, but I don't quite believe it. There are some unit tests that run in parallel, but each is its own exec. There are integration tests that run in parallel using the same exec (.e.g, libdap4/tests/das-test). Look for problems in this project.
Response:
Found a real bug. Your suspicion about libdap4's coverage is justified — here's the mechanism.
Setup is otherwise correct. The scan job (.travis.yml, "scan" stage) does build-wrapper → make check -j$BUILD_PROCS → gcovr --sonarqube coverage.xml → sonar-scanner, and sonar-project.properties correctly points sonar.coverageReportPaths=coverage.xml. Coverage flags (--coverage -g -O0, via coverage.mk) are applied to the actual library (libdap.la, libdapclient.la, libdapserver.la) as well as the tests, not just the test code — so this isn't the "only instrumented the tests" problem.
The bug: the integration tests corrupt their own coverage counters. tests/Makefile.am builds das-test, dds-test, expr-test, dmr-test — one binary each — then check-local runs GNU Autotest suites (DASTest.at, DDSTest.at, etc.) with (TESTSUITEFLAGS)‘,whichTravissetsto‘−j(TESTSUITEFLAGS), which Travis sets to -j (TESTSUITEFLAGS)‘,whichTravissetsto‘−jBUILD_PROCS (nproc on an x-large VM, so a high degree of parallelism). Each .at file has dozens of AT_CHECK cases that all invoke the same binary, e.g. tests/DASTest.at: AT_CHECK([$abs_builddir/das-test -p < $1 || true], ...) repeated ~40 times. Autotest runs these concurrently in separate scratch directories, but the .gcda output path is fixed to the binary's compiled object-file location — it's the same path for every one of those ~40 concurrent processes. There's no GCOV_PREFIX/GCOV_PREFIX_STRIP anywhere in the tree to give each parallel invocation its own counter file. That's a textbook concurrent-write race on .gcda, and it hits every source file compiled into das-test/dds-test/expr-test/dmr-test — which is a large chunk of the library, since these are the integration tests that exercise the parser/DDS/DAS/DMR code paths most heavily. Lost or corrupted updates there mean gcovr's summed counts (and hence what Sonar displays) understate real coverage, possibly inconsistently between runs.
Your unit tests are fine — each is its own executable/object set (unit-tests/Makefile.am, TESTS = $(UNIT_TESTS)), so there's no shared .gcda path and no collision.
Fix, cheapest first:
Serialize just the coverage-affecting step: in the scan stage only, drop -j$BUILD_PROCS for check-local (i.e., run TESTSUITEFLAGS= or -j1 when LIBDAP_BUILD=sonar) so das-test/dds-test/expr-test/dmr-test invocations don't overlap. You keep full parallelism in your normal (non-scan) CI stage where you don't care about coverage accuracy — only the scan job needs to be serial.
If serializing is too slow, isolate instead: export a distinct GCOV_PREFIX per parallel Autotest worker (Autotest already gives each check its own working directory you could derive a prefix from) and point gcovr at the resulting tree of prefixed .gcda files so it sums them rather than racing.
Sanity check either fix by diffing coverage.xml from a serial run against the current parallel one — a meaningful jump in das-test/dds-test-adjacent source files' coverage would confirm this was the cause.
Option 1 is the one I'd start with — smallest diff, and correctness matters more than speed for the one job that produces the Sonar report.
Summary: I think the code coverage metrics in sonarscan are lower than the real number. I ask Claude if it can see a problem and it finds that libdap4 is running tests in parallel and that is very likely leading to overwriting the data files used by gcov/gcovr (libdap4 uses gcovr). Fix that.
TL;DR
Here's what Claude says about test coverage reporting in libdap4:
My question: We use sonarscan (aka sonar qube) to perform a static analysis of our code. It has a feature that enables reporting on code coverage. Our code is C++ and we develop on Mac OS but run most of our CICD stack on Linux (ubuntu). In particular, the sonarscan run on Ubuntu using TravisCI. How can I configure those scans so the code coverage is accurate? Right now, one project has coverage, although I'm not confident it's correct and another project - with 1,000s of tests - has 'zero percent' coverage - clearly wrong. How do I configure code coverage for this C++ code?
Response: Sonar's C/C++ analyzer never generates coverage itself — it just imports a report from gcov/gcovr/lcov that your build produces. Nearly all "wrong coverage" problems trace back to a break somewhere in that chain: compile flags → test execution → report generation → path matching → scanner config. Since you build and scan entirely on Ubuntu in Travis, the Mac dev machine isn't a factor here — just make sure you never try to reuse coverage data generated on your Mac; always regenerate it fresh inside the same Linux CI job that runs the scan.
The pipeline you want, end to end:
-fprofile-arcs -ftest-coverage(or just--coveragefor both compile and link) on every translation unit you want measured. If any compiler/CMake flag is missing, no.gcno/.gcdafiles are produced for those files and they silently show as 0%..gcdafiles are written only when the test binary exits normally — a crashed orabort()'d test run leaves partial/missing data.gcov-9forg++-9, etc.):gcov --preserve-pathsinto a directory of.gcovfiles → feed viasonar.cfamily.gcov.reportsPath=<dir>, orgcovr --sonarqube > coverage.xml→ feed viasonar.coverageReportPaths=coverage.xml(note: this is a different property than the one above — pointinggcov.reportsPathat a gcovr XML file is a common mistake and yields 0%). gcovr 5.0 is the known-good version; 5.1 has had compatibility issues with Sonar's parser.sonar.cfamily.compile-commandspoints at thecompile_commands.jsonproduced by SonarSource'sbuild-wrapperwrapping your actual build command. Without this, the C++ analyzer can't even reliably map compiled files to source, and coverage can't attach correctly regardless of the report.Minimal working Travis example (gcov flavor, matches SonarSource's own reference repo for this exact combo):
with
sonar-project.propertiescontainingsonar.cfamily.compile-commands=build_wrapper_output_directory/compile_commands.json.For the project showing 0%, work through these in order — they cover the vast majority of real cases:
--coverage/-fprofile-arcs -ftest-coverageis actually applied to the build used for CI (not just a local dev CMake preset) — check the actual compile lines in the Travis log..gcdafiles exist after the test run (find . -name '*.gcda' | wc -lright after tests, before gcov runs). If zero, tests aren't linking/running the instrumented binaries.build/dir with CMake), rungcov/gcovrfrom a location where relative paths resolve back to your actual source tree — mismatched paths cause Sonar to silently drop the coverage data for those files (looks like 0%, not an error).gcov.reportsPath= directory of.gcovfiles only;coverageReportPaths= gcovr's--sonarqubeXML). Mixing these up is the single most common cause of "coverage shows zero."sonar.sourcesinsonar-project.propertiesactually includes the directories where the instrumented/tested code lives — files outsidesonar.sourcesnever get coverage numbers even if the report has data for them..gcdafiles, or tests run in a container with a different working directory than the build, both corrupt or orphan the coverage data.If you want, I can look at the actual Travis config and
sonar-project.propertiesfor both projects (paste them or point me at the files in your workspace) and diagnose the specific misconfiguration rather than working from general causes.And then I ask: Lets first look at the code in /Users/jhrg/src/hyrax/libdap4. That has coverage reporting that shows up in sonarscane, but I don't quite believe it. There are some unit tests that run in parallel, but each is its own exec. There are integration tests that run in parallel using the same exec (.e.g, libdap4/tests/das-test). Look for problems in this project.
Response:
Found a real bug. Your suspicion about libdap4's coverage is justified — here's the mechanism.
Setup is otherwise correct. The scan job (.travis.yml, "scan" stage) does build-wrapper → make check -j$BUILD_PROCS → gcovr --sonarqube coverage.xml → sonar-scanner, and sonar-project.properties correctly points sonar.coverageReportPaths=coverage.xml. Coverage flags (--coverage -g -O0, via coverage.mk) are applied to the actual library (libdap.la, libdapclient.la, libdapserver.la) as well as the tests, not just the test code — so this isn't the "only instrumented the tests" problem.
The bug: the integration tests corrupt their own coverage counters. tests/Makefile.am builds das-test, dds-test, expr-test, dmr-test — one binary each — then check-local runs GNU Autotest suites (DASTest.at, DDSTest.at, etc.) with
(TESTSUITEFLAGS)‘,whichTravissetsto‘−j(TESTSUITEFLAGS), which Travis sets to-j (TESTSUITEFLAGS)‘,whichTravissetsto‘−jBUILD_PROCS(nproc on an x-large VM, so a high degree of parallelism). Each .at file has dozens of AT_CHECK cases that all invoke the same binary, e.g. tests/DASTest.at: AT_CHECK([$abs_builddir/das-test -p < $1 || true], ...) repeated ~40 times. Autotest runs these concurrently in separate scratch directories, but the .gcda output path is fixed to the binary's compiled object-file location — it's the same path for every one of those ~40 concurrent processes. There's no GCOV_PREFIX/GCOV_PREFIX_STRIP anywhere in the tree to give each parallel invocation its own counter file. That's a textbook concurrent-write race on .gcda, and it hits every source file compiled into das-test/dds-test/expr-test/dmr-test — which is a large chunk of the library, since these are the integration tests that exercise the parser/DDS/DAS/DMR code paths most heavily. Lost or corrupted updates there mean gcovr's summed counts (and hence what Sonar displays) understate real coverage, possibly inconsistently between runs.Your unit tests are fine — each is its own executable/object set (unit-tests/Makefile.am, TESTS = $(UNIT_TESTS)), so there's no shared .gcda path and no collision.
Fix, cheapest first:
Serialize just the coverage-affecting step: in the scan stage only, drop -j$BUILD_PROCS for check-local (i.e., run TESTSUITEFLAGS= or -j1 when LIBDAP_BUILD=sonar) so das-test/dds-test/expr-test/dmr-test invocations don't overlap. You keep full parallelism in your normal (non-scan) CI stage where you don't care about coverage accuracy — only the scan job needs to be serial.
If serializing is too slow, isolate instead: export a distinct GCOV_PREFIX per parallel Autotest worker (Autotest already gives each check its own working directory you could derive a prefix from) and point gcovr at the resulting tree of prefixed .gcda files so it sums them rather than racing.
Sanity check either fix by diffing coverage.xml from a serial run against the current parallel one — a meaningful jump in das-test/dds-test-adjacent source files' coverage would confirm this was the cause.
Option 1 is the one I'd start with — smallest diff, and correctness matters more than speed for the one job that produces the Sonar report.