Skip to content

COMP: Make the 'Build and Test' analysis workflow run - #57

Open
gdevenyi wants to merge 1 commit into
InsightSoftwareConsortium:fix/buildyml-jobsfrom
gdevenyi:pr/ci-dashboard-rewrite
Open

gdevenyi wants to merge 1 commit into
InsightSoftwareConsortium:fix/buildyml-jobsfrom
gdevenyi:pr/ci-dashboard-rewrite

Conversation

@gdevenyi

@gdevenyi gdevenyi commented Aug 15, 2026 •

Copy link
Copy Markdown

The first commit fixes the workflow's trigger: it fires on branch
main, but this repository's default branch is master, so not one
of its jobs -- coverage, valgrind memcheck, scan-build, ASan+UBSan --
has ever executed. It also replaces the retired macos-11 runner and
the deprecated actions/checkout@v3.

The trigger alone is not enough to make 'Build and Test' work. Every job invokes cmake/travis_dashboard.cmake, which begins

set_from_env(CTEST_SITE "TRAVIS_APP_HOST" REQUIRED)

and aborts with a FATAL_ERROR because GitHub Actions does not set that
variable. The script is a half-migrated hybrid: it wants
TRAVIS_APP_HOST from Travis and AGENT_BUILDDIRECTORY from Azure
Pipelines, and the workflow supplies only the latter. All five jobs
fail in about twenty seconds having compiled nothing.

Each job now configures, builds and tests directly:

coverage --coverage build, tests, an lcov summary
valgrind ctest -T memcheck, then a step that fails on any non-zero
ERROR SUMMARY or any definite or indirect leak
use_prefix builds with NIFTI_PACKAGE_PREFIX set
sanitize ASan and UBSan with -fno-sanitize-recover, then
scan-build --status-bugs
macos Release on macos-14

Three details are recorded in comments so they do not have to be
rediscovered:

  • valgrind will not start without libc6-dbg -- it reports that it
    cannot redirect memcmp and exits;

  • its memcheck needs --trace-children=yes, because most of these tests
    are shell scripts that exec nifti_tool. Without it valgrind
    inspects the shell, finds nothing and reports a clean run;

  • ctest -T memcheck exits 0 even when valgrind reports defects, so the
    following step greps the logs. That grep matches the error count
    itself rather than the absence of a clean summary: with
    --trace-children the concurrent processes share one log and their
    lines interleave, and a real run produced

    ==3336== ERROR SUMMARY: ==3300==
    

    which an absence test would have accepted.

This also drops the submission to my.cdash.org. Posting results to the
project's public dashboard from every pull request, including ones from
forks, is not obviously wanted, and nothing else in the workflow needs
it. cmake/travis_dashboard.cmake is left in place; nothing else refers
to it.

Stated plainly: the workflow itself cannot be run locally. The YAML
parses, and every cmake, ctest, valgrind and scan-build invocation in it
was run by hand against this tree, but the jobs are unverified until
they run on a runner.


Interface impact: none. On the union of all these changes, configured with USE_FSL_CODE=ON and USE_CIFTI_CODE=ON: all 448 exported symbols across libniftiio, libnifti2, libznz, libfslio, libnifticdf and libcifti are identical to master under nm -D --defined-only, and all ten installed headers are identical under gcc -E -P. Under gcc -dM -E one macro definition differs, intentionally and only in text: #61 makes FSL_RADIOLOGICAL read (-1) so it is safe inside an expression. Its value is still -1, checked by compiling against each installed fslio.h and printing it.

Verification. This branch: builds with gcc 16.1.1, ctest unchanged from master (2 of 345 fail on master itself in this environment; #31 and #29 each fix one). The union of all the PRs: 0 errors under both gcc 16.1.1 and clang 22.1.8, ctest 345/345 under each, and the whole suite under valgrind memcheck with --trace-children=yes gives 484 traced processes with no invalid access, no uninitialised value and no leak in any nifti binary.

Coordination. Every line of every branch was compared, whitespace-normalised, against the diffs of the open PRs (#11, #21, #22, #23, #24). Where one of those already changes a line, the line was left alone, and the few deliberate overlaps are named in the text above. What survives is 17 compiler warnings, all of them on those lines: 9 -Wsign-conversion (5 in fslio.c for #22, 2 in nifti2_io.c and 2 in nifti_tester001.c for #24) and 8 -Wcalloc-transposed-args in nifti_findhdrname and nifti_findimgname, which #11 rewrites. No formatting changes appear anywhere, to stay clear of #10 and #12.

One of a set of independent, single-purpose PRs. Each bases on master and can be merged on its own, in any order.

The full set of PRs (35)

The union of all of them is on the fork as all-changes, if you want to build and test the lot at once.

CI and build

Configuration and documentation

Defects

Warning and check classes

This was referenced Aug 15, 2026
@hjmjohnson

Copy link
Copy Markdown
Member

Rebased onto current master (b4876bf) and pushed as c8d48e7, now a single commit. Partially superseded — this needs a maintainer decision before it is worth reviewing in detail.

What master has already fixed, and this PR therefore no longer contributes:

  • The main vs master trigger. master's build.yml already triggers on master, so the first commit of this PR (8dcb6e5, "Trigger the analysis workflow on the branch that exists") is fully redundant and was dropped by the rebase.
  • The Travis script. cmake/travis_dashboard.cmake is gone; master uses cmake/github_dashboard.cmake, which takes CTEST_SITE from RUNNER_OS with a default rather than requiring TRAVIS_APP_HOST. The original "all five jobs fail in twenty seconds having compiled nothing" premise no longer holds — the jobs run.
  • macos-11 is gone; master uses macos-latest.

What is still genuinely new here, and is the only reason to keep the PR open:

  • ctest -T memcheck exits 0 even when valgrind reports defects. master's valgrind job therefore cannot fail on a memory error. This PR adds a log-scanning step that does.
  • --trace-children=yes, without which valgrind inspects the shell wrapper and reports a clean run having never looked at the library, and libc6-dbg, without which valgrind will not start at all.
  • A use_prefix job exercising NIFTI_PACKAGE_PREFIX, and -fno-sanitize-recover=all on the sanitizer job.
  • actions/checkout@v3 -> @v4.
  • Failures land in the Actions log instead of only on CDash, and pull requests from forks stop posting to my.cdash.org.

No overlap with cmake-multi-platform.yml, which covers the plain build matrix, the minimal configuration, the oldest supported CMake, the exported-symbol baseline, and the missing-declarations check — none of coverage, valgrind, sanitizers, scan-build, or the package prefix.

The decision is whether to keep the CDash dashboard path or move these five jobs to direct cmake/ctest invocations. If you want to keep CDash, the valgrind gating and the two valgrind flags should be extracted from this PR and applied to the dashboard script instead; the rest can be dropped. I have not closed or retitled anything.

What changed in this update
  • Rebased onto master. The trigger commit dropped as redundant. The rewrite commit conflicted with master's Travis-to-GitHub migration in build.yml; resolved by taking this PR's version of the jobs: block, since the commit's whole purpose is to replace it, after confirming that master's version of that block contributes nothing the new one lacks (the trigger and the dashboard-script name are the only differences, and the new jobs use neither).
  • Commit message: dropped the Co-Authored-By: naming an AI tool and the Claude-Session: URL; removed the reference to "the previous commit", which no longer exists; and rewrote the rationale, which described travis_dashboard.cmake and a FATAL_ERROR that master has since removed. The message now describes the situation as it is.
  • No change to the workflow content itself beyond the rebase.
Why no test

BUILD class. A GitHub Actions workflow has no runtime seam a ctest can observe; it is only exercised by a runner. Per the commit message, the individual cmake, ctest, valgrind and scan-build invocations were run by hand, but the jobs themselves remain unverified until CI runs them.

@hjmjohnson
hjmjohnson force-pushed the pr/ci-dashboard-rewrite branch 2 times, most recently from 03b4b67 to b3c827a Compare September 23, 2026 00:41
The 'Build and Test' jobs drive everything through
cmake/github_dashboard.cmake, so a failure is reported on CDash rather
than in the job log, and the memcheck job does not fail at all: ctest
-T memcheck exits 0 even when valgrind reports defects.

Each job now configures, builds and tests directly: coverage with an
lcov summary, valgrind memcheck followed by a step that fails on any
non-zero ERROR SUMMARY or definite or indirect leak, a build with
NIFTI_PACKAGE_PREFIX set, ASan and UBSan plus scan-build, and a macOS
Release build.  The submission to my.cdash.org is dropped;
cmake/github_dashboard.cmake is left in place.

The workflow cannot be run locally.  The YAML parses and every cmake,
ctest, valgrind and scan-build invocation in it was run by hand against
this tree, but the jobs are unverified until a runner runs them.

(cherry picked from commit b3c827a)
@hjmjohnson
hjmjohnson force-pushed the pr/ci-dashboard-rewrite branch from b3c827a to 075715d Compare September 24, 2026 12:01
@hjmjohnson
hjmjohnson changed the base branch from master to fix/buildyml-jobs September 24, 2026 12:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants