Skip to content

CI: cherry-pick shell:bash + OS-split test filename convention from PR 820#12

Merged
eduralph merged 2 commits intomaintenance/gramps60from
fix/ci-shell-bash-bashism
May 6, 2026
Merged

CI: cherry-pick shell:bash + OS-split test filename convention from PR 820#12
eduralph merged 2 commits intomaintenance/gramps60from
fix/ci-shell-bash-bashism

Conversation

@eduralph
Copy link
Copy Markdown
Owner

@eduralph eduralph commented May 6, 2026

Summary

Cherry-picks two companion CI fixes from feature/ci-cd-pipeline-upstream (= upstream PR 820) onto fork's maintenance/gramps60, so the fork-internal CI stops failing/hanging while we wait for 820 to land upstream:

  • 28febdcdshell: bash on unit-test-linux + integration-test steps
  • 715e71dd — OS-split addon test filename convention (test_linux_* / test_windows_* / test_integration_*) + applying the convention to TMGimporter

Background

Issue 1: Bad substitution on Unit Tests (Linux) and Integration Tests

The container default shell on python:3.12-slim is /bin/sh (dash). The current workflow steps use bash-only parameter expansions:

mod="${f%.py}"
mod="${mod//\//.}"

Dash fails on the second line with Bad substitution, the script aborts, and unittest is never invoked. Visible in the April 19 maintenance/gramps60 run:

/__w/_temp/.../sh: 5: Bad substitution
Process completed with exit code 2

28febdcd adds shell: bash to the affected steps. Compile Check already sets it; Windows jobs inherit bash via job-level defaults.run.

Issue 2: Unit Tests (Windows) hangs for 6 hours

Visible in the same April 19 run: Unit Tests (Windows) in 6h5m1s — GitHub's job timeout. Cause: TMGimporter's tests/test_libtmg.py calls make_database("sqlite").load(":memory:", None), which deadlocks under conda-forge GTK + pip Gramps on Windows.

715e71dd introduces a filename convention (mirroring the existing test_integration_* convention) and applies it to TMGimporter:

Filename Linux Windows Integration job
test_*.py
test_linux_*.py
test_windows_*.py
test_integration_*.py

For TMGimporter:

  • 7 pure-logic test classes (string codes, date parsing, repo-type / URL inference) stay in tests/test_libtmg.py — run on every OS.
  • 13 DB-backed test classes (and the _Rec / _table / _make_db / _add_person / _MockUser helpers they use) move to tests/test_linux_libtmg.py — Linux only, no behavioural change.
  • Windows job no longer touches the deadlocking imports.

Why fork-only

Both fixes are already in upstream PR 820 (commits 28febdcd and 715e71dd on feature/ci-cd-pipeline-upstream). When 820 merges and the fork resyncs, this cherry-pick becomes a no-op merge. No upstream PR is needed.

This PR exists only to clear the immediate red/hung CI on eduralph/addons-source maintenance/gramps60 until upstream catches up.

Verification

  • yaml.safe_load parses the resulting workflow (7 jobs).
  • All three ${var//pat/repl} sites verified under bash (two via step-level shell: bash, one via Windows-job-level defaults.run.shell: bash -el {0}).
  • OS filter rules verified symmetric: Linux job skips test_windows_* + test_integration_*; Windows job skips test_linux_* + test_integration_*.
  • Reproduced the original Bad substitution failure under dash and confirmed bash succeeds on the same expansion.
  • TMGimporter: ran the testbed runner against the post-cherry-pick tree:
$ ./scripts/ubuntu/run-addon-unit.sh TMGimporter
Ran 175 tests in 8.640s
OK

Same total count as before the OS split (the 13 DB-backed classes now load via the dotted path TMGimporter.tests.test_linux_libtmg.* instead of …test_libtmg.*, with no test cases lost or added).

Remaining red on fork's maintenance/gramps60 after this lands

Out of scope for this PR (separate follow-ups; all continue-on-error: true so non-blocking):

  • 4 addons missing po/template.pot → waiting on a gramps60 backport of upstream #831.
  • 5 addons with plugin-registration errors (Collections Clipboard Gramplet, Query Quickview, lxml Gramplet, rebuild_types, TimePedigreeHTML) → real per-addon code bugs.
  • WebSearch tests/test_filetable.py bare-import bug → addressed by #11 / upstream #833.
  • Lint / Compile Check pre-existing failures → covered by 820's broader cleanup, not in this surgical cherry-pick.

🤖 Generated with Claude Code

eduralph added 2 commits May 6, 2026 23:06
Root cause of "Unit Tests (Linux)" and "Integration Tests (Gramps)"
failures was not broken test modules — the steps never invoked
unittest. The container's default shell is /bin/sh (dash on
python:3.12-slim), and the inline scripts use bash-only parameter
expansions (${f%.py}, ${mod//\//.}) to build the dotted module list.
Dash fails with "Bad substitution" on the first such line; the rest
of the script never runs. continue-on-error: true masked this as a
generic job failure for two CI rounds.

Add "shell: bash" explicitly to:
- unit-test-linux / Run per-addon unit tests (bashisms)
- integration-test / Run per-addon integration tests (bashisms)
- integration-test / Run plugin registration tests (no bashisms today,
  but consistent and future-proof)

Compile Check already sets shell: bash. Windows jobs inherit bash
via defaults.run at the job level. No other steps affected.
The Windows unit-test job hung on TMGimporter's DB-backed tests because
make_database("sqlite").load(":memory:", None) deadlocks under the
conda-forge GTK + pip Gramps combination. Rather than patch the hang,
introduce a filename convention so per-addon authors can declare OS
scope up front:

  test_*.py              general (every OS)
  test_linux_*.py        Linux-only
  test_windows_*.py      Windows-only
  test_integration_*.py  Linux-only, full-pipeline/DB-backed (pre-existing)

unit-test-linux skips test_windows_* and test_integration_*;
unit-test-windows skips test_linux_* and test_integration_*.

Applied to TMGimporter: the 13 DB-backed classes in tests/test_libtmg.py
move to tests/test_linux_libtmg.py (along with the _Rec/_table/_make_db/
_add_person/_MockUser helpers they use). The 7 pure-logic classes
(TestStripTmgCodes, TestTmgDateToGrampsDate, TestNumTo{Month,Date},
TestParseDate, TestRepoTypeFromName, TestUrlFromName) stay in
test_libtmg.py and will run on every OS.

Locally all 175 tests still pass via run-addon-unit.sh TMGimporter.
@eduralph eduralph changed the title CI: cherry-pick shell:bash fix for unit-test-linux + integration-test CI: cherry-pick shell:bash + OS-split test filename convention from PR 820 May 6, 2026
@eduralph eduralph merged commit 146b9c3 into maintenance/gramps60 May 6, 2026
1 of 7 checks passed
@eduralph eduralph deleted the fix/ci-shell-bash-bashism branch May 6, 2026 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant