Skip to content

fix: harden the v5.0.2 release gate and close the test-hygiene tail - #171

Merged
ohjonathan merged 1 commit into
mainfrom
codex/v5.0.2-audit-tail
Jul 14, 2026
Merged

fix: harden the v5.0.2 release gate and close the test-hygiene tail#171
ohjonathan merged 1 commit into
mainfrom
codex/v5.0.2-audit-tail

Conversation

@ohjonathan

Copy link
Copy Markdown
Owner

Summary

  • finalize the O4 ledger with the shipped v5.0.1 tag, PyPI publication, merged sweep PRs, and the first-run R2 gate's fail-safe propagation-race recovery
  • add bounded TestPyPI Simple-API polling that retries only an exact version not found result and fails immediately on hash or provenance mismatches
  • replace redundant subprocess help checks with recursive in-process parser/registry and golden assertions, and remove the unused legacy marker
  • bump package metadata to 5.0.2 and add the dated changelog and release note

Root cause and behavior

The v5.0.1 workflow verified the uploaded artifacts through TestPyPI metadata, then reached the Simple API before the same wheel had propagated there. The gate correctly blocked production and a targeted failed-job rerun succeeded later. This patch makes that expected lag bounded and explicit: 12 attempts at 10-second intervals, only for pip's exact unavailable-version result. Hash mismatches, manifest mismatches, unrelated pip failures, and wrong wheel bytes still fail on the first attempt.

The release workflow is read from the tag, so this path self-validates during the v5.0.2 release. Release actions and issue closure remain maintainer-owned; #148 closes only after v5.0.2 is published and verified.

Validation

  • full suite: 1,568 passed, 6 existing deprecation warnings
  • focused converted CLI/command regressions: 177 passed
  • focused release-artifact tests: 15 passed
  • doctor activation tripwire: passed
  • wheel/sdist manifest inspection and fresh non-editable install smoke: passed for 5.0.2
  • git diff --check: passed
  • Ontos activation: clean; Ontos doctor: passed
  • llm-dev manifest conformance: 4/4 passed

Remaining scope

@ohjonathan

Copy link
Copy Markdown
Owner Author

PR #171 (v5.0.2) review — Claude — approve to merge (no blockers, no majors)

Verified independently at the PR head (d0ed046).

B1 — the propagation-race fix (the one that mattered): done right, better than dispatched

  • Implemented in scripts/release_artifacts.py (testable) rather than inline YAML — which closes the "workflow not covered by PR CI" gap I flagged.
  • Fail-closed property is correct and locked by tests: _version_not_found() checks hash-failure signatures ("packages do not match the hashes" / "expected sha256") first → returns False (non-retryable); only the two exact "version not found" messages are retryable; retry-budget exhaustion also fails closed. Unit tests test_testpypi_download_never_retries_hash_mismatch, test_testpypi_download_retries_only_exact_version_not_found, and test_testpypi_download_version_not_found_retry_is_bounded pin exactly the property that mattered.
  • Bounded (12 × 10s), per-attempt fresh dirs, --require-hashes, a verify_downloaded_wheel re-check on success, atomic os.replace, and download-dir safety guards. Wired into publish.yml via download-testpypi-wheel --attempts 12; the old inline pip download is removed.

B2 — legacy marker

Removed cleanly; no registration, no @pytest.mark.legacy references; --strict-markers clean.

B3 — test hygiene: better than dispatched, no coverage lost

Every removed test was a subprocess --help test (test_*_help_parity, test_cli_phase4.py's parametrized --help); functional parity tests were kept. The new test_cli_help.py recursively matches the parser tree to the command registry, compares each command's full format_help() against goldens (so --depends-on/--count/--apply/--all remain covered), and explicitly re-asserts command-specific options — stronger than the spot-checks it replaced.

Bookkeeping

Version 5.0.2 consistent (__init__.py + pyproject.toml); golden baselines changed version-strings only; the O4 ledger finalization is accurate and honest (32/33 findings shipped through v5.0.1, the R2-gate fail-safe/propagation-race/rerun --failed recovery recorded, #148→v5.0.2 and #149→v6.0.0 tails correct). Suite 1,631→1,568 is expected (redundant subprocess help tests consolidated). CI 6/6; tripwire passed == 12 green.

Verdict: no blockers, no majors — ready to merge. After the v5.0.2 release cut, #148 closes and the entire audit-remediation program is done except the single v6.0.0 paths.py removal. v5.0.2's own release will be the live self-validation of the B1 retry. Release actions remain maintainer-deferred.

@ohjonathan
ohjonathan marked this pull request as ready for review July 14, 2026 15:34
@ohjonathan
ohjonathan merged commit 61fd4bc into main Jul 14, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant