Stage 8 exit audit: close three criteria gaps, three stale documents - #90
Open
AdamClemens wants to merge 3 commits into
Open
Stage 8 exit audit: close three criteria gaps, three stale documents#90AdamClemens wants to merge 3 commits into
AdamClemens wants to merge 3 commits into
Conversation
make ci was green throughout -- 1200 tests, 144 scenarios, 99% coverage
-- and three of Stage 8's nine Completion Criteria were marked Met
against checks that did not test what the criterion said. None needed
judgement to find; each is a sentence the criterion had already written
down. This is docs/practices.md's own "An exit audit reads each
criterion to its last sentence", recurring on the one stage whose
previous audit is why Criteria 6-9 exist at all.
Criterion 6's rendered-pixel clause ("a keyboard seek and a mouse drag
each change window.renderer.snapshot()'s content") was checked by
nothing: neither scrub test calls snapshot() at all, and the keyboard
one's own docstring says it proves PlaybackState.position. Two new
tests close it.
Criterion 8 named a scale nothing had ever run ("a real
multi-hundred-frame pyflow record run"); the CLI test used
--max-frames 20 and TASK-049's by-hand check used 25. Closed by a real
300-frame CLI run, plus a separate test pinning peak on-disk count
during the run -- the claim _advance_and_checkpoint's docstring makes
("as the recording grows, not only once it finishes") and which the
final file set cannot prove either way.
Criterion 1 rested on a by-hand check and said so in its own text.
Re-verified by hand here, then gated: a fresh-subprocess check over the
transitive closure for all four headless modules.
All three fixes mutation-tested before being trusted green. The
Criterion 6 pair took three attempts, and the second is the useful
part: comparing a seek-reached frame against an autoplay-reached frame
at the same index PASSED against a deliberately frozen field, because
both sides came from the same run and were equally frozen. The working
version takes its reference from a separate play() window launched at
the target frame, and compares the field region only, cropping away the
scrub bar. Recorded as a standing rule in docs/practices.md: a
reference the run under test produced proves consistency, not
correctness.
Documentation the stage invalidated and never updated, all in files no
Stage 8 task opened:
- docs/architecture/CLAUDE.md said "all five" criteria met, falsified
two days later by the reopening that added four more.
- docs/repository-manifest.md said TASK-046/047 were "still unbuilt" --
stale from the day it was written, since both landed alongside the
TASK-045 the same sentence credits.
- docs/architecture/sequences.md (Checked-by: stage-boundary) described
only TASK-045/046/047, and still said playback had "no
declared-field/scalar-colormap path yet" two days after TASK-051
built exactly that. All four reopening tasks landed without it being
touched. Its Maintenance section now records why a task anchor cannot
reach a task the file has never named.
Also: pyflow play --help documented none of the controls the subcommand
exists for (pause, speed, scrub) -- README was the only place to find
them, now pinned by a test. Linux CI had no DISPLAY and no xvfb, so all
8 display-guarded tests skipped there while passing on Windows, making
a green two-platform matrix prove live-window behaviour on one
platform; now run under xvfb-run. 19 stray checkpoint_*.pt from a
2026-09-07 manual run removed from the repository root, and .gitignore
widened to match the files rather than only the default directory.
Not done, deliberately, and now recorded where a decision goes rather
than in a task's Design decisions: the Goal's "scrubbed to any point"
is still satisfied only within the window one pyflow play invocation
loads. The maintainer was asked at the audit and chose to defer.
Building it needs a new task and a new criterion, not a quiet widening
of Criterion 6. A user meets the boundary in pyflow play --help.
Verified live, not only unit-tested: pyflow play --help and pyflow
record --help run and checked by eye; a real 300-frame capped record
run leaves exactly {0, 290, 295, 300} on disk from 61 written.
1200 -> 1209 tests. make ci green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
That paragraph claims every edit to it has landed in the same change as the roadmap event it describes -- so the exit audit is an edit it owes. It also needed the distinction it was inviting a reader to miss: this section staying current says nothing about the three criteria and three documents that did not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Linux CI crashed a worker outright on the first run that had a virtual
display -- no Python traceback, the hard process abort GLFW produces
rather than an exception -- while the run immediately before it passed
the identical test code. Two runs of the same tests with opposite
results is contention, not a test failure: 8 xdist workers were each
creating software-GL contexts against one Xvfb display.
--dist loadgroup plus an xdist_group("display") mark on all 10
window-opening tests keeps them on one worker. Ordinary tests still
distribute exactly as before; the group is 10 of 1209, so nothing else
slows down (make ci: 183s, against 197s and 214s on the two runs before
it).
This is a hypothesis under test, not a confirmed fix -- the crash was
never reproducible locally, so CI on this branch is the only place it
can be judged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stage 8's end-of-stage exit audit, and the fixes it produced.
make ciwas green throughout — 1200 tests, 144 scenarios, 99% coverage — and three of the nine Completion Criteria were marked Met against checks that did not test what the criterion said. None needed judgement to find; each is a sentence the criterion had already written down. This isdocs/practices.md's own "An exit audit reads each criterion to its last sentence", recurring on the one stage whose previous audit is why Criteria 6-9 exist at all.The three criteria gaps
window.renderer.snapshot()'s content"snapshot(); both assert onPlaybackState.position, and one's docstring says sopyflow recordrun"--max-frames 20; TASK-049's by-hand check used 25renderingnor anything that transitively imports it"All three were true; none were gated. Each fix is mutation-tested.
The part worth reading
The Criterion 6 pair took three attempts, and the second one is the useful finding:
play()window launched at the target frame (where that frame is index 0, drawn by the initial scene build, never by the seek path), compared over the field region only, cropping away the scrub bar. Both halves then fail correctly — against a frozen field, and against an off-by-one field, checked separately.Recorded as a standing rule in
docs/practices.md: a reference the run under test produced proves consistency, not correctness.Stale documentation, all in files no Stage 8 task opened
docs/architecture/CLAUDE.mdsaid Stage 8 was complete with "all five" criteria met — falsified two days after it was written by the reopening that added four more.docs/repository-manifest.mdsaid TASK-046/047 were "still unbuilt" — stale from the day it was written, since both landed alongside the TASK-045 the same sentence credits.docs/architecture/sequences.md(Checked-by: stage-boundary) described only TASK-045/046/047, and still said playback had "no declared-field/scalar-colormap path yet" two days after TASK-051 built exactly that. All four reopening tasks landed without it being touched. Its Maintenance section now records why a task anchor cannot reach a task the file has never named.Also
pyflow play --helpdocumented none of the controls the subcommand exists for — pause, speed, scrub. README was the only place to find them. Now in the epilog, pinned by a test, including the line saying seeking stops at the loaded window.DISPLAY, noxvfb, so all 8 display-guarded tests — every check Criterion 6 rests on — skipped there while passing on Windows. A green two-platform matrix was proving live-window behaviour on one platform. Now run underxvfb-run. This is the one change in this PR I could not verify locally; watching the Linux job is the point of the PR.checkpoint_*.ptfrom a 2026-09-07 manual run removed from the repository root;.gitignorewidened to match the files, not only the default directory.Deliberately not done
The Goal's "scrubbed to any point" is still satisfied only within the window one
pyflow playinvocation loads. The maintainer was asked at the audit and chose to defer. That is now recorded as a standing exclusion under Criterion 6 — where a decision goes — rather than in a task's Design decisions, where it was hiding. Building it needs a new task and a new criterion, not a quiet widening of Criterion 6.1200 → 1209 tests.
make cigreen locally.🤖 Generated with Claude Code