Stage 8 exit audit: close three criteria gaps, three stale documents

.github/workflows/ci.yml

@@ -77,6 +77,34 @@ jobs:
       - name: make install
         run: make install
 
+      # **No `xvfb` here, and that is a recorded decision rather than an
+      # omission** -- so the 10 display-guarded tests
+      # (`@_needs_a_real_display` in `tests/integration/
+      # test_interactive_window.py` and `test_playback_cli.py`) skip on
+      # Linux and run only on Windows. Without a `DISPLAY`,
+      # `_display_available()` short-circuits before GLFW is ever
+      # constructed (deliberately: GLFW hard-aborts the process rather
+      # than raising, see `tests/integration/CLAUDE.md`).
+      #
+      # **Tried and reverted 2026-09-11/12 by the Stage 8 exit audit.**
+      # Installing `xvfb` and running `xvfb-run -a make ci` does work --
+      # Linux skips dropped from 29 to 18, matching a local run, so all
+      # 10 tests genuinely ran. It was reverted because it made the
+      # Linux job flaky: across three attempts on identical test code,
+      # two passed and one crashed an xdist worker outright, with no
+      # Python traceback (the hard process abort GLFW produces), and the
+      # crashed one then passed on a plain re-run. 8 workers creating
+      # software-GL contexts against one Xvfb display concurrently is the
+      # working hypothesis, never reproduced locally.
+      #
+      # **Serialising the display tests onto one worker
+      # (`--dist loadgroup`) is not the fix, and made things worse**: it
+      # hung the *Windows* job for 5h45m until GitHub's own 6-hour limit
+      # killed it, at 95% with ~6 tests outstanding and an orphaned
+      # python process at cleanup. One process creating ~15 GLFW windows
+      # in sequence is the failure that buys; 1-2 per process spread
+      # across workers is what has always worked. See
+      # `docs/planning/backlog.md` for the open item.
       - name: make ci
         env:
           # Runs with more test parallelism than a local `make test`

.gitignore

@@ -46,3 +46,16 @@ checkpoints/
 # `replay.py`) -- README's own documented example uses `--cache cache`
 # from the repository root; same reasoning as `checkpoints/` above.
 cache/
+
+# The same run output as `checkpoints/` above, but written straight into
+# whatever directory `pyflow record`/`resume` was invoked from -- what
+# `--output-dir .` produces, and what a user experimenting from the
+# repository root gets. **Added 2026-09-11 by the Stage 8 exit audit**,
+# which found 19 of these sitting untracked in the repository root from
+# a 2026-09-07 manual run: the `checkpoints/` rule above only ever
+# matched the default directory, never the files themselves.
+checkpoint_*.pt
+
+# `pyflow play --cache`'s own materialized windows, for the same reason
+# -- `window_{from}_{to}.pt` written to a cache directory given as `.`.
+window_*.pt

README.md

@@ -148,12 +148,25 @@ the roadmap rather than restated here:
 
 **Stage 8's own record, for anyone tracking how reliably this section
 stays current**: opened and closed in one day (2026-09-07), reopened
-two days later, and reclosed the same day it was reopened -- each edit
-to this paragraph has so far landed in the same change as the roadmap
-event it describes, unlike the multi-day staleness windows the two
-paragraphs below describe for Stages 7 and 8's own *earlier* drafts.
-Don't read this as the pattern solved; read Stage 9's own eventual
-entry here as the next real test of it.
+two days later, reclosed the same day it was reopened, and audited at
+its exit on 2026-09-11 -- each edit to this paragraph has so far landed
+in the same change as the roadmap event it describes, unlike the
+multi-day staleness windows the two paragraphs below describe for
+Stages 7 and 8's own *earlier* drafts. Don't read this as the pattern
+solved; read Stage 9's own eventual entry here as the next real test of
+it.
+
+**And read the exit audit before trusting that record too far.** This
+paragraph tracks whether *this section* stays current, which it has.
+What the audit found is that three other places did not: three of Stage
+8's nine Completion Criteria were marked Met against checks that did not
+test what the criterion said, and three documents outside this file
+still described the pre-reopening stage -- one of them a
+`Checked-by: stage-boundary` document that all four reopening tasks
+landed without touching. All six are fixed; `docs/planning/roadmap.md`'s
+own **Exit audit, 2026-09-11** section is the record. **A section that
+keeps itself current is not evidence that the repository around it
+did.**
 
 **This sentence said "not yet started" for Stage 8 itself, twice, while
 that stage was still open** -- once for the same reason a fourth time as

docs/architecture/CLAUDE.md

@@ -42,12 +42,20 @@ all three of that stage's pieces in real code:
 `src/pyflow/checkpoint.py`/`recording.py`/`simulation_run.py`
 (TASK-045, checkpointing), and `src/pyflow/replay.py`/`playback.py`
 (TASK-046/047, windowed replay and interactive playback, built the same
-day). Stage 8 itself is complete, all five of its own Completion
+day). Stage 8 itself is complete, all **nine** of its own Completion
 Criteria met -- see `docs/planning/roadmap.md`'s own Stage 8 Status
 section for the one real course-correction along the way (its Golden
 Demo moved from Heat Diffusion to Lid-Driven Cavity once TASK-047's own
 scope decision made the first choice incompatible).
 
+**This paragraph said "all five" until 2026-09-11**, correct when
+written on 2026-09-07 and falsified two days later when the stage was
+reopened and Criteria 6-9 were added (TASK-048/049/050/051). Found by
+that stage's own exit audit, not by anything that runs -- a criteria
+*count* is exactly the decaying-number shape `docs/practices.md`
+warns about under "A stage's documentation sweep is a grep, not a diff
+review", restated here in a file no Stage 8 task ever opened.
+
 **Section 3's anchor history is worth keeping, because it is why the
 subsection existed to be finished at all.** It used to say the
 subsection was "anchored to the specific roadmap task that will build it

docs/architecture/sequences.md

@@ -384,6 +384,9 @@ sequenceDiagram
         alt frame_count % checkpoint_interval == 0, or final frame
             recording->>checkpoint: write_checkpoint(frame_count, config, state.fields)
             checkpoint->>Disk: checkpoint_{frame_count:08d}.pt
+            opt max_checkpoints_retained set (TASK-049)
+                recording->>Disk: _prune_checkpoints() -- keep frame 0 + newest N
+            end
         end
     end
 ```
@@ -443,6 +446,9 @@ sequenceDiagram
         alt frame_count % checkpoint_interval == 0, or final frame
             recording->>checkpoint: write_checkpoint(frame_count, config, state.fields)
             checkpoint->>Disk: checkpoint_{frame_count:08d}.pt
+            opt max_checkpoints_retained set (TASK-049)
+                recording->>Disk: _prune_checkpoints() -- keep frame 0 + newest N
+            end
         end
     end
 ```
@@ -472,6 +478,18 @@ nothing. Those are Stage 8's own second and third bullets, and both are
 built now (TASK-046/047, 2026-09-07): the rest of this subsection
 covers them.
 
+**Retention prunes after every write, not once at the end** (TASK-049,
+added to both diagrams above 2026-09-11) -- which is what makes
+`max_checkpoints_retained` bound peak disk use *as a long run grows*
+rather than only its final footprint. A 300-frame run at `interval=5`
+with a cap of 3 writes 61 checkpoints and never holds more than 5 at
+once. Opt-in: unset (the default) deletes nothing, so every config
+written before TASK-049 produces exactly the files it always did. Frame
+0 is excluded from the count itself rather than merely surviving by
+being newest, so a capped recording always keeps a point to restart
+from. `resume` prunes the whole `output_dir`, including checkpoints an
+earlier `record` left there, not only the files it wrote itself.
+
 ### Built today: windowed replay and interactive playback (`pyflow play`)
 
 ```mermaid
@@ -487,6 +505,10 @@ sequenceDiagram
     playback->>replay: materialize_or_load_window(...)
     alt --cache given and an exact-range match exists
         replay->>Disk: read_materialized_window(cache_path)
+    else --cache given and a cached window is a strict superset (TASK-050)
+        replay->>replay: _find_superset_window(cache_dir, from_frame, to_frame)
+        replay->>Disk: read_materialized_window(superset_path)
+        replay->>replay: slice out [from_frame, to_frame] -- no re-simulation
     else materialize fresh
         replay->>replay: find_checkpoint_at_or_before(checkpoints_dir, from_frame)
         replay->>checkpoint: read_checkpoint() + restore_simulation_state()
@@ -499,13 +521,23 @@ sequenceDiagram
     end
     replay-->>playback: MaterializedWindow (config, frames[])
     playback->>Window: RenderWindow(config.rendering), build_vector_field_arrows(frames[0])
+    opt config declares fields (TASK-051)
+        playback->>Window: field_visualization.panel_colors/build_panel_legend per panel
+    end
+    playback->>Window: scrub bar track + thumb (gfx.Line + gfx.Points)
     Window-->>Window: window.playback_state = PlaybackState()
     playback->>Window: canvas.add_event_handler(_on_key, "key_down")
+    playback->>Window: canvas.add_event_handler(pointer_down/move/up) -- scrub drag (TASK-048)
     loop each real draw
-        Window->>playback: on_frame()
-        playback->>playback: advance_playback_position() -- position += speed unless paused
+        alt a seek event arrived (TASK-048)
+            Window->>playback: _on_key ArrowLeft/Right -> seek_relative, Home/End -> seek_to
+            Window->>playback: pointer drag on the bar -> frame_index_from_fraction -> seek_to
+        else no input
+            Window->>playback: on_frame()
+            playback->>playback: advance_playback_position() -- position += speed unless paused
+        end
         opt materialized frame index changed
-            playback->>Window: remove old arrows, build_vector_field_arrows(frames[index])
+            playback->>Window: rebuild arrows, panels and thumb from frames[index]
         end
     end
 ```
@@ -517,20 +549,43 @@ window` (what `play()` actually calls) is the only place a cache is
 read or written, and only when `--cache DIR` is given. Nothing is
 written to disk by a bare `pyflow play`.
 
-**Scoped to solved-velocity-only rendering for this first cut** --
+**Arrows and declared-field panels, from the same materialized frame.**
 `playback.py` builds `gfx` arrows from `MaterializedWindow.frames[i]`'s
 `velocity.0`/`velocity.1` tensors the same way `bootstrap.py`'s own
-`_add_solved_velocity_rendering` does from a live `SimulationState`, but
-has no declared-field/scalar-colormap path yet
-(`UnsupportedPlaybackConfigError` otherwise) -- see `src/pyflow/
-CLAUDE.md`'s own `playback.py` entry for the full reasoning and the
-scene-rebuild-cost measurements that shaped the speed mechanism
-(`position += speed`, not more draws per second).
-
-**Every subsection in this section is now built.** `sequences.md`'s own
-Maintenance note, below, no longer names a task to re-read this file
-for -- update it again the next time Stage 8 gains a fourth piece, or
-whenever any task named in this section is touched.
+`_add_solved_velocity_rendering` does from a live `SimulationState`, and
+-- since TASK-051 -- colours any declared field's panels from that same
+frame, through the `rendering/field_visualization.py` helpers
+`bootstrap.py`'s own live path calls rather than a second copy of them.
+A config with **no** solved velocity at all is still rejected
+(`UnsupportedPlaybackConfigError`); see `src/pyflow/CLAUDE.md`'s own
+`playback.py` entry for the full reasoning and the scene-rebuild-cost
+measurements that shaped the speed mechanism (`position += speed`, not
+more draws per second).
+
+**Seeking is bounded by the window loaded at launch.** `seek_relative`/
+`seek_to` clamp to `[0, max_index]` of the materialized window, so
+Home/End reach that window's own edges and not the recording's --
+reaching anything outside it still needs a fresh `pyflow play` with
+different `--from-frame`/`--to-frame`. A stated Stage 8 scope boundary,
+not an oversight: `docs/planning/roadmap.md`'s own Completion Criterion
+6 records it, and `pyflow play --help` tells a user so directly.
+
+**Every subsection in this section is built.** Seven Stage 8 tasks are
+covered here now: TASK-045 (recording, `resume`), TASK-049 (retention),
+TASK-046 (windowed replay) and TASK-050 (subset cache reuse) above,
+TASK-047 (playback) and TASK-048 (live scrub) and TASK-051 (declared
+fields in playback) in this subsection.
+
+**This subsection described only TASK-046/047 until 2026-09-11**, and
+one of its paragraphs ("scoped to solved-velocity-only rendering for
+this first cut... no declared-field/scalar-colormap path yet") had been
+false since TASK-051 landed on 2026-09-09. It was found by that stage's
+own exit audit rather than by the re-read this file's own closing note
+below asked for -- the note said to "update it again the next time
+Stage 8 gains a fourth piece", Stage 8 then gained four pieces, and
+nothing re-read it. **A `Checked-by: stage-boundary` declaration is a
+promise about the stage boundary, not about the tasks in between**,
+which is exactly how four tasks landed without touching this file.
 
 ---
 
@@ -653,3 +708,30 @@ playback are both real, built sequences in Section 3 now, not Planned in
 substance under a different heading; that section's own closing note
 says so and names no further task, since Stage 8 has nothing left
 undrafted to anchor to.
+
+**And then the mechanism failed a second time, in the one way the
+paragraphs above did not anticipate: no anchor existed to fail.**
+Stage 8 was reopened on 2026-09-09 and gained four more tasks
+(TASK-048/049/050/051 -- live scrub, checkpoint retention, subset cache
+reuse, combined declared-field playback). All four landed without this
+file being touched, and one of Section 3's own paragraphs was left
+saying playback had "no declared-field/scalar-colormap path yet" two
+days after TASK-051 built exactly that. Found 2026-09-11 by the Stage 8
+exit audit; Section 3 now covers all seven tasks.
+
+**Why the existing lesson did not reach it.** Every mechanism described
+above keys off a *named task* in this file -- "grep this file's own
+TASK-NNN mentions the next time any named task is touched". TASK-048-051
+were new tasks this file had never named, so there was nothing to grep
+and nothing to notice. The paragraph that should have caught them was
+the closing note's own "update it again the next time Stage 8 gains a
+fourth piece", which is a standing instruction addressed to nobody in
+particular and fired for no one.
+
+**So the rule this file actually needs is the one its own header already
+declares:** `Checked-by: stage-boundary` means this document is re-read
+when a stage opens or closes, and a stage that *reopens* is a stage
+boundary twice over. The reopening on 2026-09-09 was the moment to
+re-read this file, and the audit two days later was the second. Neither
+a task anchor nor a "next time" note substitutes for that, because both
+depend on a reader who already knows to look here.

docs/planning/backlog.md

@@ -2427,6 +2427,54 @@ here.):
       actually built (recording, replay, playback's own first cut) and
       does not track the reopening's own progress.
 
+- [ ] **Run the display-guarded tests on Linux CI.** Raised 2026-09-12 by
+      the Stage 8 exit audit, which found the gap, tried two fixes, and
+      reverted both. The 10 tests carrying `@_needs_a_real_display`
+      (`tests/integration/test_interactive_window.py`,
+      `test_playback_cli.py`) skip on `ubuntu-latest` and run only on
+      `windows-latest`, so **a green two-platform matrix proves
+      live-window behaviour on one platform** -- including every check
+      Stage 8's own Completion Criterion 6 rests on.
+
+      **Not blocking**: the tests do run, on Windows, every CI run, and
+      they are the platform the maintainer develops on. What is lost is
+      the second platform's confirmation, which for GLFW/wgpu behaviour
+      is exactly where the two have diverged before
+      (`tests/integration/CLAUDE.md`'s own 2026-08-19 SIGABRT entry).
+
+      **Both obvious approaches are already known to fail, with
+      measurements** -- read `tests/integration/CLAUDE.md` and
+      `.github/workflows/ci.yml`'s own comment before attempting this,
+      rather than rediscovering them:
+
+      1. **`xvfb` works and is flaky.** `xvfb-run -a make ci` plus an
+         `xvfb` apt package dropped Linux skips from 29 to 18, matching a
+         local run -- so all 10 tests genuinely ran. Across three
+         attempts on *identical* test code, two passed and one crashed an
+         xdist worker with no Python traceback (GLFW's hard process
+         abort), and that attempt then passed on a plain re-run. Roughly
+         a third of attempts, on a gating check.
+      2. **Serialising them onto one worker is worse.** `--dist
+         loadgroup` plus an `xdist_group("display")` mark hung the
+         *Windows* job for 5h45m until GitHub's own 6-hour limit killed
+         it -- 95% complete, ~6 tests outstanding, an orphaned python
+         process at cleanup. It passed `make ci` locally in 183s, so
+         nothing short of CI could have caught it.
+
+      **The unblock condition is a hypothesis worth testing, not a
+      design**: both failures point the same way from opposite
+      directions -- approach 1 raised *concurrent* window creation across
+      processes, approach 2 raised *sequential* window creation within
+      one process, and each broke a different platform. What has always
+      worked is 1-2 GLFW windows per process, spread across workers. So
+      the likely route is **reducing the number of windows rather than
+      redistributing them**, starting with the two
+      `*_rerenders_the_field_in_real_pixels` tests, which open two each
+      (a main window plus an independent reference window) and are the
+      highest count in the suite. Whether that alone takes the Linux
+      flake rate to zero is unmeasured; the flake was never reproducible
+      locally, so CI is the only place any of this can be judged, which
+      is what makes it a task rather than a quick fix.
 ---
 
 # Part III — Audit history