Log every stop/nudge intervention (CL-6938) - #561
Merged
TheGreatAxios merged 1 commit intoAug 23, 2026
Merged
Conversation
TheGreatAxios
force-pushed
the
cl-6937-shell-based-reads-and-edits-count-as-zero-evidence-so-real
branch
from
August 23, 2026 19:49
5c8466a to
bdb9ced
Compare
TheGreatAxios
force-pushed
the
cl-6938-salvage-false-positive-rate-is-unmeasurable-log-every
branch
from
August 23, 2026 19:50
7390f7e to
ee51560
Compare
Fourteen stop reasons and ~20 injected-text interventions decide when a run is stuck, and there was no way to tell how often any of them was wrong. Four threshold judgments in this tree were later reverted, one on a justification the file itself retracts. interventions.jsonl in the firing worker's trace dir now records each intervention with its measured value beside the threshold it crossed, the provider/model/family, and the run state at that moment — turns used vs budget, tool calls, read and edit counts. A refused parent re-dispatch is recorded on the parent side, where no leaf run exists to record it. Writes are fire-and-forget and swallow their own errors: a diagnostic must not be able to fail a run. The sink defaults to a no-op, so nothing depends on logging being wired. scripts/intervention-forensics.ts aggregates across local sessions: per intervention counts by family, measured-value distribution against threshold, and two false-positive proxies — stops that fired on runs which had already edited files, and stops that fired before half the turn budget was spent. It lstats and skips symlinks so the `latest` session link cannot double-count. Stacked on cl-6937.
TheGreatAxios
force-pushed
the
cl-6937-shell-based-reads-and-edits-count-as-zero-evidence-so-real
branch
from
August 23, 2026 19:57
bdb9ced to
4fd8c94
Compare
TheGreatAxios
force-pushed
the
cl-6938-salvage-false-positive-rate-is-unmeasurable-log-every
branch
from
August 23, 2026 19:57
ee51560 to
7b8d247
Compare
TheGreatAxios
changed the base branch from
cl-6937-shell-based-reads-and-edits-count-as-zero-evidence-so-real
to
main
August 23, 2026 20:00
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes CL-6938. Stacked on #559 (which is stacked on #558) — same files.
This is the gate the project's standing rule points at: no threshold retuning ships before CL-6938 lands.
What it records
One JSONL record per intervention, in the firing worker's own trace dir (
interventions.jsonl):stop/nudge/block/clear)consecutiveIdentical=5 threshold=5,silenceMs=…,turnsCompleted=…Wired at:
no-progress,turn-budget,never-acted,never-edited,no-ship,incomplete-report(nudge and stop),report-forced,tool-failure-recovery,stall-nudge,stalled, and the run-loop-ownedrepetition/deadline/cancelledsalvages. Refused parent re-dispatches are recorded fromtask-tool.ts, since no leaf run exists there to record them.Writes are fire-and-forget and swallow their own errors — a diagnostic must not be able to fail a run — and the sink defaults to a no-op, so nothing depends on logging being wired.
Reading it back
bun run scripts/intervention-forensics.tsaggregates across local sessions:model-family-policy.tshas no data for today —kimiis a verbatim copy ofdefault)Neither proxy is proof alone; both are cheap and directional, which is what the tuning history has been missing. It
lstats and skips symlinks, so thelatestsession link cannot double-count.Verification
bun run checkgreen: 5277 pass / 0 fail. New tests cover the record shape, append ordering, failure-swallowing, and that the director actually emits a nudge and a stop with value ≥ threshold.Review
needs-sawyer-review— Guardrail 4: touches director/stop-policy internals. Not for auto-merge.Review fix applied: two changes. (a) Relabeled
editedWork/earlyBudgetinscripts/intervention-forensics.tsand the ARCHITECTURE.md paragraph as neutral context columns, not false-positive proxies — either is equally consistent with a correct stop or a wrong one. (b) Added a real outcome signal: the parent now appends oneoutcomeintervention record per completed dispatch (salvage kind or clean-complete, plus dispatch count), addedoutcometoInterventionClass, and taughtintervention-forensics.tsto report outcome counts by kind. CHANGELOG and ARCHITECTURE updated to describe the outcome records.