Count shell reads and edits as evidence (CL-6937) - #559
Merged
TheGreatAxios merged 1 commit intoAug 23, 2026
Merged
Conversation
TheGreatAxios
force-pushed
the
cl-6936-re-read-thrash-hard-stop-at-rereadlimit-4-false-positives-on
branch
from
August 23, 2026 19:49
055690c to
928447d
Compare
TheGreatAxios
force-pushed
the
cl-6937-shell-based-reads-and-edits-count-as-zero-evidence-so-real
branch
from
August 23, 2026 19:49
5c8466a to
bdb9ced
Compare
The stop policy measured whether a worker did work by counting typed tool calls only. A worker that edited with `sed -i`, a heredoc, or `>` redirection had editedPaths empty and salvaged as never-edited — a HARD_BLOCK_SALVAGES class, so the parent was then refused an identical re-dispatch for the session. One that read with cat/head salvaged as incomplete-report. Both are real work classified as no work. The prompt does prohibit shell file work, but buildGrokLeafAntiThrashNote documents grok reaching for shell first anyway, and that is the family we run. A prompt violation should produce a correction, not a verdict that the work never happened. classifyShellFileEvidence lives in run-shell-authz.ts and reuses expandShellSubjects, so bash -c / env -S / xargs payloads are inspected rather than trusted. Writes are recognized from redirection as well as from the program name, since a missed write is exactly the false salvage this prevents, while a missed read costs a worker nothing. Stacked on cl-6936 (same files).
TheGreatAxios
force-pushed
the
cl-6936-re-read-thrash-hard-stop-at-rereadlimit-4-false-positives-on
branch
from
August 23, 2026 19:57
928447d to
d9efb08
Compare
TheGreatAxios
force-pushed
the
cl-6937-shell-based-reads-and-edits-count-as-zero-evidence-so-real
branch
from
August 23, 2026 19:57
bdb9ced to
4fd8c94
Compare
TheGreatAxios
changed the base branch from
cl-6936-re-read-thrash-hard-stop-at-rereadlimit-4-false-positives-on
to
main
August 23, 2026 20:00
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes CL-6937. Stacked on #558 (
cl-6936-…base) — both touchthrash.ts/stop-policy.ts, so they cannot be disjoint lanes. Merge #558 first; this PR then retargets tomain.evaluateSubAgentStopmeasured real work by typed tool calls only:requireEdit→editedPaths.size === 0→never-edited, aHARD_BLOCK_SALVAGESclass, so the parent is refused an identical re-dispatch for the rest of the session.requireEvidence→readCounts.size === 0→incomplete-report.So
sed -i, a heredoc, or>redirection was work classified as no work. The prompt does prohibit shell file work — butbuildGrokLeafAntiThrashNotedocuments grok reaching for shell first anyway, and grok is the family every sampled workbench session runs. A prompt violation should earn a correction, not a verdict that the work never happened.What changed
classifyShellFileEvidenceinsrc/shell/run-shell-authz.ts, reusingexpandShellSubjects— the same subject expansion the auto-shell policy uses — sobash -c,env -S, and xargs payloads are inspected rather than trusted. No new parser.sed -i,sed -i.bak,perl -pi -e),>/>>redirection including fused (>out.txt) and heredoc forms,tee,cp,mv,install,touch,truncate,patch,ln.cat,head,tail,grep/rg,sedwithout an in-place flag,awk,diff,find,jq, and friends. Flag values are skipped, sohead -n 40 freadsf, not5.shell:<program>— the checks only test non-emptiness.nextThrashStatefolds that intoreadCounts/editedPaths.The ticket's second option — demoting missing evidence from a hard block to a nudge — is deliberately not done here; the sticky-hard-block machinery is CL-6939 / CL-6704 / CL-6710.
Verification
bun run checkgreen: 5272 pass / 0 fail. 14 new tests covering the classifier and the two salvage paths (a shell-only run with a full envelope now completes under bothrequireEditandrequireEvidence).Review
needs-sawyer-review— Guardrail 4: touches stop/nudge policy. Not for auto-merge, and wants an eval matrix run before merge.Review fix applied: moved
classifyShellFileEvidence(and its program/flag tables and helpers) out ofsrc/shell/run-shell-authz.tsinto a newsrc/subagent/shell-evidence.ts— the authz module is the security boundary and evidence detection is a different concern with a single consumer (src/subagent/thrash.ts). ExportedtokenizeSegment,programBasename,HEAD_TAIL_VALUE_FLAGS, andGREP_VALUE_FLAGSfrom run-shell-authz.ts so the new module keeps reusing the shared parsing primitives instead of reimplementing them. Moved the corresponding tests tosrc/subagent/shell-evidence.test.ts.