feat: run-integrity leak detection over finished transcripts - #84

Closed

dmorosanu wants to merge 31 commits into

mainfrom

feat/leak-detection

Commits on Aug 5, 2026

Add IntegrityInfo telemetry with run.json wiring and an INTEGRITY_MODE switch
dmorosanu
committed
5f0a8da
Count sub-agents whose inner tool calls could not be recovered
dmorosanu
committed
6579279
Add a graded-material read detector for finished transcripts
dmorosanu
committed
35aa79a
Void a passing row whose transcript read graded material
dmorosanu
committed
2b291db
Stop handing the scenario folder and the raw task YAML to the container
dmorosanu
committed
18b2397
Derive graded material from the task's declared mock and fixture directories
dmorosanu
committed
53891fc
Match a grader script by its resolved task-dir path and without an underscore
dmorosanu
committed
b2c8d8a
Stop classifying writes, removals and loop keywords as reads of graded material
dmorosanu
committed
06ebe51
Classify the file-editing tools as reads and the producing tools as neutral
dmorosanu
committed
4d147d8
Exclude voided replicates from the experiment pass rate and report their count
dmorosanu
committed
b2995a9
Classify git by subcommand so git show of an answer key is a read
dmorosanu
committed
5155625
Anchor a basename glob to a whole filename so it cannot match inside a longer name
dmorosanu
committed
e36c92a
Exclude voided rows from the headline pass rate, score mean and CI instead of counting them as failures
dmorosanu
committed
bf578cf
Split shell commands on unquoted separators only and drop comments and quoted heredoc bodies
dmorosanu
committed
4566af9
Parse redirects as unquoted operators that consume one word instead of partitioning on any angle bracket
dmorosanu
committed
7ca282c
Match literal reference paths and directories as whole paths instead of substrings
dmorosanu
committed
a373464
Match structured tool calls on their path parameters instead of every value joined
dmorosanu
committed
b1fea2b
Excuse basename-glob reads of paths the agent itself created earlier in the transcript
dmorosanu
committed
8699ae5
Classify a leading search utility by its flags before the nested-reader token sweep
dmorosanu
committed
e4885d8
Parse search-utility options with end-of-options and value operands instead of sweeping every token
dmorosanu
committed
a2e884e
Match a protected directory typed bare, letting utility semantics decide listing versus emitting
dmorosanu
committed
329a8a9
Keep a staged template mount exact instead of widening it to its first path component
dmorosanu
committed
26a0285
Match every task-dir spelling of a declared reference and strip quotes before matching
dmorosanu
committed
8ef049f
Stop flagging mock-shim execution and protected segments inside installed libraries
dmorosanu
committed
fdd3b15
Classify a Codex apply_patch by its change kinds so an update of graded material is a read
dmorosanu
committed
3c00d19
Classify command-substitution bodies as commands in their own right
dmorosanu
committed
7950174
Treat shell-state builtins as neutral so a PATH export naming the mock dir is not a read
dmorosanu
committed
3a107c1
Exclude voided replicates from the variant report's bootstrap confidence interval
dmorosanu
committed
aeeb7b8
Treat a mock path in the invocation prefix as shim execution, not a read
dmorosanu
committed
1a5f1ee
Unwrap a shell wrapper's command string so its redirects and pipelines classify as the command that ran
dmorosanu
committed
c9c47b9
Decide the mock-shim invocation carve-out before the utility rules so a quote-swallowed pipeline tail cannot turn execution into a read
dmorosanu
committed
7f82cb8