docs(eval): record forward-only rescore posture in spec 9.6 - #13
Merged
Merged
Conversation
The one-shot raw outputs runs/out/2026-09-03 and runs/out/2026-09-05 are confirmed absent and unrecoverable, so the historical one-shot rows from those dates are permanently non-rescorable. State that a rescore reads only the row's own embedded artifact.output, that there is no external runs/out/ ingestion path, and that a pre-embedding row is refused by the ordinary rule with no date or era special case.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The rescoring contract in spec 9.6 did not state whether the historical one-shot rows can be rescored. A reader could not tell whether the deferred
eval rescorecommand would ever ingest a recoveredruns/out/raw output. This records the settled posture so no recovery/ingestion path is expected.What changed
Added one bullet to spec 9.6 (Reports and cost guard). It records that:
artifact.output, so rescoring is forward-only, with no path that ingests an externalruns/out/raw file back into a row;runs/out/2026-09-03andruns/out/2026-09-05are confirmed absent and unrecoverable, so the historical one-shot rows from those two dates are permanently non-rescorable;artifact.outputembedding is refused by the ordinary rule with no date or era special case, the same way section 9.5 treats a missing artifact as incomparable.Documentation only. No code, no CLI, no snapshot contract, and no committed evidence changed. The
eval rescorecommand remains deferred and unimplemented.How I checked
PASS encoding(crlf: 0,doubleEncoded: 0,nul: 0). No gate reads this prose as a threshold or lint target. Confirmed the edit touches only the spec 9.6 rescore bullet and leaves the section 9.5 S3-max and trajectory-selector prose intact.What I left alone
No weakened gate, no
vitest -u, no rewritten committed evidence, no refactor. Did not add or spec a rescore command, and did not touch the C2 changes (exact trajectory selector, S3 max gate) merged in #12. Spends no money and changes no public CLI or snapshot contract.