fix: 2026-08-10 audit follow-up — count/flag/link accuracy, semantic passages - #9
Merged
Merged
Conversation
…passages
Accuracy fixes from the full library audit, each with paired clean +
hallucinated corpus cases:
- Comma-grouped numbers ("1,234") are one claim; decimals ("94.53") and
version components ("Python 3.10") are no longer claims at all.
- Short count-source labels ("api", "eps") are no longer silently dead;
short words match on exact word boundaries and only when the label has
no longer word, so stopwords in mixed labels ("number of tests") can't
drag unrelated numbers into the source.
- Flags inside single-span commands (`mytool --flag`) and bash/sh/shell/
console/zsh fences are now checked — previously only a flag backticked
alone was seen, despite a comment claiming otherwise.
- Markdown link titles ([text](path "Title")) and <angle-bracket> targets
no longer break path existence checks.
- The semantic layer judges against caller-declared VerifyContext.passages
instead of the content itself (vacuously faithful); with no passages the
judge is skipped with a warning. Bad-judge and no-judge warnings are
distinct messages.
- Import resolution passes the module name via argv (defense in depth).
- Corpus flag cases use the realistic single-span form.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Accuracy fixes from the 2026-08-10 full library audit: three numeric-claim false-positive classes stopped, two silent false-negative classes closed, markdown link titles handled, and the semantic layer now judges against real source passages instead of the content itself. Every fix carries paired clean + hallucinated corpus cases.
Fixed
tests=1234; grouped year-valued counts ("2,026 widgets") stay checked.`mytool --flag`and```bashfences were entirely unchecked; only a flag backticked alone was seen, despite a code comment claiming otherwise. Corpus flag cases updated to the realistic single-span form.<angle-bracket>targets no longer break path existence checks; dead paths with titles are still flagged.VerifyContext.passages(new field) instead of the content itself, which was vacuously faithful by construction; with no passages the judge is skipped with a warning. No-judge vs. protocol-mismatch warnings are now distinct messages.python -c(defense in depth).Test plan
--flag=value, console-fence help output, and year/comma interplay behave correctly.🤖 Generated with Claude Code