Skip to content

release: 0.3.0 — beta, with five verifier-accuracy fixes - #10

Merged
silversurfer562 merged 1 commit into
mainfrom
claude/attune-verify-beta-review-c6965c
Aug 11, 2026
Merged

release: 0.3.0 — beta, with five verifier-accuracy fixes#10
silversurfer562 merged 1 commit into
mainfrom
claude/attune-verify-beta-review-c6965c

Conversation

@silversurfer562

Copy link
Copy Markdown
Member

Promotes attune-verify to Development Status :: 4 - Beta and states an API compatibility promise for the deterministic core.

Why not just flip the classifier

A review pass over the library found five defects, all in the extractor, all in the two classes that actually matter for a fact-checker. Shipping "beta" over a checker that silently skips a whole shape of code block would have been the wrong claim to make.

Silent false negatives — a hallucination passing unchecked

  • Indented fences were invisible. A code block nested under a list item — how LLMs routinely write install steps — matched nothing, so every import and shell flag inside one went unchecked. The extractor is now a line scanner that accepts a fence at any indentation and strips that indent from the body; uniformly indented code otherwise fails ast.parse, which was the silent skip. This was the largest remaining hole.
  • Tilde fences (~~~) were unrecognized. CommonMark-legal, never extracted.

False positives — a real entity flagged

  • Percent-encoded link targets were checked literally, so a link to a file whose name contains a space (docs/my%20file.md) was flagged though the file exists. The raw form is still tried first, so a file genuinely named a%20b.md resolves, and a decoded target that escapes project_root is still caught.
  • Balanced parens in a link target were truncated at the first )docs/a(1).md was checked as docs/a(1.
  • Link syntax shown as an example inside a code span or fence was checked as a live link. No renderer resolves those. Found by running verify over its own README.

The scanner also enforces what the old regex ignored: an unclosed fence is not a fence (its body was the rest of the document), a closing run must match the opening character and length, and a line-leading inline span no longer opens one.

Beta hygiene

  • py.typed — the package is fully annotated but shipped no PEP 561 marker, so type checkers ignored it downstream. Verified present in the built wheel.
  • Version drift guard — the release flow bumps pyproject.toml and __version__ by hand; a test now pins them equal.
  • README — a per-checker table of what each checker settles and which truth source it needs, plus an explicit known-limitations list (reference-style links, short flags, bare-flag attribution, count keyword matching).

Verification

Gate Result
Tests 130 passed (was 99)
Coverage 96%, _extract.py 100%
Corpus precision/recall passing, 9 new regression cases
Mutation score 80.7% against the 75% gate
Lint ruff + black clean
Wheel builds, Development Status :: 4 - Beta, ships py.typed

Every fix carries a regression corpus case, per this repo's convention.

Not published — this stops at the PR.

🤖 Generated with Claude Code

Promotes attune-verify to Development Status :: 4 - Beta and states an
API compatibility promise for the deterministic core.

A beta-review pass over the library found five defects, all in the
extractor, and all of the two classes that matter for a fact-checker:

Silent false negatives (a hallucination passing unchecked)
- Indented fences were invisible. A code block nested under a list item
  — how LLMs routinely write install steps — matched nothing, so every
  import and shell flag inside one went unchecked. The extractor is now
  a line scanner that accepts a fence at any indentation and strips that
  indent from the body; uniformly indented code otherwise fails
  ast.parse, which was the silent skip. Largest remaining hole.
- Tilde fences (~~~) were unrecognized.

False positives (a real entity flagged)
- Percent-encoded link targets were checked literally, so a link to a
  file whose name contains a space was flagged though the file exists.
- Balanced parens in a link target were truncated at the first ')'.
- Link syntax shown as an example inside a code span or fence was
  checked as a live link. Found by running verify over its own README.

The scanner also enforces what the old regex ignored: an unclosed fence
is not a fence, a closing run must match the opening character and
length, and a line-leading inline span no longer opens one.

Beta hygiene: ship py.typed (the package is fully annotated but PEP 561
told type checkers to ignore it), pin __version__ against the pyproject
version (the release flow bumps both by hand), and document per-checker
behavior plus known limitations in the README.

Every fix carries a regression corpus case. 130 tests pass, coverage
96% (_extract.py 100%), mutation score 80.7% against a 75% gate.
@silversurfer562
silversurfer562 merged commit 5cd55d1 into main Aug 11, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant