Skip to content

Tell a caller when a readable page still hides unread visual content - #170

Open
silverstein wants to merge 1 commit into
masterfrom
claude/unread-vector-routing
Open

silverstein wants to merge 1 commit into
masterfrom
claude/unread-vector-routing

Conversation

@silverstein

Copy link
Copy Markdown
Member

The gap

convert_pdf_to_markdown already reports VECTOR_CONTENT_NOT_INTERPRETED in its coverage gaps — in prose. Nothing structured carried it, so a vision-capable host reading the response saw pages_needing_vision: [] and was never told there was anything to look at.

Those two outputs are not in conflict, and I first misread them as such. pages_needing_vision answers "can this page be read at all without vision?" (path_segment_count >= 1000 && textLength < 30). Shannon page 40 is mostly text, so it correctly stays out.

What had no signal at all is the middle case: a page that reads fine and still contains something the converter did not interpret. Shannon's Table I is exactly that — five small plots nobody is ever told about.

What this adds

pages_with_unread_visual_content republishes the gap codes that page already reported, in a field a caller can act on. It forms no new opinion about the content, carries no geometry it cannot derive, and never claims the thing is a chart.

A separate field rather than a new reason inside pages_needing_vision, because that field means "unreadable without vision" and these pages are readable — saying otherwise would be a false statement made to fix a missing one, and it feeds the documentKind classifier.

Population — measured, and thin

Across the first three pages of 35 fixture documents plus the Shannon paper: exactly one document, one page.

Worth stating plainly rather than burying. But the fixture corpus is almost entirely synthetic — forms and small generated PDFs built to exercise code paths — so it contains nearly nothing resembling the documents where this matters (papers with figures, reports with charts). The corpus cannot really answer the question. The failure is demonstrated; the breadth is not.

I'd rather hand you that than a number dressed up to justify the change. If you'd sooner not carry a field on this evidence, closing it is a reasonable call and I won't re-litigate it.

Verification

160 tests green across the five affected suites, including a new live test derived from the real document. Tool contract digest re-pinned — the structured output changed; no tool name, description, input schema, or annotation did. Layout oracle regenerated: source-identity digests only, no scored case changed. Share mirror byte-identical across all three server files.

🤖 Generated with Claude Code

convert_pdf_to_markdown already reports VECTOR_CONTENT_NOT_INTERPRETED
in its coverage gaps, in prose. Nothing structured carried it, so a
vision-capable host reading the response saw pages_needing_vision: []
and was never told there was anything to look at.

Those two outputs are not in conflict and I first misread them as such.
pages_needing_vision answers "can this page be read at all without
vision" - path_segment_count >= 1000 and textLength < 30 - and Shannon
page 40 is mostly text, so it correctly stays out. What had no signal at
all was the middle case: a page that reads fine and still contains
something the converter did not interpret.

pages_with_unread_visual_content republishes the gap codes that page
already reported, in a field a caller can act on. It forms no new opinion
about the content, carries no geometry it cannot derive, and does not
claim the thing is a chart. A separate field rather than a new reason
inside pages_needing_vision, because that field means "unreadable without
vision" and these pages are readable - saying otherwise would be a false
statement made to fix a missing one, and it feeds the documentKind
classifier.

Population, measured rather than assumed: across the first three pages of
35 fixture documents plus the Shannon paper, exactly one document and one
page hit this case. That is thin, and worth saying plainly. The fixture
corpus is almost entirely synthetic - forms and small generated PDFs
built to exercise code paths - so it holds nearly nothing resembling the
documents where this matters, and it cannot really answer the question.
The honest position is that the failure is demonstrated and the breadth
is not.

Tool contract digest re-pinned; the structured output changed and no tool
name, description, input schema or annotation did. Layout oracle
regenerated, source-identity digests only, no scored case changed. Share
mirror byte-identical.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant