Tell a caller when a readable page still hides unread visual content - #170
Open
silverstein wants to merge 1 commit into
Open
silverstein wants to merge 1 commit into
silverstein wants to merge 1 commit into
Conversation
convert_pdf_to_markdown already reports VECTOR_CONTENT_NOT_INTERPRETED in its coverage gaps, in prose. Nothing structured carried it, so a vision-capable host reading the response saw pages_needing_vision: [] and was never told there was anything to look at. Those two outputs are not in conflict and I first misread them as such. pages_needing_vision answers "can this page be read at all without vision" - path_segment_count >= 1000 and textLength < 30 - and Shannon page 40 is mostly text, so it correctly stays out. What had no signal at all was the middle case: a page that reads fine and still contains something the converter did not interpret. pages_with_unread_visual_content republishes the gap codes that page already reported, in a field a caller can act on. It forms no new opinion about the content, carries no geometry it cannot derive, and does not claim the thing is a chart. A separate field rather than a new reason inside pages_needing_vision, because that field means "unreadable without vision" and these pages are readable - saying otherwise would be a false statement made to fix a missing one, and it feeds the documentKind classifier. Population, measured rather than assumed: across the first three pages of 35 fixture documents plus the Shannon paper, exactly one document and one page hit this case. That is thin, and worth saying plainly. The fixture corpus is almost entirely synthetic - forms and small generated PDFs built to exercise code paths - so it holds nearly nothing resembling the documents where this matters, and it cannot really answer the question. The honest position is that the failure is demonstrated and the breadth is not. Tool contract digest re-pinned; the structured output changed and no tool name, description, input schema or annotation did. Layout oracle regenerated, source-identity digests only, no scored case changed. Share mirror byte-identical. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gap
convert_pdf_to_markdownalready reportsVECTOR_CONTENT_NOT_INTERPRETEDin its coverage gaps — in prose. Nothing structured carried it, so a vision-capable host reading the response sawpages_needing_vision: []and was never told there was anything to look at.Those two outputs are not in conflict, and I first misread them as such.
pages_needing_visionanswers "can this page be read at all without vision?" (path_segment_count >= 1000 && textLength < 30). Shannon page 40 is mostly text, so it correctly stays out.What had no signal at all is the middle case: a page that reads fine and still contains something the converter did not interpret. Shannon's Table I is exactly that — five small plots nobody is ever told about.
What this adds
pages_with_unread_visual_contentrepublishes the gap codes that page already reported, in a field a caller can act on. It forms no new opinion about the content, carries no geometry it cannot derive, and never claims the thing is a chart.A separate field rather than a new reason inside
pages_needing_vision, because that field means "unreadable without vision" and these pages are readable — saying otherwise would be a false statement made to fix a missing one, and it feeds thedocumentKindclassifier.Population — measured, and thin
Across the first three pages of 35 fixture documents plus the Shannon paper: exactly one document, one page.
Worth stating plainly rather than burying. But the fixture corpus is almost entirely synthetic — forms and small generated PDFs built to exercise code paths — so it contains nearly nothing resembling the documents where this matters (papers with figures, reports with charts). The corpus cannot really answer the question. The failure is demonstrated; the breadth is not.
I'd rather hand you that than a number dressed up to justify the change. If you'd sooner not carry a field on this evidence, closing it is a reasonable call and I won't re-litigate it.
Verification
160 tests green across the five affected suites, including a new live test derived from the real document. Tool contract digest re-pinned — the structured output changed; no tool name, description, input schema, or annotation did. Layout oracle regenerated: source-identity digests only, no scored case changed. Share mirror byte-identical across all three server files.
🤖 Generated with Claude Code