Skip to content

feat(vision): derive API-engine vision from the models.dev catalog#270

Merged
cukas merged 1 commit into
mainfrom
fix/api-vision-from-catalog
Jul 17, 2026
Merged

feat(vision): derive API-engine vision from the models.dev catalog#270
cukas merged 1 commit into
mainfrom
fix/api-vision-from-catalog

Conversation

@cukas

@cukas cukas commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

"kimi-for-coding-k3 can't see images" was a config gap, not a model limit: image gates checked only the engine JSON's declared capabilities:["vision"], and engines created from the models.dev catalog picker never carry capabilities — so every catalog API engine was treated as blind even when the catalog itself says the model accepts image input.

New single source of truth in core models-registry:

  • lookupCatalogModelAttachment(engineId): models.dev attachment via the same exact sanitized providerId-modelId match as the pricing lookup — CLI engines and hand-named engines are untouched; null = catalog doesn't know, never "no vision".
  • engineSupportsVision(engine): declared 'vision' capability ALWAYS wins (the user override for stale catalog rows — kimi-for-coding/k3 is attachment:false in the catalog while the underlying kimi-k3 is multimodal), otherwise catalog attachment === true.

All five vision gates now route through it: visionSupportNote (attach- time warning), session-resume (API base64 parts), session-companion (codex localImage), adapter structured API dispatch, and the API-agent execution plan (the last two found blocking 0.96/0.95 by codex in review — the first cut missed them). Exported from the core barrel.

Tests: catalog vision suite (override wins, catalog-true inherits, unknown stays declaration-only, missing cache degrades) + adapter regression for a catalog-attachment engine with NO declared capabilities; the adapter core-mock now defines engineSupportsVision explicitly (the spread actual resolves to built dist and may predate the helper).

Review: agon review --risk auto, 6/6 engines — codex's 2 blocking + 1 important all fixed; agy's missing-fs-imports claim refuted (imports exist, tests green).

⚔️ Forged by Agon

"kimi-for-coding-k3 can't see images" was a config gap, not a model
limit: image gates checked only the engine JSON's declared
capabilities:["vision"], and engines created from the models.dev catalog
picker never carry capabilities — so every catalog API engine was
treated as blind even when the catalog itself says the model accepts
image input.

New single source of truth in core models-registry:
- lookupCatalogModelAttachment(engineId): models.dev `attachment` via the
  same exact sanitized providerId-modelId match as the pricing lookup —
  CLI engines and hand-named engines are untouched; null = catalog
  doesn't know, never "no vision".
- engineSupportsVision(engine): declared 'vision' capability ALWAYS wins
  (the user override for stale catalog rows — kimi-for-coding/k3 is
  attachment:false in the catalog while the underlying kimi-k3 is
  multimodal), otherwise catalog attachment === true.

All five vision gates now route through it: visionSupportNote (attach-
time warning), session-resume (API base64 parts), session-companion
(codex localImage), adapter structured API dispatch, and the API-agent
execution plan (the last two found blocking 0.96/0.95 by codex in
review — the first cut missed them). Exported from the core barrel.

Tests: catalog vision suite (override wins, catalog-true inherits,
unknown stays declaration-only, missing cache degrades) + adapter
regression for a catalog-attachment engine with NO declared
capabilities; the adapter core-mock now defines engineSupportsVision
explicitly (the spread `actual` resolves to built dist and may predate
the helper).

Review: agon review --risk auto, 6/6 engines — codex's 2 blocking + 1
important all fixed; agy's missing-fs-imports claim refuted (imports
exist, tests green).

⚔️ Forged by [Agon](https://github.com/KERNlang/agon)

Co-Authored-By: agon (KERN) <292465531+KERN-Agon@users.noreply.github.com>
@cukas
cukas merged commit a84e104 into main Jul 17, 2026
2 checks passed
@cukas
cukas deleted the fix/api-vision-from-catalog branch July 17, 2026 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants