[Feature Request] Auto-switch to a vision-capable model (e.g. `gemini-3.5`) when an image is uploaded

[Rosecheng7]

Harness: dsh 0.1.0-rc.6 (dsh web profile)
Environment: macOS (Apple Silicon), Node v25.9.0

Summary

When a user uploads an image in the web UI, the harness does not automatically switch to a vision-capable model. The current model (e.g. a text-only model) stays selected, so any subsequent request that references the image either fails or produces a useless response — the user has to manually switch to a model that can see images (e.g. gemini-3.5) before the image is actually usable.

Current behavior

1. User is on a text-only model
2. User uploads an image via the UI.
3. The model stays unchanged; the image attachment is effectively ignored (or the request errors out).
4. User must notice the failure, manually open the model picker, and select a vision-capable model like gemini-3.5, then retry.

Expected behavior

• When an image is uploaded, the harness automatically switches to a vision-capable model (e.g. gemini-3.5) — or at minimum shows a clear, one-click prompt: "This model can't see images. Switch to gemini-3.5?"
• The switch should be non-destructive to the current conversation: keep the conversation history and the uploaded image, just swap the model for subsequent turns.
• If multiple vision-capable models are configured, pick the first available one (or the one the user last used), and surface a hint in the UI showing which model was auto-selected.
• Conversation history and the uploaded image are preserved across the switch.
• The UI indicates which model is now active after the auto-switch.
• Users can opt out of auto-switching if they prefer manual control.

[VickyXAI]

Real problem, but the half of it that belongs to this plugin is already shipped, and the other half is the harness's model picker rather than the route.

What this plugin owns: telling the truth about which models can see. Every model this route publishes carries inputModalities, and the image modality is declared from a measured list, not the gateway's vision tag. Ten tagged models were sent the same inline PNG: Google's and Moonshot's answered correctly, OpenAI's returned HTTP 400 after taking payment, xAI's returned 503, and Anthropic's charged and streamed nothing at all. So the verified set is:

google/gemini-2.5-flash
google/gemini-3.5-flash
google/gemini-3.6-flash
moonshot/kimi-k3

(visionModels widens this without waiting for a release, as the gateway improves.) Note that gemini-3.5 is not a model id here — it is google/gemini-3.5-flash.

What this plugin cannot do: change the session's model. Selecting a model is a DSH decision, and a plugin reaching across to swap it mid-conversation would be the wrong place for that logic. The information the picker needs is already on the models it lists, so the UI can gate the upload or offer the switch without any further work on this side.

On auto-switching itself, I would push back on the "automatically" part. Changing the model changes the price, the tone, the tool behaviour, and the reasoning support of every subsequent turn — silently, in response to a drag-and-drop. Your fallback wording is the better default: "This model can't see images. Switch to google/gemini-3.5-flash?" — one click, conversation and attachment preserved, and the active model shown afterwards. That is exactly what your own expected-behaviour list asks for at minimum, and I think it should be the maximum too, with opt-in auto-switch for people who want it.

The right home for the switch is the DSH web UI (deepseek-ai/deepseek-harness), since it owns the picker, the attachment surface, and the session state. Worth filing there — the modality data it would need is already published by this route, which makes it a UI change rather than a capability one. Happy to be pointed at that issue if you open it.

[VickyXAI]

Followed up upstream, and checked the harness source while I was there — the current behaviour is better than described, but your ask still stands.

DSH does not silently ignore the attachment. It refuses the send:

• packages/host/apiproxy/src/api-proxy.ts:2488 returns attachment-error / MODEL_DOES_NOT_SUPPORT_IMAGES when the active model's inputModalities lacks image
• packages/host/apiproxy/src/api-proxy.ts:2299 returns model-unavailable if you switch to a text-only model while the session already holds images
• packages/fs/tool-fs/src/read-image.ts:73 refuses the read_image tool on the same basis

So the image is never quietly dropped. What is missing is exactly the second half of your request: the refusal never names a model that would work, and there is no one-click switch — which is why it reads as "the harness did nothing useful."

Filed there: deepseek-ai/deepseek-harness#2911 — proposing the refusal carry candidate image-capable models in its details, the web UI render that as "switch to X and resend" with history and attachment preserved, and the model picker mark image-capable entries. Your issue is linked in it. I argued against silent auto-switching as the default for the reason in my earlier comment; a one-click confirm gets you the same result without changing price and behaviour behind a drag-and-drop.

Leaving this open until the upstream thread lands. The data side is already done on this route: inputModalities is declared from the measured set, so anything the picker needs is published today.