Area
Provider adapter
What are you trying to accomplish?
I use several text-only routed models through the vision sidecar, which describes images before feeding them to the text-only model. Right now my only choice for the "describer" is an OpenAI-native or Anthropic model that runs through the ChatGPT forward backend. I'd like to be able to pick any configured provider's vision model — for instance a Doubao/volcengine vision model I already have configured in opencodex — as the describer.
What prevents this today?
The vision sidecar (visionSidecar.model) sends the describe request through forwardProvider (the ChatGPT forward backend) to /responses, so model is effectively limited to OpenAI-native (or Anthropic) model ids. A model hosted on a custom provider (cliproxy, volcengine/doubao, etc.) can't be selected because the describe path doesn't know how to reach that provider.
What should OpenCodex do?
Allow the vision-sidecar description model to be resolved against any configured provider, not just the forward/ChatGPT backend — for example, per-provider or a routed model id for the describer, so a Doubao/volcengine (or any other) vision-capable model can act as the image describer for text-only routed models. This mirrors how noVisionModels lets operators choose which models get the sidecar; the companion ask is letting them choose who does the describing.
Checks
Area
Provider adapter
What are you trying to accomplish?
I use several text-only routed models through the vision sidecar, which describes images before feeding them to the text-only model. Right now my only choice for the "describer" is an OpenAI-native or Anthropic model that runs through the ChatGPT forward backend. I'd like to be able to pick any configured provider's vision model — for instance a Doubao/volcengine vision model I already have configured in opencodex — as the describer.
What prevents this today?
The vision sidecar (
visionSidecar.model) sends the describe request throughforwardProvider(the ChatGPT forward backend) to/responses, somodelis effectively limited to OpenAI-native (or Anthropic) model ids. A model hosted on a custom provider (cliproxy, volcengine/doubao, etc.) can't be selected because the describe path doesn't know how to reach that provider.What should OpenCodex do?
Allow the vision-sidecar description model to be resolved against any configured provider, not just the forward/ChatGPT backend — for example, per-provider or a routed model id for the describer, so a Doubao/volcengine (or any other) vision-capable model can act as the image describer for text-only routed models. This mirrors how
noVisionModelslets operators choose which models get the sidecar; the companion ask is letting them choose who does the describing.Checks