Skip to content

feat: B6 — first-class OSS model support (Kimi K3, GLM-5.2, DeepSeek V4, Qwen3.7) #296

Description

@zjshen14

Roadmap B6. The B1 transport plumbing (--provider, --base-url) already shipped, and src/providers/openai.ts is a generic chat-completions client, so OSS models are reachable today. But nothing in the codebase knows about them, which produces four defects.

Defects

1. Silent wrong-provider fallback. detectProvider() in src/providers/factory.ts returns "gemini" for any unrecognised model name. --model kimi-k3 or --model glm-5.2 silently builds a Gemini client and fails with a confusing error. Every OSS model hits this.

2. Context windows are wrong by 10x. MODEL_CONTEXT_WINDOWS in src/core/compact.ts has no OSS entries, so they fall back to DEFAULT_CONTEXT_WINDOW = 100_000. Kimi K3, GLM-5.2, DeepSeek V4 and Qwen3.7-Max all ship 1M-token windows. Auto-compact fires at ~75k on a model with 13x that headroom, destroying context for no reason.

3. API key collision. resolveApiKey() in src/cli/keys.ts is a hardcoded 3-way branch. Using Kimi today means --provider openai --base-url https://api.moonshot.ai/v1 with a Moonshot key in OPENAI_API_KEY — which breaks as soon as you also want real OpenAI.

4. Redundant think tool. hasNativeThinking() only matches Gemini patterns. Kimi K3 has thinking always-on; injecting a think tool wastes tokens and muddies tool selection.

Proposal

A single provider/model registry (src/providers/registry.ts) that becomes the one source of truth for: wire format, base URL, API-key env vars, context window, and native-thinking capability. detectProvider(), contextWindowFor(), hasNativeThinking() and resolveApiKey() all read from it instead of maintaining four scattered tables.

Target: --provider moonshot / --model glm-5.2 work out of the box with the right key, URL, and context window.

Providers in scope

Preset Wire Base URL Key env
moonshot openai https://api.moonshot.ai/v1 MOONSHOT_API_KEY
zai openai https://api.z.ai/api/coding/paas/v4 ZAI_API_KEY
deepseek openai https://api.deepseek.com DEEPSEEK_API_KEY
dashscope openai https://dashscope.aliyuncs.com/compatible-mode/v1 DASHSCOPE_API_KEY
openrouter openai https://openrouter.ai/api/v1 OPENROUTER_API_KEY
ollama openai http://localhost:11434/v1 (none)

Design doc to follow at docs/design/b6-oss-models.md.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions