Skip to content

Offer the models the account can actually reach in the model picker - #241

Merged
pufit merged 1 commit into
mainfrom
pufit/model-picker-discovery
Aug 1, 2026
Merged

Offer the models the account can actually reach in the model picker#241
pufit merged 1 commit into
mainfrom
pufit/model-picker-discovery

Conversation

@pufit

@pufit pufit commented Aug 1, 2026

Copy link
Copy Markdown
Member

The bug

claude-fable-5 doesn't appear in the web composer's model selector, even though the configured API key can use it.

GET /api/models built the Claude list from config alone: agent.model + agent.models, and when agent.models was unset, a hardcoded DEFAULT_CLAUDE_MODELS tuple (claude-opus-5, claude-sonnet-4-6, claude-haiku-4-5-20251001). Every model Anthropic ships after that constant is written is invisible until someone edits code or config. On this install the API key exposes 11 models; the picker offered 3.

The fix

Ask the Anthropic Models API (GET /v1/models) which models the configured credentials actually reach, and offer those.

New nerve/models_catalog.py:

  • Primed at gateway startup (fire-and-forget task in the lifespan, after the proxy is up since discovery goes through it when proxy.enabled), cached in-process for 6h, then refreshed in the background — a stale entry is still served immediately, so only a cold cache ever waits on the network.
  • Never raises, never empties the picker. A failure is remembered for 5 min so an unreachable API isn't retried per request.
  • Filters to Claude IDs. With proxy.enabled the request is served by CLIProxyAPI, whose catalog can include non-Anthropic upstreams (Ollama, ...) that the picker lists separately.
  • Cache is fingerprinted on provider/base-URL/key-tail, so a config reload that swaps credentials invalidates it instead of serving another account's catalog.

Resolution order is now, in NerveConfig.selectable_claude_models():

  1. agent.models — an explicit list still wins outright
  2. the discovered catalog
  3. DEFAULT_CLAUDE_MODELS (the previous behaviour)

agent.model always leads the list, as before. Discovery is skipped — falling through to 3 — on Bedrock (the Bedrock client has no Models API, and Bedrock IDs are region-prefixed), without an API key (OAuth-token installs), or when the API is unreachable. agent.model_discovery: false turns it off entirely.

config.claude_models stays as the config-only view, so nothing else that reads it changes.

Verification

  • Full suite: 2938 passed.
  • New tests/test_models_catalog.py (10 tests): ordering, non-Claude filtering + dedup, Bedrock/no-key short-circuits, error swallowing, client cleanup, caching, failure backoff, credential-change invalidation, stale-serve-while-refreshing.
  • tests/test_models_route.py extended (6 tests): discovered models reach the picker, configured default leads, explicit agent.models wins, model_discovery: false and discovery failure both fall back to the built-in list.
  • Against the live API on this install, the picker list goes from 3 entries to 11 — including claude-fable-5 and claude-sonnet-5.

No frontend change: the composer renders whatever the endpoint returns.

Generated by Nerve

The Claude side of GET /api/models was a hardcoded current-generation
list plus whatever agent.model/agent.models named, so every model shipped
after that constant was written stayed invisible in the composer even
though the configured credentials could use it.

Ask the Anthropic Models API (GET /v1/models) instead: primed once at
gateway startup, cached in-process for 6h and refreshed in the
background, so a newly released model needs no code change or config
edit. Best-effort throughout — an explicit agent.models still wins, and
Bedrock (no Models API on that client), a missing API key or an
unreachable API all fall back to the built-in list. agent.model_discovery
turns it off.
@pufit
pufit merged commit c0e1f5f into main Aug 1, 2026
2 checks passed
@pufit
pufit deleted the pufit/model-picker-discovery branch August 1, 2026 04:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant