feat(routing): same routing surface on all four clients — Solana, message lists, virtual ids - #52
Merged
Merged
Conversation
…sage lists, virtual ids 1.11.0 put Router Core in the Python SDK but wired it only into the Base sync client's single-prompt path. Three gaps came out of that: - SolanaLLMClient and AsyncSolanaLLMClient had no routing at all. A Solana user got no model selection, while the TypeScript SDK has offered it on both chains since 3.12.0. - There was no way to route a message list, so an agent transcript — the case where tools, response_format and transcript size actually drive the decision — could not be routed at all. - `blockrun/auto` | `blockrun/eco` | `blockrun/premium` did nothing in Python; in the TS SDK they select a routing profile from ordinary chat calls. All four clients now expose route(), smart_chat() and smart_chat_completion(). Both chains run the same engine over the same catalog, so the same request picks the same model; only the x402 floor in the cost metadata differs ($0.002 Base, $0.001 Solana). smart_chat_completion routes on the whole request, not a prompt string: tools and tool_choice make it a tool-agent decision, response_format forces the structured-output tier, image parts force vision, and capacity is checked against the entire transcript rather than the last message — an agent conversation can be 100x its final turn and a context overflow is a non-transient error no fallback chain rescues. Solana chat()/chat_completion() gain fallback_models. The parameter existed only on the Solana streaming path, so a routed Solana call carried a recovery chain it could not walk. The walk reuses _should_fallback_solana, which refuses anything already tagged as settled — the next model cannot sign a second transfer for one call. The /v1/models -> pricing conversion moved to router_adapter.build_model_pricing so the four clients cannot drift; unavailable catalog rows are now skipped everywhere, not just in the Base sync client. Also fixes a 429 ending a call outright: both clients counted only 5xx as retriable, so a saturated upstream failed the request with capable models still in the chain. Found live — a rate-limited free model answered 429 and the three remaining free models were never tried. The TS adapter has always treated 429 as transient. Settled and permanently-failed payments are still refused first.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
1.11.0 put Router Core in the SDK but wired it into exactly one place: the Base sync client's single-prompt path. This closes the three gaps that left.
What was missing
SolanaLLMClient/AsyncSolanaLLMClientroute(),smart_chat(),smart_chat_completion()AsyncLLMClientsmart_chat_completion()on all fourblockrun/auto|eco|premiumchat()/chat_completion()chat()/chat_completion()fallback_modelsA Solana user got no model selection at all, while the TypeScript SDK has offered it on both chains since 3.12.0.
Why
smart_chat_completionmatters separatelysmart_chatroutes a prompt string. The decisions that actually need routing are the ones a prompt string cannot express:tools+tool_choice→ a tool-agent decision, and a model that cannot call tools is filtered out before scoring.response_format→ forces the structured-output tier.Cross-chain equivalence
Both chains run one engine over one catalog, so the same request picks the same model. Only the x402 floor in the cost metadata differs — $0.002 Base, $0.001 Solana. Pinned by tests that route the same prompts through both clients and compare model, tier, task type and the full candidate chain.
Fixed along the way: a 429 killed the call
Both clients counted only 5xx as retriable, so a saturated upstream ended the request with capable models still in the chain. This showed up in live testing: a rate-limited free model answered 429 and the three remaining free models were never tried. The TypeScript adapter has always treated 429 as transient — this upstream is saturated, the next model is a different upstream. Settled payments and permanent payment failures are still refused before the status check, so no call can pay twice.
Also
/v1/models→ pricing-map conversion moved intorouter_adapter.build_model_pricing(), shared by all four clients rather than written out per client. Catalog rows markedavailable: falseare now skipped everywhere (1.11.0 did that only in the Base sync client).Verification
test_routing_parity.py: surface parity across the four clients, virtual-id resolution, cross-chain decision equality, the free-profile guarantee on Solana, and the retriable-status matrix).chat_completion("blockrun/auto", ...)routed togoogle/gemini-2.5-flashand answered; a tool-required message list routed toopenai/gpt-5-miniwithtask_type: tool_agent; the free profile through the message-list path landed onnvidia/step-3.7-flashat $0.route()returneddeepseek/deepseek-v4-pro(REASONING, portfolio) andsmart_chat(routing_profile="free")was served bynvidia/step-3.7-flashat $0 with a three-model chain behind it.Version bumped to 1.12.0.