diff --git a/docs/products/franklin.md b/docs/products/franklin.md index 3c56f4b..3808f1e 100644 --- a/docs/products/franklin.md +++ b/docs/products/franklin.md @@ -59,7 +59,7 @@ Franklin is the autonomous agent on top of the BlockRun stack — it uses the sa ::::cards :::card{title="ClawRouter" href="routing/clawrouter.md" icon="Route"} -The router Franklin uses — 14-dimension scoring picks the cheapest capable model. +The router Franklin uses — 15-dimension scoring plus portfolio ranking picks the cheapest capable model. ::: :::card{title="BlockRun MCP" href="../mcp/blockrun-mcp.md" icon="Terminal"} diff --git a/docs/products/routing/clawrouter.md b/docs/products/routing/clawrouter.md index 8c5398c..01ff7bb 100644 --- a/docs/products/routing/clawrouter.md +++ b/docs/products/routing/clawrouter.md @@ -57,18 +57,23 @@ For the full technical deep-dive, see [Inside ClawRouter's Decision Layer](https ### The Decision Pipeline (<1ms, fully local) ``` -1. Lexical scoring → 14 weighted dimensions, score ∈ [-1, 1] each +1. Lexical scoring → 15 weighted dimensions, score ∈ [-1, 1] each 2. Tier mapping → SIMPLE / MEDIUM / COMPLEX / REASONING 3. Confidence calibration → sigmoid; below 0.7 → AMBIGUOUS → defaults to MEDIUM -4. Profile resolution → auto / eco / premium → primary + ordered fallback -5. Capability filtering → context window, tool calling, vision +4. Task classification → chat / code_edit / code_agent / tool_agent / + reasoning_math / long_context / vision / … +5. Profile resolution → auto / eco / premium → primary + ordered fallback +6. Capability filtering → context window, output length, tool calling, vision +7. Portfolio ranking → task affinity × cost × speed × reliability ``` No external API calls. No LLM inference in the classification step. Pure keyword matching and arithmetic. -### 14-Dimension Scoring +Steps 4 and 7 are the V3 portfolio layer: the tier says how much capability the request needs, the task type says what *kind* of work it is, and the portfolio ranks the eligible models against calibrated per-task evidence. The tier primary is a starting point, not the answer — a code-agent turn and a multiple-choice question in the same tier get different models. -The classifier reads the prompt and scores it across 14 weighted dimensions. Weights sum to 1.0: +### 15-Dimension Scoring + +The classifier reads the prompt and scores it across 15 weighted dimensions: | Dimension | Weight | Detects | |---|---|---| @@ -86,6 +91,7 @@ The classifier reads the prompt and scores it across 14 weighted dimensions. Wei | simpleIndicators | 0.02 | "what is", "hello", "define" | | referenceComplexity | 0.02 | "the code above", "the API docs" | | domainSpecificity | 0.02 | "quantum", "FPGA", "genomics" | +| negationComplexity | 0.01 | "don't", "avoid", "except", "without" | **Multilingual:** Every keyword list ships in 9 languages (EN, ZH, JA, RU, DE, ES, PT, KO, AR). "证明这个定理" triggers the same reasoning classification as "prove this theorem." @@ -116,25 +122,29 @@ Each tier × profile combination resolves to a primary model plus an ordered fal We descend by **quality first**, then trade quality for speed. Example COMPLEX-tier fallback under `auto`: ``` -gemini-3.1-pro IQ 48, 1,352ms ← primary -gemini-3.5-flash IQ 46, 1,398ms -grok-4.3 IQ 41, 1,348ms -glm-5.2 1,294ms -claude-sonnet-4.6 IQ 52, 2,110ms -deepseek-chat IQ 32, 1,431ms -minimax-m3 IQ 20, 1,238ms -gpt-5.4 IQ 57, 6,213ms ← last resort +google/gemini-3.1-pro ← primary +google/gemini-3-flash-preview +xai/grok-4-0709 +google/gemini-2.5-pro +anthropic/claude-sonnet-5 +anthropic/claude-sonnet-4.6 +deepseek/deepseek-chat +google/gemini-2.5-flash +openai/gpt-5.6-terra +openai/gpt-5.5 +openai/gpt-5.4 ← last resort ``` GPT-5.4 sits last despite the highest IQ — its 6.2s latency creates a worse compounded experience across multi-step workflows than a slightly-lower-IQ model that completes in 1.4s. ### Runtime Capability Filtering -Before any model is dispatched, the candidate set is filtered against three hard constraints: +Before any model is dispatched, the candidate set is filtered against four hard constraints: -1. **Context window fit** — must hold (input + estimated output) × 1.10 safety buffer -2. **Tool calling** — if request includes tools, only function-calling models stay -3. **Vision** — if request includes images, only vision-capable models stay +1. **Context window fit** — must hold (input + estimated output) × 1.10 safety buffer, measured against the *whole* conversation, not the last message +2. **Output length** — must be able to emit the requested `max_tokens` +3. **Tool calling** — if request includes tools, only function-calling models stay +4. **Vision** — if request includes images, only vision-capable models stay A "cheaper" model lacking a required capability is removed from the candidate set, **never silently substituted.** This prevents the classic multi-step failure mode where a tool-call step gets routed to a model that can't actually call tools. @@ -146,14 +156,14 @@ Every request is its own settled x402 transaction. There is no session state to Default `auto` profile primaries (cost-balanced; switch to `free` profile for $0 routing across the free tier): -| Tier | Model (auto) | Cost | Free-tier fallback | Use Case | -|------|-------|------|--------------------|----------| -| **SIMPLE** | moonshot/kimi-k2.7 | $0.95/M in / $4.00/M out | free-tier model (FREE) | Q&A, summaries, simple tasks | -| **MEDIUM** | google/gemini-3.5-flash | $1.50/M in / $9.00/M out | free-tier model (FREE) | Analysis, writing, coding | -| **COMPLEX** | google/gemini-3.1-pro | $2.00/M in / $12.00/M out | free-tier model (FREE) | Advanced reasoning, research | -| **REASONING** | deepseek/deepseek-reasoner | $0.14/M in / $0.28/M out | free-tier model (FREE) | Math, logic, proofs | +| Tier | Primary (auto) | Use Case | +|------|-------|----------| +| **SIMPLE** | google/gemini-2.5-flash | Q&A, summaries, simple tasks | +| **MEDIUM** | moonshot/kimi-k2.7 | Analysis, writing, coding | +| **COMPLEX** | google/gemini-3.1-pro | Advanced reasoning, research, long documents | +| **REASONING** | xai/grok-4-1-fast-reasoning | Math, logic, proofs | -*Prices shown per 1M tokens (after 5% BlockRun markup)* +The primary is where the tier starts, not where the request necessarily lands: the portfolio ranks every capability-eligible candidate for the detected task, so a tool-calling turn and a proof in the same tier resolve to different models. Switch to the `free` profile for $0 routing across the 5 free models. ## Smart Routing Examples @@ -169,7 +179,7 @@ Default `auto` profile primaries (cost-balanced; switch to `free` profile for $0 ### 100% Local Routing -- 14-dimension weighted scoring runs on your machine in <1ms +- 15-dimension weighted scoring runs on your machine in <1ms - No external API calls for routing decisions - Full privacy - your prompts never leave your machine for routing @@ -185,7 +195,7 @@ Access all major providers through one wallet: - **Z.AI**: GLM-5.2 (flagship, 1M context), GLM-5.1, GLM-5, GLM-5 Turbo - **Moonshot**: Kimi K3 (flagship, 1M context, image + text), Kimi K2.7 (256K, image + video) - **MiniMax**: MiniMax M3 -- **Free tier (all FREE)**: 10 reasoning, coding, and vision models with no per-token charge +- **Free tier (all FREE)**: 5 NVIDIA-hosted chat, reasoning and vision models with no per-token charge [View all models →](../intelligence/pricing.md) diff --git a/docs/sdks/python.md b/docs/sdks/python.md index d4bec5d..ab9ec27 100644 --- a/docs/sdks/python.md +++ b/docs/sdks/python.md @@ -115,11 +115,17 @@ address = client.get_wallet_address() print(f"Paying from: {address}") ``` -## Smart Routing (ClawRouter) +## Smart Routing (Router Core) -**Save 88% on LLM costs automatically.** +**Save 88% on LLM costs automatically.** -The `smart_chat()` method uses ClawRouter's 14-dimension scoring algorithm to route each request to the optimal model. Routing decisions run locally in <1ms — your prompts never leave your machine for routing. +Routing runs on [Router Core](https://github.com/BlockRunAI/router-core) — the same engine the TypeScript SDK and the BlockRun gateway use, so an identical request routes identically everywhere. Decisions are local (<1ms, no extra model call): your prompts never leave your machine to be routed. + +Three stages: + +1. **Classify** — 15 weighted dimensions map the request onto a capability tier, and a task classifier labels the shape of the work (`chat`, `code_edit`, `code_agent`, `tool_agent`, `reasoning_math`, `long_context`, `extraction`, `vision`, …). +2. **Filter** — capability constraints are hard filters. A model that cannot hold the conversation, emit the requested `max_tokens`, call tools, or read images is dropped *before* scoring, so the router never picks a model the request would fail on. +3. **Rank** — survivors are scored on task affinity, cost, speed and reliability. The winner serves the request; the rest become the fallback chain, walked automatically on a timeout, a saturated upstream (429) or a 5xx. ### Basic Usage @@ -128,99 +134,142 @@ from blockrun_llm import LLMClient client = LLMClient() -# Let ClawRouter pick the best model automatically -result = client.smart_chat("What is 2+2?") +result = client.smart_chat("Summarize this changelog entry in one line") -print(result.response) # "4" -print(result.model) # "deepseek/deepseek-chat" (cheap model for simple query) +print(result.response) +print(result.model) # "google/gemini-2.5-flash" print(result.routing.tier) # "SIMPLE" -print(result.routing.savings) # 0.94 (94% savings vs baseline) +print(result.routing.task_type) # "chat" +print(result.routing.savings) # 0.90 (90% savings vs the baseline flagship) +``` + +### Inspect a decision without paying + +`route()` runs the same routing and returns the decision only — no model call, no payment. + +```python +decision = client.route("Prove that the square root of 2 is irrational") + +print(decision.model) # "deepseek/deepseek-v4-pro" +print(decision.tier) # "REASONING" +print(decision.task_type) # "reasoning" +print(decision.method) # "portfolio" +print(decision.candidates) # ordered chain; smart_chat walks it on a transient failure +print(decision.reasoning) # human-readable explanation of the pick +``` + +### Routing a full message list + +`smart_chat_completion()` is the routing counterpart of `chat_completion()`. Tools, `tool_choice` and `response_format` are inputs to the *decision*, not just the request, and capacity is checked against the whole transcript rather than the last message. + +```python +result = client.smart_chat_completion( + [{"role": "user", "content": "Cancel order B-42 using the tool."}], + tools=[{"type": "function", "function": {"name": "cancel_order", "parameters": {}}}], + tool_choice="required", +) + +print(result.model) # "openai/gpt-5-mini" — tool-capable +print(result.routing.task_type) # "tool_agent" +print(result.response.choices[0].message.content) +``` + +### Virtual model ids + +Passing `blockrun/auto`, `blockrun/eco` or `blockrun/premium` to the ordinary chat methods routes the turn instead of calling a model by that name — one string change to opt existing OpenAI-compatible code into routing. + +```python +response = client.chat_completion("blockrun/auto", messages) ``` ### Routing Profiles | Profile | Behavior | Best For | |---------|----------|----------| -| `"free"` | Always uses free NVIDIA models | Development, testing | -| `"eco"` | Maximizes cost savings | Bulk processing | +| `"free"` | Only the 5 $0 NVIDIA models — no wallet needed | Development, testing | +| `"eco"` | Cheapest capable model per tier | Bulk processing | | `"auto"` | Balances quality and cost (default) | Production workloads | -| `"premium"` | Always uses top-tier models | Critical tasks | +| `"premium"` | Top-tier models | Critical tasks | ```python -# Force free models (great for development) -result = client.smart_chat( - "Explain recursion", - routing_profile="free" -) -print(result.model) # "nvidia/qwen3-next-80b-a3b-instruct" (cheapest capable for SIMPLE tier) +# Free models only — a paid model can never leak into this profile +result = client.smart_chat("Explain recursion", routing_profile="free") +print(result.model) # "nvidia/step-3.7-flash" +print(result.routing.cost_estimate) # 0.0 -# Maximum savings mode -result = client.smart_chat( - "Summarize this article: ...", - routing_profile="eco" -) +# Maximum savings +result = client.smart_chat("Summarize this article: ...", routing_profile="eco") +print(result.model) # "google/gemini-3.1-flash-lite" -# Premium mode for critical tasks -result = client.smart_chat( - "Review this contract for legal issues...", - routing_profile="premium" -) -print(result.model) # "anthropic/claude-opus-4.6" +# Premium for critical tasks +result = client.smart_chat("Review this contract for legal issues...", routing_profile="premium") ``` -### 4-Tier Model Selection +### Capability tiers + +The classifier places every request in one of 4 tiers. Under `auto`, the tier primary is the starting point — the portfolio then ranks the eligible candidates and may promote a better-suited model for the task. -ClawRouter classifies prompts into four tiers: +| Tier | Auto primary | Use Case | +|------|--------------|----------| +| **SIMPLE** | `google/gemini-2.5-flash` | Q&A, summaries, simple tasks | +| **MEDIUM** | `moonshot/kimi-k2.7` | Analysis, writing, coding | +| **COMPLEX** | `google/gemini-3.1-pro` | Advanced reasoning, research, long documents | +| **REASONING** | `xai/grok-4-1-fast-reasoning` | Math, logic, proofs | -| Tier | Models | Use Case | -|------|--------|----------| -| **SIMPLE** | DeepSeek, Gemini Flash | Q&A, summaries, simple tasks | -| **MEDIUM** | GPT-5.5, Claude Sonnet 4.6 | Analysis, writing, coding | -| **COMPLEX** | Claude Opus 4.6, GPT-5.4 Pro | Advanced reasoning, research | -| **REASONING** | DeepSeek Reasoner, o1, o3 | Math, logic, proofs | +Under uncertainty the router fails **upward**: a score too close to a tier boundary is treated as ambiguous and defaults to MEDIUM, never SIMPLE. ### Routing Decision Details ```python -result = client.smart_chat("Prove that √2 is irrational") - -# Access full routing decision +result = client.smart_chat("Prove that the square root of 2 is irrational") routing = result.routing -print(f"Model: {routing.model}") # "deepseek/deepseek-reasoner" -print(f"Tier: {routing.tier}") # "REASONING" -print(f"Confidence: {routing.confidence}") # 0.97 -print(f"Reasoning: {routing.reasoning}") # "Detected: math proof request..." -print(f"Estimated cost: ${routing.cost_estimate:.4f}") -print(f"Baseline cost: ${routing.baseline_cost:.4f}") -print(f"Savings: {routing.savings:.0%}") # "97%" + +print(routing.model) # the model that served the request +print(routing.tier) # "REASONING" +print(routing.task_type) # "reasoning" +print(routing.method) # "portfolio" ("rules" for the free profile) +print(routing.router_version) # "v3-portfolio" +print(routing.confidence) # 0.85 +print(routing.reasoning) # why this model won +print(routing.candidates) # ordered candidate chain +print(routing.candidate_scores) # per-model quality / cost / speed / reliability +print(routing.fallbacks) # candidates[1:], the runtime retry chain +print(f"${routing.cost_estimate:.4f} vs ${routing.baseline_cost:.4f}") +print(f"Savings: {routing.savings:.0%}") ``` ### Smart Routing Types ```python from blockrun_llm import ( - RoutingProfile, # Literal["free", "eco", "auto", "premium"] - RoutingTier, # Literal["SIMPLE", "MEDIUM", "COMPLEX", "REASONING"] - RoutingDecision, # Full routing details - SmartChatResponse, # Response + model + routing + RoutingProfile, # Literal["free", "eco", "auto", "premium"] + RoutingTier, # Literal["SIMPLE", "MEDIUM", "COMPLEX", "REASONING"] + RoutingDecision, # Full routing details + CandidateScore, # One row of routing.candidate_scores + SmartChatResponse, # response + model + routing + SmartChatCompletionResponse, # ChatResponse + model + routing ) ``` -### Async Smart Routing +### Every client routes + +`LLMClient`, `AsyncLLMClient`, `SolanaLLMClient` and `AsyncSolanaLLMClient` all expose `route()`, `smart_chat()` and `smart_chat_completion()`. Both chains run the same engine over the same catalog, so the same request picks the same model; only the x402 minimum in the cost estimate differs ($0.002 on Base, $0.001 on Solana). ```python import asyncio -from blockrun_llm import AsyncLLMClient +from blockrun_llm import AsyncLLMClient, SolanaLLMClient +# Async, Base async def main(): async with AsyncLLMClient() as client: - result = await client.smart_chat( - "What's the weather like?", - routing_profile="eco" - ) + result = await client.smart_chat("What's the weather like?", routing_profile="eco") print(result.response) asyncio.run(main()) + +# Solana — same routing, USDC on Solana +solana = SolanaLLMClient() +print(solana.route("Prove this theorem").model) ``` ## Specialized clients diff --git a/docs/sdks/typescript.md b/docs/sdks/typescript.md index 5704a36..a919f21 100644 --- a/docs/sdks/typescript.md +++ b/docs/sdks/typescript.md @@ -130,7 +130,7 @@ console.log(`Paying from: ${address}`); **Save 88% on LLM costs automatically.** -The `smartChat()` method uses ClawRouter's 14-dimension scoring algorithm to route each request to the optimal model. Routing decisions run locally in <1ms — your prompts never leave your machine for routing. +The `smartChat()` method routes each request on [Router Core](https://github.com/BlockRunAI/router-core) — 15 weighted dimensions classify the request, capability constraints are applied as hard filters, and the surviving candidates are ranked on task affinity, cost, speed and reliability. Decisions run locally in <1ms — your prompts never leave your machine for routing, and no extra model call is made to decide. ### Basic Usage @@ -164,7 +164,7 @@ console.log(result.routing.savings); // 0.94 (94% savings) const result = await client.smartChat('Explain recursion', { routingProfile: 'free' }); -console.log(result.model); // "nvidia/qwen3-next-80b-a3b-instruct" (cheapest capable for SIMPLE tier) +console.log(result.model); // "nvidia/step-3.7-flash" (a live $0 model; the free lineup rotates as NVIDIA retires SKUs) // Maximum savings mode const result2 = await client.smartChat('Summarize this article: ...', { diff --git a/docs/sdks/xrpl.md b/docs/sdks/xrpl.md index 8f691d9..c9fae05 100644 --- a/docs/sdks/xrpl.md +++ b/docs/sdks/xrpl.md @@ -141,7 +141,7 @@ print(f"Paying from: {address}") **Save up to 94% on LLM costs automatically.** -The `smart_chat()` method uses ClawRouter's 14-dimension scoring algorithm to route each request to the optimal model. Routing decisions run locally in <1ms — your prompts never leave your machine for routing. +The `smart_chat()` method routes each request on [Router Core](https://github.com/BlockRunAI/router-core) — 15 weighted dimensions classify the request, capability constraints are applied as hard filters, and the surviving candidates are ranked on task affinity, cost, speed and reliability. Decisions run locally in <1ms — your prompts never leave your machine for routing, and no extra model call is made to decide. ### Basic Usage