Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions crates/switchyard-runner/src/algorithm.rs
Original file line number Diff line number Diff line change
Expand Up @@ -258,6 +258,16 @@ pub enum AlgorithmSpec {
#[serde(default)]
subagents: Option<SubagentRouteConfig>,
},
/// Picks a routing strategy automatically, preset with recommended knobs.
/// Currently a `stage_router` with `picker = "efficient_first"` and
/// `confidence_threshold = 0.5`; change `build_algorithm`'s `Auto` arm to
/// repoint it at a different algorithm or preset.
Auto {
/// The capable tier.
capable_target: String,
/// The efficient tier.
efficient_target: String,
},
/// A judge picks the tier at each user turn; a stage router runs the turns within it.
Composite {
/// Judge that picks the tier. Called through its own target.
Expand Down Expand Up @@ -464,6 +474,10 @@ impl AlgorithmSpec {
}
names
}
Self::Auto {
capable_target,
efficient_target,
} => vec![capable_target.as_str(), efficient_target.as_str()],
Self::Composite {
stage, subagents, ..
} => {
Expand Down Expand Up @@ -553,6 +567,7 @@ impl AlgorithmSpec {
| Self::Passthrough { .. }
| Self::LlmClassifier { .. }
| Self::StageRouter { .. }
| Self::Auto { .. }
| Self::Composite { .. }
| Self::PrefillRouter { .. } => None,
}
Expand Down Expand Up @@ -1023,6 +1038,21 @@ fn build_algorithm(
let parent: Arc<dyn Algorithm> = Arc::new(algorithm);
attach_subagent_router(route_name, parent, subagents.as_ref(), targets)
}
AlgorithmSpec::Auto {
capable_target,
efficient_target,
} => {
let capable = resolve_target_model_id(route_name, capable_target, targets)?;
let efficient = resolve_target_model_id(route_name, efficient_target, targets)?;
let config = StageRouterConfig::new(PickerMode::EfficientFirst, 0.5);
let algorithm = StageRouter::new(capable, efficient, config).map_err(|error| {
AlgorithmConfigError::with_source(
format!("auto route {route_name}: {error}"),
error,
)
})?;
Ok(Arc::new(algorithm))
}
AlgorithmSpec::Composite {
classifier,
stage,
Expand Down
16 changes: 16 additions & 0 deletions crates/switchyard-runner/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -982,6 +982,22 @@ confidence_threshold = 0.5
assert!(error_message(&config).contains("unknown field"));
}

#[test]
fn auto_route_builds_a_stage_router_with_no_extra_fields() -> RunnerResult<()> {
let config = format!(
r#"{VALID_CONFIG}
[routes.auto]
id = "switchyard/auto"
type = "auto"
capable_target = "strong"
efficient_target = "weak"
"#
);
let runner = runner_from_toml(&config)?;
assert!(runner.route("switchyard/auto").is_some());
Ok(())
}

#[test]
fn composite_stage_block_rejects_an_unknown_field() {
let config = composite_config().replace(
Expand Down
16 changes: 7 additions & 9 deletions docs/getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ Cargo builds the release binary and installs it into `~/.cargo/bin` by default.

The Rust server reads an explicit TOML file.

Create `routes.toml` with an LLM-classifier route:
Create `routes.toml` with an auto route:

```toml
schema_version = 1
Expand All @@ -84,12 +84,9 @@ llm_client = "openrouter"

[routes.smart]
id = "switchyard"
type = "llm_classifier"
mode = "capability"
classifier_target = "weak"
strong_target = "strong"
weak_target = "weak"
base_threshold = 0.5
type = "auto"
capable_target = "strong"
efficient_target = "weak"
```

`format` selects the upstream protocol and must be `openai_chat`,
Expand Down Expand Up @@ -131,11 +128,12 @@ curl http://localhost:4000/v1/chat/completions \

#### Choose a route type

This guide uses `llm_classifier`, which asks a classifier target whether each
request should use the weak or strong target. The Rust server also supports:
This guide uses `auto`, which routes with Switchyard's recommended default
settings. The Rust server also supports:

| Algorithm | Use it when | Config |
|---|---|---|
| Auto | You want a recommended default instead of picking a strategy yourself. | `auto` |
| [Random](routing_algorithms/random_routing.md) | You need a weighted split for A/B tests or baselines. | `random` |
| [LLM classifier](routing_algorithms/llm_classifier_routing.md) | Request content should decide whether to use the weak or strong target. | `llm_classifier` |
| [Stage router](routing_algorithms/stage_router_routing.md) | Tool-result and progress signals should select an efficient or capable target. | `stage_router` |
Expand Down
15 changes: 15 additions & 0 deletions docs/reference/toml_schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,21 @@ optional `handoff_notes` and `classifier` tables and for tuning.
| `classifier.response_format_type` | No | `json_schema` | Structured-output mode for the optional classifier judge. Use `json_object` when the classifier provider does not support JSON Schema; Switchyard adds the schema to the prompt and validates the verdict locally. |
| `subagents` | No | unset | Nested `passthrough` or custom `llm_classifier` policy used only for delegated sub-agent work. See [Sub-Agent-Aware Routing](../routing_algorithms/subagent_routing.md). |

### `auto`

Uses Switchyard's recommended default routing strategy instead of one you pick
yourself: a `stage_router` preset with `picker = "efficient_first"` and
`confidence_threshold = 0.5`, no classifier. See
[Stage-Router Routing](../routing_algorithms/stage_router_routing.md) for a
deeper dive on the current default, or the
[strategy table](../routing_algorithms/overview.md#choose-a-strategy) to pick
one manually.

| Key | Required | Default | Meaning |
|---|:---:|---|---|
| `capable_target` | Yes | — | Capable tier. |
| `efficient_target` | Yes | — | Efficient tier. |

### `composite`

Composes other algorithms, letting one set another's
Expand Down
1 change: 1 addition & 0 deletions docs/routing_algorithms/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ configuration and tuning. For the vocabulary these pages use, see
| [Random Routing](random_routing.md) | You need a fixed traffic split for A/B tests, baselines, or cost experiments. | `random` |
| [LLM Classifier Routing](llm_classifier_routing.md) | Request content should decide whether a turn needs the weak or strong tier. | `llm_classifier` |
| [Stage-Router Routing](stage_router_routing.md) | Tool-result and agent-progress signals should route most turns without an extra classifier call. | `stage_router` |
| Auto Routing | You want a recommended default instead of picking a strategy yourself. For a deeper dive on the current default, see [Stage-Router Routing](stage_router_routing.md); for full control, pick one of the strategies above instead. | `auto` |
| [Composite Routing](composite_routing.md) | Routing algorithms are composed, one setting the configuration of another before handing off. Today an LLM classifier sets a stage router's default tier. | `composite` |
| [Escalation-Router Routing](escalation_router_routing.md) | Start every task on the weak tier and escalate to strong when an LLM judge detects trouble. | `llm_classifier` with `escalation` |
| [Advisor-Gate Routing](advisor_gate_routing.md) | One model should serve every turn, with a stronger reviewer approving its "done" claims or sending back a redo plan. | `advisor` |
Expand Down
Loading