Smart Router

The Smart Router picks a model for each request. You set the policy once, in Your workspace, and your code sends auto or a Router address instead of a model name.

resp = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Summarise this contract clause."}],
)
print(resp.model)  # the model that actually served

Routing works on /v1/chat/completions, /v1/messages and /v1/responses.

Three ways to address a model

model="auto"                          # your workspace's default Router
model="@acme/eu-only"                 # a Router your workspace created
model="anthropic/claude-sonnet-4-5"   # a pinned model — no routing

auto runs your workspace's default Router. Change the default in Your workspace → Connect → Routers and every auto call follows, with no deploy.

A Router is addressed as @workspace-slug/router-slug. It resolves only inside your own workspace:

A Router that is not your default never falls back to anything else. If its pool is empty you get an error.

A pinned model skips routing. Your workspace policy and sovereignty floor still apply. A key restricted to a Router cannot pin a model; it may only send auto or one of its Routers.

How a decision is made

1. Build the candidate pool

Start from the models your workspace may use, then remove any model that:

• is below your sovereignty floor (see Sovereignty)
• is below the quality floor, if one is set (the strictest of workspace, key and Router wins)
• has no active Provider Endpoint
• cannot serve a chat request
• fails the Router's own filters: allowed or blocked models, allowed creators, required capabilities, blocked provider countries, allowed provider jurisdictions, its own sovereignty floor (it can only tighten the workspace floor), and BYOK-only mode
• lacks a capability the request needs (below)
• has a context window too small for the request's estimated input, tool definitions included

2. Score what survives

Default weights are quality 40 / cost 30 / latency 30. Each Router carries its own weights. A workspace set to prefer sovereign models adds a bonus to sovereign open-source models.

2a. What a Router pays for

A Router can set two prices instead of three weights: what a good answer is worth and what one second of waiting costs, in your workspace currency (Custom in the Router's Objective). The router then picks the model with the best value: the chance of a good answer times its worth, minus the expected cost, minus the waiting. Presets (Balanced, Lowest cost, Lowest latency, Highest quality) are fixed pairs of these two prices. Routers that set weights keep working.

2b. Pick the reasoning level

If you sent no reasoning level, the router chooses one for the model it picks, from the levels that model's Provider Endpoint accepts — never above the model's default unless the Router allows it, and never a level whose thinking would not fit under your max_tokens when a lower one fits. Your Router's Reasoning setting (Auto / Never / At most a level) bounds it. See Reasoning.

3. Serve and keep the runners-up

The top model serves. The next three are kept as the failover chain.

Shortcuts

• Scoring cache. Rankings are cached per policy and request shape. A hit replays the earlier ranking and its failover chain. X-Stav-Route-Reason starts with Auto-routed (cached).
• Continuity lock. In an agent loop, a request that returns tool results for calls the previous model made — or carries signed thinking — stays on that model if it is still eligible. Switching mid-loop would break the loop. X-Stav-Route-Lock names the trigger (tool_loop, signed_thinking) or says none.

Capability filtering

A hard filter, applied before scoring:

The same flags are in stav_capabilities on GET /v1/models. If no model qualifies, the request fails naming the capability.

The classifier

On routed requests Stav first runs a small classifier over the start of the prompt. It fails open: on timeout or error, routing continues without it. Pinned-model requests skip it.

Its main output is a complexity band, which becomes the expected output length. That number drives the cost and speed estimates — output tokens usually dominate the bill, so a one-line answer and a full document can have different cheapest models. Without the classifier, expected output falls back to a multiple of the input size. max_tokens always caps the estimate.

Reasoning scales the estimate up: low ×2, medium ×4, high ×6, more for xhigh and max.

Domain, sensitivity and language are recorded on the request log. They do not change the score today.

When no policy applies

If auto or your default Router yields no eligible model, Stav re-scores across every active chat model. The sovereignty floor, quality floor and capability filters still apply. The reason header reads:

Auto-routed (fallback - no workspace model policy, scored across N active chat models): <slug>

A non-default Router never does this. If even the fallback finds nothing, the request fails with 404 no_eligible_models.

Failover

On /v1/chat/completions, if the chosen model fails, Stav tries the next candidate from the same ranked list. The list was built from your policy, so failover never leaves it. Pinned models have no chain; their errors reach you directly.

Failover triggers on:

• upstream 5xx
• upstream 429 (the provider client already retried and lost)
• Stav's account being refused by the provider
• connect, read, write and pool timeouts
• an open circuit breaker for the provider

Other 4xx do not fail over — a different model would fail the same way.

Streaming: failover happens only before the first chunk is sent. One case is handled for you: a reasoning model that returns an empty stream is replaced silently, because nothing has reached you yet.

What you can see

Response headers:

X-Stav-Request-Id:        chatcmpl-8f3a…
X-Stav-Model:             mistralai/mistral-small-4-119b-2603
X-Stav-Provider:          mistral
X-Stav-Sovereignty-Level: L3
X-Stav-Route-Reason:      Picked mistralai/mistral-small-4-119b-2603 for code work with the best score, 78.4; reasoning level none chosen by the router.
X-Stav-Reasoning-Chosen-By: router
X-Stav-Route-Time-Ms:     41.3
X-Stav-Route-Lock:        none
X-Stav-Router:            @acme/eu-only

response.model is the model that served, not the string you sent. Log that one.

The full decision — candidates, which filter removed what, scores, classification, estimated tokens — is on the request in Your workspace → Monitor → Request log. The Routers page also shows what routed traffic would have cost on a single frontier model, at catalogue price.

Why this pick, in the request log

Open a request under Your workspace → Monitor → Request log and expand Why it went here. It shows the kind of work the router read from the request (code, agent loop, extraction, …), the needs it matched (tool calling, JSON output, image input, …), the model and endpoint that served, the reasoning level and who chose it, whether the pick was an explored one, and the runners-up with the quality, expected cost and expected time they were scored on. The same data is GET /api/inference/logs/{request_id}/routing.

Router examples

• Compliance lane. A sovereignty floor and a list of cleared provider jurisdictions. Regulated code calls @acme/regulated.
• Cost lane. Cost weight high, a quality floor, five qualified models. Batch jobs call @acme/bulk.
• Latency lane. Latency weight high. Autocomplete calls @acme/interactive.

The point is that your code says @acme/bulk, so changing models is a change in Your workspace, not a release.

When to pin instead

• Reproducing an evaluation.
• A contract names the model.
• You depend on one model's exact output format.

Next steps

• Routing hints — per-request session identity and provider filters
• Sovereignty
• Reasoning
• Response headers