Auto Router

Overview

The Auto Router picks a different model per request based on what the prompt actually asks for. Instead of choosing between a cheap model (fast, low quality on hard tasks) and a premium one (expensive on simple tasks), you describe the tradeoff once and let Mizan decide per call. Auto-routed requests are always billed from your wallet — they never fall back to a connected BYOK key.

How it works

The router analyzes your prompt and selects from a curated set of high-quality models. The selection pipeline runs in four steps:

1. Classify the task. A fast classifier (gpt-oss-120b on Cerebras) maps the prompt to one of ~30 fine-grained task types — for example code:debugging, agent:multi_step_planning, math, or research_report. Classification results are cached by prompt hash, and a classification failure never fails the request — it just routes on the general pool.
2. Rank by real spend. For that task type, Mizan ranks models by the share of spend they actually received over a trailing 7-day window across auto-routed traffic. This is a live signal: when developers migrate a workload to a newer model, the router follows within days — no retraining or manual curation. Until a task type accrues enough traffic, a curated default pool stands in.
3. Apply your dial. The cost / quality tradeoff filters the ranked pool by cost, and your allowed-models patterns narrow it further. A request carrying image content is restricted to image-capable models.
4. Route with fallbacks. The top surviving models become the primary pick plus fallbacks. If the primary errors before the response starts, the next candidate serves the request.

Usage

Set model to mizan/auto (the bare auto alias also works). Any OpenAI SDK works as a client.

curl https://api.app-mizan.com/v1/chat/completions \
  -H "Authorization: Bearer mizan_rt_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mizan/auto",
    "messages": [{ "role": "user", "content": "Explain quantum entanglement in simple terms" }]
  }'

The response echoes the model that was actually selected in the model field:

{
  "id": "chatcmpl-...",
  "model": "anthropic/claude-sonnet-5",
  "choices": [
    { "message": { "role": "assistant", "content": "..." } }
  ],
  "usage": { "prompt_tokens": 15, "completion_tokens": 150, "total_tokens": 165 }
}

Plugin configuration

Control the Auto Router per request with the plugins parameter. The Auto Router reads the plugin with id: "auto-router".

{
  "model": "mizan/auto",
  "messages": [
    { "role": "user", "content": "Explain quantum entanglement in simple terms" }
  ],
  "plugins": [
    {
      "id": "auto-router",
      "allowed_models": ["anthropic/*", "openai/gpt-5.1"],
      "cost_quality_tradeoff": 3
    }
  ]
}

allowed_models

Wildcard patterns that restrict which models the Auto Router can route to. Separate patterns with commas or newlines. Patterns match against the full provider/model slug:

Leave empty to allow all supported models. Defaults configured on the Auto-Routing settings page apply to every request unless overridden here — and if Prevent overrides is enabled, per-request plugins are ignored entirely and account defaults always win.

cost_quality_tradeoff

An integer 0–10 controlling how aggressively to optimize for cost vs. quality. 0 is pure quality (most capable model regardless of cost), 10 is the cheapest model wins. Intermediate values blend quality and cost signals continuously. The default is 9, balancing cost savings with strong output quality.

cost_tier

A named shorthand that selects a contiguous cost-percentile band of the candidate pool. If both are provided, the numeric cost_quality_tradeoff takes precedence.

{
  "model": "mizan/auto",
  "plugins": [{ "id": "auto-router", "cost_tier": "medium" }],
  "messages": [{ "role": "user", "content": "Draft an email to my team" }]
}

Session stickiness

The Auto Router pins both the selected model and provider so subsequent requests in the same conversation route to the same place. This keeps behavior consistent across turns and maximizes prompt-cache hits.

• Explicit: pass a session_id in the body (or an x-session-id header) to pin from the first successful response. Recommended for multi-turn conversations and agent workflows.
• Implicit: Mizan derives a fingerprint from the first system and first user message and pins the model after a successful call — no configuration needed.

In both cases the pin expires after 5 minutes of inactivity and each successful request resets the timer. If the pinned provider errors, the pin is not refreshed and the next request re-routes.

This is configurable per account on the Auto-Routing page under Mid-conversation routing. Stay on one model is the default behavior described above. Reclassify every message turns stickiness off entirely: every message is classified and routed fresh, which can pick a better-fit model as a conversation's difficulty changes, at the cost of an extra classifier call per message and losing that conversation's prompt-cache streak whenever the model actually changes turn to turn.

Fallbacks

The top surviving models form an ordered chain: the first is the primary, the rest are fallbacks. If the primary fails before the response has started streaming — a network error, an upstream 4xx/5xx, or an unpriced model — the router retries the next candidate automatically. Once response bytes have reached the client, no fallback happens: an in-flight stream either succeeds or fails as-is. Every attempt is logged individually on Audit Logs with its task type and outcome.

This chain is picked for you by the classifier. To name your own primary model and fallback chain instead, see Model Fallbacks.

Savings

Every auto-routed call records what it would have cost on the most expensive eligible model in its task type's pool (the "premium baseline") alongside what it actually cost. The difference is your savings, summed per calendar month as a callout on Overview and the Auto-Routing page — no extra fee for using the Auto Router; you pay the standard rate for whichever model was selected.

The savings chart on Analytics and Auto-Routing takes a second, complementary view: it replays your actual auto-routed token usage against a small set of fixed reference models (currently Claude Sonnet 5, GPT-5.6 Sol, and Grok 4.6) to show what the same traffic would have cost had it all gone to one of those instead — a concrete point of comparison rather than a per-task-type baseline that shifts with the pool.

Limitations

• The Auto Router requires the messages format (not prompt).
• Streaming is supported, and standard features like tool calling work with the selected model.
• Rankings are built from Mizan's own auto-routed traffic, so a brand-new task type uses curated defaults until enough spend accrues.
• Image-content requests are restricted to image-capable models in the pool.
• Auto-routed calls always bill the wallet — BYOK keys are never used on this path.