Model Fallbacks

Pick your own primary model and try other models automatically if it's down, rate-limited, or refuses to reply. Pass an ordered models array alongside model and Mizan walks the chain for you. Two related features pick the chain for you instead: Auto Router classifies each prompt and ranks models per request, and Switchboard pins a whole key to a dashboard-configured chain. Use Model Fallbacks when you want to name the models yourself, per request, in the order you want them tried.

Overview

Provide model as usual for your primary choice, and an optional models array of further provider/model strings to try, in order, if the primary errors out. Any error can trigger a fallback — rate-limiting, downtime, content-moderation refusals, an unpriced model — as long as it happens before the response has started streaming back to you. Once bytes have reached the client, an in-flight response either succeeds or fails as-is; nothing retries mid-stream.

Each candidate resolves its own BYOK vs. wallet credentials independently — a fallback chain can mix providers you've connected your own account for with providers billed from your wallet, in the same request. Every attempt, including failed ones, is logged individually on Audit Logs.

Usage

Works with the standard chat-completions endpoint — no separate route, no plugin configuration. The OpenAI SDK doesn't know about models natively, so pass it as a top-level extra field on the request body.

cURL

curl https://api.app-mizan.com/v1/chat/completions \
  -H "Authorization: Bearer mizan_rt_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-4-5",
    "models": ["openai/gpt-4o", "gemini/gemini-2.5-flash"],
    "messages": [{ "role": "user", "content": "What is the meaning of life?" }]
  }'

TypeScript (OpenAI SDK)

import OpenAI from "openai"

const mizan = new OpenAI({
  baseURL: "https://api.app-mizan.com/v1",
  apiKey: process.env.MIZAN_KEY, // "mizan_rt_..."
})

const completion = await mizan.chat.completions.create({
  model: "anthropic/claude-sonnet-4-5",
  // @ts-expect-error — "models" is a Mizan-only extra field
  models: ["openai/gpt-4o", "gemini/gemini-2.5-flash"],
  messages: [{ role: "user", content: "What is the meaning of life?" }],
})

console.log(completion.choices[0].message.content)

Response & pricing

The response's model field always reflects whichever model actually served the request, so you can tell when a fallback fired:

{
  "id": "chatcmpl-...",
  "model": "openai/gpt-4o",
  "choices": [
    { "message": { "role": "assistant", "content": "..." } }
  ],
  "usage": { "prompt_tokens": 15, "completion_tokens": 150, "total_tokens": 165 }
}

You're billed the standard rate for whichever model was ultimately used — no extra fee for the fallback mechanism itself, and failed attempts before a response starts are never billed.

Limitations

models accepts at most 4 entries; a longer list returns a 400 error.
Every entry must use the same provider/model format as model — a malformed entry returns a 400 error rather than being silently skipped.
Can't be combined with mizan/auto or mizan/switchboard — both already build their own fallback chain; sending models alongside either returns a 400 error.
Streaming is supported for the primary and every fallback candidate.