Cookbook

Small, copy-pasteable patterns for everyday gateway work.

Cheap-first cascade

Try a budget model first and escalate to a flagship only when needed — list them cheapest-first in the models array.

{
  "models": [
    "deepseek/deepseek-v3.2",
    "google/gemini-3.8-flash",
    "anthropic/claude-sonnet-5"
  ],
  "messages": [{"role": "user", "content": "..."}]
}

JSON mode, guaranteed

Pair a json_schema response_format with a system prompt that states the contract. Validate on your side and retry once on parse failure.

{
  "model": "openai/gpt-5.5-pro",
  "messages": [
    {"role": "system", "content": "Reply only with JSON matching the schema."},
    {"role": "user", "content": "..."}
  ],
  "response_format": {"type": "json_object"}
}

Streaming with fallback

Start the stream from your first-choice model; if the connection errors before the first token, re-issue against the next slug in your list.

for slug in ["moonshotai/kimi-k3", "qwen/qwen3.8-max"]:
    try:
        for chunk in client.chat.completions.create(
            model=slug, messages=msgs, stream=True
        ):
            emit(chunk)
        break
    except Exception:
        continue

More recipes land with each docs release — suggest one at info@jackz.ai.