Docs / Features

Latency & performance

Practical guidance for keeping time-to-first-token low.

The gateway terminates TLS in Japan and routes to the nearest healthy upstream. First-hop latency across East and Southeast Asia is low; total latency is dominated by the provider and model you choose.

Recommendations

Stream responses, keep prompts lean, prefer cache-friendly layouts, and test flagship and flash-tier variants — smaller models often answer 3–5× faster at a fraction of the price.