AI Gateway Benchmarks

These benchmarks measure what Zilla adds when it sits between AI applications and their model providers. Each one runs against a mock provider with a fixed response time, so any latency above it comes from the gateway, the network, or the load generator.

Each benchmark has a Run the Benchmark page with the steps to reproduce it.

Benchmarks

LLM Gateway Benchmark

Zilla routing OpenAI-compatible requests at 5,000 req/s, with a median of 0.27 ms spent inside Zilla.