AI Gateway
Gateway
Secure LLM access, control agent connectivity, and govern tool use: without changing your application code. MCP-native, Kafka-backed, and observable by default.
Route apps to any LLM provider through one gateway, and turn a REST API into agent tools without writing an MCP server.
Capabilities
LLM Gateway
One endpoint for the OpenAI and Anthropic APIs. Applications keep their SDK, and Zilla translates between APIs when needed.
Route each request by model to OpenAI, Anthropic, AWS Bedrock, Azure AI Foundry, or Google Vertex AI, with the caller's own key or the key the gateway holds.
Fall back to another model when the deployment a request is routed to can't be reached.
Reject prompts and replies that contain secrets, personal data, or jailbreak attempts, per model, before they reach the provider.
Detect secrets by pattern, personal data with a Presidio analyzer, or content by meaning with an embedding, and reject it with the classify model.
Token usage, duration, and time to first token for every LLM exchange, per provider. Exported via Prometheus or OTLP.
MCP Gateway
First-class MCP support with server, proxy, and client bindings. MCP is a native stream type alongside http and kafka.
Aggregate multiple MCP servers behind one endpoint. Route tool calls by toolkit name with a configurable listing cache.
JWT bearer authentication at the gateway, plus client credentials and token exchange for Zilla's own outbound calls to an upstream. Auth enforced before tool calls reach upstream servers.
Cache and aggregate tool, prompt, and resource listings at the gateway, in memory or shared across replicas via Redis or Hazelcast. Upstream servers are hit only on TTL expiry.
Derive MCP tools and resources directly from an OpenAPI document. No manual schema authoring required.
Per-method counters and histograms for all MCP calls, dimensioned by tool and outcome. Exported via Prometheus or OTLP.
Comparison
Most MCP gateways ship as a dedicated MCP/AI gateway service, or a plugin added to an existing API gateway, either way a new component with its own config surface. The rows below reflect patterns common across that landscape, not any single product.
| What You Need | Other MCP Gateways | Zilla MCP Gateway |
|---|---|---|
| Tool aggregation | Separate aggregator, or custom merge logic | Unified tool list, add an upstream, done |
| Streaming data as tools | Custom wrapper service, or a bespoke MCP server | Native Kafka toolkit, produce/consume as MCP tools directly |
| REST-to-MCP | Translation plugin, or a proxy component | One route, schema-validated, no new service |
| OpenAPI-to-MCP | Hand-config, or external codegen, every read modeled as a tool | Point at the spec, tools and native resources auto-generated |
| Tool discovery at scale | Full schema load, or custom search | Eager + cold tools, keyword search, load only what's needed |
| Schema-governed messages | Custom validation middleware, or none at all | Schema registry-backed, enterprise data contracts enforced |
| Binary wire formats | Agent-side codec handling, or a translation service | JSON ↔ Avro/Protobuf, mediated at the gateway |
| RBAC and auth | Auth plugin, plus custom scoping code | One guard, role-based visibility |
| Human-in-the-loop | Build it into the agent, or run a callback service | Elicitation relay, built into the gateway |
| Stateless scaling | Sticky sessions, or a custom shared cache | Shared store, no sticky sessions |
| Operational footprint | Separate service, database, and cache per protocol | MCP, HTTP, and Kafka, one config file, one runtime |