Guides
Some guides below pick up where Get Started leaves off; others run their own self-contained stack. Each one is the runnable companion to a use case, hands-on steps for a capability the use case pages describe narratively.
LLM Gateway
LLM Gateway Setup
Build an LLM Gateway from scratch that serves OpenAI and Anthropic requests and routes them by model.
Route by Model
Add model routes and see how route order decides which deployment serves a request.
Model Fallback
Configure a fallback model on a route and watch it take over when a deployment can't be reached.
LLM Guardrails
Add guardrails that reject secrets and personal data in prompts before they reach the provider.
Bring Your Own Key
Forward the caller's own API key to each provider with an
inline guard.Track Token Usage
Record the
llm.* metrics and read them in Prometheus.Secure LLM Access with JWT
Validate callers with a
jwt guard, hold the provider key in the gateway, and route by role.OpenAI SDK on AWS Bedrock
Serve OpenAI SDK requests from Amazon Bedrock with Zilla Plus.
MCP Gateway
HTTP APIs as Tools
Route a toolkit straight to a REST API with
mcp-http · proxy, no MCP server in front of it.MCP Gateway Setup
Add a third upstream, a plain REST API with no MCP server in front of it, to the gateway from Get Started using
mcp-http · proxy.Secure MCP with OAuth
Add a
jwt guard so every tool call is authenticated at the gateway before it reaches an upstream.Kafka as MCP Tools
Chain
mcp-http · proxy in front of a Kafka-backed REST API so an AI agent can produce and fetch Kafka messages as MCP tools.Centralized Auth
Run one
jwt guard in front of two independent toolkits and see the same policy enforced for both.Secure Agent Access
Connect once and reach three upstream toolkits behind a single entrypoint.
OpenAPI Specs as Tools
Compile a small OpenAPI document straight into an MCP tool with
mcp-openapi · client.Observable AI Agents
Make a few tool calls and watch the per-tool, per-outcome metrics counters climb.