Troubleshooting Guides
These guides cover common issues encountered while operating an AI Gateway deployment, for both the MCP Gateway and the LLM Gateway. Each entry lists the Symptom you'll observe, the likely Cause, and the Fix to apply.
Quick reference
Match what you're seeing to jump straight to the fix. See Log Events for the full event reference.
Authentication & authorization
Why do tool calls fail with an authorization error?
Symptom: Zilla raises GUARD_JWT_AUTHORIZATION_FAILED.
Cause: the gateway enforces inbound auth once, directly on the mcp server binding's own options.authorization, using an OAuth guard such as JWT Bearer. The logged reason narrows down the cause: an invalid alg or unknown key, a signature that doesn't verify, an expired token, or an issuer or audience that doesn't match the guard's configuration. This is distinct from an outbound Client Credentials or Token Exchange failure, which would instead be logged against the mcp client or mcp-http proxy exit binding calling the upstream.
Fix: compare the caller's token against the guard's configured issuer, audience, and keys in zilla.yaml.
Why do replicas disagree on tool listings or auth state?
Symptom: two requests to the same gateway, load-balanced across replicas, see a different tools/list response or one replica re-prompts for auth that another already satisfied.
Cause: the listing cache and OAuth guard auth state live in a store. The default memory store keeps that state local to one process, so each replica in a horizontally scaled deployment builds its own copy independently instead of sharing one.
Fix: move the store binding backing the cache and the guard from memory to redis or hazelcast so every replica reads and writes the same state. See Scale MCP Gateway for the full walkthrough.
Discovery & routing
Why don't a new upstream's tools show up in tools/list?
Symptom: a newly added upstream's tools are missing from tools/list.
Cause: two independent things have to be true after adding an upstream: the route has to exist, and the listing cache has to have picked it up. If the cache tried and failed to refresh from that upstream, Zilla raises BINDING_MCP_HYDRATE_FAILED and keeps retrying with backoff, so the tool never appears until a retry succeeds.
Fix: check both things that have to be true for a tool to appear:
- Confirm the new upstream has both a binding (an
mcp · clientormcp-http · proxy) and a matching route onmcp · proxy, keyed bytoolkit. A binding with no route, or a route with the wrongtoolkitvalue, never gets aggregated intotools/list. - If the route is correct but the tool still doesn't appear, the merged listing may simply be serving a cached response from before the upstream was added. The cache TTL defaults to
PT5M(5 minutes); wait for it to expire, or restart the gateway to force a refresh. - If
BINDING_MCP_HYDRATE_FAILEDis logged, the upstream itself is the problem: confirm it's reachable and returning a well-formedtools/list,prompts/list, orresources/listresponse for the affected route.
Why does a tool call route to the wrong upstream, or fail to route at all?
Symptom: a tool call is routed to the wrong upstream, or gets no route.
Cause: Toolkit routing matches on the toolkit prefix of the tool name, using the <toolkit>__<tool> convention (for example bluesky__get_weather routes on toolkit: bluesky). This happens when the prefix the agent sends doesn't match any configured routes[].when[].toolkit value.
Fix: check the exact toolkit name configured on each route against what the agent is actually calling, since this comparison is exact, not a partial or case-insensitive match. If two upstreams are meant to share one toolkit name, only the first matching route is ever taken.
Sessions & elicitation
Why did a client's MCP session close unexpectedly?
Symptom: Zilla raises BINDING_MCP_SESSION_CLOSED, sometimes paired with a BINDING_MCP_SESSION_ESTABLISHED shortly after on a reconnect.
Cause: in a horizontally scaled deployment (see Scale MCP Gateway), there are no sticky sessions by design, a client's connection can land on a different replica after a reconnect, closing the session the previous replica held. Outside that pattern, frequent open/close pairs usually point at the client not keeping its connection alive, an idle timeout between the client and the gateway, or a load balancer resetting the connection.
Fix: if the gateway is running multiple replicas, confirm this isn't just expected reconnect behavior under a load balancer before treating it as a fault. Otherwise, check the client's keep-alive behavior and any idle-timeout configuration in front of the gateway.
Why did an elicitation request time out?
Symptom: Zilla raises BINDING_MCP_ELICITATION_TIMEOUT.
Cause: an upstream server's elicitation request went unanswered by the client past the configured timeout. This is frequently a human-in-the-loop delay, a form left unfilled, an OAuth consent screen never completed, rather than a gateway or upstream defect.
Fix: check the timeout configured under options.elicitation on the mcp · server binding, and confirm the client actually surfaced the elicitation request to the user instead of silently dropping it.
Requests & validation
Why does an mcp-http tool call fail with "unresolved expression"?
Symptom: Zilla raises BINDING_MCP_HTTP_SCHEMA_ACCESSOR_UNRESOLVED.
Cause: a ${...} accessor in the tool's HTTP request template, used to map a field from the incoming MCP tool call onto the outbound HTTP request (see HTTP APIs as Tools), didn't resolve against the tool call the agent actually sent.
Fix: check that the tool's schema declares the field the accessor points to, and that the agent's tool call actually included that field. A tool call missing an optional argument that an accessor assumes is present is the most common cause.
Why does Guardrails reject a well-formed tool call or response?
Symptom: a tool call or response that looks well-formed is rejected.
Cause: a Guardrails model configured on the http binding's request or response content rejects a payload when it doesn't match that model, for example a content.model: json field rejecting a request body that isn't valid JSON, or a schema-backed model (avro, protobuf) rejecting a payload that doesn't conform to the schema resolved from its catalog.
Fix: check the exact model and, if configured, catalog on the matching requests[]/responses[] entry in the http binding against the payload the agent or upstream actually sent. See Validate for the full configuration shape.
Connectivity
Why can't the gateway reach an upstream MCP server or HTTP API?
Symptom: BINDING_TCP_DNS_FAILED if the upstream hostname doesn't resolve, or one of the BINDING_TLS_* events if the TLS handshake fails.
Cause: both mcp · client and mcp-http · proxy exit over tcp, and usually tls, to reach an upstream. See the full event reference for every connectivity event.
Fix: confirm the upstream hostname and port in the client or proxy's options are reachable from the gateway, and that the upstream's certificate is trusted if TLS is in the path.
LLM Gateway
Why does an LLM request get no reply?
Symptom: a request to the OpenAI or Anthropic API on the gateway waits and never gets a reply, until the caller's own timeout ends it.
Cause: no route on llm · proxy matches the request's model. Routes match the model name exactly, or by a pattern such as gpt-*, and the first match wins. See Model Routing.
Fix: compare the model the application sends with the when[].model patterns on llm · proxy, and add a route for it. A dated model name such as claude-haiku-4-5-20251001 needs a route that matches it, such as claude-*.
Why does an LLM request fail with 401 Unauthorized?
Symptom: the caller receives 401 in the error format of its API.
Cause: either Zilla or the provider rejected the API key:
- When
llm · serverhasoptions.authorization, Zilla rejects a request that doesn't carry the key in the header the credentials template describes, before it reaches the provider. The OpenAI API carries the key asAuthorization: Bearer <key>, and the Anthropic API asx-api-key: <key>. - When the key is forwarded to the provider, as with an
inlineguard, the provider checks it, and Zilla returns its401in the caller's error format.
Fix: send the key in the header of the API the application calls, check the credentials templates on llm · server and llm · client, and check that the key is valid for the provider the model routes to. A request for gpt-* needs an OpenAI key, even when it's sent to the Anthropic API. See LLM Credentials.
Why does an Anthropic request fail with an anthropic-version error?
Symptom: a request to the Anthropic API gets 400 with the message anthropic-version: header is required or anthropic-version: unsupported version <version>.
Cause: Anthropic API requests must carry the header anthropic-version: 2023-06-01. Zilla rejects a request without it, or with another version.
Fix: send anthropic-version: 2023-06-01. The official Anthropic SDKs send it for you. See Request Checks.
Why does an LLM request fail with 415 Unsupported Media Type?
Symptom: the caller receives 415.
Cause: the request's content type isn't application/json. For example, curl -d sends application/x-www-form-urlencoded unless you set the header.
Fix: send Content-Type: application/json.
Why is a prompt rejected by a guardrail?
Symptom: the caller receives 400 with a message naming a section, such as user-text rejected.
Cause: an LLM guardrail configured for the requested model rejected that section of the prompt, so Zilla didn't send it to the provider. A classify check also rejects the section when its classifier can't check it, for example because the Presidio analyzer or the embedding service can't be reached.
Fix: check the guardrails entry for the model on llm · proxy, in order, against the content of the named section. If the prompt should have passed, check that the services your classifiers use are running and reachable from Zilla. See Classify.
Why does an LLM request fail with 502 Bad Gateway?
Symptom: the caller receives 502 with the message Bad Gateway.
Cause: Zilla couldn't connect to the provider: the server host in llm · client doesn't resolve or isn't reachable, or the TLS handshake failed. A failed handshake raises BINDING_TLS_HANDSHAKE_FAILED, logged by sys:tls_client when llm · client exits to sys:http_client.
Fix: check the server URL on llm · client, and that the host is reachable from where Zilla runs. sys:http_client trusts only the standard certificate authorities, so a server with a certificate signed by your own certificate authority needs your own tls · client. See Connecting to LLM Providers.
Why didn't a request fall back to another model?
Symptom: a route lists a fallback model, but the caller receives the provider's error, such as 429 Too Many Requests or 503, instead of a reply from the fallback model.
Cause: fallback covers a deployment that can't be reached. Once the deployment accepts the request, its reply is final, including an error.
Fix: see When Fallback Happens.
Why are settings like temperature ignored for some models?
Symptom: a setting such as temperature, top_p, or stop has no effect for some models, or the request is rejected with 400 naming the setting.
Cause: when the provider uses a different API from the request, Zilla carries the settings the APIs share, such as temperature, top_p, stop, seed, tool_choice, and response_format, in the provider's form. A setting the provider's API has no equivalent for is dropped, and one it can't represent, such as stop for an OpenAI Responses request, is rejected. An llm · client can also drop or reject settings that its provider or model doesn't accept, as set by its unsupported and models options. Requests to a provider with the same API keep every setting.
Fix: check which settings are carried in Dialect Translation, and whether the llm · client lists the setting in Model Capabilities. To keep a setting that has no equivalent, send the request through the API the provider uses, for example the Anthropic API for claude-* models.
Log Events
Named events raised by the mcp and mcp-http bindings show up in the log format as <event name> - <details>. For events raised by the surrounding tcp, tls, http, and guard bindings in an AI Gateway pipeline, see the full event reference.
The mcp binding defines five named events of its own, in addition to being instrumented through the mcp.* metrics (mcp.initialize, mcp.tools.list, mcp.tools.call, and their .duration histograms):
| Event | What to check |
|---|---|
| BINDING_MCP_AUTHORIZATION_FAILED | An authorization failure raised by the mcp binding itself. Compare against GUARD_JWT_AUTHORIZATION_FAILED above to see whether the guard or the mcp binding rejected the request. |
| BINDING_MCP_SESSION_ESTABLISHED | An MCP session was established. Expected on every new client connection; see Why did a client's MCP session close unexpectedly? if it's paired with an unexpected close. |
| BINDING_MCP_SESSION_CLOSED | See Why did a client's MCP session close unexpectedly? |
| BINDING_MCP_ELICITATION_TIMEOUT | See Why did an elicitation request time out? |
| BINDING_MCP_HYDRATE_FAILED | See Why don't a new upstream's tools show up in tools/list? |
The mcp-http binding, used when an mcp · proxy maps MCP tool calls onto HTTP requests, raises one event of its own:
| Event | What to check |
|---|---|
| BINDING_MCP_HTTP_SCHEMA_ACCESSOR_UNRESOLVED | A ${...} expression accessor referenced by a tool's HTTP request template could not be resolved against the incoming tool call. Check that the tool's schema declares the field the accessor points to and that the caller actually supplied it in the request. |
The llm binding doesn't raise named events of its own. Each LLM request is logged as BINDING_HTTP_REQUEST_ACCEPTED by the http · server in front of llm · server, and a failed TLS handshake with a provider is logged by the tls binding. To see each provider's requests, tokens, and latency, use the llm.* metrics.