Metrics

AI Gateway instruments every MCP method call and every LLM exchange automatically, no instrumentation required in the agent, the application, or the upstream server. To collect a metric, add its name to the telemetry.metrics property in your zilla.yaml config. The metrics can then be collected and exported via exporters, such as Prometheus.

MCP Metrics

Metric	Type	Description
mcp.initialize	counter	The number of initialize requests handled.
mcp.initialize.duration	histogram	The duration of initialize requests in nanoseconds.
mcp.tools.list	counter	The number of tools/list requests handled.
mcp.tools.list.duration	histogram	The duration of tools/list requests in nanoseconds.
mcp.tools.call	counter	The number of tools/call requests handled.
mcp.tools.call.duration	histogram	The duration of tools/call requests in nanoseconds.

telemetry:
  metrics:
    - mcp.initialize
    - mcp.initialize.duration
    - mcp.tools.list
    - mcp.tools.list.duration
    - mcp.tools.call
    - mcp.tools.call.duration

You can also enable every MCP metric at once with the mcp.* wildcard, which is useful while a gateway is still adding new MCP methods such as prompts/list or resources/list:

telemetry:
  metrics:
    - mcp.*

MCP Attributes

Add telemetry.attributes to an mcp · server binding to dimension its metrics by the MCP method, the tool being called, and the outcome of the call. Each attribute value is a variable populated by the gateway per request:

north_mcp_server:
  type: mcp
  kind: server
  exit: north_mcp_proxy
  telemetry:
    attributes:
      method: ${mcp.method}
      tool: ${mcp.tool}
      outcome: ${mcp.outcome}

Attribute	Description
method	The MCP method invoked, such as initialize, tools/list, or tools/call.
tool	The name of the tool being called, when the method is tools/call.
outcome	Whether the call succeeded or failed.

Use the zilla metrics command to see the raw metrics that AI Gateway has collected.

Tips

The synthetic tools Tool Discovery adds, zilla__search_tools, zilla__describe_tool, and zilla__execute_tool, are ordinary tools from the metrics system's point of view. A call to any of them counts toward mcp.tools.call and its duration histogram with tool set to the tool's own name, the same as any other tools/call.

LLM Metrics

The LLM Gateway records one sample per exchange at each llm binding whose telemetry.metrics includes them.

Metric	Type	Description
llm.interactions	counter	The number of exchanges, counted when the reply ends or the exchange is aborted.
llm.active.requests	gauge	The number of exchanges in progress.
llm.duration	histogram	The duration of exchanges in nanoseconds, from the request to the end of the reply.
llm.time.to.first.token	histogram	The time in nanoseconds from the request to the first data of the reply.
llm.usage.incomplete	counter	The number of exchanges that ended without a complete token usage report from the provider.
llm.tokens.input	histogram	Input tokens per exchange.
llm.tokens.output	histogram	Output tokens per exchange.
llm.tokens.total	histogram	Total tokens per exchange.
llm.tokens.cache.read	histogram	Input tokens read from the provider's prompt cache per exchange.
llm.tokens.cache.write	histogram	Input tokens written to the provider's prompt cache per exchange.
llm.tokens.reasoning	histogram	Reasoning output tokens per exchange.

Enable them per binding with the llm.* wildcard, and list the ones to collect under telemetry.metrics:

telemetry:
  metrics:
    - llm.interactions
    - llm.duration
    - llm.time.to.first.token
    - llm.tokens.input
    - llm.tokens.output

bindings:
  north_llm_server:
    type: llm
    kind: server
    telemetry:
      metrics:
        - llm.*

Token counts come from the usage the provider reports. A count the provider doesn't report is not recorded, rather than recorded as 0. For example, Anthropic reports no total, so llm.tokens.total stays empty for Anthropic.

Where Metrics Are Recorded

A request that goes through an llm · server, an llm · proxy, and an llm · client is counted at all three, so don't add up the same metric across bindings. Use the llm · client bindings to see each provider, and the llm · server bindings to see each API callers use.

Prometheus Names

The Prometheus exporter reports durations in milliseconds, and adds the usual suffixes:

Metric	Prometheus name
llm.duration	llm_duration_milliseconds (_bucket, _sum, _count)
llm.time.to.first.token	llm_time_to_first_token_milliseconds (_bucket, _sum, _count)
llm.interactions	llm_interactions_total
llm.usage.incomplete	llm_usage_incomplete_total
llm.tokens.input	llm_tokens_input (_bucket, _sum, _count)
llm.active.requests	llm_active_requests

Each series carries a binding label with the name of the binding that recorded it.

LLM Attributes

An llm binding's telemetry.attributes label every llm.* series it records with values from the caller's authorization, or with the model of the exchange:

Value	Label
${guarded['<guard>'].identity}	The identity the named guard authorized
${guarded['<guard>'].attributes.<name>}	An attribute of that authorization, such as a jwt claim
${llm.request.model}	The model the request asked for
${llm.response.model}	The model that served the request, as the provider reports it in the reply

For example, to label usage per tenant from a jwt claim:

guards:
  jwt0:
    type: jwt
    options:
      issuer: https://auth.example.com
      audience: llm-proxy
      attributes:
        tenant: tenant

bindings:
  north_llm_server_openai:
    type: llm
    kind: server
    telemetry:
      attributes:
        tenant: ${guarded['jwt0'].attributes.tenant}
      metrics:
        - llm.*

To label usage per model, use the model values:

bindings:
  south_llm_client_openai:
    type: llm
    kind: client
    telemetry:
      attributes:
        requested: ${llm.request.model}
        served: ${llm.response.model}
      metrics:
        - llm.*

llm_tokens_output_count{binding="south_llm_client_openai",requested="gpt-4o-mini",served="gpt-4o-mini-2024-07-18"} 2

The served model can differ from the requested one: providers often report a dated version, such as gpt-4o-mini-2024-07-18 for gpt-4o-mini. After a fallback, the llm · server and llm · proxy series keep the model the caller asked for as requested, while the client that served the request records the fallback model.

${llm.response.model} is known only once the reply starts, so it doesn't label llm.active.requests, and it is left out when the reply reports no model or the exchange ends before the reply starts, such as when the deployment can't be reached.

Only these forms are supported. Each distinct value creates its own series, so avoid attributes with unbounded values.

Detecting Issues

Each metric points at a different layer of the pipeline, depending on which one moves and how it's dimensioned:

• mcp.tools.call failures isolated to one tool (outcome attribute): that upstream or toolkit is degraded, not the gateway itself. See Why does a tool call route to the wrong upstream, or fail to route at all?

• Rising mcp.tools.list.duration with no connectivity events in the logs: cache or store latency, not an upstream problem. See Listing Cache and, for a multi-replica deployment, the shared-store guidance in Scale MCP Gateway.

• Rising mcp.initialize counts without a matching rise in traffic: session churn. Cross-check against BINDING_MCP_SESSION_ESTABLISHED/BINDING_MCP_SESSION_CLOSED in the logs to confirm whether sessions are opening and closing more often than expected.

• llm.interactions rising on a client binding whose provider isn't answering: requests are reaching a deployment that can't be reached, and falling back to another model if their route lists one.

• Rising llm.usage.incomplete: replies are ending without a token usage report, so token metrics undercount. Check whether the provider reports usage for streamed replies.

• Rising llm.time.to.first.token on one client binding: that provider is slow to start replying, not the gateway.

There's no rate-limiting or throttling binding today, so a volume spike on mcp.tools.call by itself isn't something the gateway can act on directly, it's a signal to look at the upstream tool provider, not a gateway-side control.

See Troubleshooting Guides for the full symptom-to-fix reference.