LLM Metrics

TelemetryMetrics

LLM Metrics

Zilla runtime telemetry type

telemetry:
  metrics:
    - llm.active.requests
    - llm.interactions
    - llm.duration
    - llm.time.to.first.token
    - llm.tokens.input
    - llm.tokens.output
    - llm.tokens.total
    - llm.tokens.cache.read
    - llm.tokens.cache.write
    - llm.tokens.reasoning
    - llm.usage.incomplete

LLM metrics are recorded by the llm server, proxy and client bindings, one sample per LLM request and response exchange. Token counts are taken from the usage reported by the upstream LLM server. A token count that the upstream dialect does not report is not recorded, rather than recorded as 0. For example, anthropic reports no total or reasoning tokens.

LLM metrics can carry attributes resolved from the guard that authorized the request, see Attributes.

Configuration

Metrics

metrics

* required

metrics

The list of metric names available to record and export.

llm.active.requests

The number of active llm requests, from the request until both the request and response are complete.

llm.interactions

The number of llm exchanges, counted once per exchange when its response completes, or when the exchange fails.

llm.duration

The duration of llm exchanges in nanoseconds, from the request to the end of the response. Exchanges that are reset or aborted are not recorded.

llm.time.to.first.token

The time in nanoseconds from the llm request to the first data of its response. Exchanges that end before any response data are not recorded.

llm.tokens.input

The number of llm input tokens per exchange.

llm.tokens.output

The number of llm output tokens per exchange.

llm.tokens.total

The number of llm total tokens per exchange.

llm.tokens.cache.read

The number of llm input tokens read from the prompt cache per exchange.

llm.tokens.cache.write

The number of llm input tokens written to the prompt cache per exchange.

llm.tokens.reasoning

The number of llm reasoning output tokens per exchange.

llm.usage.incomplete

The number of llm exchanges whose response reports no token usage, or that are aborted after the response has started, for example when a response section is rejected by an llm proxy guardrail.

Attributes

LLM metrics carry the telemetry.attributes of the binding that records them, when an attribute value resolves from the guard that authorized the request.

telemetry:
  attributes:
    user: ${guarded['my_api_key_guard'].identity}
    tenant: ${guarded['my_api_key_guard'].attributes.tenant}
  metrics:
    - llm.*

Value	Resolves to
`${guarded['<guard>'].identity}`	The identity of the request's authorization by the named guard.
`${guarded['<guard>'].attributes.<name>}`	The named attribute of the request's authorization by the named guard.

Values are resolved when the request starts. Other attribute values are not applied to LLM metrics, and an attribute whose value does not resolve is omitted.