Skip to main content

LLM Credentials

The LLM Gateway handles credentials at both ends of a request. llm · server reads the API key each caller sends and passes it to a guard. llm · client adds an API key to each request it sends to a provider, in the header that provider expects. Both use the same options.authorization setting, naming a guard and a credentials template.

Credentials Templates

A template describes how the API key appears in the header, with {credentials} standing for the key itself. The header depends on the API:

APIHeaderTemplate
OpenAIauthorization"Bearer {credentials}"
Anthropicx-api-key"{credentials}"

The default template is Bearer {credentials} for both APIs, so set credentials: "{credentials}" explicitly for the Anthropic API.

Inbound Authorization

On llm · server, options.authorization passes the caller's API key to the named guard:

north_llm_server:
  type: llm
  kind: server
  options:
    dialect: openai
    authorization:
      api_key:
        credentials: "Bearer {credentials}"
  exit: north_llm_proxy

A request whose header is missing, doesn't match the template, or whose key the guard rejects, gets 401 Unauthorized in the error format of its API.

Bring Your Own Key

To forward each caller's own provider key, name the same inline guard on the llm · server and on each llm · client. No provider key is written to zilla.yaml:

guards:
  api_key:
    type: inline

bindings:
  north_llm_server:
    type: llm
    kind: server
    options:
      dialect: openai
      authorization:
        api_key:
          credentials: "Bearer {credentials}"
    exit: north_llm_proxy

  anthropic_llm_client:
    type: llm
    kind: client
    options:
      dialect: anthropic
      server: https://api.anthropic.com
      authorization:
        api_key:
          credentials: "{credentials}"
    exit: sys:http_client

The key moves between headers when Zilla translates: here an OpenAI SDK caller sends Authorization: Bearer <key>, and the request reaches Anthropic with x-api-key: <key>.

The inline guard doesn't validate the key: the provider does. Send the key for the provider that serves the model you ask for.

Holding the Provider Key in the Gateway

When callers authenticate with their own tokens, such as JWTs, rather than provider keys, the provider key is held by the gateway instead. Leave options.authorization off the llm · client, so it doesn't forward the caller's token, and exit to an http · client that adds the provider key to every request it sends:

openai_llm_client:
  type: llm
  kind: client
  options:
    dialect: openai
    server: https://api.openai.com
  exit: openai_http_client

openai_http_client:
  type: http
  kind: client
  options:
    overrides:
      authorization: Bearer ${{env.OPENAI_API_KEY}}
  exit: sys:tls_client

For the Anthropic API, override x-api-key instead. The key is read from the OPENAI_API_KEY environment variable, so it isn't written to zilla.yaml.

Restricting Models by Role

A route on llm · proxy can require roles from the guard that authorized the request. A request without the roles falls through to the next route. Here callers with the premium role in their JWT are sent to their own client, and every other caller to the default client:

north_llm_proxy:
  type: llm
  kind: proxy
  routes:
    - when:
        - model:
            - gpt-*
      guarded:
        llm_jwt:
          - premium
      exit: openai_llm_client_premium
    - when:
        - model:
            - gpt-*
      exit: openai_llm_client

See Secure LLM Access with JWT for a complete example.

AWS Bedrock ★Zilla Plus

An llm · client with dialect: aws-bedrock doesn't use options.authorization. Zilla Plus signs each request with AWS Signature Version 4, using the AWS credentials it runs with. See AWS Bedrock.

An llm · server that accepts the Bedrock Converse API checks the request's Authorization header against the guard in options.authorization, like any other request. It doesn't verify AWS Signature Version 4. See Accepting the Bedrock Converse API.

Next Steps