Skip to main content

Bring Your Own Key

The Bring Your Own Key use case describes forwarding each caller's own provider key instead of storing keys in the gateway. This guide runs an Anthropic API endpoint that routes gpt-* models to OpenAI, and shows the caller's key moving from the Anthropic x-api-key header to the OpenAI Authorization header.

Prerequisites

docker-compose.yaml

Download docker-compose.yaml

name: zilla-llm-bring-your-own-key
services:
  zilla:
    image: ghcr.io/aklivity/zilla:${ZILLA_VERSION:-2.5.0-rc5}
    restart: unless-stopped
    ports:
      - 7162:7162
    healthcheck:
      interval: 5s
      timeout: 3s
      retries: 15
      start_period: 15s
      start_interval: 250ms
      test: ["CMD", "sh", "-c", "test -f /var/run/zilla/ready"]
    volumes:
      - ./zilla.yaml:/etc/zilla/zilla.yaml:ro
    command: start -v -e
zilla.yaml

Download zilla.yaml

name: llm-bring-your-own-key
#region guard
guards:
  api_key:
    type: inline
#endregion guard
bindings:
  north_tcp_server:
    type: tcp
    kind: server
    options:
      host: 0.0.0.0
      port: 7162
    exit: north_http_server
  north_http_server:
    type: http
    kind: server
    exit: north_llm_server
  #region llm_server
  north_llm_server:
    type: llm
    kind: server
    options:
      dialect: anthropic
      authorization:
        api_key:
          credentials: "{credentials}"
    exit: north_llm_proxy
  #endregion llm_server
  north_llm_proxy:
    type: llm
    kind: proxy
    routes:
      - when:
          - model:
              - gpt-*
        exit: south_llm_client_openai
  #region llm_client
  south_llm_client_openai:
    type: llm
    kind: client
    options:
      dialect: openai
      server: https://api.openai.com
      authorization:
        api_key:
          credentials: "Bearer {credentials}"
    exit: sys:http_client
  #endregion llm_client

One inline guard accepts any key and hands it back unchanged:

guards:
  api_key:
    type: inline

llm · server reads the key from the Anthropic x-api-key header, which carries it with no prefix:

north_llm_server:
  type: llm
  kind: server
  options:
    dialect: anthropic
    authorization:
      api_key:
        credentials: "{credentials}"
  exit: north_llm_proxy

llm · client writes the same key into the OpenAI Authorization header, with the Bearer prefix:

south_llm_client_openai:
  type: llm
  kind: client
  options:
    dialect: openai
    server: https://api.openai.com
    authorization:
      api_key:
        credentials: "Bearer {credentials}"
  exit: sys:http_client

No provider key appears anywhere in zilla.yaml.

Start the Stack

docker compose up -d --wait

Send Your Key

Call the Anthropic API on port 7162 for gpt-4o-mini, with your OpenAI key in x-api-key:

curl -s http://localhost:7162/v1/messages \
  -H "x-api-key: $OPENAI_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","max_tokens":60,"messages":[{"role":"user","content":"In one sentence, what is an LLM gateway?"}]}'

OpenAI answers: Zilla sent your key to OpenAI as Authorization: Bearer <key>.

Send a Wrong Key

curl -s http://localhost:7162/v1/messages \
  -H "x-api-key: sk-not-a-real-key" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","max_tokens":60,"messages":[{"role":"user","content":"Hello"}]}'
{"type":"error","error":{"type":"authentication_error","message":"Unauthorized"}}

OpenAI rejected the key, and Zilla returned its 401 in the Anthropic error format.

Send No Key

Leave out x-api-key:

curl -s http://localhost:7162/v1/messages \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","max_tokens":60,"messages":[{"role":"user","content":"Hello"}]}'
{"type":"error","error":{"type":"authentication_error","message":"invalid x-api-key"}}

This time Zilla rejected the request itself with 401, before it reached OpenAI.

Stop the Stack

docker compose down

Next Steps