Unified API

Zilla accepts requests in the OpenAI, Anthropic, and Gemini APIs. Applications keep using the official SDKs and change only the base URL. Each API is a dialect of the llm binding.

Supported APIs

Dialect	API	Request path	API key header
`openai`	OpenAI Chat Completions and Responses	`POST /v1/chat/completions`, `POST /v1/responses`	`authorization`
`anthropic`	Anthropic Messages	`POST /v1/messages`	`x-api-key`
`gemini`	Google Gemini generateContent	`POST /v1beta/models/{model}:generateContent` or `:streamGenerateContent`, or the same paths under `/v1`	`x-goog-api-key`

All three dialects work on an llm · server, to accept requests, and on an llm · client, to call a provider. Zilla Plus adds the aws-bedrock dialect, for calling AWS Bedrock and for accepting requests in the Bedrock Converse API, POST /model/{modelId}/converse and /converse-stream, the azure-foundry dialect, for calling Azure AI Foundry and for accepting requests in the Azure OpenAI API, and the gcp-vertex dialect, for calling Google Vertex AI and for accepting requests in the Vertex AI APIs.

Point an SDK at the Gateway

SDK	Base URL	API key
OpenAI	`http://<gateway-host>:<port>/v1`	Sent as `Authorization: Bearer <key>`
Anthropic	`http://<gateway-host>:<port>`	Sent as `x-api-key: <key>`
Gemini	`http://<gateway-host>:<port>`	Sent as `x-goog-api-key: <key>`
Azure OpenAI ★Zilla Plus	`http://<gateway-host>:<port>`, as `azure_endpoint`	Sent as `api-key: <key>`
AWS SDK, Bedrock Runtime ★Zilla Plus	`http://<gateway-host>:<port>`, as `endpoint_url`	None: the request is signed with AWS Signature Version 4, and the gateway holds the provider keys

See Client Setup for complete examples with each SDK.

Dialect Detection

llm · server resolves each request's dialect in one of three ways.

Fixed Dialect

Set options.dialect to accept one dialect on a listener. With openai, both OpenAI APIs are served, told apart by the request path, and options.apis narrows them to chat/completions or responses. With anthropic, every request is treated as that dialect, whatever its path or method. With gemini, a request that isn't a POST to one of the Gemini paths gets 404 Not Found:

north_llm_server:
  type: llm
  kind: server
  options:
    dialect: openai
  exit: north_llm_proxy

A Chosen Set of Dialects

Set options.dialects to accept some of the dialects on one listener, told apart by the request path. Each entry is a dialect name, or, for openai, an object that narrows its APIs:

north_llm_server:
  type: llm
  kind: server
  options:
    dialects:
      - openai:
          apis:
            - chat/completions
      - anthropic
  exit: north_llm_proxy

This listener accepts OpenAI Chat Completions and Anthropic Messages requests, and rejects OpenAI Responses and Gemini requests. dialects and dialect can't be set together. To give each API its own port instead, use one tcp · server, http · server, and llm · server per API, and send them all to the same llm · proxy.

Automatic Detection

Leave options.dialect unset to accept every API on one listener. Each request must be recognized by exactly one dialect:

Dialect	Recognizes
`openai`	`POST /v1/chat/completions` or `POST /v1/responses` with the content type `application/json`
`anthropic`	`POST /v1/messages`, or any request with an `anthropic-version` header, or with an `x-api-key` header and no `authorization` header
`gemini`	`POST /v1beta/models/{model}:generateContent` or `:streamGenerateContent`, or the same paths under `/v1`

A request that no dialect recognizes, or that more than one does, is rejected.

Request Checks

llm · server checks each request in this order:

The content type must be application/json, with or without parameters. Otherwise the request gets 415 Unsupported Media Type.
When options.authorization is set, the API key must be accepted by the guard. Otherwise the request gets 401 Unauthorized. A Gemini API request instead gets 403 Forbidden when it has no key, and 400 Bad Request when the guard rejects its key. See LLM Credentials.
Anthropic API requests must carry the header anthropic-version: 2023-06-01. A missing header or another version gets 400 Bad Request.
The body must be valid JSON in the dialect's format, with a model. Otherwise the request gets 400 Bad Request. A Gemini API request takes the model from its path instead.

The official SDKs send the right content type and headers.

Streaming

Set "stream": true in the request body to receive the reply as Server-Sent Events (text/event-stream). Without it, the reply is a single JSON body (application/json).

API	Stream ends with
OpenAI Chat Completions	A `[DONE]` data line
OpenAI Responses	A `response.completed` event
Anthropic	A `message_delta` event, then `message_stop`

The Gemini API chooses streaming by path instead: :streamGenerateContent streams the reply, as Server-Sent Events when the query string has alt=sse and as a JSON array of chunks otherwise, and :generateContent returns a single JSON body.

Headers

The provider receives the request's content type, the API key header that llm · client adds, and, for the Anthropic API, anthropic-version: 2023-06-01 and the caller's anthropic-beta header, if sent. No other request headers are forwarded.

The caller receives the reply's status, content type, and body. No other response headers are returned.

Errors

Errors are returned in the error format of the request's API.

Status	Cause
`400 Bad Request`	The request body isn't valid, the Anthropic `anthropic-version` header is missing or unsupported, an LLM guardrail rejected it, or it can't be translated to the provider's API.
`401 Unauthorized`	The API key was missing or rejected by the guard in `options.authorization`. A Gemini API request gets `403` for a missing key and `400` for a rejected one.
`404 Not Found`	A Gemini API request isn't a `POST` to a `generateContent` or `streamGenerateContent` path.
`415 Unsupported Media Type`	The content type is not `application/json`.
`502 Bad Gateway`	The provider couldn't be reached.
Provider status	The provider rejected the request, for example with `429 Too Many Requests`. Its status, error type, and message are kept, even when the provider uses a different API, and so is its OpenAI error `code` when it sends one.

OpenAI errors use the OpenAI format, here for a 401:

{
  "error": {
    "message": "Incorrect API key provided.",
    "type": "invalid_request_error",
    "param": null,
    "code": "invalid_api_key"
  }
}

Anthropic errors use the Anthropic format, here for a 401:

{
  "type": "error",
  "error": {
    "type": "authentication_error",
    "message": "invalid x-api-key"
  }
}

Gemini errors use the Google API format, with the status name, here for a request rejected by a guardrail:

{
  "error": {
    "code": 400,
    "message": "user-text rejected",
    "status": "INVALID_ARGUMENT"
  }
}

When a stream fails after it has started, the error arrives in the stream: an OpenAI chunk with an error object, or an Anthropic error event.

Next Steps

Dialect Translation: what changes when a request is sent to a provider with a different API.
Model Routing: send each model to the right provider.
llm · server: the full llm · server reference.