Unified API

Zilla accepts requests in the OpenAI and Anthropic APIs. Applications keep using the official SDKs and change only the base URL. Each API is a dialect of the llm binding.

Supported APIs

Dialect	API	Request path	API key header
`openai`	OpenAI Chat Completions	`POST /v1/chat/completions`	`authorization`
`anthropic`	Anthropic Messages	`POST /v1/messages`	`x-api-key`

Both dialects work on an llm · server, to accept requests, and on an llm · client, to call a provider. Zilla Plus adds the aws-bedrock dialect, for calling AWS Bedrock and for accepting requests in the Bedrock Converse API, POST /model/{modelId}/converse and /converse-stream.

Point an SDK at the Gateway

SDK	Base URL	API key
OpenAI	`http://<gateway-host>:<port>/v1`	Sent as `Authorization: Bearer <key>`
Anthropic	`http://<gateway-host>:<port>`	Sent as `x-api-key: <key>`

See Client Setup for complete examples with each SDK.

Dialect Detection

llm · server resolves each request's dialect in one of two ways.

Fixed Dialect

Set options.dialect to accept one API on a listener. Every request is treated as that dialect, whatever its path or method:

north_llm_server:
  type: llm
  kind: server
  options:
    dialect: openai
  exit: north_llm_proxy

To accept both APIs with fixed dialects, give each its own tcp · server port, http · server, and llm · server, and send both to the same llm · proxy.

Automatic Detection

Leave options.dialect unset to accept both APIs on one listener. Each request must be recognized by exactly one dialect:

Dialect	Recognizes
`openai`	`POST /v1/chat/completions` with the content type `application/json`
`anthropic`	`POST /v1/messages`, or any request with an `anthropic-version` header, or with an `x-api-key` header and no `authorization` header

A request that no dialect recognizes, or that more than one does, is rejected.

Request Checks

llm · server checks each request in this order:

The content type must be application/json, with or without parameters. Otherwise the request gets 415 Unsupported Media Type.
When options.authorization is set, the API key must be accepted by the guard. Otherwise the request gets 401 Unauthorized. See LLM Credentials.
Anthropic API requests must carry the header anthropic-version: 2023-06-01. A missing header or another version gets 400 Bad Request.
The body must be valid JSON in the dialect's format, with a model. Otherwise the request gets 400 Bad Request.

The official SDKs send the right content type and headers.

Streaming

Set "stream": true in the request body to receive the reply as Server-Sent Events (text/event-stream). Without it, the reply is a single JSON body (application/json).

API	Stream ends with
OpenAI	A `[DONE]` data line
Anthropic	A `message_delta` event, then `message_stop`

Headers

The provider receives the request's content type, the API key header that llm · client adds, and, for the Anthropic API, anthropic-version: 2023-06-01 and the caller's anthropic-beta header, if sent. No other request headers are forwarded.

The caller receives the reply's status, content type, and body. No other response headers are returned.

Errors

Errors are returned in the error format of the request's API.

Status	Cause
`400 Bad Request`	The request body isn't valid, the Anthropic `anthropic-version` header is missing or unsupported, an LLM guardrail rejected it, or it can't be translated to the provider's API.
`401 Unauthorized`	The API key was missing or rejected by the guard in `options.authorization`.
`415 Unsupported Media Type`	The content type is not `application/json`.
`502 Bad Gateway`	The provider couldn't be reached.
Provider status	The provider rejected the request, for example with `429 Too Many Requests`. Its status, error type, and message are kept, even when the provider uses a different API, and so is its OpenAI error `code` when it sends one.

OpenAI errors use the OpenAI format, here for a 401:

{
  "error": {
    "message": "Incorrect API key provided.",
    "type": "invalid_request_error",
    "param": null,
    "code": "invalid_api_key"
  }
}

Anthropic errors use the Anthropic format, here for a 401:

{
  "type": "error",
  "error": {
    "type": "authentication_error",
    "message": "invalid x-api-key"
  }
}

When a stream fails after it has started, the error arrives in the stream: an OpenAI chunk with an error object, or an Anthropic error event.

Next Steps

Dialect Translation: what changes when a request is sent to a provider with a different API.
Model Routing: send each model to the right provider.
llm · server: the full llm · server reference.