Dialect Translation
When an llm · client speaks a different dialect from the request, Zilla translates the request into the client's API and the reply back into the request's API. The application sees only its own API, whichever provider answers.
When Translation Happens
Translation happens when the request's dialect differs from the dialect of the llm · client its route leads to, for example an OpenAI SDK request for claude-haiku-4-5 routed to an Anthropic client, or an Anthropic SDK request for gpt-4o-mini routed to an OpenAI client.
When the two dialects match, Zilla forwards the request and the reply unchanged, with every setting the caller sent.
What Is Translated in a Request
Zilla translates the conversation, section by section:
| Section | OpenAI API | Anthropic API |
|---|---|---|
| System instruction | A system or developer message | The top-level system |
| User text | Message content | Text blocks |
| Images | image_url, a URL or data: URL | image blocks with a URL or base64 source |
| Documents | file parts | document blocks |
| Assistant text | Message content | Text blocks |
| Tool definitions | tools[].function with parameters | tools[] with input_schema |
| Tool calls | tool_calls, with arguments as a JSON string | tool_use blocks, with input as an object |
| Tool results | tool messages | tool_result blocks |
The output token limit is carried across too:
| Setting | OpenAI API | Anthropic API |
|---|---|---|
| Output token limit | max_completion_tokens or max_tokens | max_tokens |
When a request without a limit is sent to Anthropic, Zilla sets max_tokens to 4096. With Zilla Plus, the limit is also carried to and from the AWS Bedrock Converse API, as inferenceConfig.maxTokens.
What Isn't Translated in a Request
Other top-level settings are not carried across, including temperature, top_p, stop, n, tool_choice, response_format, metadata, and thinking.
Some content can't be translated:
| Content | From OpenAI to Anthropic | From Anthropic to OpenAI |
|---|---|---|
| Audio input | Rejected | — |
| Video input | Rejected | Rejected |
| Typed server tools, such as a built-in web search tool | — | Rejected |
Reasoning (thinking), server tool results, citations | — | Dropped |
A request with content that is rejected gets 400 Bad Request, with a message naming the section, such as tool-definition cannot be converted from anthropic to openai.
What Is Translated in a Reply
Text, tool calls, refusals, and the finish reason are returned in the request's API.
| Finish reason | OpenAI caller | Anthropic caller |
|---|---|---|
| Natural end | stop | end_turn |
| Token limit | length | max_tokens |
| Tool call | tool_calls | tool_use |
| Refusal or content filter | content_filter | refusal |
A refusal is returned as message.refusal to an OpenAI caller, and as a text block with stop_reason: "refusal" to an Anthropic caller. Reasoning is returned to an Anthropic caller as a thinking block, and not returned to an OpenAI caller. Citations are not returned.
Only one choice is returned, even when n is greater than 1, and logprobs are not returned.
Token Usage
Token counts are passed through from the provider into the request's API:
| Count | OpenAI caller | Anthropic caller |
|---|---|---|
| Input | prompt_tokens | input_tokens |
| Output | completion_tokens | output_tokens |
| Total | total_tokens, input plus output when the provider doesn't report it | Not returned |
| Cache read | prompt_tokens_details.cached_tokens | cache_read_input_tokens |
| Cache write | Not returned | cache_creation_input_tokens |
| Reasoning | completion_tokens_details.reasoning_tokens | Not returned |
The llm.* metrics record the counts the provider reports, whether or not the caller's API can show them.
Streaming
A request that asks for a stream receives one in its own API's event format:
- OpenAI caller: a chunk with the role, content deltas, tool calls as a chunk with the id and name followed by argument fragments, a chunk with the
finish_reason, then[DONE]. When the request setsstream_options.include_usagetotrue, a usage chunk with emptychoicescomes before[DONE]. - Anthropic caller:
message_start, thencontent_block_start,content_block_delta, andcontent_block_stopfor each block, thenmessage_deltawith the stop reason and token usage, thenmessage_stop. Themessage_startevent reports zero tokens.
Errors
When the provider rejects a translated request, its status, error type, and message are kept, in the error format of the request's API. See Unified API.
Next Steps
- Unified API: the APIs Zilla accepts.
- Providers: the clients that call each provider.
- llm · client: the full
llm · clientreference.