Unified LLM API
An application written against the OpenAI SDK speaks the OpenAI API. Moving it to a Claude model, or adding Claude next to GPT, means adding the Anthropic SDK, rewriting requests and replies in another format, and handling a second set of errors, in every application that wants the change.
The Problem
Each LLM provider has its own API, SDK, request format, reply format, and error format. An organization that uses more than one provider ends up with the provider choice baked into application code: switching a feature from one model to another is a code change, a test cycle, and a deployment, repeated in every application, and every team maintains its own copy of the same adapter code.
How Zilla Solves It
The LLM Gateway accepts both the OpenAI and the Anthropic API, and sends each request to the provider that serves its model, in that provider's API. When the two APIs differ, Zilla translates the request and the reply. The application keeps its SDK and changes only the base URL: which provider answers is decided by the model it asks for and the gateway's routes.
Architecture
What Changes in the Application
Only the base URL:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:7161/v1", api_key="<your-provider-api-key>")Asking for gpt-4o-mini reaches OpenAI. Asking for claude-haiku-4-5 reaches Anthropic, with the reply returned in the OpenAI format the SDK expects. Errors come back in the OpenAI format too.
What Gets Translated
Messages, system instructions, tool definitions, tool calls, and streaming replies are translated between the two APIs. Some request settings and reply content aren't translated. See Dialect Translation for exactly what is carried across.
Try It
The LLM Gateway demo runs an OpenAI and an Anthropic frontend side by side, so you can send the same prompt through both SDKs to both providers.
Try the guide
Walk through LLM Gateway Setup to build the gateway from scratch.