Anthropic SDK

The Anthropic SDK talks to the LLM Gateway through an llm · server with dialect: anthropic. Set the SDK's base URL to the gateway, without a path, and keep the rest of the code as it is.

Configure the Client

from anthropic import Anthropic

client = Anthropic(
    base_url="http://localhost:7162",
    api_key="<your-provider-api-key>")

The SDK sends the key as x-api-key: <key>. With credential pass-through, send the key for the provider that serves the model you ask for.

Send a Request

resp = client.messages.create(
    model="claude-haiku-4-5",
    max_tokens=512,
    messages=[{"role": "user", "content": "In one sentence, what is an LLM gateway?"}])
print(resp.content[0].text)

The model decides which provider answers, through the routes on llm · proxy.

Stream the Reply

with client.messages.stream(
        model="claude-haiku-4-5",
        max_tokens=512,
        messages=[{"role": "user", "content": "In one sentence, what is an LLM gateway?"}]) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

Call Tools

resp = client.messages.create(
    model="claude-haiku-4-5",
    max_tokens=512,
    messages=[{"role": "user", "content": "What is the weather in Paris?"}],
    tools=[{
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "input_schema": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    }])
for block in resp.content:
    if block.type == "tool_use":
        print(block.name, block.input)

Handle Errors

Zilla returns errors in the Anthropic error format, so the SDK raises them as anthropic.APIStatusError:

import anthropic

try:
    resp = client.messages.create(
        model="claude-haiku-4-5",
        max_tokens=512,
        messages=[{"role": "user", "content": "My email is jane.doe@example.com"}])
except anthropic.APIStatusError as error:
    print(error.status_code, error.message)

A request rejected by an LLM guardrail, for example, raises a 400 whose message names the rejected section, such as user-text rejected.