Gemini SDK

The Google Gen AI SDK talks to the LLM Gateway through an llm · server with dialect: gemini. Set the SDK's base URL to the gateway, without a path, and keep the rest of the code as it is.

Configure the Client

from google import genai
from google.genai import types

client = genai.Client(
    api_key="<your-provider-api-key>",
    http_options=types.HttpOptions(base_url="http://localhost:7163", api_version="v1beta"))

The SDK sends the key as x-goog-api-key: <key>, and the model in the request path. With credential pass-through, send the key for the provider that serves the model you ask for.

Send a Request

resp = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="In one sentence, what is an LLM gateway?")
print(resp.text)

The model decides which provider answers, through the routes on llm · proxy, so the same client can ask for a claude-* or gpt-* model.

Stream the Reply

for chunk in client.models.generate_content_stream(
        model="gemini-2.5-flash",
        contents="In one sentence, what is an LLM gateway?"):
    print(chunk.text, end="", flush=True)

Call Tools

resp = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="What is the weather in Paris?",
    config=types.GenerateContentConfig(
        tools=[types.Tool(function_declarations=[types.FunctionDeclaration(
            name="get_weather",
            description="Get the current weather for a city",
            parameters_json_schema={
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            })])],
        automatic_function_calling=types.AutomaticFunctionCallingConfig(disable=True)))
for call in resp.function_calls or []:
    print(call.name, call.args)

Handle Errors

Zilla returns errors in the Gemini error format, so the SDK raises them as google.genai.errors.APIError:

from google.genai import errors

try:
    resp = client.models.generate_content(
        model="gemini-2.5-flash",
        contents="My email is jane.doe@example.com")
except errors.APIError as error:
    print(error.code, error.message)

A request rejected by an LLM guardrail, for example, raises a 400 whose message names the rejected section, such as user-text rejected.