Azure AI Foundry

An llm · client with dialect: azure-foundry sends requests to Azure OpenAI chat completions in Azure AI Foundry. Applications keep using the OpenAI, Anthropic, or Gemini SDK: Zilla Plus converts each request for Azure OpenAI and returns the reply in the caller's API. An llm · server can also accept requests in the Azure OpenAI API, so an application written for Azure OpenAI calls Zilla the same way it calls its Azure OpenAI resource.

Available in Zilla Plus

Configuration

azure_llm_client:
  type: llm
  kind: client
  options:
    dialect: azure-foundry
    server: https://my-resource.openai.azure.com/openai/v1
    authorization:
      api_key:
        credentials: "{credentials}"
  telemetry:
    metrics:
      - llm.*
  exit: sys:http_client

server is the Azure OpenAI resource, with the /openai/v1 path of the v1 API. Each request is sent to /openai/v1/chat/completions, with the model in the request body, so callers ask for a model by its deployment name. The client connects through sys:http_client, which uses TLS when the URL is https://.

Route the deployment names to it from llm · proxy:

north_llm_proxy:
  type: llm
  kind: proxy
  routes:
    - when:
        - model:
            - gpt-4o-mini
      exit: azure_llm_client

Classic Deployments API

Resources that use the classic Azure OpenAI API take the deployment in the request path and an api-version query parameter. Set options.classic to send requests that way:

azure_llm_client:
  type: llm
  kind: client
  options:
    dialect: azure-foundry
    server: https://my-resource.openai.azure.com
    classic:
      api-version: 2024-10-21
      deployments:
        gpt-4o: my-gpt-4o-deployment
    authorization:
      api_key:
        credentials: "{credentials}"
  exit: sys:http_client

Each request is sent to /openai/deployments/{deployment}/chat/completions?api-version={api-version}, and the path of server is ignored. deployments maps the model the caller asked for to the deployment name, and a model with no entry uses its own name as the deployment name. api-version defaults to 2024-06-01. The model isn't sent in the request body.

Credentials

API Key

options.authorization sends a key to Azure OpenAI. The header depends on the credentials template:

Template	Header sent
`"{credentials}"`	`api-key: <key>`
`"Bearer {credentials}"`	`authorization: Bearer <key>`

The key is the one the named guard returns, such as each caller's own key with an inline guard. See LLM Credentials for forwarding the caller's key, or holding one key in the gateway.

When the request was accepted by an llm · server with dialect: azure-foundry and authorized by a guard, the client sends the key in the header the caller used, api-key or authorization, and the template is not applied.

Microsoft Entra ID

To authenticate with a Microsoft Entra ID access token instead of a key, set options.entra and leave out authorization:

azure_llm_client:
  type: llm
  kind: client
  options:
    dialect: azure-foundry
    server: https://my-resource.openai.azure.com/openai/v1
    entra:
      scope: https://ai.azure.com/.default
  exit: sys:http_client

Zilla Plus requests the token when the binding starts, sends it as authorization: Bearer <token>, and refreshes it five minutes before it expires, so no key is stored in zilla.yaml. The credentials come from the first of these that the environment configures:

Workload identity: AZURE_TENANT_ID, AZURE_CLIENT_ID, and AZURE_FEDERATED_TOKEN_FILE.
A client secret: AZURE_TENANT_ID, AZURE_CLIENT_ID, and AZURE_CLIENT_SECRET.
A managed identity. AZURE_CLIENT_ID selects a user-assigned identity when it is set.

entra.client-id selects a user-assigned managed identity, entra.scope defaults to https://ai.azure.com/.default, and AZURE_AUTHORITY_HOST overrides the default authority, https://login.microsoftonline.com. When no token is available within two seconds, the request fails with 503 Service Unavailable and the message The Microsoft Entra access token is not available.

What Is Converted

Azure OpenAI uses the Chat Completions format. When the caller's API differs, each part of the conversation is converted section by section, as with any translation: system instructions, user and assistant text, tool definitions, tool calls, and tool results.

The output token limit is sent as max_completion_tokens.
OpenAI settings, such as temperature, n, logprobs, and parallel_tool_calls, are sent as the caller gave them, as are the Azure settings data_sources, user_security_context, and enhancements. Settings of other APIs are dropped.
stream_options is sent as the caller gave it, so a streaming caller must set stream_options.include_usage to true to receive token usage, and for the llm.* token metrics to record it.

Some content can't be converted:

Images, documents, audio, video, and tools the provider runs are rejected with 400 Bad Request, naming the section, such as user-image cannot be converted from openai to azure-foundry.
Reasoning and citations in the request are dropped.

Replies and Content Filtering

Finish reasons are the same as OpenAI's. Azure's content filter results are kept with the content they describe: a choice's content_filter_results, content_filter_offsets, and logprobs, and the response's prompt_filter_results and prompt_annotations. They are returned to a caller that uses the OpenAI or Azure OpenAI API, are not translated to other APIs, and are dropped when larger than 4096 bytes. Reasoning, refusals, annotations, and audio in a reply are dropped.

When the content filter blocks the prompt, the caller receives 400 Bad Request with the code content_filter and the param prompt. When it blocks the completion, the reply ends with the finish reason content_filter.

Accepting the Azure OpenAI API

An llm · server with dialect: azure-foundry, or with no dialect, also accepts requests in the Azure OpenAI API, as a POST with content-type: application/json to either path:

Path	API
`/openai/v1/chat/completions`	v1, with the model in the request body
`/openai/deployments/{deployment}/chat/completions?api-version=...`	Classic, with the deployment as the model

north_llm_server:
  type: llm
  kind: server
  options:
    dialect: azure-foundry
    authorization:
      api_key:
        credentials: "{credentials}"
  exit: north_llm_proxy

An Azure OpenAI request is handled as an OpenAI request, so llm · proxy routes it like any other, to Azure OpenAI or to another provider, and matches it with when.dialect: openai. On the classic path, the deployment in the path is the model, and a model in the request body is ignored. The classic path needs a non-empty api-version.

Credentials: the key is read from the authorization header when the template starts with Bearer, and from the api-key header otherwise. A request that sends the key in the other header is also accepted. A rejected request gets 401 Unauthorized with the code invalid_api_key.
Message content: each message's content must be a string. A request with content parts, such as an image, gets 400 Bad Request.
Other paths: with dialect: azure-foundry, a request to any other path, or to the classic path without api-version, gets 404 Not Found.

See Azure OpenAI SDK to call the gateway with the SDK's AzureOpenAI client.

Next Steps

LLM Credentials: forward the caller's key, or hold one key in the gateway.
Model routing: route deployment names to Azure OpenAI.
llm · client: the full llm · client reference.