OpenAI Apps on AWS Bedrock

Use CaseLLMAWSAI Gateway

An organization standardizing on Amazon Bedrock often has applications already written against the OpenAI SDK. Moving them means rewriting each one for the Bedrock Converse API, and for AWS request signing.

The Problem

The Bedrock Converse API differs from the OpenAI API in its request and reply formats and in authentication: every request is signed with AWS Signature Version 4 from IAM credentials. Each application that moves to Bedrock needs new client code, and access to AWS credentials it didn't need before.

How Zilla Solves It

An llm · client with dialect: aws-bedrock serves OpenAI and Anthropic API requests from Bedrock. Zilla Plus converts each request for Bedrock, signs it with the AWS credentials it runs with, and returns the reply in the caller's API. Applications keep their SDK and point it at the gateway, and they never hold AWS credentials.

Architecture

Configuration

north_llm_proxy:
  type: llm
  kind: proxy
  routes:
    - when:
        - model:
            - anthropic.*
            - amazon.*
            - us.anthropic.*
            - us.amazon.*
      exit: bedrock_llm_client

bedrock_llm_client:
  type: llm
  kind: client
  options:
    dialect: aws-bedrock
    server: https://bedrock-runtime.us-east-1.amazonaws.com
  exit: bedrock_http_client

bedrock_http_client:
  type: http
  kind: client
  exit: bedrock_tls_client

bedrock_tls_client:
  type: tls
  kind: client
  options:
    sni:
      - bedrock-runtime.us-east-1.amazonaws.com
  exit: bedrock_tcp_client

bedrock_tcp_client:
  type: tcp
  kind: client
  options:
    host: bedrock-runtime.us-east-1.amazonaws.com
    port: 443

Zilla Plus takes AWS credentials from the standard AWS default credentials chain, such as an EKS pod's IAM role, an ECS task role, or an EC2 instance profile, so no AWS key is stored in zilla.yaml. See AWS Bedrock.

Limits

A request must fit in one of Zilla's buffer slots, 32 KiB by default.
Images and documents sent by URL, and tool definitions for tools the provider runs, can't be sent to Bedrock.

See AWS Bedrock for what is converted, and for accepting requests in the Bedrock Converse API.

Editions

Component	Zilla	Zilla Plus
`aws-bedrock` dialect	—	✓

Try It

Try the guide

Walk through OpenAI SDK on AWS Bedrock to serve OpenAI SDK requests from Bedrock.