TLS

AI Gateway is client-facing, so TLS here mainly encrypts the inbound connection from AI agents. The LLM Gateway also connects out to LLM providers over TLS; see Connecting to LLM Providers. Insert a tls server binding between the tcp server binding and the http server binding in zilla.yaml, so Zilla terminates TLS at the edge before the request reaches the http server and mcp server bindings. For client certificate authentication on top of encryption, see mTLS.

Terminating Inbound TLS

Configure a filesystem vault with AI Gateway's private key and signed certificate in its keystore, and reference it, along with the certificate alias to present, from the tls server binding:

vaults:
  server_vault:
    type: filesystem
    options:
      keys:
        store: ${{env.KEYSTORE_PATH}}
        type: ${{env.KEYSTORE_TYPE}}
        password: ${{env.KEYSTORE_PASSWORD}}

bindings:
  north_tcp_server:
    type: tcp
    kind: server
    options:
      host: 0.0.0.0
      port: 8443
    exit: north_tls_server

  north_tls_server:
    type: tls
    kind: server
    vault: server_vault
    options:
      keys:
        - ${{env.SERVER_CERT_ALIAS}}
      sni:
        - ${{env.SERVER_HOSTNAME}}
    exit: north_http_server

  north_http_server:
    type: http
    kind: server
    options:
      access-control:
        policy: cross-origin
    routes:
      - when:
          - headers:
              ":path": /mcp
        exit: north_mcp_server

  north_mcp_server:
    type: mcp
    kind: server
    exit: north_mcp_proxy

See the tls server binding reference for the full set of options fields, including options.sni for serving multiple hostnames from the same binding.

Connecting to LLM Providers

An llm · client whose server is an https:// URL connects over TLS when it exits to sys:http_client. Zilla trusts the standard certificate authorities and uses the URL's host for SNI, so public providers such as OpenAI and Anthropic need no TLS configuration:

south_llm_client_openai:
  type: llm
  kind: client
  options:
    dialect: openai
    server: https://api.openai.com
  exit: sys:http_client

When the handshake fails, Zilla logs BINDING_TLS_HANDSHAKE_FAILED on sys:tls_client, and the caller receives 502 Bad Gateway.

An AWS Bedrock client ★Zilla Plus connects through its own tls · client instead, with the Bedrock host as its sni. It also trusts the standard certificate authorities.

Trusting a Private Certificate Authority

To reach a model server whose certificate is signed by your own certificate authority, such as an OpenAI-compatible server inside your network, exit to your own http · client, tls · client, and tcp · client bindings, and trust the certificate authority from a filesystem vault:

vaults:
  client_vault:
    type: filesystem
    options:
      trust:
        store: truststore.p12
        type: pkcs12
        password: ${{env.TRUSTSTORE_PASSWORD}}

bindings:
  south_llm_client_private:
    type: llm
    kind: client
    options:
      dialect: openai
      server: https://llm.internal.example.com:8443
    exit: south_http_client
  south_http_client:
    type: http
    kind: client
    exit: south_tls_client
  south_tls_client:
    type: tls
    kind: client
    vault: client_vault
    options:
      trust:
        - llm-ca
    exit: south_tcp_client
  south_tcp_client:
    type: tcp
    kind: client

truststore.p12 holds your certificate authority's certificate under the alias llm-ca. For example, create it with keytool:

keytool -importcert -noprompt -alias llm-ca -file ca.crt \
  -keystore truststore.p12 -storetype pkcs12 -storepass "$TRUSTSTORE_PASSWORD"

With options.trust set, the tls · client trusts only the listed certificates, not the standard certificate authorities. Keep sys:http_client for public providers, and use your own bindings only for the clients that need them. See the tls client binding reference for the full set of options.