Whole-Payload Encryption Benchmark
This benchmark measures the cost of each layer Zilla Plus can add on the same Kafka cluster: the proxy hop alone with TLS passthrough, whole-payload encryption, and structured field encryption (SFE) of selected JSON fields.
Summary
- Every setup held the target rate. All four setups sustained the workload for the full 60 minutes with zero publish errors.
- The proxy hop adds 0.9 ms at p50. Whole-payload encryption adds another 3.4 ms, and structure-aware processing for SFE another 11.4 ms.
- SFE costs more than whole-payload encryption because each record is parsed, validated against its schema, and re-serialized. In return, fields that aren't sensitive stay readable downstream.
- In a separate single-topic run on one c7i.xlarge gateway, whole-payload encryption sustained 60,085 msg/s with a produce p99 of 41.7 ms.
Setup
| Setting | Value |
|---|---|
| Deployment | m5.xlarge: 3 Zilla Plus, 6 benchmark clients, 3 controllers, 1 Prometheus |
| Cluster | Apache Kafka 3.6.0 with KRaft on 6 m5.xlarge brokers, GP3 storage at 500 GB, 3,000 IOPS, and 125 MB/s per broker |
| Security | TLS end to end: SSL for direct and passthrough, SASL_SSL (PLAIN) for the encryption setups so the Zilla Plus guard has an identity to evaluate |
| Traffic | About 24 MB/s, 45 topics with 200 partitions each, 1 producer and 3 consumer groups per topic |
| Messages | 1 KiB JSON events |
| Producer | acks=all, linger.ms=10, replication factor 3, min.insync.replicas=2 |
| Encryption | AES-GCM with a 12-byte IV and 16-byte tag, keys cached from a filesystem vault |
| Duration | 30-minute warm-up, 60-minute measurement |
| Measurement | Publish and end-to-end latency from the OpenMessaging Benchmark |
Results
Publish latency, in milliseconds:
| Setup | p50 | p95 | p99 | p99.9 |
|---|---|---|---|---|
| Direct to Kafka | 27.1 | 62.0 | 86.3 | 139.4 |
| TLS passthrough | 28.0 | 64.9 | 96.5 | 157.1 |
| Whole-payload encryption | 31.4 | 81.3 | 134.4 | 220.0 |
| Structured field encryption | 42.8 | 98.5 | 152.6 | 250.5 |
End-to-end latency, in milliseconds:
| Setup | p50 | p95 | p99 | p99.9 |
|---|---|---|---|---|
| Direct to Kafka | 27 | 62 | 86 | 139 |
| TLS passthrough | 28 | 66 | 99 | 159 |
| Whole-payload encryption | 32 | 82 | 135 | 221 |
| Structured field encryption | 45 | 105 | 161 | 262 |
Each layer's share of publish latency, in milliseconds:
| Layer | p50 | p95 | p99 |
|---|---|---|---|
| Proxy hop, passthrough over direct | +0.9 | +2.9 | +10.2 |
| Encryption, whole-payload over passthrough | +3.4 | +16.4 | +37.9 |
| Structure-aware processing, structured over whole-payload | +11.4 | +17.2 | +18.2 |
Caveats
- Encryption keys are cached from a filesystem vault, so no KMS calls are on the measured path.
- The single-topic run used a different topic layout and gateway instance type, so its results aren't directly comparable to the main runs.
- Larger records, higher rates, or more encrypted fields per record change the results. Measure your own workload before sizing a deployment.