Apache Kafka Passthrough Benchmark
This benchmark compares connecting to Apache Kafka directly with connecting to the same cluster through Zilla Plus.
Summary
- Lower tail latency through Zilla Plus. Publish p99 was 565 ms through Zilla Plus against 751 ms direct, and p99.9 was 718 ms against 1,113 ms.
- Lower end-to-end latency at every percentile. End-to-end p99 was 552 ms through Zilla Plus against 730 ms direct.
- Each Zilla Plus instance carried about 200 MB/s, inbound and outbound, at about 65% CPU across 4 vCPUs, while Kafka carried the 100 MB/s workload.
- Flatter latency profile. Kafka alone showed spikes near 1,000 to 1,200 ms, while Zilla Plus held around 600 ms at p99.
Setup
| Setting | Value |
|---|---|
| Deployment | m5.xlarge: 3 Zilla Plus, 6 benchmark clients, 3 controllers, 1 Prometheus |
| Cluster | Apache Kafka 3.6.0 with KRaft on 6 m5.xlarge brokers, GP3 storage at 500 GB, 3,000 IOPS, and 125 MB/s per broker |
| Security | SSL |
| Traffic | 100 MB/s, 45 topics with 200 partitions each, 1 producer and 3 consumer groups per topic |
| Messages | 2 KB |
| Producer | acks=all, linger.ms=10 |
| Duration | 30-minute warm-up, 1-hour measurement |
| Measurement | Publish and end-to-end latency from the OpenMessaging Benchmark |
Results
Publish latency, in milliseconds:
| Setup | p50 | p95 | p99 | p99.9 |
|---|---|---|---|---|
| Apache Kafka | 340 | 591 | 751 | 1113 |
| Through Zilla Plus | 268 | 456 | 565 | 718 |
| Difference | −72 | −135 | −186 | −395 |
End-to-end latency, in milliseconds:
| Setup | p50 | p95 | p99 | p99.9 |
|---|---|---|---|---|
| Apache Kafka | 332 | 576 | 730 | 1073 |
| Through Zilla Plus | 264 | 447 | 552 | 701 |
| Difference | −68 | −129 | −178 | −372 |
Caveats
- Results cover one cluster size, 6 brokers, at a 100 MB/s workload of 2 KB messages.
- Latency depends on hardware, message size, partition count, and client settings. Measure your own workload before sizing a deployment.