Structured Field Encryption Benchmark
This benchmark holds the mode at structured field encryption and varies only the encoding. The same record, at the same rate on the same cluster, is carried as JSON, as Avro, and as Protobuf, with one field tagged sensitive in each. Zilla Plus encrypts that field on produce and decrypts it on fetch in all three, with no client changes on either side.
It extends the Whole-Payload Encryption Benchmark, where structured field encryption over JSON was the most expensive of the four setups measured.
Summary
- Throughput was unaffected by the encoding. All three encodings held 24,576 msg/s with the consume rate at exactly 3× publish, the backlog back to zero, and zero publish errors across the 60-minute measurement.
- Avro cut end-to-end latency against JSON by 12 ms at p50, 26 ms at p95, 71 ms at p99, and 757 ms at p99.9.
- Protobuf landed within 2 to 12 ms of Avro at every percentile through p99.9, inside this setup's run-to-run variance.
- The encoding matters more than the mode. At p99, the gap between JSON and Avro (71 ms) is larger than the gap between whole-payload and structured field encryption in the Whole-Payload Encryption Benchmark (26 ms).
Setup
| Setting | Value |
|---|---|
| Deployment | m5.xlarge: 3 Zilla Plus, 6 benchmark clients, 3 controllers |
| Cluster | Apache Kafka 3.6.0 with KRaft on 6 m5.xlarge brokers, gp3 storage |
| Security | TLS end to end, with clients reaching Zilla Plus over SASL_SSL with PLAIN |
| Traffic | 24,576 msg/s, about 24 MB/s, 45 topics with 200 partitions each, 1 producer and 3 consumer groups per topic |
| Messages | The same record as 1,024 B JSON, 988 B Avro, or 992 B Protobuf |
| Producer | acks=all, linger.ms=10, replication factor 3, min.insync.replicas=2 |
| Encryption | AES-GCM with a fresh 12-byte IV per operation and a 16-byte tag, keys cached from a filesystem vault on each gateway |
| Duration | 30-minute warm-up, 60-minute measurement |
| Measurement | Publish and end-to-end latency from the OpenMessaging Benchmark |
Each encoding runs on its own Zilla Plus listener, so all three run against one cluster without interfering. The record has four fields, one of them sensitive:
| Field | Value | Encrypted |
|---|---|---|
id | 36-character identifier | No |
status | ACTIVE | No |
ssn | 123-45-6789 | Yes |
notes | 930 characters | No |
Read directly off the broker, id, status, and notes are in plain text, and the 11-byte ssn is replaced by an AES-GCM envelope of about 167 bytes. A consumer fetching through Zilla Plus gets the record back byte-identical to what was produced. With three consumer groups per topic, every record is encrypted once on produce and decrypted three times on fetch.
A run counts only if the consume rate settles at exactly 3× the publish rate and the backlog returns to zero. Every result below met both, with zero publish errors.
Results
Publish latency, in milliseconds:
| Encoding | p50 | p75 | p95 | p99 | p99.9 | Max |
|---|---|---|---|---|---|---|
| JSON | 36.3 | 49.5 | 79.3 | 146.2 | 882.2 | 1,495.9 |
| Avro | 26.4 | 36.3 | 58.0 | 85.7 | 164.3 | 1,206.9 |
| Protobuf | 28.3 | 39.6 | 64.4 | 98.6 | 169.0 | 604.3 |
Publish latency added over Avro, the fastest of the three, in milliseconds:
| Encoding | p50 | p75 | p95 | p99 | p99.9 |
|---|---|---|---|---|---|
| JSON | +9.9 | +13.2 | +21.3 | +60.5 | +717.9 |
| Protobuf | +1.9 | +3.3 | +6.4 | +12.9 | +4.7 |
End-to-end latency, in milliseconds:
| Encoding | p50 | p75 | p95 | p99 | p99.9 | Max |
|---|---|---|---|---|---|---|
| JSON | 38 | 52 | 85 | 162 | 923 | 1,746 |
| Avro | 26 | 36 | 59 | 91 | 166 | 1,219 |
| Protobuf | 28 | 40 | 65 | 103 | 171 | 987 |
End-to-end latency added over Avro, in milliseconds:
| Encoding | p50 | p75 | p95 | p99 | p99.9 |
|---|---|---|---|---|---|
| JSON | +12 | +16 | +26 | +71 | +757 |
| Protobuf | +2 | +4 | +6 | +12 | +5 |
End-to-end latency tracks publish latency closely, because the added work sits in the gateway rather than in the broker. JSON costs 40 to 50% more than Avro at every percentile up to p99, and 5.5× more at p99.9. The tail shows in the backlog too: peak consumer backlog was 2,520 records for JSON, 1,309 for Avro, and 580 for Protobuf, all draining back to zero.
Why the Encoding Matters
Structured field encryption does four things per record: it parses the value against its schema, validates it, locates the tagged fields, and re-serializes the record after encrypting them. Only the encryption is a fixed cost. The other three scale with how expensive the encoding is to decode.
JSON is text. Finding ssn means scanning the document, and writing the record back means re-serializing all four fields, including the 930-character notes field that was never touched. Avro and Protobuf decode into a field layout directly, so Zilla Plus reaches the tagged field without materializing the rest, and copies the untouched bytes rather than rebuilding them. Zilla Plus does this once on produce and three times on fetch, so the per-record difference is multiplied by four.
The binary encodings also move about 3.5% fewer bytes at the same message rate. That saving is too small to account for a 5.5× difference at p99.9.
Caveats
- This benchmark ran separately from the Whole-Payload Encryption Benchmark, on a different cluster. Compare the differences measured within each run, not absolute numbers across them: structured field encryption over JSON reached a p99.9 end-to-end latency of 262 ms there and 923 ms here.
- The 2 to 12 ms difference between Avro and Protobuf is inside this setup's run-to-run variance.
- Results describe one 11-byte sensitive field in a 1 KiB record, three consumer groups per topic, and m5.xlarge gateways. A record whose sensitive fields make up most of the payload narrows the gap between whole-payload and structured field encryption, and a different number of consumer groups changes the ratio of decryption to parsing work.
- Measure your own records before sizing a deployment.