MQTT Kafka Proxy
MQTT Kafka Proxy
The Zilla MQTT Kafka Proxy manages MQTT Pub/Sub connections and messages on and off of Kafka.
An MQTT server acts as a broker between publishers and subscribers. This requires a complex protocol to manage the wide range of IoT devices and use cases. By proxying these messages on and off of Kafka with the mqtt-kafka binding, IoT devices can transmit data to a wider range of tech stacks, adapting to more business needs.
Zilla uses specific Kafka topics to store and route MQTT messages, meaning the Kafka architecture can be optimized for MQTT Pub/Sub. MQTT client subscribers and publishers will communicate with Zilla the same as any broker.
An MQTT Broker
A Zilla MQTT server can manage client sessions and broker all traffic, adhering to the official MQTT protocol.
An MQTT client can use any Quality of Service flag.
- QoS 0 - At most once
- QoS 1 - At least once
- QoS 2 - Exactly once
MQTT over WebSocket
The tcp binding defines the ports Zilla will accept traffic for both MQTT and WebSocket connections. Zilla natively handles WebSockets and can manage the MQTT protocol over an active connection.
Last Will and Testament
An MQTT client can specify a last will and testament (LWT) message and topic that is delivered when the client disconnects abruptly and fails to reconnect before session timeout.
An MQTT client can use the v5 request-response paradigm to send messages with a response topic and correlated data. A requesting MQTT client can send a message on one topic and receive a response on another, while a responding MQTT client or any Kafka workflow can handle the message's journey.
An MQTT client reconnecting with the same client-id, even to a different Zilla instance, will automatically remain subscribed to MQTT topics previously subscribed while previously connected.
An MQTT client connecting with the same client-id, even to a different Zilla instance, will automatically disconnect the original MQTT client and take over the session.
An MQTT client can be redirected to a specific Zilla instance, sharding client session state across Zilla instances without needing to replicate every client's session state on each Zilla instance.
Pub/Sub with Kafka
Zilla manages MQTT pub/sub to Kafka using three Kafka topics. The specific topic names can be configured using the options.topics property.
Messages on Kafka
All MQTT messages brokered by Zilla are published on the
messages Kafka topic. The MQTT message topic becomes the Kafka key.
By defining routes in Zilla, you can direct MQTT publish and subscribe connections to specific kafka topics other than the
messages Kafka topic. The
retained topics are not affected by routing.
An MQTT client can Publish messages to any configured Kafka topics, marking specific messages with the retain flag. These messages will have a copy published on the
retained Kafka topic. When a client subscribes with replay-on-subscribe, Zilla will deliver the retained messages.
MQTT connect, disconnect, and other session messages are maintained on the
sessions Kafka topic.
Any connection Zilla handles can be secured using the tls binding. This means both MQTT and MQTT over WebSocket can be encrypted. Additionally, A client connection to the MQTT server can be guarded by the jwt guard supporting JWT access tokens, with fine-grained privileges enforced on publish or subscribe to MQTT topics.