Build the Todo Application

Zilla
Home GitHub Support About
Search…
What is Zilla?
Get Started
Build the Todo Application
Secure the Todo Application
Bring your own Kafka
Reference
Build the Todo Application
In this getting started exercise, you will create a simple Todo application using a CQRS design pattern that's backed by Apache Kafka and Zilla as the event-driven API gateway. Zilla lets you focus on your applications and business logic instead of spending time writing tons of code and this demo helps to ease CQRS complexity. This guide gives a basic introduction to Zilla and describes some straightforward capabilities.
This Todo Application guide has the following goals:
Provide a list of Todo tasks that is shared by all clients
Support optimistic locking with conflict detection when attempting to update a Todo task
Deliver updates in near real-time when a Todo task is created, modified, or deleted
Demonstrate a user interface driving the Tasks API
Support scaling Todo task reads and writes
Prerequisites
Docker 20.10.14
Git 2.32.0
npm 8.3.1  and above
Step 1: Kafka
In this step, you will set up basic infrastructure components for your event-driven architecture.
Run docker swarm init if you already haven't done to initiateSwarm orchestrator.
Let's create stack.yml and add Apache Kafka.
stack.yml
1
version: "3"
2
​
3
networks:
4
  net0:
5
    driver: overlay
6
​
7
services:
8
​
9
  kafka:
10
    image: "bitnami/kafka:3.1.0"
11
    hostname: "kafka.internal.net"
12
    networks:
13
      - net0
14
    command:
15
      - 'sh'
16
      - '-c'
17
      - '/opt/bitnami/scripts/kafka/setup.sh && kafka-storage.sh format --config "${KAFKA_CONF_FILE}" --cluster-id "lkorDA4qT6W1K_dk0LHvtg" --ignore-formatted  && /opt/bitnami/scripts/kafka/run.sh' # Kraft specific initialise
18
    environment:
19
      - ALLOW_PLAINTEXT_LISTENER=yes
20
      - KAFKA_CFG_NODE_ID=1
21
      - KAFKA_CFG_BROKER_ID=1
22
      - [email protected]:9093
23
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,INTERNAL:PLAINTEXT,CONTROLLER:PLAINTEXT
24
      - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
25
      - KAFKA_CFG_LOG_DIRS=/tmp/logs
26
      - KAFKA_CFG_PROCESS_ROLES=broker,controller
27
      - KAFKA_CFG_LISTENERS=CLIENT://:9092,INTERNAL://:29092,CONTROLLER://:9093
28
      - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=INTERNAL
29
      - KAFKA_CFG_ADVERTISED_LISTENERS=CLIENT://localhost:9092,INTERNAL://kafka.internal.net:29092
30
    ports:
31
      - "9092:9092"
32
  init-kafka:
33
    image: "bitnami/kafka:3"
34
    networks:
35
      - net0
36
    deploy:
37
      restart_policy:
38
        condition: none
39
        max_attempts: 0
40
    depends_on:
41
      - kafka
42
    entrypoint: [ '/bin/sh', '-c' ]
43
    command: |
44
      "
45
      # blocks until Kafka becomes reachable
46
      /opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server kafka.internal.net:29092 --list --topic 'task-.*'
47
      
48
      echo '## Creating the Kafka topics'
49
      /opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server kafka.internal.net:29092 --create --if-not-exists --topic task-commands --partitions 1
50
      /opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server kafka.internal.net:29092 --create --if-not-exists --topic task-replies --partitions 1
51
      /opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server kafka.internal.net:29092 --create --if-not-exists --topic task-snapshots --config cleanup.policy=compact --partitions 1
52
      echo ''
53
      echo '## Created the Kafka topics'
54
      /opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server kafka.internal.net:29092 --list --topic 'task-.*'
55
      "
Copied!
Now let's run
1
docker stack deploy -c stack.yml example --resolve-image never
Copied!
 to spin up Apache Kafka and create the following topics.
task-commands
Queues commands to be processed by the Todo service
task-replies
Queues the response from the Todo service after processing each command
task-snapshots
Captures the latest snapshot of each task entity
Now verify that Kafka has fully started and that the topics have been successfully created.
1
docker service logs example_init-kafka --follow --raw
Copied!
Make sure you see this output at the end of the example_init-kafka service logs.
1
## Creating the Kafka topics
2
Created topic task-commands.
3
Created topic task-replies.
4
Created topic task-snapshots.
5
​
6
## Created the Kafka topics
7
task-commands
8
task-replies
9
task-snapshots
Copied!
Step 2: Todo Service
Next, you will need to build a todo service that is implemented using Spring boot + Kafka Streams to process commands and generate relevant output.  This Todo service can deliver near real-time updates when a Task is created, renamed, or deleted, and produces a message to the Kafka task-snapshots topic with the updated value.
Combining this with cleanup-policy: compact for the task-snapshots topic causes the topic to behave more like a table, where only the most recent message for each distinct message key is retained.
This approach is used as the source of truth for the current state of our Todo service, setting the Kafka message key to the Task identifier to retain all the distinct Tasks.
When a Task is deleted, you will produce a tombstone message (null value) to the task-snapshots topic causing that Task identifier to no longer be retained in Kafka.
Commands arrive at the Tasks service via the task-commands topic and correlated replies are sent to the task-replies topic with the same zilla:correlation-id value that was received with the inbound command message.
Implementing the Todo domain using these topics gives us the following Kafka Streams topology.
The ValidateCommand Kafka Streams processor implements optimistic locking by ensuring that conditional requests using if-match are allowed to proceed only if the latest etag for the Task matches, otherwise the command is rejected.
Let's build the service by running the below command in the root directory.
1
git clone https://github.com/aklivity/todo-service && cd todo-service
2
./mvnw clean install && cd ..
Copied!
This will checkout and build todo-service:latest image.
Open stack.yml file and add the Todo service into the stack:
stack.yml
1
  ...
2
  
3
  todo-service:
4
    image: "todo-service:latest"
5
    networks:
6
      - net0
7
    environment:
8
      SPRING_KAFKA_APPLICATION_ID: todo-service
9
      SPRING_KAFKA_BOOTSTRAP_SERVERS: kafka.internal.net:29092
10
      TASK_COMMANDS_TOPIC: task-commands
11
      TASK_SNAPSHOTS_TOPIC: task-snapshots
12
      TASK_REPLIES_TOPIC: task-replies
Copied!
Run the command below to deploy the todo-service to your existing stack.
1
docker stack deploy -c stack.yml example --resolve-image never
Copied!
1
Creating service example_todo-service
2
Updating service example_kafka (id: st4hq1bwjsom5r0jxnc6i9rgr)
Copied!
Now, you have a running to-do service that can process incoming commands, send a response and take snapshots of the task.
Step 3: Zilla
Next, the most exciting and most challenging part of implementing the CQRS design pattern is where you need to build and deploy your API with a real-time response.
In a traditional approach, you would have to set up and build multiple layers of service implementation to expose the API that is based on Kafka Streams.
However, Zilla is designed to solve these architectural challenges, requiring only declarative configuration as shown below.
Let's design the Tasks API. You need to define a Tasks API to send commands to the Todo service via Kafka and retrieve task queries from Kafka as needed by the Tasks UX.
​
​
​
​
​
​
​
​
Configure Zilla
The Zilla engine configuration defines a flow of named bindings representing each step in the pipeline as inbound network traffic is decoded and transformed then encoded into outbound network traffic as needed.
Let's configure Zilla for the Tasks API to interact with the Todo Kafka Streams service via Kafka topics.
You will add the following bindings to support the Tasks API as shown zilla.json below. To understand each binding type in more detail please visit Zilla Runtime Configuration.
tcp_server0
listens on port 8080
routes to http_server0
http_server0
decodes HTTP protocol
routes Tasks API to http_kafka_proxy0
http_kafka_proxy0
transforms HTTP to Kafka
routes POST, PUT and DELETE Tasks API requests to task-commands topic with task-replies reply-to topic via kafka_cache_client0
routes GET Tasks API requests to task-snapshots topic via kafka_cache_client0
kafka_cache_client0
reads from local Kafka topic message cache
routes to kafka_cache_server0
kafka_cache_server0
writes to local Kafka topic message cache as new messages arrive from Kafka
routes to kafka_client0
kafka_client0
encodes Kafka protocol
routes to kafka_client0
tcp_client0
connects to Kafka brokers
Copy the contents of zilla.json shown below to your local zilla.json file.
zilla.json
Now let's add the zilla service to the docker stack, mounting the zilla.json configuration.
1
  ...
2
  
3
  zilla:
4
    image: "ghcr.io/aklivity/zilla:latest"
5
    hostname: "zilla"
6
    command: ["start", "-v", "-e"]
7
    volumes:
8
      - ./zilla.json:/zilla.json:ro
9
    networks:
10
      - net0
11
    ports:
12
      - "8080:8080"
13
  ...
Copied!
Run the below command as this will deploy the zilla service to the existing stack.
1
docker stack deploy -c stack.yml example --resolve-image never
Copied!
1
Updating service example_kafka (id: st4hq1bwjsom5r0jxnc6i9rgr)
2
Updating service example_todo-service (id: ojbj2kbuft22egy854xqv8yo8)
3
Creating service example_zilla
Copied!
Make sure that Zilla is fully started by checking container logs where you see started at the end of the log.
Now we can verify the Tasks API using curl as shown below.
Let's first get all the current tasks.
1
curl http://localhost:8080/tasks
Copied!
This shows an empty array of tasks.
1
[]
Copied!
Now let's create a new task, and get all the tasks again.
1
curl -X "POST" "http://localhost:8080/tasks" \
2
    -H "Idempotency-Key: 9417E83E-313E-468E-AC7C-1BCE0BAF9712" \
3
    -H "Content-Type: application/json" \
4
    -d "{\"name\":\"Read the docs\"}"
Copied!
1
curl http://localhost:8080/tasks
Copied!
This now shows an array with the task you just created.
1
[{"name":"Read the docs"}]
Copied!
As you can see, the GET /tasks API works as expected, but you would still need to poll the server to detect changes made by other clients.
Let's modify the zilla.json configuration to use Server-Sent Events (SSE) instead to support both initial delivery of the current list of all Todo Tasks, as well as incremental updates when a Task is created, renamed, or deleted by the same client or a different client.
First, add the following bindings to zilla.json inside the bindings property of the configuration.
zilla.json (sse)
Then modify the http_server0 binding in zilla.json to route the GET /tasks API to exit to sse_server0 instead of http_kafka_proxy0.
zilla.json (http routes)
Now run the command below to update the zilla service and force a restart.
1
docker service update --force \
2
  $(docker stack services example -q -f name=example_zilla)
Copied!
Let's verify the Tasks API using curl as shown below.
1
curl http://localhost:8080/tasks
Copied!
1
id:["OTQxN0U4M0UtMzEzRS00NjhFLUFDN0MtMUJDRTBCQUY5NzEy","AQIAAg==/1"]
2
data:{"name":"Read the docs"}
3
​
Copied!
As you can see, the GET /tasks API now delivers a continuous stream of tasks starting with the initial tasks as expected.
Now create a new Todo task while keeping the GET /tasks stream open as shown above.
1
curl -X POST http://localhost:8080/tasks \
2
    -H "Idempotency-Key: 5C1A90A3-AEB5-496F-BA00-42D1D805B21B" \
3
    -H "Content-Type: application/json" \
4
    -d "{\"name\":\"Join the Slack community\"}"
Copied!
The GET /tasks stream automatically receives the update when the new task is created.
1
id:["NUMxQTkwQTMtQUVCNS00OTZGLUJBMDAtNDJEMUQ4MDVCMjFC","AQIABA==/1"]
2
data:{"name":"Join the Slack community"}
3
​
Copied!
Each new update arrives automatically, even when changes are made by other clients.
Step 4: Web App
Next, you will build the Todo app that's implemented using VueJs framework.  Run the commands below in the root directory.
1
git clone https://github.com/aklivity/todo-app && cd todo-app
2
npm install
3
npm run build
4
cd ..
Copied!
This will generate dist folder with necessary artifacts. Now you can configure Zilla to host the app so that both API and app can be served under the same hostname and port.
First, add the following bindings to zilla.json inside the bindings property of the configuration.
zilla.json
The last step is to mount the dist folder into the Zilla container.
Open stack.yml file and add - ./todo-app/dist:/app/dist:ro to the zilla service volumes.
stack.yml
1
  ...
2
  
3
  zilla:
4
    image: "ghcr.io/aklivity/zilla:latest"
5
    hostname: "zilla"
6
    command: ["start", "-v", "-e"]
7
    volumes:
8
      ...
9
      - ./todo-app/dist:/app/dist:ro
10
​
11
  ...
Copied!
Finally, run
1
docker stack deploy -c stack.yml example --resolve-image never
Copied!
Make sure that zilla.json  config changes got applied after restarting the Zilla service. Check the example_zilla service log.
Step 5: Test Drive!
Open the browser and enter http://localhost:8080/ to see the Todo Application.
Notice that there is no need to Login as the Tasks API is initially available to anonymous clients.
Next up: Secure the Todo Application Tasks API with JWT access tokens.
Get Started
Secure the Todo Application
Last modified 22d ago
Copy link
Contents