Kafka - Stream Word Count demo

Kafka Commit Log Messaging Process


Stream Word count demo adapted from the stream quickstart.

Prerequisites   broker   zookeeper


Start the services

cd /kafka/docker/stream
export COMPOSE_PROJECT_NAME="stream-demo"
docker-compose up -d
Creating network "streamdemo_default" with the default driver
Creating streamdemo_zookeeper_1 ...
Creating streamdemo_zookeeper_1 ... done
Creating streamdemo_broker_1 ...
Creating streamdemo_broker_1 ... done

  • Service status
docker-compose ps
Name                     Command            State                     Ports
streamdemo_broker_1      /etc/confluent/docker/run   Up>9092/tcp
streamdemo_zookeeper_1   /etc/confluent/docker/run   Up>2181/tcp, 2888/tcp, 3888/tcp

Create a topic

# Enter in the broker container and start bash
docker-compose exec broker bash
# Create the topic
kafka-topics --create \
    --zookeeper zookeeper:2181 \
    --replication-factor 1 \
    --partitions 1 \
    --topic streams-plaintext-input
Created topic "streams-plaintext-input".

  • Create the output topic
kafka-topics --create \
    --zookeeper zookeeper:2181 \
    --replication-factor 1 \
    --partitions 1 \
    --topic streams-wordcount-output \
    --config cleanup.policy=compact
Created topic "streams-wordcount-output".

  • Describe them
kafka-topics \
    --zookeeper zookeeper:2181 \
Topic:streams-plaintext-input   PartitionCount:1        ReplicationFactor:1     Configs:
        Topic: streams-plaintext-input  Partition: 0    Leader: 1       Replicas: 1     Isr: 1

Topic:streams-wordcount-output  PartitionCount:1        ReplicationFactor:1     Configs:cleanup.policy=compact
        Topic: streams-wordcount-output Partition: 0    Leader: 1       Replicas: 1     Isr: 1


Pom full example


java -cp ./target/kafka-streams-0.1-standalone.jar com.gerardnico.kafka.demo.WordCountDemo

Discover More
Kafka Commit Log Messaging Process
Kafka - Tutorials

A list of tutorials that I made on windows. They were my path to learn Kafka confluent In this tutorial: you start all services in a docker container you write and read data into a topic...

Share this page:
Follow us:
Task Runner