Kafka - Stream Word Count demo


Stream Word count demo adapted from the stream quickstart.

Prerequisites   broker   zookeeper


Start the services

cd /kafka/docker/stream
export COMPOSE_PROJECT_NAME="stream-demo"
docker-compose up -d
Creating network "streamdemo_default" with the default driver
Creating streamdemo_zookeeper_1 ...
Creating streamdemo_zookeeper_1 ... done
Creating streamdemo_broker_1 ...
Creating streamdemo_broker_1 ... done
  • Service status
docker-compose ps
         Name                     Command            State                     Ports
streamdemo_broker_1      /etc/confluent/docker/run   Up>9092/tcp
streamdemo_zookeeper_1   /etc/confluent/docker/run   Up>2181/tcp, 2888/tcp, 3888/tcp

Create a topic

# Enter in the broker container and start bash
docker-compose exec broker bash
# Create the topic
kafka-topics --create \
    --zookeeper zookeeper:2181 \
    --replication-factor 1 \
    --partitions 1 \
    --topic streams-plaintext-input
Created topic "streams-plaintext-input".
  • Create the output topic
kafka-topics --create \
    --zookeeper zookeeper:2181 \
    --replication-factor 1 \
    --partitions 1 \
    --topic streams-wordcount-output \
    --config cleanup.policy=compact
Created topic "streams-wordcount-output".
  • Describe them
kafka-topics \
    --zookeeper zookeeper:2181 \
Topic:streams-plaintext-input   PartitionCount:1        ReplicationFactor:1     Configs:
        Topic: streams-plaintext-input  Partition: 0    Leader: 1       Replicas: 1     Isr: 1

Topic:streams-wordcount-output  PartitionCount:1        ReplicationFactor:1     Configs:cleanup.policy=compact
        Topic: streams-wordcount-output Partition: 0    Leader: 1       Replicas: 1     Isr: 1



java -cp ./target/kafka-streams-0.1-standalone.jar com.gerardnico.kafka.demo.WordCountDemo

Powered by ComboStrap