Kafka - Stream Word Count demo

Stream Word count demo adapted from the stream quickstart.

• Install docker: Docker - Installation
• Map the hostname zookeeper and broker to your docker host ip in your host file. Example:

192.168.99.100   broker
192.168.99.100   zookeeper

• Start the kafka server and zookeeper

cd /kafka/docker/stream
export COMPOSE_PROJECT_NAME="stream-demo"
docker-compose up -d

Creating network "streamdemo_default" with the default driver
Creating streamdemo_zookeeper_1 ...
Creating streamdemo_zookeeper_1 ... done
Creating streamdemo_broker_1 ...
Creating streamdemo_broker_1 ... done

• Service status

docker-compose ps

         Name                     Command            State                     Ports
-------------------------------------------------------------------------------------------------------
streamdemo_broker_1      /etc/confluent/docker/run   Up      0.0.0.0:9092->9092/tcp
streamdemo_zookeeper_1   /etc/confluent/docker/run   Up      0.0.0.0:2181->2181/tcp, 2888/tcp, 3888/tcp

• Create the input topic

# Enter in the broker container and start bash
docker-compose exec broker bash
# Create the topic
kafka-topics --create \
    --zookeeper zookeeper:2181 \
    --replication-factor 1 \
    --partitions 1 \
    --topic streams-plaintext-input

Created topic "streams-plaintext-input".

• Create the output topic

kafka-topics --create \
    --zookeeper zookeeper:2181 \
    --replication-factor 1 \
    --partitions 1 \
    --topic streams-wordcount-output \
    --config cleanup.policy=compact

Created topic "streams-wordcount-output".

• Describe them

kafka-topics \
    --zookeeper zookeeper:2181 \
    --describe

Topic:streams-plaintext-input   PartitionCount:1        ReplicationFactor:1     Configs:
        Topic: streams-plaintext-input  Partition: 0    Leader: 1       Replicas: 1     Isr: 1

Topic:streams-wordcount-output  PartitionCount:1        ReplicationFactor:1     Configs:cleanup.policy=compact
        Topic: streams-wordcount-output Partition: 0    Leader: 1       Replicas: 1     Isr: 1

Pom full example

java -cp ./target/kafka-streams-0.1-standalone.jar com.gerardnico.kafka.demo.WordCountDemo