Kafka - Architecture

Warehouse

• into secondary indexes like Elasticsearch
• into batch systems such as Hadoop for offline analysis.

Workers lend themselves well to being run in containers in managed environments such as Kubernetes, Mesos, Docker Swarm, or YARN as all state is stored in Kafka, making the local processes themselves stateless.