About
Installation, starting Kafka
Articles Related
Steps
Installation
- With zip file
<path-to-confluent>/bin/ # Driver scripts for starting/stopping services
<path-to-confluent>/etc/ # Configuration files
<path-to-confluent>/share/java/ # Jars
- With package, If you installed from deb or rpm packages, the contents are installed globally and you’ll need to adjust the paths used below:
/usr/bin/ # Driver scripts for starting/stopping services, prefixed with <package> names
/etc/<package>/ # Configuration files
/usr/share/java/<package>/ # Jars
Windows
!!! Important on WINDOWS Version 3.3.0 !!!
- With MINGW, in ./bin/kafka-run-class if running on Windows with Mingwin, replace the CYGWIN test at the begining:
# if [[ $(uname -a) =~ "CYGWIN" ]]; then
# becomes
if [[ $(uname -a) =~ "CYGWIN" || $(uname -a) =~ "MINGW" ]]; then
- Schema registry file (./bin/schema-registry-run-class) must be modified if you are using bash on Windows. At the end of the CLASSPATH and SCHEMA_REGISTRY_LOG4J_OPTS init, add the below line
# Before the setting of SCHEMA_REGISTRY_LOG4J_OPTS
# The block above the assignment is adpated in order to have the variable SCHEMA_REGISTRY_LOG4J_OPTS that contains only a directory
# (ie -Dlog4j.configuration=file: is suppressed)
# Then we modifed the path to turn them in Windows form wikth cygpath
if [[ $(uname -a) =~ "CYGWIN" || $(uname -a) =~ "MINGW" ]]; then
LOG_DIR=$(cygpath --path --mixed "${LOG_DIR}")
SCHEMA_REGISTRY_LOG4J_OPTS=$(cygpath --path --mixed "${SCHEMA_REGISTRY_LOG4J_OPTS}")
fi
SCHEMA_REGISTRY_LOG4J_OPTS="-Dschema-registry.log.dir=$LOG_DIR -Dlog4j.configuration=file:$SCHEMA_REGISTRY_LOG4J_OPTS"
# before the call of the java command
if [[ $(uname -a) =~ "CYGWIN" || $(uname -a) =~ "MINGW" ]]; then
CLASSPATH=$(cygpath --path --mixed "${CLASSPATH}")
fi
- Rest file (./bin/kafka-rest-run-class) must be modified if you are using bash on Windows. At the end of the CLASSPATH and KAFKAREST_LOG4J_OPTS init, add the below line
# After the setting of KAFKAREST_LOG4J_OPTS
# The block above the assignment is adapted in order to have the variable KAFKAREST_LOG4J_OPTS that contains only a directory
# (ie -Dlog4j.configuration=file: is suppressed)
# Then we modifed the path to turn them in Windows form wikth cygpath
# CYGINW == 1 if Cygwin is detected, else 0.
if [[ $(uname -a) =~ "CYGWIN" || $(uname -a) =~ "MINGW" ]]; then
KAFKAREST_LOG4J_OPTS=$(cygpath --path --mixed "${KAFKAREST_LOG4J_OPTS}")
KAFKAREST_LOG4J_OPTS="-Dlog4j.configuration=file:${KAFKAREST_LOG4J_OPTS}"
fi
# before the call of the java command
if [[ $(uname -a) =~ "CYGWIN" || $(uname -a) =~ "MINGW" ]]; then
CLASSPATH=$(cygpath --path --mixed "${CLASSPATH}")
fi
Start the services
With the CLI
- Default Start with the cli
$ export PATH=<path-to-confluent>/bin:$PATH
$ confluent start
# confluent help
Starting zookeeper
zookeeper is [UP]
Starting kafka
kafka is [UP]
Starting schema-registry
schema-registry is [UP]
Starting kafka-rest
kafka-rest is [UP]
Starting connect
connect is [UP]
- To start just Zookeeper, Kafka and Schema Registry run:
$ export PATH=<path-to-confluent>/bin:$PATH
$ confluent start schema-registry
One by one
- Start Zookeeper in its own terminal. Zookeeper is used to track the health of the nodes.
./bin/zookeeper-server-start ./etc/kafka/zookeeper.properties
- Start Kafka in its own terminal.
./bin/kafka-server-start ./etc/kafka/server.properties
- Start the Schema Registry in its own terminal.
./bin/schema-registry-start ./etc/schema-registry/schema-registry.properties
- Start the Rest Services
./bin/kafka-rest-start ./etc/kafka-rest/kafka-rest.properties
- Start Connect in distributed mode. Run this command in its own terminal.
$ ./bin/connect-distributed ./etc/schema-registry/connect-avro-distributed.properties
Support
Log
confluent current
/tmp/confluent.tbVCwQps
cd /tmp/confluent.tbVCwQps
ls
connect kafka kafka-rest schema-registry zookeeper
- Querying the connect log for instance
less connect/connect.stderr