Table of Contents

Kafka - Confluent Installation and services

About

Installation, starting Kafka

https://docs.confluent.io/current/quickstart.html

Steps

Installation

<path-to-confluent>/bin/        # Driver scripts for starting/stopping services
<path-to-confluent>/etc/        # Configuration files
<path-to-confluent>/share/java/ # Jars


/usr/bin/                  # Driver scripts for starting/stopping services, prefixed with <package> names
/etc/<package>/            # Configuration files
/usr/share/java/<package>/ # Jars

Windows

!!! Important on WINDOWS Version 3.3.0 !!! 

# if [[ $(uname -a) =~ "CYGWIN" ]]; then
# becomes
if [[ $(uname -a) =~ "CYGWIN" || $(uname -a) =~ "MINGW" ]]; then

# Before the setting of SCHEMA_REGISTRY_LOG4J_OPTS
# The block above the assignment is adpated in order to have the variable SCHEMA_REGISTRY_LOG4J_OPTS that contains only a directory
# (ie -Dlog4j.configuration=file: is suppressed)
# Then we modifed the path to turn them in Windows form wikth cygpath
if [[ $(uname -a) =~ "CYGWIN" || $(uname -a) =~ "MINGW" ]]; then
  LOG_DIR=$(cygpath --path --mixed "${LOG_DIR}")
  SCHEMA_REGISTRY_LOG4J_OPTS=$(cygpath --path --mixed "${SCHEMA_REGISTRY_LOG4J_OPTS}")
fi
SCHEMA_REGISTRY_LOG4J_OPTS="-Dschema-registry.log.dir=$LOG_DIR -Dlog4j.configuration=file:$SCHEMA_REGISTRY_LOG4J_OPTS"

# before the call of the java command
if [[ $(uname -a) =~ "CYGWIN" || $(uname -a) =~ "MINGW" ]]; then
  CLASSPATH=$(cygpath --path --mixed "${CLASSPATH}")
fi

# After the setting of KAFKAREST_LOG4J_OPTS
# The block above the assignment is adapted in order to have the variable KAFKAREST_LOG4J_OPTS that contains only a directory
# (ie -Dlog4j.configuration=file: is suppressed)
# Then we modifed the path to turn them in Windows form wikth cygpath
# CYGINW == 1 if Cygwin is detected, else 0.
if [[ $(uname -a) =~ "CYGWIN" || $(uname -a) =~ "MINGW" ]]; then
  KAFKAREST_LOG4J_OPTS=$(cygpath --path --mixed "${KAFKAREST_LOG4J_OPTS}")
  KAFKAREST_LOG4J_OPTS="-Dlog4j.configuration=file:${KAFKAREST_LOG4J_OPTS}"
fi

# before the call of the java command
if [[ $(uname -a) =~ "CYGWIN" || $(uname -a) =~ "MINGW" ]]; then
  CLASSPATH=$(cygpath --path --mixed "${CLASSPATH}")
fi

Start the services

With the CLI

$ export PATH=<path-to-confluent>/bin:$PATH
$ confluent start
# confluent help

Starting zookeeper
zookeeper is [UP]
Starting kafka
kafka is [UP]
Starting schema-registry
schema-registry is [UP]
Starting kafka-rest
kafka-rest is [UP]
Starting connect
connect is [UP]


$ export PATH=<path-to-confluent>/bin:$PATH
$ confluent start schema-registry

One by one

./bin/zookeeper-server-start ./etc/kafka/zookeeper.properties

./bin/kafka-server-start ./etc/kafka/server.properties

./bin/schema-registry-start ./etc/schema-registry/schema-registry.properties

./bin/kafka-rest-start ./etc/kafka-rest/kafka-rest.properties

$ ./bin/connect-distributed ./etc/schema-registry/connect-avro-distributed.properties

Support

Log

confluent current

/tmp/confluent.tbVCwQps


cd /tmp/confluent.tbVCwQps
ls

connect  kafka  kafka-rest  schema-registry  zookeeper


less connect/connect.stderr

Documentation / Reference