Kafka Connect - Standalone Mode

Converter Basics


The runtime standalone mode of connect when running/starting a worker

Standalone mode is best suited for:

  • testing,
  • one-off jobs
  • or single agent (such as sending logs from webservers to Kafka)

You can still test a distributed mode locally by setting a different rest port. See Kafka Connect - Distributed Worker



To run a worker in standalone:

./bin/connect-standalone [-daemon] worker.properties connector1.properties [connector2.properties connector3.properties ...]

# Example
./bin/connect-standalone ./etc/schema-registry/connect-avro-standalone.properties ./etc/kafka-connect-jdbc/source-quickstart-sqlite.properties


Users may optionally provide connector configurations at the command line as only a single worker instance exists and no coordination is required in standalone mode.


When started as a deamon, the log is located to /logs/connectStandalone.out

  • Check the log
cat /logs/connectStandalone.out | grep -i "finished"
[2017-10-18 12:43:26,306] INFO Finished creating connector test-source-sqlite-jdbc-autoincrement (org.apache.kafka.connect.runtime.Worker:225)
[2017-10-18 12:43:26,425] INFO Source task WorkerSourceTask{id=test-source-sqlite-jdbc-autoincrement-0} finished initialization and start (org.apache.kafka.connect.runtime.WorkerSourceTask:143)



Kafka Connect - Worker in standalone mode.

See Work Config Reference If you run multiple standalone instances on the same host, there are a couple of settings that must be unique between each instance:

  • offset.storage.file.filename - storage for connector offsets, which are stored on the local filesystem in standalone mode; using the same file will lead to offset data being deleted or overwritten with different values
  • rest.port - the port the REST interface listens on for HTTP requests



Discover More
Converter Basics
Kafka - Connect

Kafka Connect is a framework service based around connector to move data into and out of Kafka. The service is a worker. By starting a worker, you are starting connect. The management is done through...
Kafka Commit Log Messaging Process
Kafka - Tutorials

A list of tutorials that I made on windows. They were my path to learn Kafka confluent In this tutorial: you start all services in a docker container you write and read data into a topic...
Converter Basics
Kafka Connect - Connector Plugin

Connector is a component of the connect framework that coordinates data streaming by managing tasks A connector instance is a logical job. Each connector instance coordinates a set of tasks that actually...
Kafka Commit Log Messaging Process
Kafka Connect - Sqlite in Standalone Mode

Windows where: connection.url is the JDBC URL mode indicates how we want to query the data. incrementing means that each query for new data will only return rows with IDs larger than the maximum...
Converter Basics
Kafka Connect - Worker

A worker is a connect component. It is the running process (JVM processes) that execute tasks of a connector. A worker may run several connectors. There is two types of runtime workers: standalone:...

Share this page:
Follow us:
Task Runner