Kafka Connect - Connector Plugin


Connector is a component of the connect framework that coordinates data streaming by managing tasks

A connector instance is a logical job. Each connector instance coordinates a set of tasks that actually copy the data.

A connector is run by a worker.



The easiest way to create, configure, and manage connectors is with Confluent Control Center.


The following states are possible for a connector or one of its tasks:

  • UNASSIGNED: The connector/task has not yet been assigned to a worker.
  • RUNNING: The connector/task is running.
  • PAUSED: The connector/task has been administratively paused.
  • FAILED: The connector/task has failed (usually by raising an exception, which is reported in the status output).



Built-in consumer, producer

Kafka Connect uses the standard Java producer and consumers to communicate with Kafka.

All producer configs and consumer configs can be overridden by prefixing them with producer. or consumer., respectively.


Actual Connector

See list connector available for a worker (in distributed mode).

Externally Available

Dev Guide / Build

mvn archetype:generate -DarchetypeGroupId=com.github.jcustenborder.kafka.connect -DarchetypeArtifactId=kafka-connect-quickstart -DarchetypeVersion=


Connector configuration file are given:

Common properties:

  • name - Unique name for the connector. Attempting to register again with the same name will fail.
  • connector.class - The Java class for the connector
  • tasks.max - The maximum number of tasks that should be created for this connector. The connector may create fewer tasks if it cannot achieve this level of parallelism.
  • key.converter and value.converter - (optional) to override the default converter configuration

Sink connectors:

  • topics - A list of topics to use as input for this connector


Even if you delete and recreate the connector, if the connector has the same name it will retain the same offsets previously stored.

Documentation / Reference

Powered by ComboStrap