Table of Contents

Kafka Connect - Transform (Single Message Transform - SMT)

About

Transform is a concept of connect to apply simple transformation.

A transform is a simple function that accepts one record as input and outputs a modified record. Transform are chained.

More complex transformations and operations that apply to multiple messages are best implemented with Kafka Streams.

Implementation

You can implement the Transformation interface with your own custom logic, package them as a Kafka Connect plugin, and use them with any connectors.

Example

"transforms": "convert_op_my_field", // Name of the transform
"transforms.convert_op_my_field.type": "org.apache.kafka.connect.transforms.TimestampConverter$Value", // Timestamp converter
"transforms.convert_op_my_field.target.type": "Timestamp",  // Desired type (i.e. string, long, Date, Time, Timestamp)
"transforms.convert_op_my_field.field": "field_name", //  the field name 
"transforms.convert_op_my_field.format": "yyyy-MM-dd HH:mm:ss.SSSSSS" // in case converting to or from a string, a SimpleDateFormat-compatible format string

Documentation / Reference