Idempotent

The idempotent producer strengthens Kafka's delivery semantics from at least once to exactly once delivery. The producer can only guarantee idempotence for messages sent within a single session.

retries will no longer introduce duplicates.

To enable idempotence, set the following properties enable.idempotence=true. If set, the following properties are set:

• retries=Integer.MAX_VALUE
• acks=all.

Transactional

The transactional producer allows an application to send messages to multiple partitions (and topics) atomically.

To enable, the transaction mode, the following properties must be set:

• transactional.id
• replication.factor=3
• min.insync.replicas=2
• read only committed messages

The purpose of the transactional.id is to enable transaction recovery across multiple sessions of a single producer instance.

Example:

Properties props = new Properties();
 props.put("bootstrap.servers", "localhost:9092");
 props.put("transactional.id", "my-transactional-id");
 Producer<String, String> producer = new KafkaProducer<>(props, new StringSerializer(), new StringSerializer());

 producer.initTransactions();

 try {
     producer.beginTransaction();
     for (int i = 0; i < 100; i++)
         producer.send(new ProducerRecord<>("my-topic", Integer.toString(i), Integer.toString(i)));
     producer.commitTransaction();
 } catch (ProducerFencedException | OutOfOrderSequenceException | AuthorizationException e) {
     // We can't recover from these exceptions, so our only option is to close the producer and exit.
     producer.close();
 } catch (KafkaException e) {
     // For all other exceptions, just abort the transaction and try again.
     producer.abortTransaction();
 }
 producer.close();