Exactly once

Configures the semantic guarantees

K2K can be configured for exactly-once semantics (EOS), providing the strongest possible delivery guarantee. When enabled, K2K leverages Kafka's transactional capabilities to ensure every record from the source is delivered to the target precisely one time. This prevents both data loss and data duplication, even in the event of network failures or application restarts.

Activating EOS requires coordinating settings across the source consumer, the target producer, and a global feature flag. All three properties described below must be configured correctly to achieve the full EOS guarantee.

Configuration Parameters

Property Path	Description	Required to Enable EOS
features.exactlyOnce	The main feature flag to enable transactional processing in K2K. It must be set to the string enabled.	Yes
target.kafka.producer."enable.idempotence"	This standard Kafka producer property ensures that retries do not create duplicate messages. It is a mandatory prerequisite for enabling transactions. It must be set to true.	Yes
source.kafka.consumer."isolation.level"	This standard Kafka consumer property ensures the replicator only reads records from transactions that have been fully committed on the source cluster. Set this to read_committed to prevent K2K from replicating data from an aborted transaction.	Yes

Note on the Transactional Mechanism: EOS in K2K is achieved by atomically committing the consumed source offsets and the produced target records within a single transaction on the target cluster. The configuration properties ensure that both the consumer and producer are correctly configured to participate in this transactional process. K2K manages the producer's transactional.id automatically when features.exactlyOnce is enabled.

---

Example

The following example shows the required properties set across the features, source, and target configuration blocks to enable exactly-once semantics.

# ---------------------------------------------------
# K2K Configuration for Exactly-Once Semantics (EOS)
# ---------------------------------------------------

# 1. Global feature flag to enable transactional processing.
features:
  exactlyOnce: enabled

# 2. Source consumer must be configured to only read committed records.
source:
  kafka:
    consumer:
      # Prevents "dirty reads" from the source cluster.
      "isolation.level": "read_committed"
      
      # ... other source consumer properties
      "group.id": "k2k-eos-replicator-group"

# 3. Target producer must be configured for idempotence.
target:
  kafka:
    producer:
      # Enables the idempotent producer, a prerequisite for EOS.
      "enable.idempotence": true
      
      # ... other target producer properties
      "acks": "all"