Bidirectional replication
Replicate in both directions without loops.
In active/active setups, two Kafka clusters replicate to each other. Without a loop breaker, a record copied from the source Kafka cluster to the target Kafka cluster would be copied back indefinitely. Bidirectional replication marks replicated records with a header and skips records that already carry it.
Run one pipeline per direction with features.bidirectionalReplication: enabled. The header value identifies replicated records either by pipeline name or by source Kafka cluster ID:
pipeline
Pipeline name
cluster
Source cluster ID
Prerequisites
Topic selection and data routing configured
One pipeline per direction (each with bidirectional filtering enabled)
Example
Use the following pipeline configuration:
name: "my_pipeline"
features:
bidirectionalReplication: enabled
source:
kafka:
common:
"bootstrap.servers": "source:9092"
consumer:
"group.id": "my-k2k-group"
target:
kafka:
common:
"bootstrap.servers": "target:9092"
bidirectional:
mode: pipeline
replication:
- source:
topic: "orders.*"
- sink:
topic: source
partition: sourceConfiguration keys
features.bidirectionalReplication
Enable bidirectional replication filtering
No
disabled
Replicator
bidirectional.mode
How replicated records are identified (pipeline or cluster)
No
cluster
Replicator
bidirectional.headers.bidirectional
Header name marking replicated records
No
__k2k_bidirectional
Replicator
Last updated
Was this helpful?

