Skip to content
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Bidirectional replication

Replicate in both directions without loops.

This feature is disabled by default. Set the corresponding features toggle to enabled in your pipeline file for the behavior described on this page to take effect.

In active/active setups, two Kafka clusters replicate to each other. Without a loop breaker, a record copied from the source Kafka cluster to the target Kafka cluster would be copied back indefinitely. Bidirectional replication marks replicated records with a header and skips records that already carry it.

Run one pipeline per direction with features.bidirectionalReplication: enabled. The header value identifies replicated records either by pipeline name or by source Kafka cluster ID:

Mode
Header value

pipeline

Pipeline name

cluster

Source cluster ID

Prerequisites

Example

Use the following pipeline configuration:

name: "my_pipeline"
features:
  bidirectionalReplication: enabled
source:
  kafka:
    common:
      "bootstrap.servers": "source:9092"
    consumer:
      "group.id": "my-k2k-group"
target:
  kafka:
    common:
      "bootstrap.servers": "target:9092"
bidirectional:
  mode: pipeline
replication:
  - source:
      topic: "orders.*"
  - sink:
      topic: source
      partition: source

Configuration keys

Key
Description
Required
Default
Used by

features.bidirectionalReplication

Enable bidirectional replication filtering

No

disabled

Replicator

bidirectional.mode

How replicated records are identified (pipeline or cluster)

No

cluster

Replicator

bidirectional.headers.bidirectional

Header name marking replicated records

No

__k2k_bidirectional

Replicator

Last updated

Was this helpful?