Records routing

Configures the target topic for a replicated Kafka message.

The sink.topic configuration within each replication flow determines how destination topic names are generated on the target cluster. K2K provides several routing strategies to accommodate different use cases, from direct name preservation to complex, pattern-based mapping. The following sections detail each of the mutually exclusive strategies.

Direct Mapping (1-to-1)

This strategy preserves the original source topic names. Each source topic is replicated to a target topic with the exact same name. This is the simplest routing method.

Use Cases:

• Creating an identical mirror of topics for disaster recovery or blue-green deployments.

• Simple replication scenarios where no name translation is needed.

Example

To enable this, set the value of topic to the special keyword source.

# sink configuration for direct 1-to-1 mapping
sink:
  name: "direct-mapping-sink"
  topic: source
  partition: source

Resulting Mapping:

Source Topic	Target Topic
prod-topic-1	prod-topic-1
prod-topic-2	prod-topic-2

---

Prefix and Suffix Naming

This strategy modifies the source topic name by adding a specified prefix and/or suffix. Both prefix and suffix are optional, but at least one must be provided.

Use Cases:

• Avoiding topic name collisions on a shared target cluster.

• Clearly identifying replicated topics (e.g., prefixing all with replicated-).

• Versioning replicated topics (e.g., adding a .v2 suffix).

Example

# sink configuration for adding a prefix and a suffix
sink:
  name: "prefix-suffix-sink"
  topic:
    prefix: "eu."
    suffix: ".copy"  
  partition: source

Resulting Mapping:

Source Topic	Target Topic
prod-topic-1	eu.prod-topic-1.copy
prod-topic-2	eu.prod-topic-2.copy

---

Static Topic Routing (Many-to-1)

This strategy routes all source topics selected in the flow to a single, statically named target topic.

Use Cases:

• Aggregating multiple source topics into a single destination topic for simplified consumption.

• Replicating a single source topic to a target topic with a different, arbitrary name.

Example

# sink configuration for routing all topics to a single destination
sink:
  name: "aggregation-sink"
  topic:
    static: "landing-topic" 
  partition: source

Resulting Mapping:

Source Topic	Target Topic
prod-topic-1	landing-topic
prod-topic-2	landing-topic

---

Pattern-Based Mapping

This is the most flexible routing strategy, allowing you to define explicit mapping rules from source topics or patterns to specific target topics. The keys in the staticMapping object are regular expressions that are matched against the names of the topics selected by the source block of the flow.

Note: The rules in staticMapping are applied in order. The first pattern that matches a source topic name determines its destination. It is a best practice to place more specific rules before more generic ones.

Use Cases:

• Consolidating topics based on team or service names (e.g., all team-red-.* topics go to team-red-data).

• Applying specific, one-off name translations that don't fit a simple prefix/suffix model.

• Implementing a "catch-all" rule to route any non-matching topics to a default destination.

Example

# sink configuration for custom, pattern-based mapping
sink:
  name: "custom-mapping-sink"
  topic:
    staticMapping:
      # 1-to-1 mapping for specific topics
      prod-topic-1: "copied-from-prod.1"
      prod-topic-2: "prod-two-copy"
      # Many-to-1 mapping using a regex pattern
      "team-red.*": "team-red-combined-data"
      "team-blue.*": "team-blue-combined-data"
      # A catch-all pattern for any other topic
      ".*": "other-teams-combined-data"
  partition: source

Resulting Mapping:

Original Topic	Destination Topic
prod-topic-1	copied-from-prod.1
prod-topic-2	prod-two-copy
team-red-topic-1	team-red-combined-data
team-blue-topic-2	team-blue-combined-data
another-topic-1	other-teams-combined-data