Features

A list of currently implemented features

Guaranteed Exactly-Once Semantics

By default, the replicator provides at-least-once delivery semantics, ensuring no data is lost during transfer between source and target Kafka clusters. For use cases that cannot tolerate duplicate messages, the replicator can be configured for exactly-once processing. This guarantees that each message from the source is written to the target precisely one time, providing the highest level of data integrity.

Integrated Schema Replication

The replicator can automatically propagate schemas from your source to your target cluster alongside the message data. When enabled, it inspects the key and value of each message and ensures the associated schema is registered in the target's Schema Registry before the message is written. This maintains data compatibility and usability across your entire ecosystem.

• Default State: This feature is disabled by default.

• Compatibility: Currently supports any registry that is compatible with the Confluent Schema Registry API.

Flexible Topic Routing

Gain granular control over how topics are named in the target cluster. The replicator supports several powerful routing strategies to suit different architectural needs, such as data aggregation, multi-tenancy, or blue-green deployments.

Supported naming strategies include:

• Direct Mapping: Replicate to a topic with the same name as the source.

• Static Naming: Route a source topic to a single, predefined topic name on the target.

• Dynamic Prefixing/Suffixing: Automatically add a consistent prefix or suffix to the source topic name (e.g., source-topic becomes replicated-source-topic).

• Explicit Mapping: Define custom name-to-name mappings for replicating multiple topics to specific destinations.

Automatic Topic Creation

When this feature is active, the replicator will create any destination topic on the target cluster if it does not already exist. Crucially, the replicator mirrors the source topic’s configuration, including the number of partitions, replication factor, and other topic-level settings. This ensures environmental consistency and removes the need for manual pre-provisioning of topics.

• Default State: This feature is disabled by default to prevent unintended modifications to the target cluster.

Broad Kafka API Compatibility

As a standard Kafka client application, the replicator is fundamentally vendor-agnostic. It is designed to work seamlessly with any service that implements the Kafka API. This ensures maximum flexibility and freedom from vendor lock-in.

Verified compatible implementations include:

• Vanilla Apache Kafka

• Confluent Platform & Kora

• Redpanda

• Amazon MSK

• Aiven for Apache Kafka

• IBM Event Streams

Tracing Headers

The feature introduces additional headers for target messages, enabling correlation with the original topic, partition, and offset. Users can control whether to enable these headers, which include partition, topic, offset, and pipeline access. By default, these headers are disabled.

Bidirectionality

The solution allows linking two clusters, A and B, ensuring data can be copied from A to B and vice versa without duplication. Records already transferred from A to B will not be copied back to A.