Glossary
Glossary of terms used across K2K documentation.
This page defines the terms used across K2K documentation.
For high-level architecture and how K2K components fit together, see Architecture. For task-oriented configuration guides, see Configuration reference. For which application to deploy, see Deployment.
Naming conventions
K2K
The product—data replication between Kafka clusters
k2k
The replicator application binary (k2k start)
k2k-offset-mapping
The offset mapper application binary (k2k-offset-mapping start)
k2k-tool
The administration CLI and web UI for offset management
Pipeline
The YAML configuration file that defines a replication deployment
K2K
K2K (Kafka to Kafka)
The suite of applications that replicate data between Kafka clusters. K2K covers data replication, schema migration, and consumer offset translation as a single operational surface.
K2K replicator
The application that reads records from a source Kafka cluster and writes them to a target Kafka cluster. The replicator also handles schema migration when enabled in the pipeline.
K2K offset mapper
The application that reads offset mappings produced during replication and translates consumer group offsets from the source Kafka cluster to the target Kafka cluster. Must run alongside the replicator whenever offset mapping is enabled.
See Deployment.
Clusters
Kafka cluster
A deployment of Apache Kafka brokers that store and serve topic data. Applications connect to a cluster through bootstrap servers.
Source cluster
The Kafka cluster K2K reads from. In most setups this is the cluster holding the data to be copied (for example, production).
Target cluster
The Kafka cluster K2K writes to. In most setups this is the cluster receiving the replicated data (for example a standby, migration destination, or downstream environment).
Kafka clients
Kafka consumer
A client that reads records from one or more topic partitions and tracks its position through committed offsets. K2K uses consumers on the source Kafka cluster to read data for replication.
Kafka producer
A client that writes records to topic partitions. K2K uses producers on the target Kafka cluster to publish replicated data.
Kafka admin client
A client that manages cluster metadata and administrative operations (for example listing topics, describing consumer groups, or committing offsets). K2K uses admin clients for topic creation, offset mapping, and coordination between instances.
Consumer groups
Consumer group
A named set of Kafka consumers that cooperate to read from a topic. Kafka assigns each partition to one consumer in the group. Applications track their progress through offsets committed on behalf of the group. K2K's offset mapper watches consumer groups on the source Kafka cluster and translates their committed offsets to the target Kafka cluster.
K2K group
The consumer group the K2K replicator belongs to on the source Kafka cluster. It is configured through source.kafka.consumer."group.id". All replicator instances running the same pipeline share the K2K group. Kafka uses it to distribute source topic partitions across instances. K2K does not commit offsets on the source Kafka cluster; replication progress is tracked through control topics on the target Kafka cluster.
See Consumer group.
Offsets
Source offset
The position of a record within a topic partition on the source Kafka cluster. Consumer groups on the source Kafka cluster track their progress using source offsets.
Target offset
The position of a record within a topic partition on the target Kafka cluster. The same record has a different offset on the target Kafka cluster than on the source Kafka cluster, because each Kafka cluster maintains its own offset sequence.
Offset mapping
The correspondence between a source offset and a target offset for a given topic partition. K2K records offset mappings during replication and uses them to translate consumer group positions from the source Kafka cluster to the target Kafka cluster.
See Offset mapping.
Coordination
K2K tracks replication progress and instance ownership through control topics on the target Kafka cluster, rather than committing offsets on the source Kafka cluster.
Control topics (internal topics)
Topics on the target Kafka cluster used by K2K for internal coordination. They are not application data. Control topics include those used for exclusive assignment, commit, offset mapping, and metrics.
Control consumer
The Kafka consumer on the target Kafka cluster that reads control topics. It keeps each replicator instance aware of partition ownership and replication progress across the K2K group.
Assignment
The process by which Kafka's consumer group protocol distributes source topic partitions across replicator instances in the K2K group. Each instance consumes from the partitions assigned to it.
Exclusive assignment
A K2K mechanism that ensures only one replicator instance actively processes a given source topic partition at a time. During consumer group rebalances, overlapping assignments can occur temporarily. K2K uses a control topic on the target Kafka cluster to grant and enforce exclusive ownership before replication proceeds.
Commit
The act of recording how far replication has progressed on each source topic partition. K2K writes commit records to a control topic on the target Kafka cluster, tracking the last replicated offset per partition.
See Exactly once delivery and Horizontal scaling.
Delivery semantics
Exactly once semantics
A delivery guarantee where each record is processed and written once, with no duplicates. In replication, this means each record on the source Kafka cluster appears exactly once on the target Kafka cluster, in the same order.
At least once semantics
A delivery guarantee where each record is delivered one or more times. Duplicates are possible if a failure occurs between writing and committing progress. This is the default when exactly once is not enabled.
Replication
Replication pipeline
The unit of configuration in K2K. A pipeline names a source Kafka cluster, a target Kafka cluster, the features to enable, and the replication stages that define what to copy and where records land. Each K2K application runs one pipeline definition.
Replication stage
A step in the pipeline's replication definition. The first stage is always a source stage, which declares which topics to read from the source Kafka cluster. The following stages are sink stages, which declare how records are routed to the target Kafka cluster (topic name, partition assignment). A pipeline requires at least one source stage and one sink stage.
Last updated
Was this helpful?

