Change log

Release notes for Lenses K2K (Kafka-to-Kafka replicator).

Release notes for Lenses K2K, the Kafka-to-Kafka replicator. Newer versions appear first.

October 10, 2026

NewImprovementsFixed

2.1.0

Version 2.1.0 speeds up replication by overlapping reads, writes and offset commits, and turns the K2K Tool web UI into a live view of both record replication and consumer offset mapping.

Faster replication

Replication no longer waits for each step to finish before starting the next. The source consumer reads ahead while earlier batches are still being processed. In at-least-once mode, records are sent to the target cluster asynchronously, and source offsets are committed in batches once the target cluster has acknowledged every record in the batch.

Three new settings tune this behavior (see Replication performance tuning):

• configuration.maxPrefetchBatches (default 3): how many record batches the source consumer may prefetch per partition while earlier batches are still being processed. Values below 2 are treated as 2.
• configuration.pollInterval (default 0 milliseconds): the minimum interval between Kafka consumer poll calls.
• coordination.commit.pendingAckQueueSize (default 5): how many commit batches can wait for acknowledgement in at-least-once mode. Up to pendingAckQueueSize × coordination.commit.batchSize records can be awaiting acknowledgement at once, so raising either setting raises peak memory.

coordination.commit.batchTimeout now defaults to 30 seconds (previously 8 seconds). It sets how long K2K waits for coordination.commit.batchSize records to accumulate before committing the offsets of the records collected so far.

Live K2K Tool web UI

The K2K Tool web UI (k2k-tool server) now receives updates over a WebSocket connection, so the pipeline list, committed offsets and cluster health refresh on their own.

• Record replication and offset mapping views. Each pipeline has a K2K Record Replication view and a K2K Offset Mapping view. The offset mapping view lists the consumer groups matched by the pipeline's offset mapping group filter, with source and target committed offsets and an Offset Mapping Lag (estimate) column, and can be filtered by consumer group and by topic.
• Partition detail. Topics expand to show the committed offset, last offset and lag of each partition. Topics can be searched and sorted by name or lag, and updates can be paused without closing the connection. Selecting a topic or consumer group opens an Information sidebar with per-partition detail, including the target topic name when sink rules rename topics. Drag its left edge to resize it; the width is remembered.
• Set offsets. In the record replication view, set new committed offsets for several partitions of a topic in one dialog, with success or failure reported per partition.
• Safer resets. Reset dialogs warn you to stop every replicator instance for the pipeline first; otherwise the change may be ignored.
• Bounded memory. The tool keeps offset mapping ranges in memory according to each pipeline's offsetMapping.retention settings, so memory stays bounded in long-running server sessions.

New server options control how often the tool reads from Kafka:

Option	Default	Controls
--healthcheck-interval / -H	30s	How often cluster health is checked
--topic-offset-interval / -O	1.5s	How often source topic end offsets are refreshed
--topic-offset-batch-size / -B	100	Topic-partitions fetched per end offset request
--consumer-group-offset-interval / -G	5s	How often committed offsets of offset mapping consumer groups are refreshed
--websocket-min-refresh-interval / -R	500ms	Minimum time between live updates
--kafka-fetch-jitter / -J	0	Random delay added between background Kafka reads, as a fraction of the interval (0.2 adds up to 20%)

K2K Tool command line

• describe output is now grouped by consumer group and then topic, and shows the last replicated offset, source end offset and replication lag of each partition. When offset mapping is enabled, it also prints the offset mapping consumer groups.
• describe --live has been removed. Use the web UI for live updates.
• When a command fails, the tool prints Program did not terminate successfully and the error message to stderr as well as to the log.

Fixed

• describe no longer shows blank source end offsets when the offset cache has not been populated yet, and a missing offset mapping topic no longer blocks it. If a control topic partition returns no end offset, the command now fails instead of waiting indefinitely.
• Transient Kafka errors no longer stop the K2K Tool's background refreshes. A failed read is logged and skipped, and consumer groups and partitions that no longer exist are removed from the live view.

Security updates

Third-party dependencies have been updated to address known security advisories.

July 23, 2026

NewImprovements

2.0.1

Lower memory use

K2K now uses a newer Kafka consumer library. Consumer memory use no longer grows with the number of assigned partitions.

Parallelism settings

• configuration.pipelineParallelism now sets the maximum number of concurrent streams processing the assigned source partitions, rather than the number of concurrent tasks per pipeline step. Its default changes from 1 to 15. If you set it explicitly, review the value before upgrading.
• The new configuration.controlParallelism setting (default 10) limits how many concurrent streams process control messages. The K2K Tool reads it from the pipeline configuration too.

April 29, 2026

New

2.0.0

Version 2.0.0 makes consumer group failover between clusters a first-class capability: K2K now continuously tracks source-to-target offset translations and a new companion application applies them to consumer groups on the target cluster, so consumers can switch clusters without a rewind or a manual offset reset. This release also changes how replication produces and commits records.

Consumer offset mapping for failover

K2K now publishes a continuous record of source-to-target offset translations to a dedicated control topic (__ k2k_offset_mappings by default) on the target cluster. A new companion application, lensesio/k2k-offset-mapper, reads this topic and periodically commits the translated offsets directly to consumer groups on the target cluster. When a consumer group needs to move to the target cluster, its offsets are already in place and the consumer can resume from the equivalent position without any manual intervention.

Continuous commits

Previously, replication accumulated records into an application-level batch (controlled by coordination.commit.batchSize and coordination.commit.batchTimeout ) before sending them to the target. K2K now passes each record to the Kafka producer immediately as it is ready. The result is lower end-to-end replication latency and a smaller heap footprint, especially on high-throughput deployments.

January 20, 2026

New

1.2.0

K2K tooling

The K2K Docker container now includes an additional binary: k2k-tool. This new app allows resetting and visualizing the K2K consumer

Bidirectional replication

K2K now allows bidirectional record replication by using the flag bidirectionalReplication . Advanced control regarding headers used and strategy can be done using the bidirectional top level configuration.

November 27, 2025

NewImprovements

1.1.0

Free version restriction

The free version now includes a restriction on the total number of topic-partitions that the replicator can process.

• The maximum limit for the sum of topic partitions across all replicated topics is 5.
• Any combination of topics can be replicated as long as their combined partition count is less than or equal to this limit.
• Topics with a partition count greater than 5 cannot be replicated without a valid license.

For licensing inquiries, please contact [e-mail].

Schema Mapping feature

The Schema Mapping feature is now an enterprise feature and requires a valid license for use.

Improvements

• Added validation for Producer/Consumer configuration when exactly-once semantics are enabled
• Reduced Docker image size by removing unnecessary deb files
• Fixed issue where empty license caused application termination without error message

October 30, 2025

New

1.0.0

The K2K data replicator is now available for use in production environments.

JVM Support

The software is now packaged to run on JVM 25, enabling the application to benefit from improvements made to object allocations in this JVM version.

Licensing

License restrictions have been implemented for the exactly-once delivery feature. This feature can only be activated through a license.

Distributed Tracing

Trace headers are now stored as strings instead of binary format, improving compatibility with observability and monitoring tools.

Kafka Configuration

This release introduces a new configuration parameter, target.kafka.topicPropsExclusionList, which provides finer control over topic creation behavior. This parameter allows you to exclude specified topic configurations when the replicator creates topics on the target cluster.

The primary use case addresses compatibility issues with Kafka implementations that don't expose certain configurations as read-only in their metadata but reject them when included in topic creation requests.

October 24, 2025

Improvements

0.5.0

• Application failure reasons can now be written to a configurable location. Command line argument
• Improved K2K startup time. Control topics are now read and processed faster at startup.
• Addressed bug that would cause the app to stop replicating data when feature flag optimizeOffsetCommitPartition was disabled
• Increased default topic sync timeout. By default the app can take up to 90s to read the whole control topic.
• Improved error messages when the app takes too long to read the control topic(s).

October 8, 2025

Fixed

0.4.0

• At-least-once data loss: Fixed an issue that could lead to data loss when producers failed to reliably receive timely write acknowledgements from the broker (e.g., due to quota limits or network instability). K2K now includes mitigations to minimize this risk, and an explicit error is returned before data loss can occur.
• Dependency Packaging:
• Enhanced Auto-topic Creation: Prevent failures when replicating newer clusters to older versions, ensuring smooth topic creation despite configuration differences.
• Metrics Exposure: Committed offsets are now exposed as metrics for integration with tools like Prometheus.
• Connectivity Suport: Connectivity troubleshooting module is disabled by default
• License: The free license requirement has been removed
• Google Managed Kafka: This release ensures all essential Google Managed Kafka dependencies are included. When topic auto creation is enabled, and Google Managed Kafka is the target environment, failures might occur. This is due to Google Managed Kafka not allowing the setting of some properties like segment.bytes and local.retention.ms. It's therefore advised at the moment to provide the following config values for topicCreation.replication.common.config:\

  topicCreation:
    replication:
      common:
        config:
          "segment.bytes": null
          "local.retention.ms": null

September 16, 2025

Improvements

0.3.0

• Centralized Schema registry and Kafka connectivity configuration

  • Schema registry is now specified in source|target.registry.config|headers|cacheSize
  • Kafka configuration is now specified in source|target.kafka.[common|consumer|admin]
• Kafka connectivity validation (on by default, can be toggled off using features.checkKafkaConnectionOnStartup)
• com.google.cloud.google-cloud-managedkafka is now packaged with K2K.
• Added fine grain control for schema replication.

  • configured through schemaMapping.topics
• Metrics messages and control message are now all part of the same ControlMessage definition hierarchy.
• Added support for injection of environment variables and/or file contents in the pipeline definition file.

  • the pipeline definition file supports substituting expressions (e.g: ${env:base64:ENV_VARIABLE} by their values.
• Performance improvements.

Migration

v0.2.0	v0.3.0	Change
.parameters	.configuration	renamed
.coordination.kafka.consumer	.target.kafka.consumer	moved
.coordination.kafka.charset	removed
.coordination.kafka.commit.group	removed	source consumer's group.id is now used .
.coordination.kafka.assignement	.coordination.assignement	removed kafka path segment
.coordination.kafka.commit	.coordination.commit	removed kafka path segment
.source.kafka.connection.servers	.source.kafka.common."bootstrap.servers"	moved
.source.kafka.consumer		inherits all properties defined under source.kafka.common.
.source.kafka.registry.supportedTypes	.schemaMapping.supportedTypes	source and target SR use the same values
.source.kafka.registry.url	.source.registry.config."schema.registry.url"
.source.kafka.registry	.source.registry
.target.kafka.connection.servers	.target.kafka.common."bootstrap.servers"
.target.kafka.consumer	.target.kafka.consumer	inherits all properties defined under target.kafka.common
.target.kafka.producer	.target.kafka.producer	inherits all properties defined under target.kafka.common
.target.kafka.registry.supportedTypes	.schemaMapping.supportedTypes	source and target SR use the same values
.target.kafka.registry.url	.target.registry.config."schema.registry.url"
.target.kafka.registry	.target.registry
.features.offsetCommitOptimizePartition	.features.optimizeOffsetCommitPartition

Example:

name: "simple_pipeline"
source:
  kafka:
    common:
      "bootstrap.servers": "localhost:9092"
      "sasl.mechanism": "PLAIN"
      "sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=userName password=pwd;"
      "ssl.truststore.type": "JKS"
      "ssl.truststore.location": "/etc/some/path"
      "ssl.truststore.password": "some-password"
      "security.protocol": "SASL_SSL"
    consumer:
      "group.id": "my-group"
  registry:
    config:
      "schema.registry.url": "${env:raw:REGISTRY_URL}"
      "schema.registry.ssl.truststore.type": "JSK"
      "schema.registry.ssl.truststore.location": "/some/other/path"
      "schema.registry.ssl.truststore.password": "some-sr-password"
target:
  kafka:
    common:
      "bootstrap.servers": "localhost:9099"
      "sasl.mechanism": "PLAIN"
      "sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=userName password=pwd;"
      "ssl.truststore.type": "JKS"
      "ssl.truststore.location": "/etc/some/path"
      "ssl.truststore.password": "some-password"
      "security.protocol": "SASL_SSL"
  registry:
    config:
      "schema.registry.url": "localhost:8081"
      "schema.registry.ssl.truststore.type": "JSK"
      "schema.registry.ssl.truststore.location": "/some/other/path"
      "schema.registry.ssl.truststore.password": "some-sr-password"
replication:
  - source:
      name: source
      topic: topic.*
  - sink:
      topic: source
      name: sink-source-topic
      partition: source

August 9, 2025

0.1.0

Warning

Please note that from this version, a free license token is required. Ensure you obtain one to continue using the service without interruptions. To obtain a free license, send an email to [e-mail]. You will receive an automatic reply with the license token information.

Please note that from this version, a free license token is required. Ensure you obtain one to continue using the service without interruptions. To obtain a free license, send an email to [e-mail]. You will receive an automatic reply with the license token information.

Retain Original Message Timestamp

The replication process now includes a feature called keepRecordCreationTimestamp, which retains the original message timestamp. This feature is enabled by default.

Fixes

• Ensure topic creation waits for Kafka to finish the action before proceeding.
• Auto-created control topics will use "compact" as the cleanup.policy to help lower costs and decrease startup latency.