> For the complete documentation index, see [llms.txt](https://docs.lenses.io/latest/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.lenses.io/latest/k2k/configuration-reference/full-reference.md).

# Full reference

All K2K pipeline configuration keys and command-line options in one place.

This page lists every K2K pipeline configuration key and command-line option in one place. For task-oriented guides with examples and context, see [Configuration Reference](https://docs.lenses.io/latest/k2k/configuration-reference.md). For minimal pipeline structure and file loading, see [Minimal Configuration](https://docs.lenses.io/latest/k2k/minimal-configuration.md).

Keys use dot notation. `[]` marks a list element (for example `replication[].source.topic`). Kafka and Schema Registry client properties use quoted keys (for example `source.kafka.consumer."group.id"`).

## Conventions

| Column      | Meaning                                                      |
| ----------- | ------------------------------------------------------------ |
| Key         | YAML path using dot notation                                 |
| Description | What the setting controls                                    |
| Required    | Yes, No, or Conditional (required when a feature is enabled) |
| Default     | Value applied when the key is omitted                        |
| Used by     | Replicator, Offset mapper, or Both                           |

**Value types**

| Type   | Format                                                         |
| ------ | -------------------------------------------------------------- |
| Toggle | `enabled` or `disabled`                                        |
| Time   | Duration string (for example `15 seconds`, `750 milliseconds`) |
| Regex  | Java regular expression                                        |

Pipeline files can be merged at startup. Later files override earlier ones for the same key. See [Loading pipeline files](https://docs.lenses.io/latest/k2k/minimal-configuration.md#loading-pipeline-files). Secrets can be injected with `${...}` templates — see [Secrets and templates](https://docs.lenses.io/latest/k2k/configuration-reference/secrets-and-templates.md).

## Command-line options

Applies to `k2k start`, `k2k dry-run`, and `k2k-offset-mapping start` unless noted. See [Deployment](https://docs.lenses.io/latest/k2k/deployment.md).

| Flag                        | Short | Default                | Description                                                                                                     |
| --------------------------- | ----- | ---------------------- | --------------------------------------------------------------------------------------------------------------- |
| `--file`                    | `-f`  | *(required)*           | Pipeline definition file. Pass multiple times to merge files; later files override earlier ones.                |
| `--defaults`                | `-d`  | Classpath defaults     | HOCON file with default pipeline values. Pass multiple times to layer files; later files override earlier ones. |
| `--template-engine`         | `-g`  | `enabled`              | Enable or disable `${...}` template resolution before YAML parsing.                                             |
| `--telemetry`               | `-t`  | disabled               | Enable OpenTelemetry metric export (`k2k start` and `k2k-offset-mapping start` only).                           |
| `--termination-log`         | `-l`  | `/dev/termination-log` | File to write a shutdown or error message on exit.                                                              |
| `--termination-log-enabled` | `-w`  | `enabled`              | Write the termination log on exit. Set to `disabled` to turn this off.                                          |
| `--verbosity`               | —     | *(omitted)*            | Log level: `off`, `error`, `info`, `debug`, or `trace`.                                                         |

## Pipeline structure

| Block            | Purpose                                              | Guide                                                                                                                                                                                                                                                                                          |
| ---------------- | ---------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `name`           | Unique pipeline identifier                           | —                                                                                                                                                                                                                                                                                              |
| `license`        | License acceptance and token                         | [License](https://docs.lenses.io/latest/k2k/configuration-reference/licenses.md)                                                                                                                                                                                                               |
| `features`       | Feature toggles                                      | See [features](#features) below                                                                                                                                                                                                                                                                |
| `source`         | Source Kafka cluster and Schema Registry connection  | [Kafka connectivity](https://docs.lenses.io/latest/k2k/configuration-reference/kafka-connectivity-authentication.md), [Schema Registry](https://docs.lenses.io/latest/k2k/configuration-reference/schema-registry-connectivity-authentication.md)                                              |
| `target`         | Target Kafka cluster and Schema Registry connection  | [Kafka connectivity](https://docs.lenses.io/latest/k2k/configuration-reference/kafka-connectivity-authentication.md), [Schema Registry](https://docs.lenses.io/latest/k2k/configuration-reference/schema-registry-connectivity-authentication.md)                                              |
| `replication`    | Source and sink stages                               | [Topic selection and data routing](https://docs.lenses.io/latest/k2k/configuration-reference/topic-selection-and-data-routing.md)                                                                                                                                                              |
| `coordination`   | Control topic names, assignment, and commit batching | [Consumer group](https://docs.lenses.io/latest/k2k/configuration-reference/consumer-group.md), [Exactly once](https://docs.lenses.io/latest/k2k/configuration-reference/exactly-once-delivery.md), [Commit tuning](https://docs.lenses.io/latest/k2k/configuration-reference/commit-tuning.md) |
| `offsetMapping`  | Offset mapping topic and retention                   | [Offset mapping](https://docs.lenses.io/latest/k2k/configuration-reference/offset-mapping.md)                                                                                                                                                                                                  |
| `groups`         | Consumer groups to translate (offset mapper)         | [Offset mapping](https://docs.lenses.io/latest/k2k/configuration-reference/offset-mapping.md)                                                                                                                                                                                                  |
| `schemaMappings` | Schema replication rules                             | [Schema migration](https://docs.lenses.io/latest/k2k/configuration-reference/schema-migration.md)                                                                                                                                                                                              |
| `topicCreation`  | Auto created topic settings                          | [Topic creation](https://docs.lenses.io/latest/k2k/configuration-reference/topic-creation.md)                                                                                                                                                                                                  |
| `configuration`  | Pipeline parallelism and runtime tuning              | [Horizontal scaling](https://docs.lenses.io/latest/k2k/configuration-reference/horizontal-scaling.md), [Replication performance tuning](https://docs.lenses.io/latest/k2k/configuration-reference/replication-performance-tuning.md)                                                           |
| `errorHandling`  | Behaviour when errors occur                          | [Pipeline tuning](https://docs.lenses.io/latest/k2k/configuration-reference/pipeline-tuning.md), [Offset mapping](https://docs.lenses.io/latest/k2k/configuration-reference/offset-mapping.md)                                                                                                 |
| `metrics`        | Metric name prefixes and Kafka metrics topic         | [Observability](https://docs.lenses.io/latest/k2k/configuration-reference/observability.md)                                                                                                                                                                                                    |
| `tracing`        | Tracing header names                                 | [Record tracing](https://docs.lenses.io/latest/k2k/configuration-reference/record-tracing.md)                                                                                                                                                                                                  |
| `bidirectional`  | Bidirectional replication mode and header            | [Bidirectional replication](https://docs.lenses.io/latest/k2k/configuration-reference/bidirectional-replication.md)                                                                                                                                                                            |

## Pipeline keys

### `name`

| Key    | Description                         | Required | Default | Used by |
| ------ | ----------------------------------- | -------- | ------- | ------- |
| `name` | Unique identifier for this pipeline | Yes      | (none)  | Both    |

### `license`

| Key                  | Description                                                                              | Required | Default | Used by |
| -------------------- | ---------------------------------------------------------------------------------------- | -------- | ------- | ------- |
| `license.acceptEula` | Accept the K2K EULA (must be `true`)                                                     | Yes      | `false` | Both    |
| `license.token`      | K2K license token. Omit to run in free tier mode (5 partition cap, no licensed features) | No       | (none)  | Both    |

`features.schemaMapping`, `features.bidirectionalReplication`, and `features.offsetMapping` require a token that carries the matching entitlement. See [License](https://docs.lenses.io/latest/k2k/configuration-reference/licenses.md).

### `features`

| Key                                          | Description                                                                                                | Required | Default    | Used by       |
| -------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | -------- | ---------- | ------------- |
| `features.headerReplication`                 | Replicate record headers from the source Kafka cluster to the target Kafka cluster                         | No       | `enabled`  | Replicator    |
| `features.keepRecordCreationTimestamp`       | Preserve the record creation timestamp when replicating                                                    | No       | `enabled`  | Replicator    |
| `features.schemaMapping`                     | Replicate schemas to the target Schema Registry (requires a licensed entitlement)                          | No       | `disabled` | Replicator    |
| `features.offsetMapping`                     | Record offset mappings during replication for consumer group translation (requires a licensed entitlement) | No       | `disabled` | Replicator    |
| `features.applyMappingsGuard`                | Skip applying offset mappings when the target Kafka cluster already has a more recent committed offset     | No       | `enabled`  | Offset mapper |
| `features.exactlyOnce`                       | Exactly once delivery guarantee                                                                            | No       | `disabled` | Replicator    |
| `features.optimizeOffsetCommitPartition`     | Publish all control commit messages to a single partition per group                                        | No       | `enabled`  | Replicator    |
| `features.autoCreateTopics`                  | Auto create replicated topics on the target Kafka cluster                                                  | No       | `disabled` | Replicator    |
| `features.autoCreateControlTopics`           | Auto create internal control topics on the target Kafka cluster                                            | No       | `disabled` | Both          |
| `features.validateControlTopicSettings`      | Validate control topic configuration on startup                                                            | No       | `enabled`  | Replicator    |
| `features.filterNonUpdatableTopicProperties` | Filter topic properties that cannot be set on the target Kafka cluster when auto creating topics           | No       | `enabled`  | Replicator    |
| `features.tracingHeaders`                    | Inject tracing headers on replicated records (`enabled`, `disabled`, or per header object)                 | No       | `disabled` | Replicator    |
| `features.tracingHeaders.partition`          | Inject source partition tracing header                                                                     | No       | `disabled` | Replicator    |
| `features.tracingHeaders.offset`             | Inject source offset tracing header                                                                        | No       | `disabled` | Replicator    |
| `features.tracingHeaders.topic`              | Inject source topic tracing header                                                                         | No       | `disabled` | Replicator    |
| `features.tracingHeaders.pipeline`           | Inject pipeline name tracing header                                                                        | No       | `disabled` | Replicator    |
| `features.bidirectionalReplication`          | Enable bidirectional replication filtering (requires a licensed entitlement)                               | No       | `disabled` | Replicator    |
| `features.checkKafkaConnectionOnStartup`     | Probe Kafka connectivity before starting replication                                                       | No       | `disabled` | Replicator    |

### `source`

| Key                                       | Description                                               | Required    | Default                                             | Used by    |
| ----------------------------------------- | --------------------------------------------------------- | ----------- | --------------------------------------------------- | ---------- |
| `source.kafka`                            | Source Kafka cluster connection settings                  | Yes         | (none)                                              | Both       |
| `source.kafka.common`                     | Properties inherited by source consumer and admin clients | Conditional | `{}`                                                | Both       |
| `source.kafka.common."bootstrap.servers"` | Source Kafka cluster brokers                              | Yes         | (none)                                              | Both       |
| `source.kafka.consumer`                   | Source consumer properties                                | Conditional | See [Kafka client defaults](#kafka-client-defaults) | Replicator |
| `source.kafka.consumer."group.id"`        | K2K consumer group on the source Kafka cluster            | Yes         | (none)                                              | Replicator |
| `source.kafka.admin`                      | Source admin client properties                            | No          | `{}`                                                | Both       |
| `source.registry`                         | Source Schema Registry connection                         | Conditional | (none)                                              | Replicator |
| `source.registry.config`                  | Schema Registry client properties                         | Conditional | `{}`                                                | Replicator |
| `source.registry.headers`                 | HTTP headers added to source Schema Registry requests     | No          | `{}`                                                | Replicator |
| `source.registry.cacheSize`               | Maximum schemas cached locally on the source side         | No          | `31`                                                | Replicator |

`source.registry` is required when `features.schemaMapping` is enabled.

Any standard [Apache Kafka consumer property](https://kafka.apache.org/documentation/#consumerconfigs) can be set under `source.kafka.consumer` or `source.kafka.common`. Any standard [Schema Registry client property](https://docs.confluent.io/platform/current/schema-registry/sr-client-configs.html) can be set under `source.registry.config` with the `schema.registry` prefix.

### `target`

| Key                                       | Description                                                             | Required    | Default                                             | Used by       |
| ----------------------------------------- | ----------------------------------------------------------------------- | ----------- | --------------------------------------------------- | ------------- |
| `target.kafka`                            | Target Kafka cluster connection settings                                | Yes         | (none)                                              | Both          |
| `target.kafka.common`                     | Properties inherited by target consumer, producer, and admin clients    | Conditional | `{}`                                                | Both          |
| `target.kafka.common."bootstrap.servers"` | Target Kafka cluster brokers                                            | Yes         | (none)                                              | Both          |
| `target.kafka.consumer`                   | Target consumer properties (control consumer and offset mapper)         | No          | See [Kafka client defaults](#kafka-client-defaults) | Both          |
| `target.kafka.consumer."group.id"`        | Consumer group the offset mapper joins to read the offset mapping topic | Conditional | (none)                                              | Offset mapper |
| `target.kafka.producer`                   | Target producer properties                                              | No          | See [Kafka client defaults](#kafka-client-defaults) | Replicator    |
| `target.kafka.admin`                      | Target admin client properties                                          | No          | `{}`                                                | Both          |
| `target.kafka.topicPropsExclusionList`    | Topic properties to exclude when replicating topic config               | No          | `[]`                                                | Replicator    |
| `target.registry`                         | Target Schema Registry connection                                       | Conditional | (none)                                              | Replicator    |
| `target.registry.config`                  | Schema Registry client properties                                       | Conditional | `{}`                                                | Replicator    |
| `target.registry.headers`                 | HTTP headers added to target Schema Registry requests                   | No          | `{}`                                                | Replicator    |
| `target.registry.cacheSize`               | Maximum schemas cached locally on the target side                       | No          | `31`                                                | Replicator    |

`target.registry` is required when `features.schemaMapping` is enabled.

`target.kafka.consumer."group.id"` is required when deploying the offset mapper: it subscribes to `offsetMapping.topic` as a consumer group member and has no default group. The replicator does not need it.

When `features.exactlyOnce` is enabled, the target producer must use `enable.idempotence: true` and `acks: all` (K2K defaults).

Any standard [Apache Kafka consumer or producer property](https://kafka.apache.org/documentation/#configuration) can be set under the appropriate `target.kafka` block. See [Commonly tuned Kafka client properties](#commonly-tuned-kafka-client-properties).

### `replication`

| Key                            | Description                                               | Required | Default | Used by    |
| ------------------------------ | --------------------------------------------------------- | -------- | ------- | ---------- |
| `replication[].source.topic`   | Topics to replicate (regex string or list of topic names) | Yes      | (none)  | Replicator |
| `replication[].sink.topic`     | Target topic assignment strategy                          | Yes      | (none)  | Replicator |
| `replication[].sink.partition` | Target partition assignment strategy                      | Yes      | (none)  | Replicator |

Both sink keys must be set explicitly — there is no implied default. Use `topic: source` and `partition: source` to preserve the source topic name and partition number.

Sink strategies (`source`, `static`, `prefix`/`suffix`, `staticMapping`, `producer`) are documented in [Topic selection and data routing](https://docs.lenses.io/latest/k2k/configuration-reference/topic-selection-and-data-routing.md).

### `coordination`

| Key                                             | Description                                                        | Required | Default                  | Used by    |
| ----------------------------------------------- | ------------------------------------------------------------------ | -------- | ------------------------ | ---------- |
| `coordination.assignment.topic`                 | Control topic for exclusive assignment on the target Kafka cluster | No       | `__k2k_assignment`       | Replicator |
| `coordination.assignment.graceWindow`           | Wait time before fencing other instances                           | No       | `15 seconds`             | Replicator |
| `coordination.assignment.fencingMaxParallelism` | Max parallelism when fencing producers                             | No       | `5`                      | Replicator |
| `coordination.commit.topic`                     | Control topic for replication progress commits                     | No       | `__k2k_consumer-offsets` | Replicator |
| `coordination.commit.syncTimeout`               | Max wait when catching up on the commit control topic              | No       | `90 seconds`             | Replicator |
| `coordination.commit.batchSize`                 | Records to accumulate before committing progress                   | No       | `30000`                  | Replicator |
| `coordination.commit.batchTimeout`              | Max wait before committing a partial batch                         | No       | `30 seconds`             | Replicator |
| `coordination.commit.parallelizationLimit`      | Concurrent batches awaiting acknowledgement (exactly once)         | No       | `2`                      | Replicator |
| `coordination.commit.pendingAckQueueSize`       | Max commit batches in the acknowledgement queue (at-least-once)    | No       | `5`                      | Replicator |

### `offsetMapping`

| Key                                | Description                                                                | Required | Default                 | Used by       |
| ---------------------------------- | -------------------------------------------------------------------------- | -------- | ----------------------- | ------------- |
| `offsetMapping.topic`              | Topic storing source-to-target offset mappings on the target Kafka cluster | No       | `__k2k_offset_mappings` | Both          |
| `offsetMapping.retention.time`     | How long to retain mapping ranges (`topic`, `infinite`, or a duration)     | No       | `topic`                 | Offset mapper |
| `offsetMapping.retention.count`    | Mapping ranges to retain per topic partition (integer or `infinite`)       | No       | `1000`                  | Offset mapper |
| `offsetMapping.retention.schedule` | How often the retention process runs                                       | No       | `5 minutes`             | Offset mapper |

### `groups`

| Key                        | Description                                                                                    | Required    | Default     | Used by       |
| -------------------------- | ---------------------------------------------------------------------------------------------- | ----------- | ----------- | ------------- |
| `groups.consumerGroups`    | Consumer groups on the source Kafka cluster to translate (regex string or list of group names) | Conditional | `.*`        | Offset mapper |
| `groups.pollInterval`      | How often to poll the source Kafka cluster for consumer group changes                          | No          | `5 seconds` | Offset mapper |
| `groups.offsetGuardPrefix` | Prefix for watermark entries written to consumer groups on the target Kafka cluster            | No          | `k2k_`      | Offset mapper |

`groups.consumerGroups` is required when the offset mapper is deployed. It accepts either a regex string or a YAML list of exact consumer group names — see [Offset mapping](https://docs.lenses.io/latest/k2k/configuration-reference/offset-mapping.md#polling-and-group-selection).

### `schemaMappings`

| Key                               | Description                                   | Required | Default                            | Used by    |
| --------------------------------- | --------------------------------------------- | -------- | ---------------------------------- | ---------- |
| `schemaMappings.topics[].pattern` | Regex selecting topics this rule applies to   | No       | `.*`                               | Replicator |
| `schemaMappings.topics[].key`     | Replicate schemas referenced by record keys   | No       | `enabled`                          | Replicator |
| `schemaMappings.topics[].value`   | Replicate schemas referenced by record values | No       | `enabled`                          | Replicator |
| `schemaMappings.supportedTypes`   | Schema types K2K can replicate                | No       | `AVRO`, `JSON`, `PROTOBUF`, `YANG` | Replicator |

### `topicCreation`

Replicated and control topic settings use the same [common merging](https://docs.lenses.io/latest/k2k/configuration-reference/topic-creation.md#common-merging) pattern: defaults in a `common` block are merged with more specific settings. For replicated topics, the first matching rule in `rules` wins.

#### Replicated topics

Used when `features.autoCreateTopics` is enabled.

| Key                                                        | Description                                                | Required | Default           | Used by    |
| ---------------------------------------------------------- | ---------------------------------------------------------- | -------- | ----------------- | ---------- |
| `topicCreation.replication.common.partitions`              | Default partition count for auto created replicated topics | No       | (cluster default) | Replicator |
| `topicCreation.replication.common.replication`             | Default replication factor                                 | No       | (cluster default) | Replicator |
| `topicCreation.replication.common.config`                  | Default topic configuration                                | No       | `{}`              | Replicator |
| `topicCreation.replication.rules[].pattern`                | Regex matching topic names for this rule                   | No       | (none)            | Replicator |
| `topicCreation.replication.rules[].properties.partitions`  | Partition count for matching topics                        | No       | (none)            | Replicator |
| `topicCreation.replication.rules[].properties.replication` | Replication factor for matching topics                     | No       | (none)            | Replicator |
| `topicCreation.replication.rules[].properties.config`      | Topic configuration for matching topics                    | No       | (none)            | Replicator |

#### Control topics

Used when `features.autoCreateControlTopics` is enabled.

| Key                                               | Description                                         | Required | Default                   | Used by    |
| ------------------------------------------------- | --------------------------------------------------- | -------- | ------------------------- | ---------- |
| `topicCreation.control.common.partitions`         | Default partitions for all control topics           | No       | (cluster default)         | Both       |
| `topicCreation.control.common.replication`        | Default replication factor for all control topics   | No       | (cluster default)         | Both       |
| `topicCreation.control.common.config`             | Default configuration for all control topics        | No       | `{}`                      | Both       |
| `topicCreation.control.assignment.partitions`     | Partitions for the assignment control topic         | No       | (inherits common)         | Replicator |
| `topicCreation.control.assignment.replication`    | Replication factor for the assignment control topic | No       | (inherits common)         | Replicator |
| `topicCreation.control.assignment.config`         | Configuration for the assignment control topic      | No       | `cleanup.policy: compact` | Replicator |
| `topicCreation.control.commit.partitions`         | Partitions for the commit control topic             | No       | (inherits common)         | Replicator |
| `topicCreation.control.commit.replication`        | Replication factor for the commit control topic     | No       | (inherits common)         | Replicator |
| `topicCreation.control.commit.config`             | Configuration for the commit control topic          | No       | `cleanup.policy: compact` | Replicator |
| `topicCreation.control.metrics.partitions`        | Partitions for the metrics control topic            | No       | (inherits common)         | Replicator |
| `topicCreation.control.metrics.replication`       | Replication factor for the metrics control topic    | No       | (inherits common)         | Replicator |
| `topicCreation.control.metrics.config`            | Configuration for the metrics control topic         | No       | `{}`                      | Replicator |
| `topicCreation.control.offsetMapping.partitions`  | Partitions for the offset mapping topic             | No       | (inherits common)         | Both       |
| `topicCreation.control.offsetMapping.replication` | Replication factor for the offset mapping topic     | No       | (inherits common)         | Both       |
| `topicCreation.control.offsetMapping.config`      | Configuration for the offset mapping topic          | No       | `cleanup.policy: delete`  | Both       |

### `configuration`

| Key                                                 | Description                                                                                                               | Required | Default            | Used by    |
| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -------- | ------------------ | ---------- |
| `configuration.pipelineParallelism`                 | Max concurrent streams processing assigned source partitions                                                              | No       | `15`               | Replicator |
| `configuration.maxPrefetchBatches`                  | Record batches the source consumer may prefetch per partition while earlier batches process (minimum effective value `2`) | No       | `3`                | Replicator |
| `configuration.pollInterval`                        | Minimum interval between Kafka consumer poll calls                                                                        | No       | `0 milliseconds`   | Replicator |
| `configuration.controlParallelism`                  | Max concurrent streams processing control messages                                                                        | No       | `10`               | Replicator |
| `configuration.autoControlTopicCreationCheckDelay`  | Wait between retries when verifying a created topic exists                                                                | No       | `750 milliseconds` | Both       |
| `configuration.autoControlTopicCreationMaxAttempts` | Max retries when verifying topic creation                                                                                 | No       | `10`               | Both       |

### `errorHandling`

| Key                                                  | Description                                                                                  | Required | Default  | Used by       |
| ---------------------------------------------------- | -------------------------------------------------------------------------------------------- | -------- | -------- | ------------- |
| `errorHandling.onCommitSyncTimeout`                  | Action when commit sync times out (`fail` or `ignore`)                                       | No       | `fail`   | Replicator    |
| `errorHandling.onControlMessageDeserializationError` | Action when a control message cannot be deserialized (`fail` or `ignore`)                    | No       | `ignore` | Replicator    |
| `errorHandling.onMultipleTargetCommittedOffsets`     | Action when multiple committed offsets exist for the same group on the target Kafka cluster  | No       | `fail`   | Offset mapper |
| `errorHandling.onActiveConsumerGroupCommit`          | Action when committing offsets for a group with active consumers on the target Kafka cluster | No       | `fail`   | Offset mapper |

### `metrics`

Defaults differ per application — the offset mapper ships its own prefix defaults.

| Key                               | Description                                   | Required | Replicator default      | Offset mapper default                 |
| --------------------------------- | --------------------------------------------- | -------- | ----------------------- | ------------------------------------- |
| `metrics.prefix.sourceConsumer`   | Metric prefix for the source consumer         | No       | `k2k.consumer.source.`  | (not used)                            |
| `metrics.prefix.targetConsumer`   | Metric prefix for the target control consumer | No       | `k2k.consumer.control.` | `k2k.offsetmapping.consumer.control.` |
| `metrics.prefix.targetProducer`   | Metric prefix for the target producer         | No       | `k2k.producer.`         | `k2k.offsetmapping.producer.`         |
| `metrics.prefix.assignment`       | Metric prefix for assignment metrics          | No       | `k2k.`                  | (not used)                            |
| `metrics.prefix.offsetMappingApp` | Metric prefix for offset mapper metrics       | No       | (not used)              | `k2k.offsetmapping.`                  |

| Key                   | Description                    | Required | Default         | Used by    |
| --------------------- | ------------------------------ | -------- | --------------- | ---------- |
| `metrics.kafka.topic` | Topic for Kafka metrics export | No       | `__k2k_metrics` | Replicator |

For the metric names these prefixes are applied to, see [Observability](https://docs.lenses.io/latest/k2k/configuration-reference/observability.md#metric-names).

### `tracing`

| Key                         | Description                      | Required | Default           | Used by    |
| --------------------------- | -------------------------------- | -------- | ----------------- | ---------- |
| `tracing.headers.partition` | Header name for source partition | No       | `__k2k_partition` | Replicator |
| `tracing.headers.offset`    | Header name for source offset    | No       | `__k2k_offset`    | Replicator |
| `tracing.headers.topic`     | Header name for source topic     | No       | `__k2k_topic`     | Replicator |
| `tracing.headers.pipeline`  | Header name for pipeline name    | No       | `__k2k_pipeline`  | Replicator |

### `bidirectional`

| Key                                   | Description                                                     | Required | Default               | Used by    |
| ------------------------------------- | --------------------------------------------------------------- | -------- | --------------------- | ---------- |
| `bidirectional.mode`                  | How replicated records are identified (`pipeline` or `cluster`) | No       | `cluster`             | Replicator |
| `bidirectional.headers.bidirectional` | Header name marking replicated records                          | No       | `__k2k_bidirectional` | Replicator |

## Kafka client defaults

K2K applies these defaults when keys are omitted. Properties in `common` are inherited by client-specific blocks; more specific blocks override `common`.

| Client   | Side   | Default                                                                    |
| -------- | ------ | -------------------------------------------------------------------------- |
| Consumer | Source | `"client.id": "k2k-app"`, `"auto.offset.reset": "earliest"`                |
| Consumer | Target | `"auto.offset.reset": "earliest"`, `"isolation.level": "read_committed"`   |
| Producer | Target | `"retries": "2147483647"`, `"enable.idempotence": "true"`, `"acks": "all"` |

See [Kafka connectivity and authentication](https://docs.lenses.io/latest/k2k/configuration-reference/kafka-connectivity-authentication.md) for client inheritance and authentication properties, including AWS MSK IAM and Google Managed Service for Apache Kafka.

## Commonly tuned Kafka client properties

These standard Kafka properties are frequently adjusted for throughput and memory. See [Replication performance tuning](https://docs.lenses.io/latest/k2k/configuration-reference/replication-performance-tuning.md) for how they interact with K2K pipeline settings.

### Source consumer

| Key                                            | Description                                                  |
| ---------------------------------------------- | ------------------------------------------------------------ |
| `source.kafka.consumer."max.poll.records"`     | Maximum records returned per poll                            |
| `source.kafka.consumer."fetch.max.bytes"`      | Maximum bytes fetched per partition per request              |
| `source.kafka.consumer."fetch.min.bytes"`      | Minimum bytes the broker should accumulate before responding |
| `source.kafka.consumer."fetch.max.wait.ms"`    | Maximum time the broker waits to fill `fetch.min.bytes`      |
| `source.kafka.consumer."receive.buffer.bytes"` | TCP receive buffer size                                      |

### Target producer

| Key                                                             | Description                                                     |
| --------------------------------------------------------------- | --------------------------------------------------------------- |
| `target.kafka.producer."batch.size"`                            | Target batch size in bytes before sending                       |
| `target.kafka.producer."linger.ms"`                             | Time to wait for more records before sending a batch            |
| `target.kafka.producer."buffer.memory"`                         | Total memory available to the producer for buffering            |
| `target.kafka.producer."compression.type"`                      | Compression algorithm (`none`, `gzip`, `lz4`, `zstd`, `snappy`) |
| `target.kafka.producer."max.in.flight.requests.per.connection"` | Unacknowledged produce requests per connection                  |


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.lenses.io/latest/k2k/configuration-reference/full-reference.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
