> For the complete documentation index, see [llms.txt](https://docs.lenses.io/latest/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.lenses.io/latest/k2k/configuration-reference.md).

# Configuration Reference

Configuration reference for K2K pipeline settings.

These guides are task-oriented. Each one explains how to configure a specific capability, shows a working YAML example, and lists every related key in one place.

For file loading and minimal pipeline examples, see [Minimal Configuration](/latest/k2k/minimal-configuration.md). For what each capability achieves, see [Features](/latest/k2k/features.md). For how K2K is structured, see [Architecture](/latest/k2k/architecture.md).

Each guide example builds on the [minimal pipeline](/latest/k2k/minimal-configuration.md#minimal-pipeline), adding only the keys described on that page.

Guides are ordered from foundational setup through advanced tuning. Several guides include an **Advanced** section at the end for settings that address environment-specific quirks. K2K defaults should cover most use cases.

## Setup

| Guide                                                                                                                                 | Description                                               |
| ------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------- |
| [License](/latest/k2k/configuration-reference/licenses.md)                                                                            | License token and EULA acceptance                         |
| [Kafka connectivity and authentication](/latest/k2k/configuration-reference/kafka-connectivity-authentication.md)                     | Source Kafka cluster and target Kafka cluster connections |
| [Schema Registry connectivity and authentication](/latest/k2k/configuration-reference/schema-registry-connectivity-authentication.md) | Source and target Schema Registry connections (optional)  |
| [Consumer group](/latest/k2k/configuration-reference/consumer-group.md)                                                               | Source consumer group for load distribution               |
| [Secrets and templates](/latest/k2k/configuration-reference/secrets-and-templates.md)                                                 | Inject secrets and environment values at runtime          |
| [Security and permissions](/latest/k2k/configuration-reference/security-and-permissions.md)                                           | Network paths, ACLs, and control topic access             |

## Replication

| Guide                                                                                                       | Description                                                                   |
| ----------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- |
| [Topic selection and data routing](/latest/k2k/configuration-reference/topic-selection-and-data-routing.md) | Which topics to replicate and how records are routed                          |
| [Header replication](/latest/k2k/configuration-reference/header-replication.md)                             | Copy record headers from the source Kafka cluster to the target Kafka cluster |
| [Record creation timestamp](/latest/k2k/configuration-reference/record-creation-timestamp.md)               | Preserve original record timestamps                                           |
| [Schema migration](/latest/k2k/configuration-reference/schema-migration.md)                                 | Replicate schemas to the target Schema Registry                               |
| [Topic creation](/latest/k2k/configuration-reference/topic-creation.md)                                     | Auto provision topics on the target Kafka cluster                             |

## Delivery and failover

| Guide                                                                                         | Description                                                    |
| --------------------------------------------------------------------------------------------- | -------------------------------------------------------------- |
| [Offset mapping](/latest/k2k/configuration-reference/offset-mapping.md)                       | Translate consumer group positions to the target Kafka cluster |
| [Exactly once delivery](/latest/k2k/configuration-reference/exactly-once-delivery.md)         | Guarantee each record appears once on the target Kafka cluster |
| [Commit tuning](/latest/k2k/configuration-reference/commit-tuning.md)                         | Commit batching for exactly once replication                   |
| [Bidirectional replication](/latest/k2k/configuration-reference/bidirectional-replication.md) | Replicate in both directions without loops                     |
| [Record tracing](/latest/k2k/configuration-reference/record-tracing.md)                       | Add source lineage headers to replicated records               |

## Operations

| Guide                                                                                                   | Description                                                              |
| ------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ |
| [Horizontal scaling](/latest/k2k/configuration-reference/horizontal-scaling.md)                         | Run multiple replicator or offset mapper instances for higher throughput |
| [Observability](/latest/k2k/configuration-reference/observability.md)                                   | Metrics prefixes and exporter configuration                              |
| [Startup validation](/latest/k2k/configuration-reference/startup-validation.md)                         | Probe connectivity and configuration before replication starts           |
| [Troubleshooting](/latest/k2k/configuration-reference/troubleshooting.md)                               | Diagnose runtime replication issues                                      |
| [Pipeline tuning](/latest/k2k/configuration-reference/pipeline-tuning.md)                               | Error handling and topic creation retries                                |
| [Replication performance tuning](/latest/k2k/configuration-reference/replication-performance-tuning.md) | Consumer, producer, and commit settings for throughput and memory        |
| [Full reference](/latest/k2k/configuration-reference/full-reference.md)                                 | Every pipeline key and CLI option in one place                           |

For a single-page lookup of all keys, see [Full reference](/latest/k2k/configuration-reference/full-reference.md).

## Conventions

All guides use the same table columns:

| Column      | Meaning                                                      |
| ----------- | ------------------------------------------------------------ |
| Key         | YAML path using dot notation                                 |
| Description | What the setting controls                                    |
| Required    | Yes, No, or Conditional (required when a feature is enabled) |
| Default     | Value applied when the key is omitted                        |
| Used by     | Replicator, Offset mapper, or Both                           |

**Value types**

| Type   | Format                                                         |
| ------ | -------------------------------------------------------------- |
| Toggle | `enabled` or `disabled`                                        |
| Time   | Duration string (for example `15 seconds`, `750 milliseconds`) |
| Regex  | Java regular expression                                        |

**Conditional keys**

Some keys only apply when a feature toggle is enabled. The guide notes the condition in the Description or in the introduction.

**Advanced sections**

Guides that support optional tuning include an Advanced section at the end. K2K defaults should cover most use cases; use Advanced when you need to address a quirk in your environment.

## Pipeline name

| Key    | Description                         | Required | Default | Used by |
| ------ | ----------------------------------- | -------- | ------- | ------- |
| `name` | Unique identifier for this pipeline | Yes      | (none)  | Both    |


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.lenses.io/latest/k2k/configuration-reference.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.