> For the complete documentation index, see [llms.txt](https://docs.lenses.io/latest/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.lenses.io/latest/k2k/minimal-configuration.md).

# Minimal Configuration

YAML pipeline files and minimal examples.

K2K is configured through YAML pipeline files. A K2K application runs a single pipeline definition: which clusters to connect to, what to replicate, and which features to enable. For how applications, Kafka clients, and the pipeline file relate, see [Architecture](/latest/k2k/architecture.md).

That definition can be split across multiple files. At startup, K2K merges them into one configuration. Later files override earlier ones for the same keys.

## Loading pipeline files

Start the replicator by passing one or more pipeline files:

```shell
k2k start -f base_pipeline.yml -f secrets_overrides.yml
```

When [offset mapping](/latest/k2k/configuration-reference/offset-mapping.md) is enabled, you must also start the offset mapper with the same pipeline files:

```shell
k2k-offset-mapping start -f base_pipeline.yml -f secrets_overrides.yml
```

`base_pipeline.yml` holds the pipeline definition: cluster connections, replication rules, and feature toggles. This file can be committed to source control and shared across environments.

`secrets_overrides.yml` holds sensitive values (passwords, SASL credentials, license tokens) that override matching keys in the base file. Keep this file out of source control and inject it at deploy time.

At startup, K2K merges the files into a single configuration. Values in `secrets_overrides.yml` take precedence over `base_pipeline.yml`.

Use `k2k start --help` and `k2k-offset-mapping start --help` for the full list of CLI options, including telemetry and template engine settings.

## Minimal pipeline

Every pipeline requires a name, source and target Kafka cluster connections, and at least one replication stage:

```yaml
name: "my_pipeline"
source:
  kafka:
    common:
      "bootstrap.servers": "source:9092"
    consumer:
      "group.id": "my-k2k-group"
target:
  kafka:
    common:
      "bootstrap.servers": "target:9092"
replication:
  - source:
      topic: "orders.*"
  - sink:
      topic: source
      partition: source
```

The source stage declares which topics to read. The sink stage declares how records land on the target Kafka cluster. See [Glossary](/latest/k2k/glossary.md) for more on replication stages.

For cluster connections, consumer group, and routing options, see [Kafka connectivity and authentication](/latest/k2k/configuration-reference/kafka-connectivity-authentication.md), [Consumer group](/latest/k2k/configuration-reference/consumer-group.md), and [Topic selection and data routing](/latest/k2k/configuration-reference/topic-selection-and-data-routing.md). If you use Schema Registry, see [Schema Registry connectivity and authentication](/latest/k2k/configuration-reference/schema-registry-connectivity-authentication.md).

For which K2K application to deploy, see [Deployment](/latest/k2k/deployment.md).

## Minimal pipeline (with offset mapping)

Offset mapping requires two running processes: the replicator records source-to-target offset mappings, and the offset mapper translates consumer group positions on the target Kafka cluster. Enable the feature in `features` and declare which consumer groups to translate under `groups`:

```yaml
name: "my_pipeline"
features:
  offsetMapping: enabled
source:
  kafka:
    common:
      "bootstrap.servers": "source:9092"
    consumer:
      "group.id": "my-k2k-group"
target:
  kafka:
    common:
      "bootstrap.servers": "target:9092"
    consumer:
      "group.id": "my-k2k-offset-mapper"
groups:
  consumerGroups: ".*"
replication:
  - source:
      topic: "orders.*"
  - sink:
      topic: source
      partition: source
```

Offset mapping needs one extra connection key beyond the minimal pipeline: `target.kafka.consumer."group.id"`. The offset mapper joins that group on the target Kafka cluster to read the mapping topic, and it has no default — omitting it fails the offset mapper at startup.

The same pipeline file is used to start both the K2K replicator and the K2K offset mapper. Each application reads the sections it needs and ignores the rest. See [Deployment](/latest/k2k/deployment.md) for when to deploy each application.

```shell
k2k start -f my_pipeline.yml
k2k-offset-mapping start -f my_pipeline.yml
```

{% hint style="warning" %}
When `features.offsetMapping` is `enabled`, deploy and run both `k2k` and `k2k-offset-mapping`. Enabling the feature in the pipeline file does not translate consumer offsets by itself.
{% endhint %}

See [Offset mapping](/latest/k2k/configuration-reference/offset-mapping.md) for all related keys.

## Configuration areas

| Area             | Purpose                                                                                      |
| ---------------- | -------------------------------------------------------------------------------------------- |
| `name`           | Unique pipeline identifier                                                                   |
| `source`         | Connection to the source Kafka cluster and Schema Registry                                   |
| `target`         | Connection to the target Kafka cluster and Schema Registry                                   |
| `replication`    | Source and sink stages that define what to copy and where records land                       |
| `features`       | Feature toggles (exactly once, offset mapping, schema migration, topic creation, and others) |
| `coordination`   | Control topic names and commit batching settings                                             |
| `schemaMappings` | Schema replication rules when schema migration is enabled                                    |
| `offsetMapping`  | Consumer group offset mapping settings                                                       |
| `groups`         | Which consumer groups the offset mapper translates                                           |
| `topicCreation`  | Partition count, replication factor, and topic config for auto created topics                |
| `configuration`  | Pipeline parallelism and other runtime tuning                                                |
| `errorHandling`  | Behaviour when errors occur during replication                                               |
| `metrics`        | Metric name prefixes and optional Kafka metrics topic                                        |
| `tracing`        | Tracing header names when record tracing is enabled                                          |
| `bidirectional`  | Mode and header name for bidirectional replication                                           |
| `license`        | EULA acceptance and license token                                                            |

Most settings are optional. K2K applies sensible defaults for anything not specified in the pipeline file.

For a complete list of every key and CLI option, see [Full reference](/latest/k2k/configuration-reference/full-reference.md).

## Configuration Reference guides

For task-oriented configuration with examples and key tables, see [Configuration Reference](/latest/k2k/configuration-reference.md).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.lenses.io/latest/k2k/minimal-configuration.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.