> For the complete documentation index, see [llms.txt](https://docs.lenses.io/latest/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.lenses.io/latest/k2k/configuration-reference/consumer-group.md).

# Consumer group

Source consumer group for load distribution.

Kafka assigns each partition to one member of a consumer group. To read from multiple partitions in parallel and scale the replicator horizontally, K2K must join a group on the source Kafka cluster like any other consumer.

Set `source.kafka.consumer."group.id"` in the pipeline. All replicator instances running the same pipeline must share this value—the [K2K group](https://docs.lenses.io/latest/k2k/glossary.md#consumer-groups). Unlike application consumers, K2K does not commit offsets on the source Kafka cluster; it records replication progress in control topics on the target Kafka cluster instead. Customize the commit control topic in the [Advanced](#advanced) section below, or see [Horizontal scaling](https://docs.lenses.io/latest/k2k/configuration-reference/horizontal-scaling.md) to add instances.

## Prerequisites

* [License accepted](https://docs.lenses.io/latest/k2k/configuration-reference/licenses.md)
* [Kafka connectivity and authentication](https://docs.lenses.io/latest/k2k/configuration-reference/kafka-connectivity-authentication.md)

## Example

Use this pipeline configuration:

```yaml
name: "my_pipeline"
source:
  kafka:
    common:
      "bootstrap.servers": "source:9092"
    consumer:
      "group.id": "my-k2k-group"
target:
  kafka:
    common:
      "bootstrap.servers": "target:9092"
replication:
  - source:
      topic: "orders.*"
  - sink:
      topic: source
      partition: source
```

## Configuration keys

| Key                                | Description                                    | Required | Default | Used by    |
| ---------------------------------- | ---------------------------------------------- | -------- | ------- | ---------- |
| `source.kafka.consumer."group.id"` | K2K consumer group on the source Kafka cluster | Yes      | (none)  | Replicator |

All replicator instances running the same pipeline must use the same `group.id`. This is the [K2K group](https://docs.lenses.io/latest/k2k/glossary.md#consumer-groups).

## Advanced

### Control topics

K2K records how far replication has progressed in a commit control topic on the target Kafka cluster. Each replicator instance reads this topic to recover its position after a restart.

| Key                         | Description                                    | Required | Default                  | Used by    |
| --------------------------- | ---------------------------------------------- | -------- | ------------------------ | ---------- |
| `coordination.commit.topic` | Control topic for replication progress commits | No       | `__k2k_consumer-offsets` | Replicator |

For commit batching behaviour, see [Commit tuning](https://docs.lenses.io/latest/k2k/configuration-reference/commit-tuning.md). For automatically creating this topic with custom settings, see [Topic creation](https://docs.lenses.io/latest/k2k/configuration-reference/topic-creation.md).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.lenses.io/latest/k2k/configuration-reference/consumer-group.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
