> For the complete documentation index, see [llms.txt](https://docs.lenses.io/latest/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.lenses.io/latest/k2k/configuration-reference/topic-creation.md).

# Topic creation

Auto provision topics on the target cluster.

{% hint style="info" %}
This feature is **disabled by default**. Set the corresponding `features` toggle to `enabled` in your pipeline file for the behavior described on this page to take effect.
{% endhint %}

K2K cannot produce to a topic that does not exist on the target Kafka cluster, and internal features (exactly once, offset mapping, and metrics) need control topics as well. Manually provisioning every topic before startup does not scale, therefore auto creation lets K2K create what it needs with the right partition count and configuration.

When `partitions` or `replication` are omitted, K2K uses the target Kafka cluster defaults. Replicated topics and control topics both use the same [common merging](#common-merging) pattern: defaults in a `common` block are combined with more specific settings. Use `topicPropsExclusionList` when the target Kafka cluster does not support certain source properties. Control topic names are set under `coordination` ([Exactly once delivery](https://docs.lenses.io/latest/k2k/configuration-reference/exactly-once-delivery.md)) or `offsetMapping.topic` ([Offset mapping](https://docs.lenses.io/latest/k2k/configuration-reference/offset-mapping.md)). For retry behaviour, see [Pipeline tuning](https://docs.lenses.io/latest/k2k/configuration-reference/pipeline-tuning.md). For provisioning settings, see the [Advanced](#advanced) section below.

## Prerequisites

* [License accepted](https://docs.lenses.io/latest/k2k/configuration-reference/licenses.md)
* [Kafka connectivity and authentication](https://docs.lenses.io/latest/k2k/configuration-reference/kafka-connectivity-authentication.md)
* Target admin client permissions to create topics

## Example

Use the following pipeline configuration:

```yaml
name: "my_pipeline"
features:
  autoCreateTopics: enabled
  autoCreateControlTopics: enabled
source:
  kafka:
    common:
      "bootstrap.servers": "source:9092"
    consumer:
      "group.id": "my-k2k-group"
target:
  kafka:
    common:
      "bootstrap.servers": "target:9092"
    topicPropsExclusionList:
      - "min.insync.replicas"
topicCreation:
  replication:
    common:
      partitions: 6
      replication: 3
    rules:
      - pattern: "orders.*"
        properties:
          partitions: 12
replication:
  - source:
      topic: "orders.*"
  - sink:
      topic: source
      partition: source
```

## Configuration keys

### Feature toggles

| Key                                          | Description                                                                                      | Required | Default    | Used by    |
| -------------------------------------------- | ------------------------------------------------------------------------------------------------ | -------- | ---------- | ---------- |
| `features.autoCreateTopics`                  | Auto create replicated topics on the target Kafka cluster                                        | No       | `disabled` | Replicator |
| `features.autoCreateControlTopics`           | Auto create internal control topics on the target Kafka cluster                                  | No       | `disabled` | Both       |
| `features.validateControlTopicSettings`      | Validate control topic configuration on startup                                                  | No       | `enabled`  | Replicator |
| `features.filterNonUpdatableTopicProperties` | Filter topic properties that cannot be set on the target Kafka cluster when auto creating topics | No       | `enabled`  | Replicator |

### Target topic properties

| Key                              | Description                                               | Required | Default | Used by    |
| -------------------------------- | --------------------------------------------------------- | -------- | ------- | ---------- |
| `target.topicPropsExclusionList` | Topic properties to exclude when replicating topic config | No       | `[]`    | Replicator |

### Replicated topics

Used when `features.autoCreateTopics` is enabled. The final configuration [merges](#common-merging) `topicCreation.replication.common` with the first matching rule in `topicCreation.replication.rules`.

| Key                                                        | Description                                                | Required | Default           | Used by    |
| ---------------------------------------------------------- | ---------------------------------------------------------- | -------- | ----------------- | ---------- |
| `topicCreation.replication.common.partitions`              | Default partition count for auto created replicated topics | No       | (cluster default) | Replicator |
| `topicCreation.replication.common.replication`             | Default replication factor                                 | No       | (cluster default) | Replicator |
| `topicCreation.replication.common.config`                  | Default topic configuration                                | No       | `{}`              | Replicator |
| `topicCreation.replication.rules[].pattern`                | Regex matching topic names for this rule                   | No       | (none)            | Replicator |
| `topicCreation.replication.rules[].properties.partitions`  | Partition count for matching topics                        | No       | (none)            | Replicator |
| `topicCreation.replication.rules[].properties.replication` | Replication factor for matching topics                     | No       | (none)            | Replicator |
| `topicCreation.replication.rules[].properties.config`      | Topic configuration for matching topics                    | No       | (none)            | Replicator |

## Advanced

### Common merging

Replicated topics and control topics share the same configuration pattern:

1. Set defaults in a `common` block (`topicCreation.replication.common` or `topicCreation.control.common`).
2. Override per topic with more specific settings.

For replicated topics, K2K merges `common` with the **first** rule in `topicCreation.replication.rules` whose `pattern` matches the topic name. Rule properties override `common` for `partitions`, `replication`, and `config`.

For control topics, K2K merges `control.common` with the topic-specific block (for example `topicCreation.control.commit`). Topic-specific values override `common`.

### Control topics

Used when `features.autoCreateControlTopics` is enabled. The final configuration uses the [common merging](#common-merging) pattern described above.

| Key                                               | Description                                         | Required | Default                   | Used by    |
| ------------------------------------------------- | --------------------------------------------------- | -------- | ------------------------- | ---------- |
| `topicCreation.control.common.partitions`         | Default partitions for all control topics           | No       | (cluster default)         | Both       |
| `topicCreation.control.common.replication`        | Default replication factor for all control topics   | No       | (cluster default)         | Both       |
| `topicCreation.control.common.config`             | Default configuration for all control topics        | No       | `{}`                      | Both       |
| `topicCreation.control.assignment.partitions`     | Partitions for the assignment control topic         | No       | (inherits common)         | Replicator |
| `topicCreation.control.assignment.replication`    | Replication factor for the assignment control topic | No       | (inherits common)         | Replicator |
| `topicCreation.control.assignment.config`         | Configuration for the assignment control topic      | No       | `cleanup.policy: compact` | Replicator |
| `topicCreation.control.commit.partitions`         | Partitions for the commit control topic             | No       | (inherits common)         | Replicator |
| `topicCreation.control.commit.replication`        | Replication factor for the commit control topic     | No       | (inherits common)         | Replicator |
| `topicCreation.control.commit.config`             | Configuration for the commit control topic          | No       | `cleanup.policy: compact` | Replicator |
| `topicCreation.control.metrics.partitions`        | Partitions for the metrics control topic            | No       | (inherits common)         | Replicator |
| `topicCreation.control.metrics.replication`       | Replication factor for the metrics control topic    | No       | (inherits common)         | Replicator |
| `topicCreation.control.metrics.config`            | Configuration for the metrics control topic         | No       | `{}`                      | Replicator |
| `topicCreation.control.offsetMapping.partitions`  | Partitions for the offset mapping topic             | No       | (inherits common)         | Both       |
| `topicCreation.control.offsetMapping.replication` | Replication factor for the offset mapping topic     | No       | (inherits common)         | Both       |
| `topicCreation.control.offsetMapping.config`      | Configuration for the offset mapping topic          | No       | `cleanup.policy: delete`  | Both       |


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.lenses.io/latest/k2k/configuration-reference/topic-creation.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
