Schema Replication

Enable and configure the

To ensure that replicated data remains usable, K2K can be configured to automatically replicate schemas from the source to the target Schema Registry.

When this feature is active K2K will, for each record's schema, ensure a corresponding schema exists on the target registry and that the written record correctly references that schema.

Enabling schema replication requires providing connection details (see Connections) for both the source and target registries, in addition to activating the global feature flag.

Property Path	Description	Required to Enable
features.schemaMapping	The main feature flag that activates the automatic schema replication process. It must be set to the string enabled.	Yes

Schema Replication Behavior

Schemas identifiers are not guaranteed to be the same between source and target environments.

To handle this potential mismatch, K2K will automatically mutate the replicated record such that it references the correct identifier. The written record will then correctly reference the schema identifier present in the target environment allowing for its correct de-serialization.

---

Customizing Schema Mapping

By default, when schema mapping is enabled, K2K will assume all replicated topics to have registered schemas for both the value and schema facets. If this is not the case, or when fine grain control over which topics and facets should have their schemas propagated is required, rules can be added to enable or disable the behavior:

Rules are defined as an array of objects. Each object defines a pattern which matches against a topic name. The first rule to match a topic's name is used in order to determine if a schema should be propagated or not for each facet:

Property	Description	Required
schemaMapping.topics[].pattern	A regex used to match against the topic's name.	yes
schemaMapping.topics[].key	Can be either disabled or enabled. Controls if the key facet of the record should have its schema replicated.	yes
schemaMapping.topics[].value	Can be either disabled or enabled. Controls if the value facet of the record should have its schema replicated.	yes

Example

The following example shows the required properties set across the features, source, and target configuration blocks to enable automatic schema replication.

# ---------------------------------------------------
# K2K Configuration for Automatic Schema Replication
# ---------------------------------------------------

# 1. Global feature flag to enable the schema replication process.
features:
  schemaMapping: enabled

# 2. Source configuration must include the source Schema Registry URL.
source:
  registry:
    config:
      "schema.registry.url": "http://prod.schema-registry:8081"

# 3. Target configuration must include the target Schema Registry URL.
target:
  registry:
    config:
      "schema.registry.url": "http://dr.schema-registry:8081"
# 4. Fine grain control of 
schemaMapping:
   topics:
     - pattern: "user-.*" #rule only applies to topics matching this regex
       key: enabled #enables schema mapping of key facets
       value: enabled #enables schema mapping of value facets
     - pattern: "transaction-.*" #rule only applies to topics matching this regex
       key: disabled #disables schema mapping of key facets
       value: enabled #enables schema mapping of value facets