4.0

You are viewing documentation for an older version of Lenses.io View latest documentation here

Schema Registry

Working with records serialized in AVRO format requires a Schema Registry. Lenses supports Confluent’s Schema Registry and any other registry that implements its API.

The most simple configuration includes the Schema Registry hosts, and whether schema deletion should be allowed.

lenses.schema.registry.urls = [
  { url: "http://host1:8081" },
  { url: "http://host2:8081" }
]

# Enable schema deletion - default : false
lenses.schema.registry.delete = true
# Kafka topic that stores the state of the Schema Registry
lenses.schema.registry.topics = "_schemas"

The scheme (http:// or https://) should always be included in the registry URLs.

More advanced setups, that include authentication and access to JMX metrics are possible.

Basic Authentication requires a user and password.

lenses.schema.registry.auth = "USER_INFO"
lenses.schema.registry.username = "USERNAME"
lenses.schema.registry.password = "PASSWORD"

lenses.kafka.settings.client.basic.auth.credentials.source = USER_INFO
lenses.kafka.settings.client.basic.auth.user.info = "USERNAME:PASSWORD"

TLS support for Schema Registry is limited in Lenses 4.0 and mostly restricted to the application —SQL Processors do not support it. Please upgrade to Lenses 4.1 or later to get full support.

When the Registry is served over TLS (encryption-in-transit) a truststore is needed if the Registry’s certificate is not signed by a trusted CA. This setting should be applied to the global truststore and Kafka client settings.

The following environment variable should be exported to set the global truststore.

LENSES_OPTS="-Djavax.net.ssl.trustStore=/path/to/truststore.jks -Djavax.net.ssl.trustStorePassword=[PASSWORD]"

The configuration for the Kafka client happens within lenses.conf.

lenses.kafka.settings.client.schema.registry.ssl.truststore.location = "/path/to/truststore.jks"
lenses.kafka.settings.client.schema.registry.ssl.truststore.password = "[PASSWORD]"

Support for TLS Authentication is even more limited, restricted to record serialization within Lenses (like the Explore and SQL Studio screens), but absent from the Schema Registry screen.

The configuration for the Kafka client happens within lenses.conf.

lenses.kafka.settings.client.schema.registry.ssl.keystore.location = "/path/to/keystore.jks"
lenses.kafka.settings.client.schema.registry.ssl.keystore.password = "[PASSWORD]"
lenses.kafka.settings.client.schema.registry.ssl.key.password      = "[KEY PASSWORD]"

Lenses can read Schema Registry metrics via JMX or the Jolokia JavaAgent. A subset of the available metrics is shown in the Services screen.

The configuration is part of the registry URLs. Any settings marked as optional can be ommited.

lenses.schema.registry.urls = [
  {
    url: "http://SR_HOST_1:8081",
    metrics: {       # Optional section
       ssl: false,   # Optional, please make sure the remote JMX/HTTP
                     # certificate is accepted by the Lenses truststore
       user: "",     # Optional, the remote JMX/HTTP user
       password: "", # Optional, the remote JMX/HTTP password
       type: "JMX",  # One of 'JMX', 'JOLOKIAP' (POST), 'JOLOKIAG' (GET)
       url: "SR_HOST_1:9583"
    }
  },
  {
    url: "http://SR_HOST_2:8081",
     metrics: {      # Optional section
       ssl: false,   # Optional, please make sure the remote JMX/HTTP
                     # certificate is accepted by the Lenses truststore
       user: "",     # Optional, the remote JMX/HTTP user
       password: "", # Optional, the remote JMX/HTTP password
       type: "JMX",  # One of 'JMX', 'JOLOKIAP' (POST), 'JOLOKIAG' (GET)
       url: "SR_HOST_2:9583"
     }
  }
]

Aiven offers a Schema Registry as an optional service on top of the Kafka service. It uses Basic HTTP Authentication.

Get the address and credentials from Aiven Console.

lenses.schema.registry.urls = [{url:"https://[CLUSTER-NAME]-[PROJECT-NAME].aivencloud.com:[PORT]"}]

lenses.schema.registry.auth = "USER_INFO"
lenses.schema.registry.username = "[USERNAME]"
lenses.schema.registry.password = "[PASSWORD]"

lenses.kafka.settings.client.basic.auth.credentials.source = "USER_INFO"
lenses.kafka.settings.client.basic.auth.user.info = "[USERNAME]:[PASSWORD]"

Aiven’s TLS is signed by a known CA, and does not require extra settings.


See configuration settings.