1 of 6

Schema Registries

This page describes adding a Schema Registries to the Lenses Agent.

Overview

This page describes an overview of connecting a Lenses Agent with Schema Registries

Consider Rate Limiting if you have a high number of schemas.

Authentication

TLS and basic authentication are supported for connections to Schema Registries.

JMX Metrics

The Agent can collect Schema registry metrics via:

JMX
Jolokia

Supported formats

AVRO
PROTOBUF

JSON and XML formats are supported by Lenses but without a backing schema registry.

Schema deletion

To enable the deletion of schemas in the UI, set the following in the lenses.conf file.

lenses.conf

## Enable schema deletion in the Lenses UI
## default: false
lenses.schema.registry.delete = true

## When a topic is deleted,
## automatically delete also its associated Schema Registry subjects
## default: false
lenses.schema.registry.cascade.delete = true

IBM Event Streams supports hard deletes only

AWS Glue

This page describes connection to AWS Glue.

AWS Glue Schema Registry connection, depends on an AWS connection.

Set the following examples in provisioning.yaml

These are examples of provision Lenses with an AWS connection named my-aws-connection and an AWS Glue Schema Registry that references it.

Using AWS Access Key

aws:
  - name: my-aws-connection
    tags: ["tag1"]
    version: 1      
    configuration:
      authMode: 
        value: Access Key
      accessKeyId: 
        value: my-access-key-id
      secretAccessKey: 
        value: my-secret-access-key
      region: 
        value: eu-west-1
      
glueSchemaRegistry:
  - name: schema-registry
    tags: ["tag1"]
    version: 1      
    configuration:
      authMode: 
        value: my-aws-connection
      accessKeyId:
        reference: my-aws-connection
      secretAccessKey:
        reference: my-aws-connection
      glueRegistryArn:
          value: arn:aws:glue:region:123123123:registry/my-registry

Using AWS Credentials Chain

aws:
  - name: my-aws-connection
    tags: ["tag1"]
    version: 1      
    configuration:
      authMode: 
        value: Credentials Chain
      region: 
        value: eu-west-1
        
glueSchemaRegistry:
  - name: schema-registry
    tags: ["tag1"]
    version: 1      
    configuration:
      authMode: 
        value: my-aws-connection
      glueRegistryArn:
        value: arn:aws:glue:region:123123123:registry/my-registry

Confluent

This page describes connecting Lenses to Confluent schema registries.

Set the following examples in provisioning.yaml

Simple configuration, with JMX metrics

The URLs (nodes) should always have a scheme defined (http:// or https://).

Basic authentication

For Basic Authentication, define username and password properties.

TLS with custom truststore

A custom truststore is needed when the Schema Registry is served over TLS (encryption-in-transit) and the Registry’s certificate is not signed by a trusted CA.

TLS with client authentication

A custom truststore might be necessary too (see above).

Hard or soft delete

By default, Lenses will use hard delete for Schema Registry. To use soft delete, add the following property:

Apicurio

This page describes connecting Lenses to Apicurio.

Apicuro supports the following versions of Confluent's API:

Confluent Schema Registry API v6
Confluent Schema Registry API v7

Set the following examples in provisioning.yaml

Set the schema registry URLs to include the compatibility endpoints, for example:

http://localhost:8080/apis/ccompat/v6

provisioning.yaml

confluentSchemaRegistry:
  - name: schema-registry
    tags: ["tag1"]
    version: 1      
    configuration:
      schemaRegistryUrls:
        value:
          - http://localhost:8080/apis/ccompat/v6

IBM Event Streams Registry

This page describes connecting Lenses to IBM Event Streams schema registry.

Requires Enterprise subscription on IBM Event Streams and only hard delete is supported for IBM Event streams

To configure an application to use this compatibility API, specify the Schema Registry endpoint in the following format:

Use "token" as the username. Set the password as your API KEY from IBM Event streams

Set the following examples in provisioning.yaml

AWS Glue

This page describes connection to AWS Glue.

AWS Glue Schema Registry connection, depends on an AWS connection.

Set the following examples in provisioning.yaml

These are examples of provision Lenses with an AWS connection named my-aws-connection and an AWS Glue Schema Registry that references it.

Using AWS Access Key

aws:
  - name: my-aws-connection
    tags: ["tag1"]
    version: 1      
    configuration:
      authMode: 
        value: Access Key
      accessKeyId: 
        value: my-access-key-id
      secretAccessKey: 
        value: my-secret-access-key
      region: 
        value: eu-west-1
      
glueSchemaRegistry:
  - name: schema-registry
    tags: ["tag1"]
    version: 1      
    configuration:
      authMode: 
        value: my-aws-connection
      accessKeyId:
        reference: my-aws-connection
      secretAccessKey:
        reference: my-aws-connection
      glueRegistryArn:
          value: arn:aws:glue:region:123123123:registry/my-registry

Using AWS Credentials Chain

aws:
  - name: my-aws-connection
    tags: ["tag1"]
    version: 1      
    configuration:
      authMode: 
        value: Credentials Chain
      region: 
        value: eu-west-1
        
glueSchemaRegistry:
  - name: schema-registry
    tags: ["tag1"]
    version: 1      
    configuration:
      authMode: 
        value: my-aws-connection
      glueRegistryArn:
        value: arn:aws:glue:region:123123123:registry/my-registry

Overview

This page describes an overview of connecting a Lenses Agent with Schema Registries

Consider Rate Limiting if you have a high number of schemas.

Authentication

TLS and basic authentication are supported for connections to Schema Registries.

JMX Metrics

The Agent can collect Schema registry metrics via:

JMX
Jolokia

Supported formats

AVRO
PROTOBUF

JSON and XML formats are supported by Lenses but without a backing schema registry.

Schema deletion

To enable the deletion of schemas in the UI, set the following in the lenses.conf file.

lenses.conf

## Enable schema deletion in the Lenses UI
## default: false
lenses.schema.registry.delete = true

## When a topic is deleted,
## automatically delete also its associated Schema Registry subjects
## default: false
lenses.schema.registry.cascade.delete = true

IBM Event Streams supports hard deletes only

Confluent

This page describes connecting Lenses to Confluent schema registries.

Set the following examples in provisioning.yaml

Simple configuration, with JMX metrics

The URLs (nodes) should always have a scheme defined (http:// or https://).

Basic authentication

For Basic Authentication, define username and password properties.

confluentSchemaRegistry:
- name: schema-registry
  tags: ["tag1"]
  version: 1    
  configuration:
    schemaRegistryUrls:
      value:
        - http://my-sr.host1:8081
        - http://my-sr.host2:8081
    username: 
      value: my-username
    password: 
      value: my-password

TLS with custom truststore

A custom truststore is needed when the Schema Registry is served over TLS (encryption-in-transit) and the Registry’s certificate is not signed by a trusted CA.

confluentSchemaRegistry:
  - name: schema-registry
    tags: ["tag1"]
    version: 1      
    configuration:
      schemaRegistryUrls:
        value:
          - http://my-sr.host1:8081
          - http://my-sr.host2:8081
      sslTruststore:
        fileRef:
          filePath: /path/to/my/truststore.jks
      sslTruststorePassword: 
        value: myPassword

TLS with client authentication

A custom truststore might be necessary too (see above).

confluentSchemaRegistry:
  - name: schema-registry
    tags: ["tag1"]
    version: 1      
    configuration:
      schemaRegistryUrls:
        value:
          - http://my-sr.host1:8081
          - http://my-sr.host2:8081
      sslKeystore:
        fileRef:
          filePath: /path/to/my/keystore.jks
      sslKeystorePassword: 
        value: myPassword

Hard or soft delete

By default, Lenses will use hard delete for Schema Registry. To use soft delete, add the following property:

confluentSchemaRegistry:
  - name: schema-registry
    tags: ["tag1"]
    version: 1      
    configuration:
      schemaRegistryUrls:
        value:
          - http://my-sr.host1:8081
          - http://my-sr.host2:8081
      hardDelete:
        value: true