All pages
Powered by GitBook
1 of 4

Loading...

Loading...

Loading...

Loading...

Secret Providers

This page describes the available Apache 2.0 Connect Secret Providers from Lenses.

You are not limited to Lenses Secret Providers, you are free to use your own.

AWS Secret Manager

Secure Kafka Connect secrets with AWS Secret Manager.

Azure KeyVault

Secure Kafka Connect secrets with Azure KeyVault.

Environment Variables

Secure Kafka Connect secrets with Environment Variable.

Hashicorp Vault

Secure Kafka Connect secrets with Hashicorp Vault.

AES256

Secure Kafka Connect secrets with AES256 encryption.

Sinks

This page describes the available Apache 2.0 Sink Connectors from Lenses. Lenses can also work with any other Kafka Connect Connector.

Lenses supports any Connector implementing the Connect APIs, bring your own or use community connectors.

You need to add the connector information for them to be visible in the Topology.

Enterprise support is also offered for connectors in the Stream Reactor project, managed and maintained by the Lenses team.

AWS S3

Sink data from Kafka to AWS S3 including backing up topics and offsets.

Azure CosmosDB

Sink data from Kafka to Azure CosmosDB.

Azure Data Lake Gen2

Sink data from Kafka to Azure Data Lake Gen2 including backing up topics and offsets.

Azure Event Hubs

Load data from Azure Event Hubs into Kafka topics.

Azure Service Bus

Sink data from Kafka to Azure Service Bus topics and queues.

Cassandra

Sink data from Kafka to Cassandra.

Elasticsearch

Sink data from Kafka to Elasticsearch.

GCP PubSub

Sink data from Kafka to GCP PubSub.

GCP Storage

Sink data from Kafka to GCP Storage.

HTTP Sink

Sink data from Kafka to a HTTP endpoint.

InfluxDB

Sink data from Kafka to InfluxDB.

JMS

Sink data from Kafka to JMS.

MongoDB

Sink data from Kafka to MongoDB.

MQTT

Sink data from Kafka to MQTT.

Redis

Sink data from Kafka to Redis.

Connectors

This page describes how Lenses integrates with Kafka Connect to create, manage, and monitor connectors via multiple connect clusters.

For documentation about the available Lenses Apache 2.0 Connectors, see the Stream Reactor documentation.

For automation use the CLI.

To connect your Connect Clusters see provisioning.

Lenses connects to Connect Clusters via Connects APIs. You can deploy connectors outside of Lenses and Lenses will still be able to see and manage them

You can connect Lenses to one or more Kafka Connect clusters. Once connected, Lenses will list the available Connector plugins that are installed in each Cluster. Additionally, Connectors can automatically be and alert notifications sent.

Listing Connectors

To list the currently deployed connectors go to Workspace->Connectors. Lenses will display a list of connectors and their status.

View Connector details

Once a connector has been created, selecting the connector allows us to:

  1. View its configuration

  2. Update its configurations (Action)

  3. View individual task configurations

  4. View metrics

View a Connector as Code

To view the YAML specification as Code, select the Code tab in the Connector details page.

Download a Connector as Code

To download the YAML specification, click the Download button.

Creating a Connector

To create a new connector go to Workspace->Connectors->New Connectors.

Select the Connect Cluster you want to use and Lenses will display the plugins installed in the Connect Cluster.

Connectors are searchable by:

  1. Type

  2. Author

After selecting a connector, enter the configuration of the connector instance. Lenses will show the documentation for the currently selected option.

To deploy and start the connector, click Create.

Create a Connector as Code

Creation of a Connector as code can be done via either

  1. Selecting Configure Connector->Configure As Code from the main connector page, or

  2. Selecting a Connect Cluster and Connector, then the Code tab

Both options allow for direct input of a Connectors YAML specification or uploading of an existing file.

Managing a Connector's lifecycle

Connectors can be stopped, restarted, and deleted via the Actions button.

View exceptions.
restarted

Sources

This page describes the available Apache 2.0 Source Connectors from Lenses. Lenses can also work with any other Kafka Connect Connector.

Lenses supports any Connector implementing the Connect APIs, bring your own or use community connectors.

You need to add the connector information for them to be visible in the Topology.

Enterprise support is also offered for connectors in the Stream Reactor project, managed and maintained by the Lenses team.

AWS S3

Load data from AWS S3 including restoring topics.

Azure Data Lake Gen2

Load data from Azure Data Lake Gen2 including restoring topics.

Azure Event Hubs

Load data from Azure Event Hubs into Kafka topics.

Azure Service Bus

Load data from Azure Service Bus into Kafka topics.

Cassandra

Load data from Cassandra into Kafka topics.

GCP PubSub

Load data from GCP PubSub into Kafka topics.

GCP Storage

Load data from GCP Storage including restoring topics.

FTP

Load data from files on FTP servers into Kafka topics.

JMS

Load data from JMS topics and queues into Kafka topics.

MQTT

Load data from MQTT into Kafka topics.