This page describes how Lenses integrates with Kafka Connect to create, manage, and monitor connectors via multiple connect clusters.
For documentation about the available Lenses Apache 2.0 Connectors, see the Stream Reactor documentation.
For automation use the CLI
.
To connect your Connect Clusters see provisioning.
Lenses connects to Connect Clusters via Connects APIs. You can deploy connectors outside of Lenses and Lenses will still be able to see and manage them
You can connect Lenses to one or more Kafka Connect clusters. Once connected, Lenses will list the available Connector plugins that are installed in each Cluster. Additionally, Connectors can automatically be restarted and alert notifications sent.
To list the currently deployed connectors go to Workspace->Connectors. Lenses will display a list of connectors and their status.
Once a connector has been created, selecting the connector allows us to:
View its configuration
Update its configurations (Action)
View individual task configurations
View metrics
View exceptions.
To view the YAML specification as Code, select the Code tab in the Connector details page.
To download the YAML specification, click the Download button.
To create a new connector go to Workspace->Connectors->New Connectors.
Select the Connect Cluster you want to use and Lenses will display the plugins installed in the Connect Cluster.
Connectors are searchable by:
Type
Author
After selecting a connector, enter the configuration of the connector instance. Lenses will show the documentation for the currently selected option.
To deploy and start the connector, click Create.
Creation of a Connector as code can be done via either
Selecting Configure Connector->Configure As Code from the main connector page, or
Selecting a Connect Cluster and Connector, then the Code tab
Both options allow for direct input of a Connectors YAML specification or uploading of an existing file.
Connectors can be stopped, restarted, and deleted via the Actions button.
This page describes the available Apache 2.0 Source Connectors from Lenses. Lenses can also work with any other Kafka Connect Connector.
Lenses supports any Connector implementing the Connect APIs, bring your own or use community connectors.
You need to add the connector information for them to be visible in the Topology.
Enterprise support is also offered for connectors in the Stream Reactor project, managed and maintained by the Lenses team.
This page describes the available Apache 2.0 Sink Connectors from Lenses. Lenses can also work with any other Kafka Connect Connector.
Lenses supports any Connector implementing the Connect APIs, bring your own or use community connectors.
Enterprise support is also offered for connectors in the Stream Reactor project, managed and maintained by the Lenses team.
AWS S3
Load data from AWS S3 including restoring topics.
Azure Data Lake Gen2
Load data from Azure Data Lake Gen2 including restoring topics.
Azure Event Hubs
Load data from Azure Event Hubs into Kafka topics.
Azure Service Bus
Load data from Azure Service Bus into Kafka topics.
Cassandra
Load data from Cassandra into Kafka topics.
GCP PubSub
Load data from GCP PubSub into Kafka topics.
GCP Storage
Load data from GCP Storage including restoring topics.
FTP
Load data from files on FTP servers into Kafka topics.
JMS
Load data from JMS topics and queues into Kafka topics.
MQTT
Load data from MQTT into Kafka topics.
AWS Secret Manager
Secure Kafka Connect secrets with AWS Secret Manager.
Azure KeyVault
Secure Kafka Connect secrets with Azure KeyVault.
Environment Variables
Secure Kafka Connect secrets with Environment Variable.
Hashicorp Vault
Secure Kafka Connect secrets with Hashicorp Vault.
AES256
Secure Kafka Connect secrets with AES256 encryption.