View the latest documentation 5.5
Lenses uses Kafka Connect to manage connectors.
Multiple Kafka Connect clusters might be used - a separate Connection should be created for each. Once a Kafka Connect connection is defined, Lenses displays all its workers and their statuses (with metrics, if configured so) in Workspace->Services screen.
Workspace->Services
Users need explicit permission via their groups to see a Connect cluster. In order to view, create, or manage a connector, they further need permissions to the topics it accesses.
The name of a Kafka Connect Connections may only contain alphanumeric characters ([A-Za-z0-9]) and dashes (-). Valid examples would be dev, Prod1, SQLCluster,Prod-1, SQL-Team-Awesome.
[A-Za-z0-9]
-
dev
Prod1
SQLCluster
Prod-1
SQL-Team-Awesome
To add a connection, go to:
Admin ➡ Connections
Select the New Connection button and select Kafka Connect.
Enter the details.
To edit a connection, go to:
select the Connect connection and in actions select edit.:
Metrics, if configured, are shown in each connector’s page and in the topology view. If some workers are ommited, the metrics will be incomplete, each worker only exports its own metrics.
Like other core services, Kafka Connect is managed via Connections. See more about managing connections here.
Multiple connections allowed: YES
Deletion allowed: YES
When managing connections other than with GUI, such information are relevant for Kafka Connect connection:
Template name: KafkaConnect
The easiest way to manage Connections is via Lenses GUI under their respective pages, however it is also possible to do it directly via API, Helm or Lenses CLI. In such case, some connection type-specific values have to be used. Here are few examples of such configuration in YAML format.
The URLs (workers) should always have a scheme defined (http:// or https://).
This example uses an optional AES-256 key. The key decodes values encoded with AES-256 to enable passing encrypted values to connectors. It is only needed if your cluster uses AES-256 Decryption plugin.
your-cluster-name: tags: ["tag1"] templateName: KafkaConnect configurationObject: workers: - http://my-kc.worker1:8083 - http://my-kc.worker2:8083 aes256Key: PasswordPasswordPasswordPassword # all metrics properties are optional metricsPort: 9581 metricsType: JMX metricsSsl: false
Find more about multiple options of configuring services’ metrics (like secured JMX, Jolokia, etc) under Services Metrics
For Basic Authentication, define username and password properties.
username
password
your-cluster-name: tags: ["tag1"] templateName: KafkaConnect configurationObject: workers: - http://my-kc.worker1:8083 - http://my-kc.worker2:8083 username: my-username password: my-password
A custom truststore is needed when the Kafka Connect workers are served over TLS (encryption-in-transit) and their certificates are not signed by a trusted CA.
your-cluster-name: tags: ["tag1"] templateName: KafkaConnect configurationObject: workers: - https://my-kc.worker1:8083 - https://my-kc.worker2:8083 sslTruststore: fileRef: filePath: /path/to/my/truststore.jks sslTruststorePassword: myPassword
A custom truststore might be necessary too (see above).
your-cluster-name: tags: ["tag1"] templateName: KafkaConnect configurationObject: workers: - https://my-kc.worker1:8083 - https://my-kc.worker2:8083 sslKeystore: fileRef: filePath: /path/to/my/keystore.jks sslKeystorePassword: keystorePassword sslKeyPassword: keyPassword
As above - a custom truststore is needed when the Kafka Connect workers are served over TLS (encryption-in-transit) and their certificates are not signed by a trusted CA.
your-cluster-name: tags: ["tag1"] templateName: KafkaConnect configurationObject: workers: - https://my-kc.worker1:8083 - https://my-kc.worker2:8083 sslTruststore: fileRef: filePath: /path/to/my/truststore.jks sslTruststorePassword: myPassword username: my-username password: my-password
On this page