Overview
This page describe an overview of deploying Lenses against your Kafka clusters.
This guide walks you through manually deploying HQ and an Agent to connect to your Kafka clusters. Lenses acts as a Kafka client, it can connect to any provider exposing a Kafka-compatible API.
For more detailed guides on Helm, Docker, and Linux, see here.
How to connect to your Kafka?
To deploy Lenses against your environments, you need to:
Prerequisites
EULA acceptance
To start HQ and an Agent, you have to accept the Lenses EULA.
For HQ, in the config.yaml set:
Kafka
Any version of Apache Kafka (2.0 or newer) on-premise and in the cloud. Supported providers include:
Confluent Platform & Cloud
AWS MSK & AWS MSK Serverless
Aiven
IBM Event Streams
Azure HDInsight & EventHubs
Schema Registry
Any version of Confluent Schema Registry (5.5.0 or newer), APICurio (2.0 or newer) and AWS Glue.
Postgres
Only needed if you want to use your Postgres. The docker compose will start a local Postgres instance.
HQ and Agents can share the same instance, by either using a separate database or schema for HQ and each agent, depending on your networking needs.
Postgres server running version 9.6 or higher.
Database Role
The recommended configuration is to create a dedicated login role and database for the HQ and each Agent, setting the HQ or Agent role as the database or schema owner. Both the agent and HQ need credentials; create a role for each.
Networking
Web sockets - You may need to adjust your load balancer to allow them. See here.
JMX connectivity - Connectivity to JMX is optional (not required) but recommended for additional/enhanced monitoring of the Kafka Brokers and Connect Workers. Secure JMX connections are supported, including JOLOKIA and Open Metrics (MSK).
For more enable JMX for the Agent itself, see here.
Kafka ACLs
The agent requires access to your Kafka cluster. If ACLs are enabled, you will need to allow the Agent access.
SSO (optional)
If you want to use SSO / SAML for authentication, you will need the metadata.xml file from your provider. See Authentication for more information.
Last updated
Was this helpful?