Install
This page describes configuring and starting Lenses HQ and Agent against your Kafka cluster.
Last updated
This page describes configuring and starting Lenses HQ and Agent against your Kafka cluster.
Last updated
2024 © Lenses.io Ltd. Apache, Apache Kafka, Kafka and associated open source project names are trademarks of the Apache Software Foundation.
This guide is using the Lenses docker compose file. For non dev installations and automation see the section.
HQ is configured via by one file, config.yaml. The docker compose files loads the content of hq.config.yaml and mounts it as the HQ config.yaml file.
You only need to follow this step if you do not want to use the local postgres instance started by the docker compose file.
You must create a database and role in your postgres instance for HQ to use. See .
Edit the docker-compose.yaml and add the set the credentials for your database in the hq.config.yaml section.
Currently HQ supports:
Basic Authentication (default)
SAML
To start HQ, run the following docker command:
To create an environment in HQ:
Login into HQ and create an environment, Environments->New Environment.
At the end of the process, you will be shown an Agent Key. Copy that, keep it safe!
The environment will be disconnected until the Agent is up and configured with the key.
You can also manage environments using the CLI.
The Agent is configured via two files:
provisioning.yaml - holds the connection details to your Kafka cluster and supporting systems. can set this via the agent.provisioning.yaml key in the docker-compose file.
You only need to follow this step if you do not want to use the local postgres instance started by the docker compose file.
Update the docker-compose file agent.lenses.conf key for your Postgres instance.
The Agent Key for an environment needs to be added to the agent.provisioning.yaml key in the docker compose file.
Replace ${{LENSESHQ_AGENT_KEY}} with the Agent Key for the environment that you want to link to.
By default, the agent is configured to connect to Kafka on localhost. To change this update the agent.provisioning.yaml key. The information required here depends on how you want the Agent to authenticate against Kafka.
Add the following for a basic plaintext connection to a Kafka broker, if you are using a different authentication mechanism adjust accordingly.
Remove, or adjust the Kafka (kafka-demo), Schema Registry and Connect services in the default docker-compose file.
Replace [YOUR_BOOTSTRAP_BROKER:PORT] with the bootstrap brokers and ports for the Kafka cluster you want the Agent to connect to.
To start Agent, run the following docker command:
For none dev environments, install the agent as close as possible to your Kafka clusters and automate the installation.
Once the agent fully starts, it will report as connected in HQ, allowing you to explore your Kafka environments.
For this example we will use basic authentication, for information on configuring other methods, see and configure the hq.config.yaml key accordingly for SAML.
You can now log in your with admin/admin.
lenses.conf - holds low-level options for the agent and the database connection. You can set this via the agent.lenses.conf in the docker-compose file
You must create a database and role in your postgres instance for the Agent to use. See .
For more information on the configuration of the connection to HQ see .
See for examples of different authentication types for Kafka.
For examples of adding in other services such as Schema Registries and Kafka Connect see .