Deploying an Agent
This page describes installing Lenses Agent in Kubernetes via Helm.
Prerequisites
Kubernetes 1.23+
Helm 3.8.0+
Available local Postgres database instance.
Follow these steps to configure your Postgres database for Lenses Agent.
External Secrets Operator is the only supported secrets operator.
Configure an Agent
In order to configure an Agent, we have to understand the parameter groups that the Helm Chart offers.
Under the lensesAgent parameter there are some key parameter groups that are used to set up the connection to Lenses HQ:
Storage
HQ connection
Provision
RBAC for SQL Processors
Moving forward, in the same order you can start configuring your Helm chart.
JSON Schema Support
You can use JSON schema support to help you configure the values files for Helm. See JSON schema for support. The repository includes a JSON schema for the Agent Helm chart.
Configuring Agent chart
Configure storage (Postgres / H2 - Embedded database)
Running Agent with Postgres database
Prerequisite:
Running Postgres instance;
Created database for an Agent;
Username (and password) which has access to the created database;
In order to successfully run HQ, storage is within values.yaml has to be defined first.
The definition of storage object is as follows:
Alongside Postgres password, which can be referenced/created through Helm chart, there are a few more options which can help while setting up HQ.
There are two ways how username can be defined:
The most straightforward way, if the username is not being changed, is by just defining it within the username parameter such as
Password reference types
Postgres password can be handled in three ways using:
Pre-created secret;
Creating secrets on the spot through values.yaml;
Running Agent with H2 embedded database
In order to run Agent with H2 embedded database there are few things to be aware about:
K8s cluster Agent will be deployed on has to support Persistent Volumes;
Postgres options in Helm chart has to be left out.
Configure HQ connection (agent key)
Connection to Lenses HQ is a straight forward process which requires two steps:
Create an environment and obtain an AGENT KEY in HQ, as described in Install, if you have not already done so;
Storing that same key in Vault or as a K8s secret.
The agent communicates with HQ via a secure custom binary protocol channel. To establish this channel and authenticate the Agent needs and AGENT KEY.
Once the AGENT KEY has been copied, store it inside of Vault or any other tool that has integration with Kubernetes secrets.
There are three available options how the agent key can be used:
ExternalSecret via External Secret Operator (ESO)
Pre-created secret
Inline string
When specifying secret.type: "externalSecret", the chart will:
create an ExternalSecret in the namespace where Agent is deployed;
a secret is mounted for Agent to use.
The chart will create a secret with defined values below and the same secret will be read by Agent to connect to HQ.
This secret will be fed into the provisioning.yaml. The HQ connection is specified below, where reference ${LENSESHQ_AGENT_KEY} is being set:
Configure provisioning (Kafka / SchemaRegistry / Kafka Connect)
Provisioning offers various connections starting with:
Kafka ecosystem components such as:
More about provisioning and advanced configuration options for each of these components can be found in Provisioning.
RBAC for SQL Processors
If the deployment mode is set to KUBERNETES, the Agent's ServiceAccount needs Kubernetes RBAC permissions so Lenses can deploy and monitor SQL Processor deployments in namespaces.
rbacEnable: true and namespaceScope: true — recommended. Creates a namespace-scoped Role and RoleBinding, so the Agent can only deploy into, and read pod information from, the namespace(s) you list.
Follow this link to configure it, including the required lenses.kubernetes.namespaces setting shown further down that section.
rbacEnable: true and namespaceScope: false — creates a cluster-wide ClusterRole and ClusterRoleBinding instead.
To disable the creation of Kubernetes RBAC entirely set: rbacEnable: false
(Optional) Enable TLS connection with HQ
Enabling TLS in communication between HQ is being done in the provisioning part of values.yaml.
In order to successfully enable TLS for the Agent you would need to:
additionalVolume & additionalVolumeMounts - with which you will mount truststore with CA certificate that HQ is using and which Agent will need to successfully pass the handshake.
additionalEnv - which will be used to securely read passwords to unlock truststore.
Enable SSL in provision.
(Optional) Services
Enable a service resource in the values.yaml:
(Optional) Controlling resources
To control the resources used by the Agent:
Enabling SQL processors in K8s mode
To enable SQL processor in KUBERENTES mode and control the defaults:
To control which images a SQL Processor can be changed to, set lenses.kubernetes.processor.image.allowed via lensesAgent.append.conf. When unset, any tag or digest of the processorImage repository (lensesioextra/sql-processor when processorImage is not set) is allowed; the default image must always be allowed.
SQL Processor Role Binding
To achieve you need to create a Role and a RoleBinding resource in the namespace you want the processors deployed to.
For example:
Lenses namespace = lenses-ns
Processor namespace = lenses-proc-ns
Finally you need to define in the Agent configuration which namespaces the Agent has access to. Amend values.yaml to contain the following:
Persistence Volume
Persistence can be enabled for three purposes:
Use H2 embedded database
Logging
Provisioning
When using the Data Policies module to persist your data policies rules
When
lenses.storage.enabled: falseand an H2 local filesystem database is used instead of PostgreSQLFor non critical and NON PROD deployments
Configuration:
When you need persistent log storage across pod restarts
When you want to retain logs for auditing or debugging purposes
Configuration:
Dedicated volume for provisioning data managed via the HQ.
When to enable:
When using HQ-based provisioning workflows
Must be combined with
PROVISION_HQ_URLandPROVISION_AGENT_KEYenvironment variables
Configuration:
or Helm command execution:
Prometheus metrics
Prometheus metrics are automatically exposed on port 9102 under /metrics.
At this very moment you can scrape it only via Service under port called http-metrics.
lenses.conf
The main configurable options for lenses.conf are available in the values.yaml under the lenses object. These include:
Authentication
Database connections
SQL processor configurations
To apply other static configurations use lenses.append.conf, for example:
Data policies
For a new installation, configure the Agent to apply the data policies defined on IAM roles in Lenses HQ. Add the setting to lenses.append.conf:
If lensesAgent.append.conf already contains settings, add the line to the existing block.
If you upgrade an existing Agent, keep the default true until you recreate its data policies on IAM roles. See Configuration Reference.
Install the Chart
First, add the Helm Chart repository using the Helm command line:
Installing the Agent
Installing using cloned repository:
Installing using Helm repository:
Example Values files
You can also find examples in the Helm chart repo.
Last updated
Was this helpful?

