Deploying an Agent
This page describes installing Lenses Agent in Kubernetes via Helm.
Prerequisites
Kubernetes 1.23+
Helm 3.8.0+
Available local Postgres database instance.
Follow these steps to configure your Postgres database for Lenses Agent.
External Secrets Operator is the only supported secrets operator.
Configure an Agent
In order to configure an Agent, we have to understand the parameter groups that the Helm Chart offers.
Under the lensesAgent parameter there are some key parameter groups that are used to set up the connection to Lenses HQ:
Storage
HQ connection
Provision
Cluster RBACs
Moving forward, in the same order you can start configuring your Helm chart.
JSON Schema Support
You can use JSON schema support to help you configure the Values files for Helm. See JSON schema for support. Included in the repository is a JSON schema for the Agent Helm chart.
Configuring Agent chart
Configure storage (Postgres)
Prerequisite:
Running Postgres instance;
Created database for an Agent;
Username (and password) which has access to the created database;
In order to successfully run HQ, storage is within values.yaml has to be defined first.
The definition of storage object is as follows:
lensesAgent:
storage:
postgres:
enabled: true
host: ""
port:
username: ""
database: ""
schema: ""
params: {}
Alongside Postgres password, which can be referenced/created through Helm chart, there are a few more options which can help while setting up HQ.
There are two ways how username can be defined:
The most straightforward way, if the username is not being changed, is by just defining it within the username parameter such as
lensesAgent:
storage:
postgres:
enabled: true
host: postgres-postgresql.postgres.svc.cluster.local
port: 5432
database: lensesagent
username: lenses
Password reference types
Postgres password can be handled in three ways using:
Pre-created secret;
Creating secrets on the spot through values.yaml;
lensesAgent:
storage:
postgres:
enabled: true
host: postgres-postgresql.playground.svc.cluster.local
port: 5432
username: lenses
database: lensesagent
password: useOnlyForDemos
Configure HQ connection (agent key)
Connection to Lenses HQ is a straight forward process which requires two steps:
Creating Environment and obtaining AGENT KEY in HQ as described here, if you already have not done so;
Storing that same key in Vault or as a K8s secret.

The agent communicates with HQ via a secure custom binary protocol channel. To establish this channel and authenticate the Agent needs and AGENT KEY.
Once the AGENT KEY has been copied, store it inside of Vault or any other tool that has integration with Kubernetes secrets.
There are three available options how the agent key can be used:
ExternalSecret via External Secret Operator (ESO)
Pre-created secret
Inline string
To use this option, the External Secret Operator (ESO) has to be installed and available for usage in K8s cluster your are deploying Agent.
When specifying secret.type: "externalSecret", the chart will:
create an ExternalSecret in the namespace where Agent is deployed;
a secret is mounted for Agent to use.
lensesAgent:
hq:
agentKey:
secret:
type: "externalSecret"
# Secret name where agentKey will be read from
name: hq-password
# Key name under secret where agentKey is stored
key: key
externalSecret:
additionalSpecs: {}
secretStoreRef:
type: ClusterSecretStore # ClusterSecretStore | SecretStore
name: [secretstore_name]
This secret will be fed into the provisioning.yaml. The HQ connection is specified below, where reference ${LENSESHQ_AGENT_KEY} is being set:
lensesAgent:
provision:
path: /mnt/provision-secrets
connections:
lensesHq:
- name: lenses-hq
version: 1
tags: ['hq']
configuration:
server:
value: [LENSES_HQ_FQDN_OR_IP]
port:
value: 10000
agentKey:
# This property shouldn't be changed as it is mounted automatically
# based on secret choice for hq.agentKey above
value: ${LENSESHQ_AGENT_KEY}
sslEnabled:
value: false
Configure provisioning (Kafka / SchemaRegistry / Kafka Connect)
Provisioning offers various connections starting with:
Kafka ecosystem components such as:
lensesAgent:
provision:
path: /mnt/provision-secrets
connections:
# Kafka Connection
kafka:
- name: Kafka
version: 1
tags: [my-tag]
configuration:
kafkaBootstrapServers:
value:
- PLAINTEXT://your.kafka.broker.0:9092
- PLAINTEXT://your.kafka.broker.1:9092
protocol:
value: PLAINTEXT
# all metrics properties are optional
metricsPort:
value: 9581
metricsType:
value: JMX
metricsSsl:
value: false
# Confluent Schema Registry Connection
confluentSchemaRegistry:
- name: schema-registry
tags: ["tag1"]
version: 1
configuration:
schemaRegistryUrls:
value:
- http://my-sr.host1:8081
- http://my-sr.host2:8081
## all metrics properties are optional
metricsPort:
value: 9581
metricsType:
value: JMX
metricsSsl:
value: false
# Kafka Connect connection
connect:
- name: my-connect-cluster-name
version: 1
tags: ["tag1"]
configuration:
workers:
value:
- http://my-kc.worker1:8083
- http://my-kc.worker2:8083
metricsPort:
value: 9585
metricsType:
value: JMX
More about provisioning and more advanced configuration options for each of these components can be found on the following link.
Cluster RBACs
The Helm chart creates Cluster roles and bindings, that are used by SQL Processors, if the deployment mode is set to KUBERNETES. They are used so that Lenses can deploy and monitor SQL Processor deployments in namespaces.
To disable the creation of Kubernetes RBAC set: rbacEnabled: false
If you want to limit the permissions the Agent has against your Kubernetes cluster, you can use Role/RoleBinging resources instead. Follow this link in order to enable it.
If you are not using SQL Processors and want to limit permissions given to Agent's ServiceAccount, there are two options you can choose from:
rbacEnable: true - will enable the creation of ClusterRole and ClusterRoleBinding for service account mentioned above;
rbacsEnable: true
namespaceScope: false
rbacEnable: true and namespaceScope: true - will enable the creation of Role and RoleBinding which is more restrictive;
rbacsEnable: true
namespaceScope: true
(Optional) Enable TLS connection with HQ
In this case, TLS has to be enabled on HQ. In case you haven't yet enabled it, you can find details here to do it.
Enabling TLS in communication between HQ is being done in the provisioning part of values.yaml.
In order to successfully enable TLS for the Agent you would need to:
additionalVolume & additionalVolumeMounts - with which you will mount truststore with CA certificate that HQ is using and which Agent will need to successfully pass the handshake.
additionalEnv - which will be used to securely read passwords to unlock truststore.
Enable SSL in provision.
# Additional Volume with CA that HQ uses
additionalVolumes:
- name: hq-truststore
secret:
secretName: hq-agent-test-authority
additionalVolumeMounts:
- name: hq-truststore
mountPath: "/mnt/provision-secrets/hq"
lensesAgent:
# Additional Env to read truststore password from secret
additionalEnv:
- name: LENSES_HQ_AGENT_TRUSTSTORE_PWD
valueFrom:
secretKeyRef:
name: hq-agent-test-authority
key: truststore.jks.password
provision:
path: /mnt/provision-secrets
connections:
lensesHq:
- name: lenses-hq
version: 1
tags: ['hq']
configuration:
server:
value: [HQ_URL]
port:
value: 10000
agentKey:
value: ${LENSESHQ_AGENT_KEY}
sslEnabled:
value: true
sslTruststore:
file: "/mnt/provision-secrets/gq/truststore.jks"
sslTruststorePassword:
value: ${LENSES_HQ_AGENT_TRUSTSTORE_PWD}
(Optional) Services
Enable a service resource in the values.yaml:
# Lenses service
service:
enabled: true
annotations: {}
(Optional) Controlling resources
To control the resources used by the Agent:
# Resource management
resources:
requests:
cpu: 1
memory: 4Gi
limits:
cpu: 2
memory: 5Gi
Enabling SQL processors in K8s mode
To enable SQL processor in KUBERENTES mode and control the defaults:
lensesAgent:
sql:
processorImage: hub.docker.com/r/lensesioextra/sql-processor/
processorImageTag: latest
mode: KUBERNETES
heap: 1024M
minHeap: 128M
memLimit: 1152M
memRequest: 128M
livenessInitialDelay: 60 seconds
SQL Processor Role Binding
To achieve you need to create a Role and a RoleBinding resource in the namespace you want the processors deployed to.
For example:
Lenses namespace = lenses-ns
Processor namespace = lenses-proc-ns
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: processor-role
namespace: lenses-proc-ns
rules:
- apiGroups: [""]
resources:
- namespaces
- persistentvolumes
- persistentvolumeclaims
- pods/log
verbs:
- list
- watch
- get
- create
- apiGroups: ["", "extensions", "apps"]
resources:
- pods
- replicasets
- deployments
- ingresses
- secrets
- statefulsets
- services
verbs:
- list
- watch
- get
- update
- create
- delete
- patch
- apiGroups: [""]
resources:
- events
verbs:
- list
- watch
- get
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: processor-role-binding
namespace: lenses-proc-ns
subjects:
- kind: ServiceAccount
namespace: lenses-ns
name: default
roleRef:
kind: Role
name: processor-role
apiGroup: rbac.authorization.k8s.io
Finally you need to define in the Agent configuration which namespaces the Agent has access to. Amend values.yaml to contain the following:
lensesAgent:
append:
conf: |
lenses.kubernetes.namespaces = {
incluster = [
"lenses-processors"
]
}
Prometheus metrics
Prometheus metrics are automatically exposed on port 9102 under /metrics.
lenses.conf
The main configurable options for lenses.conf are available in the values.yaml under the lenses object. These include:
Authentication
Database connections
SQL processor configurations
To apply other static configurations use lenses.append.conf, for example:
lensesAgent:
append:
conf: |
lenses.interval.user.session.refresh=40000
Install the Chart
First, add the Helm Chart repository using the Helm command line:
helm repo add lensesio https://helm.repo.lenses.io/
helm repo update
Installing the Agent
Installing using cloned repository:
helm install lenses-agent charts/lenses-agent \
--values charts/lenses-agent/values.yaml \
--create-namespace --create lenses-agent
Installing using Helm repository:
helm install lenses-agent lensesio/lenses-agent \
--values values.yaml \
--create-namespace --namespace lenses-agent \
--version 6.0.0
Example Values files
You can also find examples in the Helm chart repo.
Last updated
Was this helpful?