To use this option, the External Secret Operator (ESO) has to be installed and available for usage in K8s cluster your are deploying HQ.
When specifying passwordSecret.type: "externalSecret", thechart will:
create an ExternalSecretin the namespace where HQ is deployed;
a secret is mounted for HQ to use.
values.yaml
lensesHq:
storage:
postgres:
enabled: true
host: postgres-postgresql.playground.svc.cluster.local
port: 5432
username: lenses
database: lenseshq
passwordSecret:
type: "externalSecret"
# Secret name where database password will be read from
name: hq-password
# Key name under secret where database password is stored
key: password
externalSecret:
additionalSpecs: {}
secretStoreRef:
type: SecretStore # or ClusterSecretStore
name: secretstore-secrets
Make sure that secret you are going to use is already created in namespace where HQ will be installed.
values.yaml
lensesHq:
storage:
postgres:
enabled: true
host: postgres-postgresql.playground.svc.cluster.local
port: 5432
username: lenses
database: lenseshq
passwordSecret:
type: "precreated"
# Secret name where database password will be read from
name: hq-password
# Key from secret's data where database password is being stored
key: postgres-password
This option is NOT for PRODUCTION usage but rather just for demo / testing.
The chart will create a secret with defined values below and the same secret will be read by HQ in order to connect to Postgres.
values.yaml
lensesHq:
storage:
postgres:
enabled: true
host: [POSTGRES_HOSTNAME]
port: 5432
username: lenses
database: lenseshq
passwordSecret:
type: "createNew"
# name of a secret that will be created
name: [K8s_SECRET_NAME]
# Database password
password: [DATABASE_USER_PASSWORD]
Advanced Postgres settings
Sometimes to form the correct connection URI special parameters are needed. You can set the extra settings using params.
SAML / SSO is available only with Enterprise license.
The second pre-requirement to successfully run HQ is setting initial authentication.
You can choose between:
password-based authentication, which requires users to provide a username and password;
and SAML/SSO (Single Sign-On) authentication, which allows users to authenticate through an external identity provider for a seamless and secure login experience.
First to cover is the users property. Users Property: The users property is defined as an array, where each entry includes a username and a password. Passwords must be hashed using bcrypt before being placed within the password property, for security purposes, ensuring that they are stored correctly and securely.
Second, to cover will be administrators. It serves as the definition of user emails have the highest level of permissionsupon authentication to HQ.
Third attribute is saml.metadata field, needed for setting SAML / SSO authentication. In this step, you will need metadata.xml file which can be set in two ways:
Referencing metadata.xml file through pre-created secret;
The third pre-requirement to successfully run HQ is the http definition. As previously mentioned, this parameter defines everything around the HTTP endpoint of the HQ itself and how users will interact with it.
Second part of HTTP definition would be enabling TLS and TLS definition itself. As previously defined for lensesHq.agents.tlssame way of configuring TLS can be used for lensesHq.http.tls definition as well.
4
Configure agent's connection endpoint
After correctly configuring the authentication strategy and connection endpoint, the agent handling is the last most important box to tick.
The Agent's object is defined as follows:
lensesHq:
agents:
# which port to listen on for agent requests
address: ":10000"
tls:
enabled: false
verboseLogs: false
cert:
privateKey:
Enabling TLS
By default TLS for the communication between Agent and HQ is disabled. In case the requirement is to enable it, fthe ollowing has to be set:
lensesHq.agents.tls - certificates to manage the connection between HQ and the Agents
lensesHq.http.tls- certificates to manage connection with HQ's API
Unlike private keys which can be referenced and obtained only through a secret, Certificates can be referenced directly in values.yamlfile as a string or as a secret.
Metrics are optionally available in a Prometheus format and by default served on port 9090.
The port can be changed in the following way:
values.yaml
lensesHq:
metrics:
prometheusAddress: ":9090"
(Optional) Configure Ingress & Services
Whilst the chart supports setting TLS on Lenses HQ itself we recommend placing it on the Ingress resource
Ingress and service resources are optionally supported.
The http ingress is intended only for HTTP/S traffic, while the agents ingress is designed specifically for TCP protocol. Ensure appropriate ingress configuration for your use case.
Enable an Ingress resource in the values.yaml:
values.yaml
ingress:
http:
enabled: true
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: websecure
host: example.com
ingressClassName: ""
tls:
enabled: false
# The TLS secret must contain keys named tls.crt and tls.key that contain the certificate and private key to use for TLS.
secretName: ""
agent:
enabled: true
agentIngressConfig:
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
name: agents
spec:
entryPoints:
- agents
routes:
- match: HostSNI(`example.com`) # HostSNI to match TLS for TCP
services:
- name: lenses-hq # Replace with your service name
port: 10000 # Agent default TCP port
tls: {}
Be aware that example of values.yaml shows only how all parameters should look at the end. Please fill them with correct values otherwise Helm installation might not be successful.
Example of values.yaml
More about default values for Lenses HQ Helm Chart can be found in values.yaml. An example is below:
lensesAgent:
storage:
postgres:
enabled: true
host: postgres-postgresql.postgres.svc.cluster.local
port: 5432
database: lensesagent
username: lenses
password: external # use "external" to manage it using secrets
additionalEnv:
- name: LENSES_STORAGE_POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: [SECRET_RESOURCE_NAME]
key: [SECRET_RESOURCE_KEY]
2
Configure HQ connection (agent key)
Connection to Lenses HQ is a straight forward process which requires two steps:
Creating Environment and obtaining AGENT KEY inHQ as described here, if you already have not done so;
Storing that same key in Vault or as a K8s secret.
The agent communicates with HQ via a secure custom binary protocol channel. To establish this channel and authenticate the Agent needs and AGENT KEY.
Once the AGENT KEY has been copied, store it inside of Vault or any other tool that has integration with Kubernetes secrets.
There are three available options how the agent key can be used:
ExternalSecret via External Secret Operator (ESO)
Pre-created secret
Inline string
To use this option, the External Secret Operator (ESO) has to be installed and available for usage in K8s cluster your are deploying Agent.
When specifying secret.type: "externalSecret", thechart will:
create an ExternalSecretin the namespace where Agent is deployed;
a secret is mounted for Agent to use.
values.yaml
lensesAgent:
hq:
agentKey:
secret:
type: "externalSecret"
# Secret name where agentKey will be read from
name: hq-password
# Key name under secret where agentKey is stored
key: key
externalSecret:
additionalSpecs: {}
secretStoreRef:
type: ClusterSecretStore # ClusterSecretStore | SecretStore
name: [secretstore_name]
Make sure that secret you are going to use is already created in namespace where Agent will be installed.
values.yaml
lensesAgent:
hq:
agentKey:
secret:
type: "precreated"
# Secret name where agentKey will be read from
name: hq-password
# Key name under secret where agentKey is stored
key: key
This option is NOT for PRODUCTION usage but rather just for demo / testing.
The chart will create a secret with defined values below and the same secret will be read by Agent to connect to HQ.
values.yaml
lensesAgent:
hq:
agentKey:
secret:
type: "createNew"
# Secret name where agentKey will be read from
name: "lenses-agent-secret-1"
# Value of agentKey generated by HQ
value: "agent_key_*"
This secret will be fed into the provisioning.yaml. The HQ connection is specified at line 30, below, where reference ${LENSESHQ_AGENT_KEY} is being set:
values.yaml
lensesAgent:
provision:
path: /mnt/provision-secrets
connections:
lensesHq:
- name: lenses-hq
version: 1
tags: ['hq']
configuration:
server:
value: [LENSES_HQ_FQDN_OR_IP]
port:
value: 10000
agentKey:
# This property shouldn't be changed as it is mounted automatically
# based on secret choice for hq.agentKey above
value: ${LENSESHQ_AGENT_KEY}
sslEnabled:
value: false
In order to enable TLS for secure communication between HQ and the Agent please refer to the following part of the page.
More about provisioning and more advanced configuration options for each of these components can be found on the following link.
4
Cluster RBACs
The Helm chart creates Cluster roles and bindings, that are used by SQL Processors, if the deployment mode is set to KUBERNETES. They are used so that Lenses can deploy and monitor SQL Processor deployments in namespaces.
To disable the creation of Kubernetes RBAC set: rbacEnabled: false
If you want to limit the permissions the Agent has against your Kubernetes cluster, you can use Role/RoleBinging resources instead. Follow this link in order to enable it.
If you are not using SQL Processors and want to limit permissions given to Agent's ServiceAccount, there are two options you can choose from:
rbacEnable: true - will enable the creation of ClusterRole and ClusterRoleBinding for service account mentioned above;
rbacEnable: true and namespaceScope: true - will enable the creation of Role and RoleBinding which is more restrictive;
(Optional) Enable TLS connection with HQ
In this case, TLS has to be enabled on HQ. In case you haven't yet enabled it, you can find details here to do it.
Enabling TLS in communication between HQ is being done in the provisioning part of values.yaml.
In order to successfully enable TLS for the Agent you would need to:
additionalVolume & additionalVolumeMounts - with which you will mount truststore with CA certificate that HQ is using and which Agent will need to successfully pass the handshake.
additionalEnv - which will be used to securely read passwords to unlock truststore.
Be aware that for the time being and alpha purposes usage of --version is mandatory when deploying Helm chart through Helm repository.
Example Values files
Be aware that example of values.yaml shows only how all parameters should look at the end. Please fill them with correct values otherwise Helm installation might not be successful.
Example of values.yaml
values.yaml
lensesAgent:
storage:
postgres:
enabled: true
host: [postgres.url]
port: 5432
username: postgres
password: changeMe
database: agent
hq:
agentKey:
secret:
type: "createNew"
name: "agentKey"
value: "agent_key_*"
provision:
path: /mnt/provision-secrets
connections:
lensesHq:
- name: lenses-hq
version: 1
tags: ['hq']
configuration:
server:
value: hq-tls-test-lenses-hq.hq-agent-test.svc.cluster.local
port:
value: 10000
agentKey:
value: ${LENSESHQ_AGENT_KEY}
kafka:
# There can only be one Kafka cluster at a time
- name: kafka
version: 1
tags: ['staging', 'pseudo-data-only']
configuration:
kafkaBootstrapServers:
value:
- PLAINTEXT://kafka-1.svc.cluster.local:9092
- PLAINTEXT://kafka-2.svc.cluster.local:9092
protocol:
value: PLAINTEXT
# Metrics are strongly suggested for better Kafka cluster observability
metricsType:
value: JMX
metricsPort:
value: 9581
