Helm

This page describes installing Lenses in Kubernetes via Helm.

Only Helm version 3 is supported.

On start-up, Lenses will be in bootstrap mode unless it has an existing Kafka Connection. Enable provisioning to automate the creation of connections.

First, add the Helm Chart repository using the Helm command line:

helm repo add lensesio https://helm.repo.lenses.io
helm repo update

Use helm to install Lenses with default values:

helm install lenses lensesio/lenses --namespace lenses --create-namespace

The default install of Lenses will place Lenses in bootstrap mode, you can add the connections to Kafka manually and upload your license or automation with provisioning. Please refer to the GitHub values.yaml for all options.

Helm

Helm Chart Repo

Provisioning

To automatically provision the connections to Kafka and other systems set the .Values.lenses.provision.connections to be the YAML definition of your connections. For a full list of the connection types supported see Provisioning.

The chart will render the full YAML specified under this setting as the provisioning.yaml file.

Alternatively you can use a second YAML file, which contains only the connections pass them at the command line when installing:

helm install lenses \
charts/lenses \
--values charts/lenses/values.yaml \
--values provisioning.yaml

You must explicitly enable provisioning via lenses.provision.enabled: true otherwise Lenses will start in bootstrap mode.

Helm Chart components

The chart uses:

1. Secrets to store Lenses Postgres credentials and authentication credentials

2. Secrets to store connection credentials such as Kafka SASL_SCRAM password or password for SSL JKS stores.

3. Secrets to hold the base64 encoded values of the JKS stores

4. ConfigMap for Lenses configuration overrides

5. Cluster roles and role bindings (optional).

Secrets and config maps are mounted as files under the mount /mnt:

1. settings - holds the lenses.conf

2. secrets - holds the secrets Lenses and license

3. provision-secrets - holds the secrets for connections in the provisioning.yaml file

4. provision-secrets/files - holds any file needed for a connection, e.g. JKS files.

Cluster RBAC

The Helm chart creates Cluster roles and bindings, these are used by SQL Processors if the deployment mode is set to KUBERENTES. They are used so that Lenses can deploy and monitor SQL Processor deployments in namespaces.

To disable the RBAC set: rbacEnabled: false

If you want to limit the permissions Lenses has against your Kubernetes cluster, you can use Role/RoleBinging resources instead.

To achieve this you need to create a Role and a RoleBinding resource in the namespace you want the processors deployed to.

For example:

• Lenses namespace = lenses-ns

• Processor namespace = lenses-proc-ns

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: processor-role
  namespace: lenses-proc-ns
rules:
- apiGroups: [""]
  resources:
    - namespaces
    - persistentvolumes
    - persistentvolumeclaims
    - pods/log
  verbs:
    - list
    - watch
    - get
    - create
- apiGroups: ["", "extensions", "apps"]
  resources:
    - pods
    - replicasets
    - deployments
    - ingresses
    - secrets
    - statefulsets
    - services
  verbs:
    - list
    - watch
    - get
    - update
    - create
    - delete
    - patch
- apiGroups: [""]
  resources:
    - events
  verbs:
    - list
    - watch
    - get
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: processor-role-binding
  namespace: lenses-proc-ns
subjects:
- kind: ServiceAccount
  namespace: lenses-ns
  name: default
roleRef:
  kind: Role
  name: processor-role
  apiGroup: rbac.authorization.k8s.io

Finally you need to define in Lenses configuration which namespaces can Lenses access. To achieve this amend values.yaml to contain the following:

lenses:
  append:
    conf: |
      lenses.kubernetes.namespaces = {
        incluster = [
          "lenses-processors"
        ]
      }      

lenses.conf

The main configurable options for lenses.conf are available in the values.yaml under the lenses object. These include:

• Authentication

• Database connections

• SQL processor configurations

To apply other static configurations use lenses.append.conf, for example:

lenses:
  append:
    conf: |
      lenses.interval.user.session.refresh=40000

Authentication secrets

Set accordingly under**lenses.security.**

lenses:
  security:
    defaultUser:
      username: admin
      password: admin

For SSO set lenses.security.saml

lenses:
  security:
    saml:
      enabled: true
      baseUrl: "https://lenses-prod.eastus2.cloudapp.azure.com"
      provider: "azure"
      keyStoreFileData: |-
                somebase64encodedvalue
      keyStorePassword: "password"
      keyPassword: "password"
      metadataFileData: |-
                somebase64encodedvalue=

Postgres

To use Postgres as the backing store for Lenses set the details in the lenses.storage.postgres object.

storage:
    postgres:
      enabled: false
      host:
      port: 
      username:
      password:
      database:
      schema:        

If Postgres is not enabled a default embedded H2 database is used. To enable persistence for this data:

persistence:
  enabled: true
  accessModes:
    - ReadWriteOnce
  size: 5Gi

External Secrets

The chart relies on secrets for sensitive information such as Passwords. Secrets can rotate and are commonly stored in an external store such as Azure KeyVault, Hashicorp Vault or AWS Secrets Manager.

If you wish to have the chart use external secrets that are synchronized with these providers, set the following for the Lenses user:

  security:
    defaultUser:  
      # username: external # "external" that tells Lenses to look for a Secret
      # password: external # Same here.
      # usernameSecretKeyRef:
      #   name: my-existing-secret
      #   key: the-username-key
      # passwordSecretKeyRef:
      #   name: my-existing-secret
      #   key: the-password-key

For Postgres, add additional ENV variables via the lenses.additionalEnv object to point to your secret and set the username and password to external in the Postgres section.

      # - name: LENSES_STORAGE_POSTGRES_PASSWORD
      #   valueFrom:
      #     secretKeyRef:
      #       name: [SECRET_RESOURCE_NAME]
      #       key: [SECRET_RESOURCE_KEY]

Ingress & Services

While the chart supports setting TLS on Lenses itself we recommend placing it on the Ingress resource

Ingress and service resources are supported.

Enabled an Ingress resource in the values.yaml:

ingress:
  enabled: false
  host:
  annotations: {}
  tls:
    enabled: false
    crt: |-
    key: |-

Enable a service resource in the values.yaml:

# Lenses service
service:
  enabled: true
  type: ClusterIP
  annotations: {}
  externalTrafficPolicy:
  loadBalancerIP: 130.211.x.x
  loadBalancerSourceRanges:
    - 0.0.0.0/0

Controlling resources

To control the resources used by Lenses:

# Resource management
resources:
  requests:
    cpu: 1
    memory: 4Gi
  limits:
    cpu: 2
    memory: 5Gi

Enabling SQL Processors in K8s mode

To enable SQL processor in KUBERENTES mode and control the defaults:

sql:
    # processorImage: eu.gcr.io/lenses-container-registry/lenses-sql-processor
    # processorImageTag: 2.3
    mode: IN_PROC
    heap: 1024M
    minHeap: 128M
    memLimit: 1152M
    memRequest: 128M
    livenessInitialDelay: 60 seconds

To control the namespace Lenses can deploy processors, use the sql.namespaces value.

Prometheus metrics

Prometheus metrics are automatically exposed on port 9102 under /metrics.

Example Values files

For Connections, see Provisioning examples. You can also find examples in the Helm chart repo.