ProductsDownload Community Edition

How to convert Wizard Mode to Provisioning Mode

Migrating from Lenses Wizard Mode to Provision

Overview

Lenses 5.0+ introduces two primary methods for configuring your Lenses instance:

  • Wizard Mode: An interactive UI-based setup that appears when no Kafka brokers are configured

  • Provision Mode: A programmatic approach using provisioning.yaml configuration with(out) sidecar containers

This guide walks you through migrating from the Wizard Mode setup to a fully automated Provision configuration, enabling GitOps workflows and Infrastructure as Code practices.

Understanding the Migration Path

When to Consider Migration

You should migrate from Wizard Mode to Provision when you need:

  • Automated deployments and Infrastructure as Code

  • GitOps workflows for configuration management

  • Consistent environments across development, staging, and production

  • Version control for your Lenses configuration

  • Scalable deployment patterns for multiple Lenses Agent instances

  • Migrating from Lenses ⅘ to Lenses 6 Agent

Key Differences

Aspect
Wizard Mode
Provision Mode

Setup Method

Interactive UI

YAML configuration

Automation

Manual

Fully automated

Version Control

Not supported

Full Git integration

Secrets Management

Manual entry

Kubernetes secrets + file references

Deployment

One-time setup

Repeatable deployments

Configuration Updates

UI-based

Code-based with CI/CD

Pre-Migration Checklist

Before starting your migration, please check Tips: Before Upgrade

1

Assess Agent Connections that have to be migrated from Lenses 5

In the picture below it is visible that three connections has to be migrated to provisioning.yaml file:

  1. Kafka

  2. SchemaRegistry

  3. KafkaConnect

in addition to that a new connection has to be made called LensesHQ.

2

Prepare Your Provision Configuration

Basic Structure

For Helm Deployment:

  • create a values.yaml file with the provision configuration enabled

For deployment from Archive create:

  • lenses-agent.conf and

  • provisioning.yaml file

Connections that provisioning is going to have:

  • Kafka;

  • SchemaRegistry;

  • KafkaConnect;

  • LensesHQ.

3

Configure Kafka Connection

values.yaml
lensesAgent:
  hq:
    agentKey:
      secret:
        type: "createNew"
        name: lenses-agent-secret
        value: <your-HQ-generated-agentKey>
  provision:
    path: /mnt/provision-secrets
    connections:
      lensesHq:
        - name: lenses-hq
          version: 1
          tags: ['hq']
          configuration:
            server:
              value: <your-HQ-address>
            port:
              value: 10000
            agentKey:
              value: ${LENSESHQ_AGENT_KEY}
            sslEnabled:
              value: false
      kafka:
        - name: kafka
          version: 1
          tags: [ "prod", "prod-1", "us"]
          configuration:
            kafkaBootstrapServers:
              value:
                - PLAINTEXT://<your-kafka-address>:9092
            metricsType:
             value: JMX
            metricsPort:
             value: 9999

For more Kafka connection details, such as using secure connections, please read Kafka

4

Configure Additional Services Connections

Last two bits are:

  • Kafka Connect and

  • Schema Registry

values.yaml
lensesAgent:
  hq:
    agentKey:
      secret:
        type: "createNew"
        name: lenses-agent-secret
        value: <your-HQ-generated-agentKey>
  provision:
    path: /mnt/provision-secrets
    connections:
      lensesHq:
        - name: lenses-hq
          version: 1
          tags: ['hq']
          configuration:
            server:
              value: <your-HQ-address>
            port:
              value: 10000
            agentKey:
              value: ${LENSESHQ_AGENT_KEY}
            sslEnabled:
              value: false
      kafka:
        - name: kafka
          version: 1
          tags: [ "prod", "prod-1", "us"]
          configuration:
            kafkaBootstrapServers:
              value:
                - PLAINTEXT://<your-kafka-address>:9092
            metricsType:
             value: JMX
            metricsPort:
             value: 9999
      confluentSchemaRegistry:
        - name: schema-registry
          version: 1
          tags: [ "prod", "global" ]
          configuration:
            schemaRegistryUrls:
              value:
                - http://<your-schema-registry-address>:8081
      connect:
        - name: datalake-connect
          version: 1
          tags: [ "prod", "us" ]
          configuration:
            workers:
              value:
                - http://<your-kafka-connect-address>:8083
5

Prepare HQ Connection

Through last few steps, we covered configuring of:

  • Kafka

  • Schema Registry

  • Kafka Connect connections.

Last but not least and probably the most important one is creating HQ connection otherwise Agent won't be useable.

values.yaml
lensesAgent:
  hq:
    agentKey:
      secret:
        type: "createNew"
        name: lenses-agent-secret
        value: <your-HQ-generated-agentKey>
  provision:
    path: /mnt/provision-secrets
    connections:
      lensesHq:
        - name: lenses-hq
          version: 1
          tags: ['hq']
          configuration:
            server:
              value: <your-HQ-address>
            port:
              value: 10000
            agentKey:
              value: ${LENSESHQ_AGENT_KEY}
            sslEnabled:
              value: false

License configuration is part of Lenses HQ from version 6.

6

Configure Database details

In case Postgres is being used:

values.yaml
lensesAgent:
  storage:
    postgres:
      enabled: true
      host: postgres-1.postgres.svc.cluster.local
      port: 5432              # optional, defaults to 5432
      username: prod          
      password: external      # use "external" to manage it using secrets
      database: agent
  additionalEnv:
    - name: LENSES_STORAGE_POSTGRES_PASSWORD
      valueFrom:
        secretKeyRef:
          name: local-postgres-pwd
          key: password
  hq:
    agentKey:
      secret:
        type: "createNew"
        name: lenses-agent-secret
        value: agent_key_*
    ssl:
      enabled: false
  provision:
    path: /mnt/provision-secrets
    connections:
      lensesHq:
        - name: lenses-hq
          version: 1
          tags: ['hq']
          configuration:
            server:
              value: lenses-hq.lenses.svc.cluster.local
            port:
              value: 10000
            agentKey:
              value: ${LENSESHQ_AGENT_KEY}
            sslEnabled:
              value: false
      kafka:
        - name: kafka
          version: 1
          tags: [ "prod", "prod-2", "eu"]
          configuration:
            kafkaBootstrapServers:
              value:
                - PLAINTEXT://testing-kafka-bootstrap.kafka.svc.cluster.local:9092
            metricsType:
             value: JMX
            metricsPort:
             value: 9999
      connect:
        - name: datalake-connect
          version: 1
          tags: [ "prod-2", "eu" ]
          configuration:
            workers:
              value:
                - http://testing-connect-connect.kafka-connect.svc.cluster.local:8083
      confluentSchemaRegistry:
        - name: schema-registry
          version: 1
          tags: [ "prod", "global" ]
          configuration:
            schemaRegistryUrls:
              value:
                - http://testing-schema-registry.schema-registry.svc.cluster.local:8081

H2 as a storage mechanism is available only from Agent v6.0.6

Be aware that H2 is not recommended for production environments

values.yaml
persistence:
  storageH2:
    enabled: true
    accessModes:
      - ReadWriteOnce
    size: 5Gi

lensesAgent:
  hq:
    agentKey:
      secret:
        type: "createNew"
        name: lenses-agent-secret
        value: agent_key_*
    ssl:
      enabled: false
  provision:
    path: /mnt/provision-secrets
    connections:
      lensesHq:
        - name: lenses-hq
          version: 1
          tags: ['hq']
          configuration:
            server:
              value: lenses-hq.lenses.svc.cluster.local
            port:
              value: 10000
            agentKey:
              value: ${LENSESHQ_AGENT_KEY}
            sslEnabled:
              value: false
      kafka:
        - name: kafka
          version: 1
          tags: [ "prod", "prod-2", "eu"]
          configuration:
            kafkaBootstrapServers:
              value:
                - PLAINTEXT://testing-kafka-bootstrap.kafka.svc.cluster.local:9092
            metricsType:
             value: JMX
            metricsPort:
             value: 9999
      connect:
        - name: datalake-connect
          version: 1
          tags: [ "prod-2", "eu" ]
          configuration:
            workers:
              value:
                - http://testing-connect-connect.kafka-connect.svc.cluster.local:8083
      confluentSchemaRegistry:
        - name: schema-registry
          version: 1
          tags: [ "prod", "global" ]
          configuration:
            schemaRegistryUrls:
              value:
                - http://testing-schema-registry.schema-registry.svc.cluster.local:8081

Complete Migration Example

Here's a complete values.yaml and lenses-agent + provisioning.yaml example for a production migration:

For more Helm options, please check lenses-helm-chart repo.

values.yaml
rbacEnable: true
namespaceScope: true

lensesAgent:
  hq:
    agentKey:
      secret:
        type: "createNew"
        name: lenses-agent-secret
        value: agent_key_*
    ssl:
      enabled: false
  provision:
    path: /mnt/provision-secrets
    connections:
      lensesHq:
        - name: lenses-hq
          version: 1
          tags: ['hq']
          configuration:
            server:
              value: lenses-hq.lenses.svc.cluster.local
            port:
              value: 10000
            agentKey:
              value: ${LENSESHQ_AGENT_KEY}
            sslEnabled:
              value: false
      kafka:
        - name: kafka
          version: 1
          tags: [ "prod", "prod-2", "eu"]
          configuration:
            kafkaBootstrapServers:
              value:
                - PLAINTEXT://testing-kafka-bootstrap.kafka.svc.cluster.local:9092
            metricsType:
             value: JMX
            metricsPort:
             value: 9999
      connect:
        - name: datalake-connect
          version: 1
          tags: [ "prod-2", "eu" ]
          configuration:
            workers:
              value:
                - http://testing-connect-connect.kafka-connect.svc.cluster.local:8083
      confluentSchemaRegistry:
        - name: schema-registry
          version: 1
          tags: [ "prod", "global" ]
          configuration:
            schemaRegistryUrls:
              value:
                - http://testing-schema-registry.schema-registry.svc.cluster.local:8081
  storage:
    postgres:
      enabled: true
      host: postgres-1.postgres.svc.cluster.local
      port: 5432              # optional, defaults to 5432
      username: prod          
      password: external      # use "external" to manage it using secrets
      database: agent
  additionalEnv:
    - name: LENSES_STORAGE_POSTGRES_PASSWORD
      valueFrom:
        secretKeyRef:
          name: local-postgres-pwd
          key: password

Deployment and Testing

1

Deploy with Helm

Add the Lenses Helm repository:

helm repo add lenses-agent https://helm.repo.lenses.io
helm repo update

Deploy Lenses with your provision configuration:

helm install lenses-agent lenses/lenses-agent \
  --namespace lenses \
  --create-namespace \
  -f values.yaml

Monitor the deployment:

kubectl get pods -n lenses -w
kubectl logs -n lenses deployment/lenses-agent

2

Deploy through Archive

Download Archive

Link to archives can be found here: https://archive.lenses.io/lenses/6.0/agent/

Extract the archive using the following command

terminal
tar -xvf lenses-agent-latest-linux64.tar.gz -C lenses

Start Agent

terminal
bin/lenses lenses-agent.conf

Best Practices for Production

Security Considerations

  • Use Kubernetes secrets for sensitive data instead of inline values

  • Enable TLS for all Lenses HQ connections

  • Implement RBAC for Kubernetes and Lenses HQ & Agent access

  • Rotate credentials regularly

Operations Best Practices

  • Monitor resource usage of sidecar containers

  • Set resource limits to prevent resource monopolization

  • Implement health checks for the provision process

  • Use GitOps workflows for configuration management

Scaling Considerations

  • Plan for multiple environments (dev, staging, prod)

  • Implement configuration templates for reusability

  • Use Helm chart dependencies for complex deployments

  • Monitor deployment metrics and success rates

Conclusion

Migrating from Lenses Wizard Mode to Provision Mode enables Infrastructure as Code practices, better security management, and automated deployments. While the initial setup requires more configuration, the long-term benefits of automated, version-controlled, and repeatable deployments make this migration worthwhile for production environments.

The provision sidecar pattern ensures that your Lenses configuration is managed alongside your infrastructure code, enabling true GitOps workflows and reducing configuration drift between environments.

PreviousSteps for SideBySide UpgradeNextDownloads

Last updated

Was this helpful?