Overview

This page describes an overview of Lenses Agent Provisioning.

Connections are defined in the provisioning.yaml file. The Agent will watch the file and resolve the desired state, applying connections defined in the file.

When deploying via Helm the provisioning.yaml is part of the Agent Values.yaml file.

Minimum configuration

The minimum configuration needed is a configuration for Lenses HQ. Once the connection is established you can use the Lenses APIs to configure and test the remaining connections, or at start up provide the full configuration.

The minimum configuration, only connects to HQ, the Agent is not fully configured yet to connect to Kafka.

Uploading a configuration

The APIs will validate the schema and connectivity, and if validate update the file used by the Agent. They update the file provided at start up.

The file is the source of truth for connection management

Defining a Connection

Connections are defined in the provisioning.yaml. This file is divided into components, each component representing a type of connection.

Each component is mandatory:

1. Name - This is the free name of the connection

2. Version set to 1

3. Configuration - This is a list of keys/values dependent on the component type.

Example provisioning.yaml

kafka:
- name: Kafka
  version: 1
  tags: [my-tag]
  configuration:
    kafkaBootstrapServers:
      value:
        - PLAINTEXT://your.kafka.broker.0:9092
        - PLAINTEXT://your.kafka.broker.1:9092
    protocol: 
      value: PLAINTEXT
    # all metrics properties are optional
    metricsPort: 
      value: 9581
    metricsType: 
      value: JMX
    metricsSsl: 
      value: falseSSL 

IDE & JSON Schema Support

To help you create a provisioning file you can use the JSON SCHEMA support. In you IDE, like VS Code, create a file called provisioning.yaml and add the following line at the top:

# yaml-language-server: $schema=./agent/provisioning.schema-6.1.json

Then start typing, for example k for Kafka and Kafka Connect, s for schema registry or just crtl+space to trigger the default templates.

Fill in the required fields, your editor should highlight issues for you.

Managing secrets

The provisioning.yaml contains secrets. If you are deploying via Helm, the chart will use Kubernetes secrets.

Support is provided for referencing environment variables. This allows you to set secrets in your environment and resolve the value at runtime.

Escape the dollar sign

sslKeystorePassword:
  value: "\${ENV_VAR_NAME}"

Referencing files

Many connections need files, for example, to secure Kafka with SSL you will need a key store and optionally a trust store.

To reference a file in the provisioning.yaml, for example, given:

    configuration:
      protocol:
        value: SASL_SSL
      sslKeystore:
        file: "my-keystore.jks"

a file called my-keystore.jks is expected in the same directory.