Deploying an Agent

This page describes deploying an Lenses Agent via Docker.

The Agent docker image can be configured via environment variables or via volume mounts for the configuration files.

Connect the Agent to HQ

You can connect the agent to HQ in two ways, all via provisioning

1. Start the Agent docker with the an AGENT_KEY via environment variables at minimum. You need to create an environment in HQ to get this key
2. Or mount a provisioning file that contains the connection to HQ, recommended for TLS enabled HQs

You can still reference environment variables if you mount the file, e.g

agentKey:
  value: ${LENSES_HQ_AGENT_KEY

Minimal Start

First deploy HQ and create an environment, then with the AGENT KEY run:

docker run \                                                                          
  --name "xxx" \
  --network=lenses \
  --restart=unless-stopped \
  -e PROVISION_AGENT_KEY=YOUR_AGENT_KEY \
  -e PROVISION_HQ_URL=YOUR_LENSES_HQ_URL \
  lensesio/lenses-agent:6.1 

This will start and connect to HQ but not to Kafka or other services. It will create a provisioning file in data/provisioning.

Set the docker network accordingly

Start and mount provisioning

terminal

docker run --name lenses-agent \
-v $(pwd)/provisioning.yaml:/mnt/provision-secrets/provisioning.yaml \
-e LENSES_PROVISIONING_PATH=/mnt/provision-secrets \
lensesio/lenses-agent:6.1

Example provisioning files:

TLS Enabled

provisioning.yaml

lensesHq:
  - name: lenses-hq
    version: 1
    tags: ['hq']
    configuration:
      server:
        value: [LENSES_HQ_URL]
      port:
        value: 10000
      agentKey:
        value: ${LENSES_HQ_AGENT_KEY}
      sslEnabled:
        value: true
      sslTruststore:
        file: "hq-truststore.jks"
      sslTruststorePassword:
        value: ${LENSES_HQ_AGENT_TRUSTSTORE_PWD}

TLS Disabled

Agent key reference

Agent key within provisioning.yaml can be referenced as a:

• environment variable shown in example above
• inline string

Database

My default the Agent will start with an embedded database, if you wish to use Progress, recommended for production, see here. Database settings are set in lenses-agent.conf

Environment Variables

Environment variables prefixed with LENSES_ are transformed into corresponding configuration options. The environment variable name is converted to lowercase and underscores (_) are replaced with dots (.). As an example set the option lenses.port use the environment variable LENSES_PORT.

Alternatively, the lenses-agent.conf can be mounted directly as

• /mnt/settings/lenses-agent.conf

---

Docker volumes

The Docker image exposes four volumes in total, where cache, logs, plugins, and persistent data are stored:

• /data/storage
• /data/plugins
• /data/logs
• /data/kafka-streams-state

Storage volume

Resides under /data/storage and is used to store persistent data, such as Data Policies. For this data to survive between Docker runs and/or Agent upgrades, the volume must be managed externally (persistent volume).

Plugins volume

Resides under /data/plugins it’s where classes that extend Agent may be added —such as custom Serdes, LDAP filters, UDFs for the Lenses SQL table engine, and custom_http implementations.

Logs volume

Resides under /data/logs, logs are stored here. The application also logs to stdout, so the log files aren’t needed for most cases.

KStreams state volume

Resides under /data/kafka-streams-state, used when Lenses SQL is in IN_PROC configuration. In such a case, Lenses uses this scratch directory to cache Lenses SQL internal state. Whilst this directory can safely be removed, it can be beneficial to keep it around, so the Processors won’t have to rebuild their state during a restart.

---

Agent TLS and Global JVM Trust Store

By default, the the serves connections over plaintext (HTTP). It is possible to use TLS instead. The Docker image offers the ability to provide the content for extra files via secrets mounted as files or as environment variables. Especially for SSL, Docker supports SSL/TLS keys and certificates in Java Keystore (JKS) formats.

This capability is optional, and users can mount such files under custom paths and configure lenses-agent.conf manually via environment variables, or lenses.append.conf.

There are two ways to use the File/Variable names of the table below.

1. Create a file with the appropriate filename as listed below and mount it under /mnt/settings, /mnt/secrets, or /run/secrets
2. Set them as environment variables.

All settings except for passwords, can be optionally encoded in base64. The docker will detect such encoding automatically.

File / Variable Name	Description
FILECONTENT_JVM_SSL_TRUSTSTORE	The SSL/TLS trust store to use as the global JVM trust store. Add to LENSES_OPTS the property javax.net.ssl.trustStore
FILECONTENT_JVM_SSL_TRUSTSTORE_PASSWORD	Τhe trust store password. If set, the startup script will add automatically to LENSESOPTS the property javax.net.ssl.trustStorePassword (**_base64 not supported**)
FILECONTENT_LENSES_SSL_KEYSTORE	The SSL/TLS keystore to use for the TLS listener for the Agent

---

Process UID/GUI

The docker does not require running as root. The default user is set to root for convenience and to verify upon start-up that all the directories and files have the correct permissions. The user drops to nobody and group nogroup (65534:65534) before starting the Agent.

If the image is started without root privileges, the agent will start successfully using the effective uid:gid applied. Ensure any volumes mounted (i.e., for the settings and data) have the correct permission set.