Deploying an Agent

The Agent docker image can be configured via environment variables or via volume mounts for the configuration files.

Running the Docker

docker run --name lenses-agent \
-v $(pwd)/provisioning.yaml:/mnt/provision-secrets/provisioning.yaml \
-v $(pwd)/lenses.conf:/data/lenses.conf \
-e LENSES_PROVISIONING_PATH=/mnt/provision-secrets \
public.ecr.aws/q8a6e1s5/public-agent:v6.0.0-alpha.0-3-g7d5519a7

File configuration

In order for command above provisioning yaml has to be configured.

There are two mandatory connections:

1. HQ which requires AgentKey (LENSESHQ_AGENT_KEY) - this key is being created once user registers "New environment" in HQ;

2. Kafka connection

lensesHq:
- configuration:
    agentKey:
      value: ${LENSESHQ_AGENT_KEY}
    port:
      value: 10000
    server:
      value: hq
  name: lenses-hq
  tags:
  - hq
  version: 1

kafka:
- configuration:
   kafkaBootstrapServers:
      value:
      - PLAINTEXT://kafka:9092
   protocol:
      value: PLAINTEXT
  name: kafka
  tags:
  - demo
  version: 1

Environment Variables

Environment variables prefixed with LENSES_ are transformed into corresponding configuration options. The environment variable name is converted to lowercase and underscores (_) are replaced with dots (.). As an example set the option lenses.port use the environment variable LENSES_PORT.

Alternatively, the lenses.conf can be mounted directly as

• /mnt/settings/lenses.conf

Docker volumes

The Docker image exposes four volumes in total, where cache, logs, plugins, and persistent data are stored:

• /data/storage

• /data/plugins

• /data/logs

• /data/kafka-streams-state

Storage volume

Resides under /data/storage and is used to store persistent data, such as Data Policies. For this data to survive between Docker runs and/or Agent upgrades, the volume must be managed externally (persistent volume).

Plugins volume

Resides under /data/plugins it’s where classes that extend Agent may be added —such as custom Serdes, LDAP filters, UDFs for the Lenses SQL table engine, and custom_http implementations.

Logs volume

Resides under /data/logs, logs are stored here. The application also logs to stdout, so the log files aren’t needed for most cases.

KStreams state volume

Resides under /data/kafka-streams-state, used when Lenses SQL is in IN_PROC configuration. In such a case, Lenses uses this scratch directory to cache Lenses SQL internal state. Whilst this directory can safely be removed, it can be beneficial to keep it around, so the Processors won’t have to rebuild their state during a restart.

Agent TLS and Global JVM Trust Store

By default, the the serves connections over plaintext (HTTP). It is possible to use TLS instead. The Docker image offers the ability to provide the content for extra files via secrets mounted as files or as environment variables. Especially for SSL, Docker supports SSL/TLS keys and certificates in Java Keystore (JKS) formats.

This capability is optional, and users can mount such files under custom paths and configure lenses.conf manually via environment variables, or lenses.append.conf.

There are two ways to use the File/Variable names of the table below.

1. Create a file with the appropriate filename as listed below and mount it under /mnt/settings, /mnt/secrets, or /run/secrets

2. Set them as environment variables.

All settings except for passwords, can be optionally encoded in base64. The docker will detect such encoding automatically.

Process UID/GUI

The docker does not require running as root. The default user is set to root for convenience and to verify upon start-up that all the directories and files have the correct permissions. The user drops to nobody and group nogroup (65534:65534) before starting the Agent.

If the image is started without root privileges, the agent will start successfully using the effective uid:gid applied. Ensure any volumes mounted (i.e., for the license, settings, and data) have the correct permission set.