Deploying an Agent

This page describes the install of the Lenses Agent via an archive on Linux.

To install the Agent from the archive you must:

1. Extract the archive

2. Configure the Agent

3. Start the Agent

---

Extracting the archive

Installation link

Link to archives can be found here: https://archive.lenses.io/lenses/6.1/agent/

Extract the archive using the following command

terminal

tar -xvf lenses-agent-latest-linux64.tar.gz -C lenses

Inside the extract archive, you will find.

   lenses
   ├── lenses.conf       ← edited and renamed from .sample
   ├── logback.xml
   ├── logback-debug.xml
   ├── bin/
   ├── lib/
   ├── licences/
   ├── logs/             ← created when you run Lenses
   ├── plugins/
   ├── storage/          ← created when you run Lenses
   └── ui/

---

Configure the Agent

To configure the agents connection to Postgres and its provisioning file. See here in the quickstart.

Once the agent files are configure you can continue to start the agent.

The configuration files are the same for docker and Linux, for docker we are simply mounting the files into the container.

Connect the Agent to HQ

You can connect the agent to HQ in two ways, all via provisioning

1. Start the Agent with the an AGENT_KEY via environment variables at minimum. You need to create an environment in HQ to get this key

2. Or provisioning file that contains the connection to HQ, recommended for TLS enabled HQs

You can still reference environment variables if you use the file, e.g

agentKey:
  value: ${LENSES_HQ_AGENT_KEY

Database

My default the Agent will start with an embedded database, if you wish to use Progress, recommended for production, see here. Database settings are set in lenses-agent.conf

---

Starting the Agent

Provisioning file path

If you configured provisioning.yaml make sure to place following property:

lenses.conf

# Directory containing the provision.yaml files
lenses.provisioning.path=/my/dir

Start Lenses by running:

terminal

bin/lenses

or pass the location of the config file:

terminal

bin/lenses lenses-agent.conf

If you do not pass the location of lenses-agent.conf, the Agent will look for it inside the current (runtime) directory. If it does not exist, it will try its installation directory.

In case agent fails with error message that security.conf does not exist and is provided just run following command under lenses directory

touch security.conf

To stop Lenses, press CTRL+C.

---

File permissions

Set the permissions of the lenses-agent.conf to be readable only by the lenses user.

chmod 0600 /path/to/lenses-agent.conf
chown [lenses-user]:root /path/to/lenses-agent.conf

The agent needs write access in 4-5 places in total:

1. [RUNTIME DIRECTORY] When the Agent runs, it will create at least one directory under the directory it is run in:

   1. [RUNTIME DIRECTORY]/logs Where logs are stored

   2. [RUNTIME DIRECTORY]/logs/sql-kstream-state Where SQL processors (when In Process mode) store state. To change the location for the processors’ state directory, use lenses.sql.state.dir option.

   3. [RUNTIME DIRECTORY]/storage Where the H2 embedded database is stored when PostgreSQL is not set. To change this directory, use the lenses.storage.directory option.

   4. /run (Global directory for temporary data at runtime) Used for temporary files. If Lenses does not have permission to use it, it will fall back to /tmp.

   5. /tmp (Global temporary directory) Used for temporary files (if access /run fails), and JNI shared libraries.

Back-up this location for disaster recovery

---

JNI libraries

The Agent and Kafka use two common Java libraries that take advantage of JNI and are extracted to /tmp.

You must either:

1. Mount /tmp without noexec

2. or set org.xerial.snappy.tempdir and java.io.tmpdir to a different location

LENSES_OPTS="-Dorg.xerial.snappy.tempdir=/path/to/exec/tmp -Djava.io.tmpdir=/path/to/exec/tmp"

---

SystemD example

If your server uses systemd as a Service Manager, then manage the Agent (start upon system boot, stop, restart). Below is a simple unit file that starts the Agent automatically on system boot.

[Unit]
Description=Run Agent service

[Service]
Restart=always
User=[LENSES-USER]
Group=[LENSES-GROUP]
LimitNOFILE=4096
WorkingDirectory=/opt/lenses
#Environment=LENSES_LOG4J_OPTS="-Dlogback.configurationFile=file:/etc/lenses/logback.xml"
ExecStart=/opt/lenses/bin/lenses /etc/lenses/lenses-agent.conf

[Install]
WantedBy=multi-user.target

---

Global Truststore

The Agent uses the default trust store (cacerts) of the system’s JRE (Java Runtime) installation. The trust store is used to verify remote servers on TLS connections, such as Kafka Brokers with an SSL protocol, JMX over TLS, and more. Whilst for some types of connections (e.g. Kafka Brokers) a separate keystore can be provided at the connection’s configuration, for some other connections (JMX over TLS) we always rely on the system trust store.

It is possible to set up a global custom trust store via the LENSES_OPTS environment variable:

export LENSES_OPTS="-Djavax.net.ssl.trustStore=/path/to/truststore.jks -Djavax.net.ssl.trustStorePassword=changeit"
bin/lenses

---

Hardware & OS

Run on any Linux server (review ulimits or container technology (docker/kubernetes). For RHEL 6.x and CentOS 6.x use docker.

Linux machines typically have a soft limit of 1024 open file descriptors. Check your current limit with the ulimit command:

ulimit -S -n     # soft limit
ulimit -H -n     # hard limit

Increase as a super-user the soft limit to 4096 with:

ulimit -S -n 4096

Use 8GB RAM /4 CPUs and 20GB disk space.