Linux

This page describes install the Lenses via a Linux archive

https://archive.lenses.io/lenses/archive.lenses.io

Lenses archive

On start-up, Lenses will be in bootstrap mode unless it has an existing Kafka Connection. See provisioning for automating.

To install Lenses from the archive you must:

1. Extract the archive

2. Configure Lenses

3. Start Lenses

Extracting the archive

Extract the archive using the following command

tar -xvf lenses.tar.gz -C lenses

Inside the extract archive, you will find.

   lenses
   ├── lenses.conf       ← edited and renamed from .sample
   ├── security.conf     ← edited and renamed from .sample
   ├── license.json
   ├── logback.xml
   ├── logback-debug.xml
   ├── bin/
   ├── lib/
   ├── licences/
   ├── logs/             ← created when you run Lenses
   ├── plugins/
   ├── storage/          ← created when you run Lenses
   └── ui/

Starting Lenses

Start Lenses by running:

bin/lenses

or pass the location of the config file:

bin/lenses lenses.conf

If you do not pass the location of the config file, Lenses will look for it inside the current (runtime) directory. If it does not exist, it will try its installation directory.

To stop Lenses, press CTRL+C.

Open Lenses in your browser, log in with admin/admin configure your brokers and add your license.

File permissions

Set the permissions of the security.conf to be readable only by the lenses user.

chmod 0600 /path/to/security.conf
chown [lenses-user]:root /path/to/security.conf

The agent needs write access in 4-5 places in total:

1. [RUNTIME DIRECTORY] When Lenses runs, it will create at least one directory under the directory it is run in:

   1. [RUNTIME DIRECTORY]/logs Where logs are stored

   2. [RUNTIME DIRECTORY]/logs/lenses-sql-kstream-state Where SQL processors (when In Process mode) store state. To change the location for the processors’ state directory, use lenses.sql.state.dir option.

   3. [RUNTIME DIRECTORY]/storage Where the H2 embedded database is stored when PostgreSQL is not set. To change this directory, use the lenses.storage.directory option.

   4. /run (Global directory for temporary data at runtime) Used for temporary files. If Lenses does not have permission to use it, it will fall back to /tmp.

   5. /tmp (Global temporary directory) Used for temporary files (if access /run fails), and JNI shared libraries.

Back-up this location for disaster recovery

JNI libraries

Lenses and Kafka use two common Java libraries that take advantage of JNI and are extracted to /tmp.

You must either:

1. Mount /tmp without noexec

2. or set org.xerial.snappy.tempdir and java.io.tmpdir to a different location

LENSES_OPTS="-Dorg.xerial.snappy.tempdir=/path/to/exec/tmp -Djava.io.tmpdir=/path/to/exec/tmp"

SystemD example

If your server uses systemd as a Service Manager, then manage Lenses (start upon system boot, stop, restart). Below is a simple unit file that starts Lenses automatically on system boot.

[Unit]
Description=Run Lenses.io service

[Service]
Restart=always
User=[LENSES-USER]
Group=[LENSES-GROUP]
LimitNOFILE=4096
WorkingDirectory=/opt/lenses
#Environment=LENSES_LOG4J_OPTS="-Dlogback.configurationFile=file:/etc/lenses/logback.xml"
ExecStart=/opt/lenses/bin/lenses /etc/lenses/lenses.conf

[Install]
WantedBy=multi-user.target

Global Truststore

Lenses uses the default trust store (cacerts) of the system’s JRE (Java Runtime) installation. The trust store is used to verify remote servers on TLS connections, such as Kafka Brokers with an SSL protocol, Secure LDAP, JMX over TLS, and more. Whilst for some types of connections (e.g. Kafka Brokers) a separate keystore can be provided at the connection’s configuration, for some other connections (e.g. Secure LDAP and JMX over TLS) we always rely on the system trust store.

It is possible to set up a global custom trust store via the LENSES_OPTS environment variable:

export LENSES_OPTS="-Djavax.net.ssl.trustStore=/path/to/truststore.jks -Djavax.net.ssl.trustStorePassword=changeit"
bin/lenses

Hardware & OS

Run on any Linux server. For RHEL 6.x and CentOS 6.x use docker.

Linux machines typically have a soft limit of 1024 open file descriptors. Check your current limit with the ulimit command:

ulimit -S -n     # soft limit
ulimit -H -n     # hard limit

Increase as a super-user the soft limit to 4096 with:

ulimit -S -n 4096

Use 6GB RAM/4 CPUs and 500MB disk space.