4.1

Docker


View on GitHub


docker pull lensesio/lenses

Lenses docker image can be configured via environment variables, or via volume mounts for the configuration files (lenses.conf, security.conf).

Environment variables prefixed with LENSES_ are transformed into corresponding configuration options. The environment variable name is converted to lowercase and undescores (_) are replaced with dots (.). As an example to set the option lenses.port use the environment variable LENSES_PORT.

Alternatively, the lenses.conf and security.conf can be mounted directly as

  • /mnt/settings/lenses.conf
  • /mnt/secrets/security.conf

License file 

The license file may be provided to the Docker image via three methods:

  • As a file, mounted at /license.json or /mnt/secrets/license.json
  • As the contents of the environment variable LICENSE
  • As a URL resource that will download on container startup via LICENSE_URL

Example of mounting a license file from host to docker.
version: '3'
services:
  lenses:
    image: "lensesio/lenses:4.1"
    container_name: "lenses"
    ports:
      - 9991:9991
    volumes:
      - /path/in/host/license.json:/mnt/secrets/license.json

Docker volumes 

The Docker image exposes four volumes in total, where cache, logs, plugins and persistent data are stored:

  • /data/storage
  • /data/plugins
  • /data/logs
  • /data/kafka-streams-state.

Storage volume 

Resides under /data/storage and is used to store persistent data, such as Data Policies. For this data to survive between Docker runs and/or Lenses upgrades, the volume must be managed externally (persistent volume).

Plugins volume 

Resides under /data/plugins it’s where classes that extend Lenses may be added —such as custom serde, LDAP filters, UDFs for the Lenses SQL table engine and custom_http implementations. Learn more about plugins.

Logs volume 

Resides under /data/logs, logs are stored here. The application also logs to stdout, so for most cases, the log files aren’t needed. Learn more about logs.

KStreams state volume 

Resides under /data/kafka-streams-state, used when Lenses SQL is in IN_PROC configuration. In such a case, Lenses takes advantage of this scratch directory to cache Lenses SQL internal state. Whilst this directory can safely be removed, it can be beneficial to keep it around, so the Processors won’t have to rebuild their state during a restart.

Kerberos and TLS 

Java keystore files, JAAS configuration, and Kerberos keytabs can be passed to Lenses Docker under specific paths or as environment variables. These inputs will be used during the automatic creation of lenses.conf and simplify the configuration process. Furthermore key and certificates in PEM format can be used and will be converted to keystore files.

This capability is optional and users can mount such files under custom paths and configure lenses.conf manually via environment variables, or lenses.append.conf.

There are two ways to use the File/Variable names of the table below.

  1. Create a file with the appropriate filename as listed bellow mount it under /mnt/settings, /mnt/secrets, or /run/secrets, or
  2. Set them as environment variables.

All settings with the exception of passwords, can be optionally encoded in base64. The docker will detect such encoding automatically.

File / Variable NameDescription
FILECONTENT_JVM_SSL_TRUSTSTOREThe SSL/TLS trust store to use as the global JVM trust store.
Add to LENSES_OPTS the property javax.net.ssl.trustStore
FILECONTENT_JVM_SSL_TRUSTSTORE_PASSWORDΤhe trust store password. If set, the startup script will add automatically to LENSES_OPTS the property javax.net.ssl.trustStorePassword (base64 not supported)
FILECONTENT_LENSES_SSL_KEYSTOREThe SSL/TLS keystore to use for the TLS listener for Lenses
FILECONTENT_LENSES_SSL_KEY_PEMThe SSL private key in PEM format for the TLS Listener for Lenses
FILECONTENT_LENSES_SSL_CERT_PEMThe SSL certificate in PEM format for the TLS Listener for Lenses
FILECONTENT_SSL_KEYSTOREThe SSL/TLS keystore to use for connecting to brokers. Please set the passwords via env vars.
FILECONTENT_SSL_TRUSTSTOREThe SSL/TLS truststore to use for verifying SSL connections to the brokers. Please set the password via env vars.
FILECONTENT_SSL_KEY_PEMThe client SSL key in PEM format to use for connecting to brokers that require authentication via SSL
FILECONTENT_SSL_CERT_PEMThe client SSL certificate in PEM format to use for connecting to brokers that require authentication via SSL.
FILECONTENT_SSL_CACERT_PEMThe CA certificate in PEM format to use for verifying SSL connections to the brokers
FILECONTENT_KEYTABThe Kerberos keytab to use for authenticating with a KDC in order to connect to brokers that require SASL/GSSAPI authentication
FILECONTENT_JAASThe JAAS Login Configuration File for setting the Kerberos (SASL/GSSAPI) authentication to the brokers
FILECONTENT_SECURITY_KEYTABThe Kerberos keytab to use for authenticating with a KDC in order to provide SPNEGO authentication
FILECONTENT_KRB5Kerberos krb5 file
FILECONTENT_SCHEMAREGISTRY_SSL_KEYSTOREThe SSL/TLS keystore to use for connecting to registry. Please set the passwords via env vars.
FILECONTENT_SCHEMAREGISTRY_SSL_TRUSTSTOREThe SSL/TLS truststore to use for verifying SSL connections to the registry. Please set the password via env vars.
FILECONTENT_SCHEMAREGISTRY_SSL_KEY_PEMThe client SSL key in PEM format to use for connecting to registry that require authentication via SSL
FILECONTENT_SCHEMAREGISTRY_SSL_CERT_PEMThe client SSL certificate in PEM format to use for connecting to registry that require authentication via SSL.
FILECONTENT_SCHEMAREGISTRY_SSL_CACERT_PEMThe CA certificate in PEM format to use for verifying SSL connections to the registry

Process UID/GUI 

The docker does not require running as root. The default user is set to root for convenience and to verify upon start up that all the directories and files have the correct permissions. The user drops to nobody and group nogroup (65534:65534) before starting Lenses.

If the image is started without root privileges, Lenses will start successfully using the effective uid:gid applied. Make sure any volumes mounted (i.e. for license, settings, data) have the correct permission set.

Docker compose example 

version: '2'
services:
  lenses:
    image: lensesio/lenses:4.1
    environment:
      LENSES_PORT: 9991
      LENSES_KAFKA_BROKERS: "PLAINTEXT://broker.1.url:9092,PLAINTEXT://broker.2.url:9092"
      LENSES_ZOOKEEPER_HOSTS: |
        [
          {url:"zookeeper.1.url:2181", jmx:"zookeeper.1.url:9585"},
          {url:"zookeeper.2.url:2181", jmx:"zookeeper.2.url:9585"}
        ]        
      LENSES_SCHEMA_REGISTRY_URLS: |
        [
          {url:"http://schema.registry.1.url:8081",jmx:"schema.registry.1.url:9582"},
          {url:"http://schema.registry.2.url:8081",jmx:"schema.registry.2.url:9582"}
        ]        
      LENSES_KAFKA_CONNECT_CLUSTERS: |
        [
          {
            name:"data_science",
            urls: [
              {url:"http://connect.worker.1.url:8083",jmx:"connect.worker.1.url:9584"},
              {url:"http://connect.worker.2.url:8083",jmx:"connect.worker.2.url:9584"}
            ],
            statuses:"connect-statuses-cluster-a",
            configs:"connect-configs-cluster-a",
            offsets:"connect-offsets-cluster-a"
          }
        ]        
      LENSES_SECURITY_USER: admin
      LENSES_SECURITY_PASSWORD: sha256:8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
    ports:
      - 9991:9991
      - 9102:9102
    volumes:
      - ./license.json:/license.json
    network_mode: host