Upgrade Notes

Upgrading from an older version of Lenses, requires some changes to the existing configuration files. Please familiriaze with the changes described below before performing an upgrade.

Some of the changes are optional currently, as both the old and the new configuration keys are supported. It is still advised to switch to the new layout as the old one may be deprecated in the future. Where the changes are mandatory, it will be explicitly stated.

For the full list of changes made for this release please check the Release Notes.

Note

Please remember that Lenses 2.2 introduced state on disk for the Data Policies module. If you use Data Policies, you have to keep your lenses.storage.directory in order for your Data Policies to transfer to 2.3.

From 2.2 to 2.3

This release is focused on improved performance, better cloud support, improving on the SQL engine and supporting more metrics configurations (i.e secured JMX, Jolokia agent). As a result, the metrics options have changed to support the new functionality. The old configuration for JMX is still supported but it may be deprecated in the future:

• Configuration entry lenses.sql.max.time is now lenses.sql.settings.max.query.time. If you don’t use the default value, it is mandatory to use the new option.

• All the configuration keys starting with lenses.interval.jmx have been renamed to lenses.interval.metrics. The change is mandatory.

• The com.landoop.kafka.lenses.connect.SimpleTopicsExtractor class, used when adding custom connectors to the topology view, was renamed to io.lenses.core.kafka.connect.SimpleTopicsExtractor. The change is mandatory.

• The Kafka broker configuration in case you setup the metrics manually. Here is the change to make:

  #Old layout
  lenses.jmx.brokers = [
      { id:"broker1 id", jmx:"broker1 JMX port" },
      { id:"broker2 id", jmx:"broker2 JMX port" },
      { id:"broker3 id", jmx:"broker3 JMX port" }
  ]
  
   #New layout
  lenses.kafka.metrics.type = JMX
  lenses.kafka.metrics.port= [
      { id:"broker1 id", port:"broker1 JMX port" },
      { id:"broker2 id", port:"broker2 JMX port" },
      { id:"broker3 id", port:"broker3 JMX port" }
  ]
  

• The new Zookeeper configuration. Here is the change to make:

  #Old layout
  lenses.zookeeper.hosts=[
    {url:"zk1:2181", jmx:"zk1:9010"},
    {url:"zk2:2181", jmx:"zk2:9010"},
    {url:"zk3:2181", jmx:"zk3:9010"},
  ]
  
  #New layout
  lenses.zookeeper.hosts = [
    {
      url: "zk1:2181",
      metrics: {
        url:"zk1:9010",
        type:"JMX"
      }
    },
    {
      url: "zk2:2181"
      metrics: {
        url:"zk2:9010",
        type:"JMX"
      }
    },
    {
      url: "zk3:2181"
      metrics: {
        url:"zk3:9010",
        type:"JMX"
      }
    }
  ]
  

• The Schema Registry configuration. Here is the change to make:

  #Old layout
  lenses.schema.registry.urls=[
    {
      url:"http://sr1:18081",
      jmx:"sr1:19395"
    },
    {
      url:"http://sr2:18081",
      jmx:"sr2:19395"
    }
  ]
  
  #New layout
  lenses.schema.registry.urls = [
    {
      url: "http://sr1:18081",
      metrics: {
        url: "sr1:19682"
        type: "JMX"
      }
    },
    {
      url: "http://sr2:28081",
      metrics: {
        url: "sr2:29682"
        type: "JMX"
      }
    }
  ]
  

• The Kafka Connect configuration. Here is the change to make:

  #Old layout
  lenses.connect.clusters = [
    {
      name: "CDH_2",
      urls: [
        {
          url:"http://worker1:18095",
          jmx:"worker1:19497"
        },
        {
          url:"http://worker2:18095",
          jmx:"worker2:19497"
        },
        {
          url:"http://worker3:18095",
          jmx:"worker3:19497"
        }
      ],
      statuses: "connect-status-2-cdh",
      configs: "connect-configs-2-cdh",
      offsets: "connect-offsets-2-cdh"
    }
  ]
  
  #New layout
  lenses.kafka.connect.clusters = [
    {
      name: "C1",
      urls: [
        {
          url:"http://worker1:18083",
          metrics: {
            url: "worker1:19684"
            type: "JMX"
          }
        },
        {
          url:"http://worker2:28083",
          metrics: {
            url: "worker2:29684"
            type: "JMX"
          }
        }
      ],
      statuses: "connect-status-2-cdh",
      configs: "connect-configs-2-cdh",
      offsets: "connect-offsets-2-cdh"
    }
  ]
  

Docker Containers

As usual the latest tag brings the latest Lenses versions. There are also tagged releases major (2.3, 2.2) and even minor (2.3.0, 2.2.9) tags to help you better manage your upgrade path.

Docker pull the latest Lenses environment for Developers

docker pull landoop/kafka-lenses-dev:latest

And update you official docker container for Lenses Enterprise with

docker pull landoop/lenses:latest

SSL and SASL for Kubernetes SQL runners

Run the script Kubernetes-relabel-2.3 located in the bin folder of your Lenses 2.3 installation. This will annotate existing pods and deployments with lenses.io/secret.kafka=lenses-sasl-secret

From 2.3.5 to 2.3.6

As a consequence of the security upgrades, users should clear their browser cache.