Upgrade notes

Lenses 5.5 is an incremental release which brings in new features and improvements.

Upgrading from 5.0 or later does not require any static configuration change but if you have automated the creation of any AWS connection, then you will have to adjust the provisioning section of your Helm chart, or your CICD, or —if you use the API directly— your API calls.

If you are upgrading from version 4.3 or older, you need to follow the upgrade procedure for Lenses 5.0 as well as the rest of the instructions that follow.

Breaking Changes and Caution Items 

Lenses upgrades (except patch releases) are not backwards compatible. It is best practice to take a backup of the Lenses database before an upgrade.

New provisioning API [caution] 

With Lenses 5.3 the provisioning API was introduced. This new API can be used to create or update the connections landscape. The old provisioning methods could only create the connection landscape (first run).

What this means, is that now the Helm chart or a CICD process can be used to manage Lenses’ connections.

For teams that are on the old provisioning method some adjustments are required to their Helm charts or other provisioning code to switch to the new API. The old methods are still available but are considered deprecated and will be removed or break in the future.

Some resources that can come handy for the switch are:

AWS and Glue Connection provisioning [breaking] 

With Lenses 5.4 IAM support was added for the AWS connection type. An AWS connection is used as an authentication provider for the Glue Schema Registry and Cloudwatch channels.

Due to this change, if you create or manage your AWS and Glue connections via the API or provisioning, you need to update your configuration to the new format.

Action required 

  • Add the new authMode property to your connections for AWS and Glue Schema Registry.

Details 

  • Lenses 5.4 adds a new required property for the AWS and Glue Schema Registry connections.
  • The property is authMode.
  • It controls how Lenses authenticates with AWS:
    • Access keys (existing feature).
    • Credentials provider chain (new feature).
  • You set the property either with the:
    • Connections API - create, update.
    • Provision YAML.

You can set authMode in 2 modes:

1. Access keys mode 

This is the existing mode where Lenses uses AWS access keys.

  • Set the authMode to Access Key.
  • Specify the access key ID and secret access key, as you had before.
{
  "authMode": { "value": "Access Key" },
  "accessKeyId": { "value": "yourAccessKeyId" },
  "secretAccessKey": { "value": "yourSecretAccessKey" }
}
2. Credentials provider chain mode (new) 

This is the new mode where Lenses uses the AWS default credentials provider chain.

  • Set the authMode to Credentials Chain.
  • No additional properties needed.
{
  "authMode": { "value": "Credentials Chain" }
}

Examples - Provision YAML 

1. Access mode 
aws:
  - name: my-aws-connection
    version: 1
    tags: [dev]
    configuration:
      authMode:
        value: Access Key
      accessKeyId:
        value: yourAccessKeyId
      secretAccessKey:
        value: yourSecretAccessKey

glueSchemaRegistry:
  - name: schema-registry
    version: 1
    tags: [dev]
    configuration:
      authMode:
        reference: my-aws-connection
      glueRegistryArn:
        value: arn:aws:glue:region:account:registry/registry-name
      accessKeyId:
        reference: my-aws-connection
      secretAccessKey:
        reference: my-aws-connection
2. Credentials provider chain mode 
aws:
  - name: my-aws-connection
    version: 1
    tags: [dev]
    configuration:
      authMode: 
        value: Credentials Chain

glueSchemaRegistry:
  - name: schema-registry
    version: 1
    tags: [dev]
    configuration:
      authMode:
        reference: my-aws-connection
      glueRegistryArn:
        value: arn:aws:glue:region:account:registry/registry-name

Examples - API JSON 

1. Access mode 
AWS connection 
{
 "name": "my-aws-connection",
 "tags": ["dev"],
 "templateName": "AWS",
 "configuration": {
   "authMode": { "value": "Access Key" },
   "accessKeyId": { "value": "yourAccessKeyId" },
   "secretAccessKey": { "value": "yourSecretAccessKey" }
 }
}
Glue Schema Registry connection 
{
    "name":"schema-registry",
    "tags": ["dev"],
    "templateName":"AWSGlueSchemaRegistry",
    "configuration": {
        "authMode": {"reference":"my-aws-connection"},
        "accessKeyId": {"reference":"my-aws-connection"},
        "secretAccessKey": {"reference":"my-aws-connection"},
        "glueRegistryArn":{"value":"arn:aws:glue:region:account:registry/registry-name"}
    }
}
2. Credentials provider chain mode 
AWS connection 
{
  "name": "my-aws-connection",
  "tags": ["dev"],
  "templateName": "AWS",
  "configuration": {
    "authMode": { "value": "Credentials Chain" }
  }
}
Glue Schema Registry connection 
{
    "name":"schema-registry",
    "tags": ["dev"],
    "templateName":"AWSGlueSchemaRegistry",
    "configuration": {
        "authMode": {"reference":"my-aws-connection"},
        "glueRegistryArn":{"value":"arn:aws:glue:region:account:registry/registry-name"}
    }
}

Docker image base change 

Starting with Lenses 5.2 the base image of Lenses and SQL Processor Dockers switched from Debian to Ubuntu. On some older systems, these docker images will fail to run, due to a combination of a recent glibc in the container, and older docker daemon on the host.

If you fall under this category, during the startup of the Lenses container, you might see errors such as Unable to identify system. Uname is required or [warning][os,thread] Failed to start thread “GC Thread#0”.

For these cases, we now offer Lenses docker images with the suffix -debian in their tags. E.g:

  • lensesio/lenses:5.5-debian
  • lensesio/lenses:5.5.0-debian
  • lensesio/lenses:latest-debian

If your host is running on an older operating system and you encounter these errors, try to use the debian equivalent tag.

Update Process 

Using the Lenses Archive 

Download the latest 5.5 archive and extract it in a new directory on your server. It is important to avoid extracting an archive over an older installation to avoid having multiple versions of libraries. Instead, you should remove (or rename) the old directory, then move the new into its place. Copy if needed and update your lenses.conf and security.conf files. If you are using the internal database instead of PostgreSQL, make sure Lenses Storage Directory (lenses.storage.directory) is kept intact. The folder is where persistent data is stored, such as users, groups, audits, data policies, connections, and more.

Make sure you have a JRE (or JDK) installed in the server running Lenses. Lenses can run on JRE 8 or greater, and the recommended version is JRE 11.

Using the Lenses Docker 

The docker image uses tags to distinguish between versions. The latest tag (lensesio/lenses:latest) brings the latest stable version of Lenses. There are minor tags to help users get the latest patch in a minor version (e.g 5.5, 5.1) and patch tags to help users pin to a specific patch (e.g 5.5.1, 5.1.2). The best practice advice is to use the minor tag (lensesio/lenses:5.5), which ensures that your installation will always get compatible updates until you made a conscious decision to upgrade the minor version.

If you use the internal database instead of PostgreSQL as the backing store of Lenses, make sure you keep the /data/storage volume to not lose your data. Other volumes supported by the docker are /data/kafka-streams-state which holds state for SQL Processors running IN-PROC and may have to be rebuilt (automatically) if lost, /data/log (log files on disk), /data/plugins (custom UDFs).

Pull the 5.5 docker:

docker pull lensesio/lenses:5.5

Stop your current container and restart with the 5.5 image, mounting any volumes you might need.

Lenses Box 

If you are a Box user, pull the latest version, preserve your /data volume and restart Lenses:

docker pull lensesio/box:5.5
docker stop [CURRENT BOX NAME or ID]
docker run -v /path/to/box/data:/data -e EULA="..." -p 3030 lensesio/box:5.5

Helm 

Download the latest charts and update your values.yaml as described below. Remember that if you are using the internal database instead of PostgreSQL as the backing store, then the Lenses Storage Directory should be stored in a persistent volume and be kept intact between updates. To support a potential downgrade, make sure this volume is backed-up before installing a newer version of Lenses.

helm repo add lensesio https://helm.repo.lenses.io/
helm repo update

If you have provisioning enabled (lenses.provision.enabled: true) in your values.yaml, and you are on provision version “1” then you have to act. Version “1” means either that lenses.provision.version is set to "1", or it is not set at all. You have two options:

  • Disable it, as Lenses already has all the information stored in the database, and version “1” does not support updating the connections and license.
    lenses:
  provision:
    enabled: false
  • Switch to provisioning version “2” which supports updating connections and licenses every time you do a helm upgrade. To do that, you must make some changes to your old provisioning section. Some resources that can come handy for the switch are:

If you don’t have your values.yaml you can download it from the Kubernetes cluster using Helm:

helm get values --namespace [LENSES_NAMESPACE] \
     --output yaml [LENSES_DEPLOYMENT] > values.yaml

Proceed to upgrade:

helm upgrade --namespace [LENSES_NAMESPACE] --values values.yaml [LENSES_DEPLOYMENT]

Alternatively, reusing the old values and turning provisioning off:

helm upgrade --namespace [LENSES_NAMESPACE] --reuse-values \
     --set lenses.provision.enabled=false [LENSES_DEPLOYMENT]

Cloud Installations 

Use the latest version available in the marketplaces. Remember that Lenses Storage Directory should be provided as a persistent volume and be kept intact between updates. If a new image does not exist, you may be able to update Lenses in-place. Our support team will be happy to go through the available options with you.

Static Configuration Changes 

No changes.

--
Last modified: April 30, 2024