Upgrade Notes¶
Upgrading from an older version of Lenses, requires some changes to the existing configuration files. Please familiarize with the changes described below before performing an upgrade.
For the full list of changes made for this release please check the Release Notes.
Note
Please remember that Lenses 2.2 introduced state on disk for the Data Policies module. You have to keep your Lenses Storage Directory in order for your Data Policies to transfer to 3.0. Furthermore, in 3.0 user management, audits, and alerts have been migrated to the Lenses Storage Directory, so you must make sure you persist these data in order to be able to retain them between upgrades in the future.
From 2.3 to 3.0¶
Application Update¶
Archive¶
Download the latest archive and
extract it in a new directory. It is important to avoid extracting an archive over
an older installation. You should instead remove (or rename) the old
directory, then move the new into its place. Copy if needed and update your
lenses.conf
and security.conf
files as described later in the
current page (Configuration Changes). Last make sure you keep
the Lenses Storage Directory intact
as this is where persistent data is stored. Up to Lenses 2.3 Data Policies
are stored there. Starting with 3.0, users, groups, audits and other data
is stored as well.
Docker¶
As usual the latest tag brings the latest Lenses versions. There are also tagged releases major (3.0, 2.3) and even minor (3.0.0, 2.3.6) tags to help you better manage your upgrade path.
Starting with 3.0, the landoop/kafka-lenses-dev
image can also be accessed
via lensesio/box
. The landoop/lenses
image can also be accessed via the
lensesio/lenses
name. In the future the old names will be deprecated, thus
it’s advised to make the switch to the new images earlier rather than later.
Docker pull the latest Lenses environment for Developers:
docker pull lensesio/box:latest
Or update you docker container for Lenses Enterprise with:
docker pull lensesio/lenses:latest
Follow the Configuration Changes section to update your settings if needed. Remember that Lenses Storage Directory should be stored in a docker volume and be kept intact between updates.
Helm¶
Download the latest charts and update your values.yaml
as described in
Configuration Changes and Helm Charts Changes sections.
Remember that Lenses Storage Directory
should be stored in a persistent volume and be kept intact between updates.
Cloud Installations¶
Use the latest version available in the marketplaces. Remember that Lenses Storage Directory should be provided as a persistent volume and be kept intact between updates.
SQL Processors¶
For kubernetes no action is required. New processors will use automatically the latest image. Old processors will keep working with the old image.
For Kafka Connect, please download the latest version of the connector and add it to your connect nodes whilst removing the old version. Your nodes need to be restarted for the change to take effect. A rolling restart is advised.
Configuration Changes¶
User and Group management logic became dynamic and moved into the Lenses API. You can manage users and groups via the Lenses web interface or the lenses-cli application. What this means for operators, is that the
security.conf
file is no longer used to setup user acounts for the BASIC authentication module, neither groups or user-group mappings.- If you are using the BASIC authentication module you must recreate your users via the web interface or the command line.
- You must recreate your groups via the Web Interface or the command line. This is also a great chance to familiarize with the new, fine grained, permission system.
- If you are using the Kerberos (SPNEGO) authentication module you must add your principals and link them to groups via the Web Interface or the command line.
- If you are using service accounts you must recreate them via the web interface or the command line.
Check User Management and Security Configuration to familiarize with the changes.
Note
This version comes with a default administrator
user. Remember to secure your setup with a custom
username and password for this user. Use SHA256 to protect the password
inside security.conf
.
In 3.0 the Alerting Subsystem became pluggable. Alertmanager integration is supported by the included Alertmanager Plugin. If you are using AM, the settings have to be migrated. Old settings that may remain in the configuration file will not affect the operation of Lenses. An example of migrating AM settings:
# Old layout lenses.alert.manager.endpoints = "http://ALERTMANAGER_HOST_1:9094,http://ALERTMANAGER_HOST_2:9094,..." lenses.alert.manager.source = "Lenses" lenses.alert.manager.generator.url = "http://LENSES_HOST:LENSES_PORT" # New layout lenses.alert.plugins = [ { class = "io.lenses.alerts.plugin.am.AlertManagerPlugin", config = { endpoints = "http://ALERTMANAGER_HOST_1:9094,http://ALERTMANAGER_HOST_2:9094,...", source = "Lenses", generator-url = "http://LENSES_HOST:LENSES_PORT" } } ]
In 3.0 the Alerting Subsystem became pluggable. Slack integration is supported by the included Slack Plugin. If you are using Slack, the settings have to be migrated. Old settings that may remain in the configuration file will prevent Lenses from starting up! An example of migrating Slack settings:
# Old layout lenses.alert.plugins.slack.enabled = true lenses.alert.plugins.slack.webhook.url = "https://hooks.slack.com/services/SECRET/YYYYYYYYYYY/XXXXXXXX" lenses.alert.plugins.slack.username = "lenses" lenses.alert.plugins.slack.channel = "alerts" # New layout lenses.alert.plugins = [ { class = "io.lenses.alerts.plugin.slack.SlackAlertsPlugin", config = { webhook-url = "https://hooks.slack.com/services/SECRET/YYYYYYYYYYY/XXXXXXXX", username = "lenses", channel = "alerts" } } ]
The class of the default LDAP plugin has been renamed. If you use LDAP you must rename the class inside
security.conf
:# Old lenses.security.ldap.plugin.class = "com.landoop.lenses.security.ldap.LdapMemberOfUserGroupPlugin" # New lenses.security.ldap.plugin.class = "io.lenses.security.ldap.LdapMemberOfUserGroupPlugin"
The default value for allowing weak SSL configuration has been changed. By default in 3.0 weak SSL settings will not be allowed. This change affects mainly the Schemas and Connectors management pages:
# Old lenses.allow.weak.SSL=true # New. Default value has been changed to false lenses.allow.weak.ssl=false
Client Changes¶
If you are using service accounts, now you must pass both the account name and token via the
X-Kafka-Lenses-Token
header:# Old Authentication Header X-Kafka-Lenses-Token: ServiceAccountToken # New Authentication Header X-Kafka-Lenses-Token: ServiceAccountName:ServiceAccountToken
There are some breaking changes in the API due to the requirements of the new, fine-grained, permissions model. If you have applications that use the API endpoints below, you must update them with the new endpoints:
¶ Endpoint Group Old Endpoint New Endpoint Information Kafka Topics GET /api/topics
GET /api/v1/kafka/topics
API reference Kafka Topic Data GET /api/sql/data?sql=...
GET /api/ws/v2/sql/execute
API reference SQL Validation GET /api/sql/validation
n/a Removed Alerts GET /api/alerts
GET /api/alerts
Response payload changedAudit GET /api/audit
GET /api/audi
Response payload changed
Helm Charts Changes¶
Zookeeper is now optional in Helm:
# Old zookeepers: chroot:
# New zookeepers: enabled: false chroot:
Security section must be adapted to the new security model of Lenses:
# Old security: mode: BASIC kerberos: principal: keytabData: |- debug: false groups: - name: adminGroup roles: - admin - write - read - name: writeGroup roles: - read - write users: - username: admin password: admin displayname: LensesAdmin groups: - adminGroup serviceAccounts: - username: jenkins token: jenkins groups: - adminGroup ...
# New security: # Change Default Lenses user credentials. # defaultUser: # Example # username: admin # password: admin kerberos: principal: keytabData: |- debug: false .....
Enabled Alert Plugins must be updated to the new format:
# Old alertManagers: enabled: false endpoints: - host: alertmanager protocol: http port: 9094 sourceName: Lenses™-k8 consumersLagThreshold: 50000 slack: enabled: false iconUrl: https://www.lenses.io/images/logos/icon_ellipse2red.png webhookUrl: "" username: Lenses channel: lenses
# New alerts: enabled: true consumersLagThreshold: 50000 plugins: - class: "io.lenses.alerts.plugin.slack.SlackAlertsPlugin" config: | { webhook-url="https://your-slack-webhook-url username=Lenses™ channel=lenses-slack-test } - class: "io.lenses.alerts.plugin.am.AlertManagerPlugin" config: | { endpoints="http://your-alert-manager-url" source=PROD generator-url="http://lenses_prod:port1" }