5.3

What’s New in Lenses 5.3

New 

1. Backup & Restore topics to AWS S3 

Lenses introduces a powerful new feature – the ability to backup and restore Kafka topics directly to AWS S3.

  • Easy. Just a couple of clicks and you’re done. Lenses takes care of the rest.
  • Multi-purpose. Backup your topic data for: disaster recovery, data retention, cluster migration, and cluster replication in an active-passive setup.
  • Based on Kafka Connect. Lenses uses your Kafka Connect clusters to automatically setup and run the appropriate connectors to backup and restore.
  • Built on open-source. Lenses leverages the power of the S3 source and sink connectors from its open-source project, Stream Reactor.
  • Powerful envelope. Lenses uses an envelope data structure to ensure that you backup the full context of your Kafka messages:
    • Key
    • Value
    • Headers
    • Kafka metadata: partition, offset, timestamp, topic name.
  • With enterprise support. Plan your backup Kafka strategy with peace of mind using the Lenses S3 enterprise support for backup/restore and S3 connectors.

Learn how to backup and restore your topics to AWS S3 here.

2. Revamped experience for Kafka Connect 

Experience an enhanced user interface for Kafka Connect within Lenses.

  • Cleaner, leaner. A more responsive and intuitive experience, with icons and columns to help you easily spot the connector you’re looking for.
  • Unified search & filtering experience. Tailor your view more easily. We brought the powerful search & filtering capabilities of the Explore screen to the Connectors.
  • Faster & more efficient. It uses the latest Connect REST API that allows to get connector and task details in a single API call.
  • Quick documentation. We’ve tried to help when creating a connector. Instantly see the documentation of every field you’re configuring in your connector, right in the editor.

3. Export data to CSV and JSON lines. 

Export & share the Kafka data you’ve found through Lenses.

  • CSV. Download data in CSV, with flexible configuration and load them into your favourite spreadsheet app.
  • JSON lines. Download multiple messages, 1 JSON object per line and import them for local analysis in popular tools like Pandas and Jupiter notebooks.

4. Subpath Deployment Support for Lenses 

You can now deploy Lenses under a URL subpath. This allows you to reuse the same domain for all your Lenses environments.

Up until now you needed separate domains for each of Lenses instances. e.g., in a 3-environment setup:

  • dev-lenses.acme.com
  • qa-lenses.acme.com
  • prod-lenses.acme.com

With 5.3 you can now reuse a single domain and place all your Lenses under it, using subpath deployments e.g.:

  • lenses.acme.com/env/dev
  • lenses.acme.com/env/qa
  • lenses.acme.com/env/prod

To configure the desired subpath, simply set the lenses.root.path setting to your preferred path. This path serves as the base from which all Lenses URLs will be served.

5. Manage and track your SSO users. 

You can now fully manage and track your enterprise users, both with SSO and LDAP:

  • Auto or manual user creation. Choose to pre-create your SSO/LDAP users or have them automatically created when the users log in.
  • Auto or manual group membership. Choose to automatically link a user’s groups to your SSO/LDAP or manually specify their membership. This offers the flexibility to decide if you want your enterprise SSO/LDAP to drive your users' permissions or you prefer Lenses to do it.
  • Login tracking. See the last time each user has logged in to Lenses. Useful for tracking engagement of your users with your Lenses environments.
  • License limits. Lenses will now inform you when you’ve reached the maximum number of users of your license, across all your authentication mechanisms (SSO/LDAP/BASIC). This helps with keeping track of your use and to plan your expansion to more users.

Enhancements 

1. GitOps support for Lenses deployments 

Lenses greatly improves its GitOps capabilities for managing your Lenses installation.

It builds upon the provision.yaml introduced in 5.0 to offer a definitive way to drive your Lenses deployments declaratively, as-code:

  • Declarative setup. Express your Lenses setup in Git as yaml, push to your repository and let the magic happen. Add, change or remove something from your yaml and Lenses will treat it as source of truth and reflect the change.
  • Flexible. Easy to hook into your CI/CD automation. Works well whether in a Kubernetes setup with Helm or with an archive installation.
  • Easy to apply to many environments. By having a fully automated Lenses deployment, as-code, it makes it much easier to deploy multiple Lenses environments from one Git repository with all the DevOps best practices.
  • Fewer restarts. Perform changes to your Lenses setup, as-code, without needing to restart and interrupting your users.
  • Connections support. Declare your Lenses integrations your Kafka ecosystem and change them without restarts.
    • Kafka
    • Schema Registry
    • Kafka Connect. Add or remove Kafka Connect clusters seamlessly without interrupting your users’ access to Lenses.
    • Cloud & Alert integrations.
  • License support. Update your license, as-code, without restarts.

Learn how to deploy Lenses with GitOps here.

2. Connector topic discovery 

See your Debezium connectors in the Lenses topology with all the topics they link to, dynamically.

Lenses can now discover the topics used by connectors at runtime. This enables Lenses to support connectors that do dynamic topic routing such as Debezium.

3. License details & report 

Find a detailed list of all the enabled features and limits of your plan.

Generate a report and share it with the Lenses support team for a more streamlined support experience.

4. User Interface Improvements 

  • Compacted topics filter. The Explore screen now offers quick filtering for compacted topics and provides table resizing capabilities for a more user-friendly experience.
  • ACLs filters. The ACLs screen has been improved with additional filtering capabilities, simplifying access control list management.

Bug Fixes 

  • Streaming SQL validation no longer incorrectly validates producer.* and consumer.* properties, ensuring a smoother experience when working with streaming data configurations.