5.1

What’s New in Lenses 5.1

The 5.1 theme is for Lenses to offer a more native AWS experience and a new, revamped documentation website.

New 

1. AWS Glue schema registry 

Lenses now supports the AWS Glue Schema Registry to manage Avro schemas (Protobuf coming soon).

This can complement your AWS-native experience for Kafka:

  • AWS MSK for a managed Kafka
  • AWS Glue for a managed Schema Registry

Lenses integrates its functionality across the board:

  • Explore Avro topics linked to Glue schemas
  • Manage and evolve Avro Glue schemas in Lenses
  • SQL Snapshot
  • SQL Streaming (SQL processors)
  • Data Policies
  • RBAC
  • Topology

Set up your Lenses with AWS Glue Schema Registry.

2. AWS IAM authentication 

Lenses can now authenticate with your AWS MSK Kafka using the native IAM access control. This is done over SSL with SASL.

Connect your Lenses to your MSK with IAM.

3. AWS S3 connector Enterprise support 

Lenses.io now offers enterprise support for its most popular AWS connectors, S3.

This allows you to choose the best of both worlds:

  • Gain control and avoid lock-in. Use a fully-open source S3 connector with the Lenses.io Stream Reactor project.
  • Have future-proof peace of mind. An S3 connector officially backed by the Lenses.io support team, offering enterprise-grade response, bug fixes and a way to channel requests for future connector features.

If you are interested in signing up please reach out to us.

4. Documentation website 

  • A single place for documentation and user guides. We’ve combined our 2 websites into 1.
  • Streamlined experience. We’ve improved the content and structure to make it easier to find what you are looking for.
  • Fresh look and feel.

Improvements 

5.1 brings productivity and UX improvements:

1. Find empty topics 

Data catalog: Quickly see only the topics and datasets that are empty.

2. Better table searchability 

Now you can search in every table-like view in Lenses. Lenses also consistently pre-sorts the table entries so things are where you expect them to be.

A good example is the topic configuration view. The view will now show the options pre-sorted, and you can search for any option.

3. Dealing with large messages 

We have seen Lenses being used to explore quite large messages. This can make it cumbersome to visualise the messages when exploring a topic. Lenses 5.1 offers a couple of improvements to help with this:

Metadata readability 

When exploring Kafka messages, the PARTITION | OFFSET | TIMESTAMP metadata will now be at the top of the message, not the bottom. This helps with dealing with large messages that can take the whole screen space. In the past you would have to scroll all the way down to see the metadata.

Customise number of messages displayed. 

Lenses, by default, displays 200 messages when exploring a topic. This can take a while to load and even overwhelm the browser when messages are MBytes+ big.

Now, you can adjust that number with a new configuration:

New ConfigDescriptionTypeDefault Value
lenses.ui.topics.row.limitThe Kafka topics screen default records limit to load automaticallyint200

See more in the Advanced configuration section.

4. Run Lenses with your own Java runtime 

(for Linux tarball installation)

Lenses can now run with any Java runtime (JRE), version 11. In the past, Lenses would ship with its own specific JRE. This could be limiting if you wanted to use a Java runtime of your own choosing (e.g. a specific distribution such as Oracle, OpenJDK, Azure). We have now lifted this limitation.

Kafka Topic screen default records limit. This release introduces lenses.ui.topics.row.limit to control the default records limit on the Kafka topics screen.

Fixes 

  1. Deleting messages in compacted Kafka topics: Lenses will now only produce 1 tombstone message per key.

  2. SQL streaming & data policies: fixed a bug where Lenses would not allow to create a SQL processor when using a field covered by a Data Policy.

  3. Data policies: Fixed broken links in Applied Data section.