Features, Improvements & Fixes

Changelog details for Lenses 6.2.0

New 🎉

SQL Studio Becomes Studio

We’ve totally revamped the global SQL Studio and renamed it to just Studio. It’s designed to deliver maximum productivity and simplicity for executing engineering workflows. The experience (including shortcuts) is inspired by your favourite IDEs.

The previous HQ views (Apps, IAM, …) in 6.0 and 6.1 have been integrated into this Studio as well as rolling up more assets from the legacy UI.

This new Studio provides a unified experience to manage all Kafka and Lenses resources including Lenses IAM roles, SQL Processors, Connectors, K2K apps, Kafka Connections, Schemas, Topics, Consumers and more.

The Studio also comes with a new search to easily find any resources across your multi-Kafka estate.

The Environments view, accessible from the top of the Explorer, has been updated to centrally view the health snapshot of your different connected Kafka environments.

Access to the global Topics catalog has also been moved to the top of the Explorer.

The Bookmarks in 6.1 have been renamed to Favourites where any Kafka or Lenses resource can be saved for easy access, avoiding the need to search or navigate the Tree Explorer.

A Form/YAML toggle will be noticed on a number of different pages – allowing users to select their preferred configuration style.

SQL Processors in HQ

SQL Processors now have a new experience in HQ, giving a global location to manage and create stream processing applications.

Listing:

• View and search all SQL Processors across all environments

• Bulk Action support to stop, start and delete processors

To create SQL Processors:

• Improved editor, with better intelli-sense, including formats, partitions, replicas and topic size and message stats

• Templates for common SQL Processing patterns

• Auto task max, that will automatically set the number of SQL Processor runners based on the topics involved

SQL Processor details:

More information is surfaced to allow you to quickly see:

• Health information: Throughput & message counts, errors and runner logs

• Consumer groups, lag and the ability to manage those offsets

• Alerts: Any alerts raised for this consumer or output topics and ability to create new alerts directly from the SQL Processor page

• Interactive topology of the processor to understand visually what it’s doing.

Connectors in HQ

Kafka Connectors also now have a new experience in HQ. The capabilities are extensive, providing:

• JSON schema support to ensure you minimise configuration mistakes

• Auto completion examples for KCQL for Stream Reactor connectors

• Auto task max, making sure you correctly scale your connectors based on topic partitions

• Automatic converter selection based on topic format

• Auto completion default snippets for

  • Converters

  • Producer and Consumer overrides

  • Error handling

  • Support resetting Connector offsets, source and sink.

Requires Kafka Connect 3.7 and above

External Apps in HQ

Last but not least, registering external applications into the Lenses App Catalog and Topology is also in HQ.

In the coming releases, we’ll be updating Topology to HQ too, to give you a global view of application and data lineage across Kafka clusters.

Secure Remote MCP Server

You loved the Lenses MCP Server released last year. But now you are looking to roll it out to more Agents and users. For this, simplicity, scalability and security are critical to you.

Now in this release, rather than having to deploy the Lenses MCP Server locally across machines and workstations, we’ve included it in the Lenses deployment running as a server. This allows local MCP Clients such as Cursor or Claude to securely connect to Lenses via MCP, without having to roll out multiple Servers.

Instead of using STDIO, the Server uses a single Streaming HTTP endpoint, as recommended by the MCP community.

And perhaps most importantly, it also allows the support of OAuth 2.1 authorization to the MCP Server (see below).

OAuth 2.1 Support

Handing an agent a long-lived API key isn't defensible. The Lenses MCP Server now supports OAuth 2.1 for delegated access, replacing static credentials with short-lived tokens tied to a named user.

Three scopes ( lenses:read, lenses:write, lenses:delete ) sit on top of the user's existing Lenses IAM role and let the agent inherit a subset of the user's IAM permissions, capped by what they consented to.

Access tokens are short-lived (minutes, not days) and refresh transparently in the background, so users aren't prompted to sign in mid-session. Every call is attributable to the user who authorized it in the audit trail, and any registered client can be revoked from within Lenses in seconds.

VSCode Plugin

You love the Lenses UI but you also often want to execute some workflows without leaving your IDE.

Now you can – we are pleased to offer the experience of Lenses within a VSCode-compatible IDE including Microsoft Visual Studio Code and Cursor.

This first release has focused on allowing users to configure Environments (connect Lenses to remote Kafka clusters), explore topics with Lenses Snapshot SQL, view and manage schemas and consumer groups, and manage Lenses IAM users and roles as-code.

But the VS Code Plugin is on a different release track than Lenses, so keep a close eye for very regular updates. Check-out the separate docs.

The VS Code Plugin can be downloaded and installed from the Microsoft VS Code Marketplace.

Kafka App Migrations

Replicating data from one Kafka cluster to another with K2K is great, but what about the consumer groups?

The committed offsets don’t match between source and target clusters, so if you blindly re-point your consumer to the new cluster, it might reprocess data.

This might be ok for your use case, but likely not.

We know that migrations can take months if not years if you have hundreds or thousands of apps to migrate, and it comes with a lot of risk and complexity.

This has now been drastically simplified thanks to the visibility and control that Lenses now offers, including a new “Migration” workflow within the Apps section of Lenses that allows you to migrate and translate the consumer group offsets to the target cluster with a single click!

Schema Inference

For topics not serialised through a schema in a Schema Registry (such as CSV or JSON topics), Lenses infers the schema. But these schemas could change over time, which breaks how Lenses is able to view the data.

In these circumstances you can now trigger schema inference on the Schema tab within Studio for a topic.

Improvements 💪

K2K Apps

Form creation flow

Some of you like YAML, some don’t — we understand. The Kafka 2 Kafka creation and edit workflows have been updated to include a more traditional approach. You can either use the form to create a replicator, or go hardcore and use the YAML editor with auto completion and intelli-sense. Pick your poison.

We also surfaced more information in the listing side drawer and details pages, improving the layout and we also added bulk actions to pause, start and delete applications.

You also now get an interactive topology of your K2K application, enabling you to perform actions and drill into the topics involved.

Consumer Group Offset Management

K2K applications now expose full control over their consumer group offsets to replay or control data to be replicated to the target cluster. You can view where replication stands – per topic, per partition – and reset it without touching Kafka directly.

From the new Consumer Groups screen you can:

• View committed offsets for all topics in the application’s consumer group

• Reset offsets by timestamp — Lenses resolves the closest available offset per partition automatically

• Reset offsets to a specific raw value (e.g. 0 to replay from the beginning)

• Operate at any scope: the entire application, a selection of topics, or a single partition

The application must be paused before offsets can be modified; all controls are disabled while it is running.

When creating a new K2K application you can also now set startFrom to control where replication begins: earliest to replay all history, latest for new messages only, or an RFC3339 timestamp to start from a specific point in time.

Surface Error states

We have improved the error reporting to give more visibility into the state of the application.

Lenses Agent Configuration

Deploying and configuring a Lenses Agent to a Kafka environment now includes more guidance on uploading JKS files when mTLS is required, and is now managed in the new Studio in HQ. The editor will also mark the gutter when a file needs to be uploaded.

Hints to change data format in SQL Snapshot

The Lenses SQL Snapshot engine tries to determine the data format in topics. It doesn’t change the underlying data; this is just metadata in Lenses. If it can’t determine the format, it defaults to BYTES. You can control this via the content menus to change the type.

If you query data with SQL with the type set to BYTES, the records are shown in the bad records tab. We have enhanced this to let you change the type, with guidance to help you choose.

Inserting messages

In Studio, you can now choose which partition to insert the message into.

Bulk Actions for Topics and Schemas

You can now perform bulk actions such as delete on multiple topics or schemas.

Info panels

On the right-hand side drawer, a per-resource-type information panel is available.

Environments

You can now go to the configuration section if your agent is disconnected. This allows you to configure the provisioning and apply to the agent at deployment time or via CI/CD.

Kafka ACLs

Filtering by operation has been added.

Fixes 🔨

• Applying tags to topics is now fixed.

• Increasing partitions for a topic no longer results in a data format error.

• Deletion of SQL Processors in a failed state now works correctly.