Features / Improvements & Fixes

Changelog details for Lenses 6.1.2

New 🎉

We're making significant strides in integrating Lenses K2K into the Lenses ecosystem, which is set to deliver enhanced governance and control over the existing standalone solution. In this release, you'll find:

• Evolving Environment Management: Initial roll-out of new capabilities in environment management, serviceability, and ACL management within HQ.

• Continued Studio Enhancements: Studio now enables topic creation, editing, and viewing configurations, progressively reducing the need for the legacy Agent UX (still accessible if needed).

• Ongoing Styling Improvements: Gradual updates to HQ's styling for a more refined user experience.

We're eager for your feedback as we continue this integration journey.

Kafka to Kafka Replication

The latest release of the Lenses K2K integration introduces enhanced Kafka connection support, extending beyond PlainText and MSK with IAM Roles for Service Accounts (IRSA). This update now includes support for mTLS and SASL/SSL, along with PLAIN, SCRAM-SHA-256, and SCRAM-SHA-512 authentication mechanisms.

Adding a New Kafka Connection

To add a new Kafka connection:

1. Navigate to the Apps menu.

2. Choose the Kafka Connections tab.

3. Click on New Kafka Connection.

A guided wizard will assist you through the necessary configuration steps.

In our ecosystem, Kafka connections are utilized by K2K and future applications provided by Lenses. These connections are managed by HQ. The Lenses agent does not directly use these connections to interact with the Kafka cluster.

Creating a K2K App

Creating a K2K app now offers enhanced control, including:

• Header replication : copies the Kafka message headers to the target

• Error handling

• Tracing headers for source record offset, partition, topic, and pipeline

• Managing underlying topic names required by K2K for offsets tracking or exactly-once semantics

• Adjusting K2K configurations such as batch size, commit size, interval, and exactly-once fencing parameters

This release introduces compatibility with various Kafka providers, including:

• Confluent

• Google

• Azure Event Hubs

• IBM Event Streams

• Azure HDInsight Kafka

• Warpstream

• Instaclustr

• Aiven

• Oracle

• RedPanda

Topic management & exploration in Studio

To create topics in SQL Studio:

1. Navigate to the desired environment node.

2. Click the ellipsis (three dots) and select Create topic.

3. Choose a creation method:

   • Use the default modal, or

   • Switch to Advanced for the native YAML editor.

4. Use breadcrumbs at the top to navigate across views.

Each topic offers accessible options for summary, data, schema, and configuration via the navigation tree.

Use the split view to get side-by-side views.

Insert messages on Studio

No need to navigate to the agent to insert messages into a topic, its now available in the HQ Studio.

Select "Insert Message" on the context menu in the Studio navigation tree and;

1. manual add messages

2. auto generate message

3. insert multiple messages

Support is also included for message headers.

Regenerate environment agent key

You can now regenerate an environment agent key from HQ and either apply it directly to the agent or update the provisioning file manually.

To apply the new key via the API, the agent must be connected. The agent will disconnect and reconnect once the key is successfully applied.

Access: Environment context menu → Regenerate agent key

Kafka ACLs management in HQ

Manage Kafka Access Control Lists (ACLs) directly from the HQ UI. View, create, and delete ACLs for your environments with full filtering and search capabilities.

Key capabilities

• View all ACLs with sortable columns (Principal, Resource Type, Resource Name, Operation, Permission, Host)

• Create new ACLs with form validation and topic autocomplete

• Delete ACLs with a confirmation dialog

• Filter by resource type, permission type, and operation

• Search across principal, resource name, and host

• Download ACLs as JSON

• Paginate through large ACL lists

Access: Environment sidebar → Kafka ACLs or environment context menu → View Kafka ACLs

Deleting or modifying ACLs can impact client access to Kafka resources. Review changes carefully before applying them in production environments.

Kafka Quotas management in HQ

Manage Kafka quotas directly from the HQ UI. View, create, and delete quotas for your environments to control producer/consumer throughput and request rates.

Key capabilities

• View all quotas with sorting on columns (Entity Type, User, Client ID, Producer Rate, Consumer Rate, Request %)

• Create quotas for specific users, client IDs, user+client combinations, or default quotas

• Delete quotas with granular property selection (remove individual rate limits)

• Filter by entity type (User, Client ID, User + Client ID)

• Search across user and client ID names

• Download quotas as JSON

• Paginate through large quota lists

Quota types supported

Type	Description
User	Quota for a specific authenticated user principal
Client ID	Quota for a specific client application identifier
User + Client	Quota for a specific user and client ID combination
Default User	Baseline quota for all users without a specific quota
Default Client	Baseline quota for all client IDs without a specific quota

Access

Environment sidebar → Kafka Quotas or environment context menu → View Kafka Quotas

Environment health dashboard

Monitor the health status of your environment's core services from a single dashboard.

Key capabilities

• Overview tab with key metrics (brokers, topics, partitions, consumer groups)

• Brokers tab with detailed broker status and configuration

• Connect tab for Kafka Connect cluster health

• Schema Registries tab for schema registry status

• Real-time status indicators (healthy, warning, error)

• Health summary in the environment listing side drawer

Access: Environment context menu → View Health

Agent logs viewer in HQ

View and filter agent log entries directly in the HQ UI for debugging and monitoring.

Key capabilities

• Real-time log streaming

• Filter by log level (DEBUG, INFO, WARN, ERROR)

• Search within log messages

• Filter by time range using timestamps

• Auto-refresh capability

Access: Environment context menu → View Logs

Agent advanced configuration

You can now access and manage advanced agent configuration settings directly from HQ. This enables you to adjust more detailed options without editing configuration files manually.

Improvements 💪

Collapsible configuration panel

Details pages now include fully collapsible side panels.

Topic details page on the Topics screen has been simplified and improved to share the basic information.

Fixes 🛠️

No user-facing fixes were introduced in this release.