Features / Improvements & Fixes

Changelog details for Lenses 6.1.1

Improvements 💪

SQL Snapshot Performance

Release Notes: SQL Engine Enhancements

Problem Addressed: The engine previously permitted unlimited concurrent queries, risking excessive CPU and memory consumption.

Solution Implemented: Introduced configurable limits for concurrent queries to ensure balanced resource usage and system stability.

New Configuration Options:

• max.concurrent.queries: Global query limit (default: 100).

• max.concurrent.queries.per.user: Per-user query limit (default: 2).

SQL Usage Enhancements: Admins can modify limits via Lenses configuration:

• lenses.sql.settings.max.concurrent.queries=200

• lenses.sql.settings.max.concurrent.queries.per.user=1

Impact:

• Prevents Out-Of-Memory (OOM) errors due to query overload.

• Ensures equitable resource distribution.

• Offers clear error messages to enhance user experience.

SQL Snapshot Performance

To alleviate CPU and memory pressure, the engine now avoids duplicating internal buffers. The default value has been adjusted from 50,000 to 5,000. You can modify this value using the following methods:

1. Through SET statements: SET query.buffer=10000; <your query>

2. In the agent configuration, either in provision.yml or lenses.conf (depending on your deployment method): lenses.sql.settings.query.buffer = 10000

Topic storage format identification

The update enhances the identification of strings, bytes, and integers/longs in a topic Key or Value format.

Fixes 🛠️

Kafka Insert Messages

Addresses a gap in the feature where inserting data for a Kafka data involving Avro schema with different Enum classes sharing the same name but different namespace.

Other fixes

• Fixed topic format identification for Confluent wire format, binary/string/emoji handling

• Fixed quota parsing for usernames containing slashes

• Fixed WebSocket status updates when processor actions fail