Using SQL to query Kafka

Understand the Lenses SQL Snapshot Engine to query data in Kafka

Lenses provides two SQL engines:

• SQL Snapshot for point-in-time queries against Kafka topic data

• SQL Processors for continuous stream processing

This page covers SQL Snapshot.

Use SQL Snapshot when you want to inspect topic data, validate assumptions, or answer an ad hoc question. The Data tab on a topic uses the same engine, but the SQL editor gives you more control and supports more advanced queries.

SQL Snapshot queries run per environment. You query topics that belong to the currently selected environment.

Open the SQL editor

Open Environments from the left navigation, expand the target environment, then select SQL.

The editor gives you:

1. auto-complete for topics, fields, and SQL keywords

2. inline validation and error highlighting

3. a timestamp selector that can generate time-bounded query filters

Write and run queries

Write a query against one or more topics in the selected environment, then run it from the editor.

For example:

SELECT *
FROM payments
LIMIT 100;

Use the timestamp selector when you want to start from a known time range instead of writing the timestamp filter manually.

Kafka is not a database. Queries that filter only on payload fields can trigger full topic scans and take time on large topics.

For query design and performance guidance, see Best practices.

Understand the results

After you run a query, the results area is split into tabs.

Results

The Results tab shows matched records.

You can:

1. switch between grid and tree or JSON views

2. sort and group rows in the grid

3. copy cell values for further analysis

Execution Details

The Execution Details tab shows query progress and execution information. Use it to understand what the engine is doing while the query runs.

Bad Records

The Bad Records tab lists records the engine could not deserialize. This usually means the topic contains unexpected payload data, the configured format is wrong, or a dependent service such as Schema Registry is unavailable.

For more on bad records and execution controls, see Best practices.

Tune Query

The Tune Query tab shows suggestions to improve query performance and reduce scan cost.

Learn the query model

Use these pages next:

1. Concepts — how SQL Snapshot models keys, values, headers, and metadata

2. Metadata fields — query offsets, timestamps, partitions, and other metadata

3. Filter by timestamp or offset — target a specific slice of topic history

4. Filtering — write effective WHERE clauses

5. Limit & Sampling — control result size and query cost

6. Managing queries — save, reuse, and organize queries

Common tasks

Use SQL Snapshot when you need to:

1. inspect recent topic records

2. verify a field value before changing an application

3. check records in a specific time range

4. compare data across topics with joins

5. troubleshoot malformed or unreadable records