Discovering Topics & Schemas

Understand how to discover Kafka topics and schemas in Lenses

Use Lenses to find topics and schemas across environments, inspect their details, and manage them where your permissions allow.

Each Lenses Agent connects to a Kafka environment and its services, including Schema Registry. The agent catalogs topics and schemas for that environment.

If a topic is not backed by Schema Registry, Lenses can sample records and infer a format. For example, it can identify a JSON structure from message data. Lenses then uses that schema for search and for the SQL snapshot engine.

You can edit schemas in Lenses only when Schema Registry is not managing them. These edits do not change data stored in Kafka. They affect how Lenses understands the topic for search and for the SQL snapshot engine.

You can discover topics and schemas in four ways:

1. Search — fastest when you know the name

2. Global topic catalogue — best for browsing across environments

3. Per environment — best when you already know the target environment

4. Legacy UX — useful only for older workflows

The legacy UX is still available. New feature work happens in the global experience.

Search

Open Search from the left navigation when you want the fastest route to a topic or schema.

Use search to:

1. find topics and schemas across environments

2. open matching results in the workspace

3. jump to the right environment when you do not know where a resource lives

Use the result context menu for follow-up actions where supported.

Global topic catalogue

Use the global catalogue when you want to browse topics across your full estate.

From the left navigation, open All Topics. You can:

1. view all topics across all environments

2. filter to a single environment

3. sort, filter, and search within the list

4. open a topic directly in the workspace

You can also reach this view from Home and from quick actions in the navigation tree.

Topics Per Environment

Use the environment view when you already know which Kafka environment you need.

Open Environments from the left navigation, then expand the target environment in the navigation tree. From there you can browse topics and schemas directly.

Use local filters and search to narrow the list. Open any topic or schema to inspect its details in the workspace.

View topic details

Topic details are split into sections. You can access them via the context menu on the tree panel nodes, the leaf nodes, or from the breadcrumbs.

1. Profile — view the main metadata and summary

2. Data — browse records in the topic and open SQL for deeper queries. See Using SQL to query Kafka.

3. Schema — view and manage the schema associated with the topic

4. Partitions — inspect partition details and the message distribution heat map

5. Consumer Groups - view and manage consumer groups for the topic

6. Configuration — inspect and manage topic settings, including overrides from broker defaults

7. Alerts — view alert events and manage alert rules for the topic. See Alerting & Monitoring.

8. Activity — inspect the audit trail for the topic. See Governance

The actions you see depend on your IAM permissions. Some users can only inspect resources. Create, edit, replicate, backup, restore, and delete actions may be restricted.

Topic Profile

The topic profile gives an overview of key metadata about the topic, for example, partitions, replication, consumers and other metrics such as records size.

You can also view records counts and averages over the last 30 days.

View schema details

Open a schema to inspect its structure and metadata.

Depending on how the schema is managed, you can:

1. review the current schema definition

2. inspect version history

3. create a new version

4. compare versions with a diff view

5. add descriptions and tags

If the schema is managed by Schema Registry, schema changes must be made there. If the schema is managed only in Lenses, you can edit it in Lenses.

The tabs are available in the navigation tree, breadcrumbs and context menu (open in tab)

Consumer group management

The consumer groups tab allows you to see and manage consumer group information for consumers reading from the topic.

You can;

1. Change the consumer offset per client

2. Change the offsets for the whole group

3. Remove offsets per client

4. Delete the whole consumer groups

5. Migrate consumer group - map consumer group offsets from one Kafka cluster to another. See Share and replicate data

Management of offsets is only possible if the consumer group is inactive

To manage the consumer group, use the context menu ellipsis (3 vertical dots) per row

You can also view the records for the start, end and current offset by clicking the cells, this will open up the query editor and run a query to fetch those records.

Topic Configuration

You can view a topic and edit configuration, the tab will also highlight those configurations differing from the broker defaults.

Create and manage topics

You can create a topic from Home, from the topic node in the navigation tree, or from the topic listing.

From a topic context menu, you can:

1. Insert and delete messages

2. Change the metadata type that Lenses uses for the SQL engine

3. Increase the number of partitions

4. Replicate the topic to another cluster

5. Back up or restore the topic

6. Delete the topic

You can also add a description and tags for the topic.

Delete, backup, restore, and replication actions can affect production systems. Use them carefully and confirm the target environment before you proceed.

Schemas per environment

Schemas can be listed at the environment level.

To list schemas, open Environments from the left navigation, expand an environment, then either expand the schema node or use the quick action on that node to open the listing.

Create and manage schemas

You can create a schema from Home, from the schema node in the navigation tree, or from the schema listing.

For each schema you can:

1. Create a new schema version, this also includes a diff editor to examine your changes

2. Infer the schema again when it is not managed by a registry and the topic structure has changed. This requires Agent 6.2 or later.

3. Add a description and tags

Legacy UX

Use the legacy UX if you prefer the older navigation model or need to follow an older workflow.

You can still open entities there from context menus in the current experience.

For more on legacy navigation, see Navigating & Discovering Lenses.

Common tasks

Use these flows as a quick reference:

1. Find a topic anywhere — open Search or All Topics

2. Browse a known environment — open Environments, then expand the target environment

3. Inspect topic records — open the topic, then select Data

4. Check or update a schema — open the topic or schema, then select Schema

5. Create a topic or schema — use Home, the navigation tree, or the relevant listing view