Elasticsearch to Lenses.io

Elasticsearch is one of the most popular integrations with Kafka. Lenses integrates with Elasticsearch to create a seamless data journey by providing a 360 view to the streaming flow. You can preview or query data present on Elasticsearch indices and assign metadata to surface relevant information faster, all while keeping your data secure and compliant.

Integration summary

  • Data Catalog index discovery: search, tag & add description
  • Index data view & health metrics metadata
  • SELECT Queries to view data
  • Data protection policies
  • Security namespaces with granular permissions
  • Versions 6 and 7

Required permission 

(Per data source)NamespacePermissions are available per connection per namespace ie. View Topics to transactions*

Data sources integrating with the catalog, are governed by granular permissions subject to namespaces. Like this you can shape the data catalog based on your teams or access constraints.

Access Management & permissions

Explore datasets 

In the data catalog, a user is able to search for terms such as index name, schema fields or descriptions and filter by tags.

Elasticsearch search on Explore

Choose the index and navigate to the detailed view. There we can see Data [1], Schema, Shards and Metadata information [2, 3]. We can specify, the number of items, we want to include in our query, view the results in Tree or Table View and see the Schema for each individual field.

Elasticsearch Details - Data

The data catalog refreshes its cache every 30s by default, but it’s subject to configuration.

Query the data 

Navigate to the SQL Studio to run SQL queries.

The Lenses SQL Studio provides a familiar query editor that allows writing Lenses SQL queries to retrieve results from individual data sources. You can also Download the results and preview them in either Tree or Table View.

To run a query for Elasticsearch data use the connection name:

USE myElasticsearch;
SELECT * FROM myIndex LIMIT 100;
Elastisearch SQL Studio


If you have any question, please refer to the following list. If you still have more, we are more than welcome to answer any question at our community channel.

How can I view the status of my connection?

Currently, Lenses UI provides a visual indication of the connection health status. Unfortunately, that is only visible in the Explore screen.However, uncovering connection problems should be fairly easy. You can inspect the Lenses Logs and search for ERROR entries related to your connection. More on Logs

What versions of Elasticsearch does Lenses support?

We offically support Elasticsearh versions ranging from 6 to 7 with all minor and patch releases for each version.

How many Elasticsearch connections can Lenses handle?

A Lenses on a Kubernetes cluster with 4GB of RAM can handle up to 10 Elasticsearch connections with approximetaly 10K Indices with good responsiveness.

Why is my Elasticsearch Node ‘Yellow’, even when it’s healthy?

Note that in a single node cluster, even a healthy index will always have a Yellow status. That doesn’t mean everything is NOT OK, however your replica rules are not satisfied. The status will update to green once your cluster contains enough nodes to allocate the replicas, even if Lenss is connected to just one node.

What Elasticsearch features are not supported currently?

The main limitation of Lenses SQL for Elasticsearch, is its ability to convert SQL Statements to the query language expected by the Elasticsearch REST query interface.

Does Lenses support write operations for Elasticsearch indices?

Support for write operations (e.g. inserting/updating records, creating/altering indices, etc) is not planned for the immediate future.