View the latest documentation 5.5
One of the main benefits of Lenses is providing a secure way to query, inspect and control your data, wherever it might be.
Lenses comes with Lenses SQL, a declarative Structured Query Language (SQL) interface, supporting industry standard ANSI joins and aggregates for querying, transforming and manipulating data at rest and data in motion.
There are two types of syntax to create queries with Lenses SQL - Snapshot SQL & Streaming SQL:
Snapshot SQL lets you create point-in-time queries to pull existing data-at-rest from a Kafka topic or other data sources.
Streaming SQL allows creating continuous real-time running queries for live data manipulations. A good fit for Kafka-to-Kafka, results and scales by running a SQL Processor app.
Some configurable Admin settings when it comes to Lenses SQL queries include:
For more regarding the configuration see configuration settings.
Query content can also be protected by adding a Quota on your Kafka cluster under the specific client ID. For more information on how to do this see content protection.
Recent queries are displayed, but only for the current session, they are not currently retained.
Click on the play button to run a previous query. If a query is already running, you will be asked if you want to stop it first.
SHOW ALL QUERIES
You can see all running queries by Lenses users using SQL:
SHOW QUERIES
You can force stop a query by another user using SQL:
KILL QUERY <id>
You can query data in other sources connected to Kafka by using the connection name. For example:
USE `lenses-postgresql`; SELECT * FROM public.metrics_series_values LIMIT 100;
or by selecting the connection in the SQL Studio.
Lenses is extendable, and the following implementations can be specified:
Lenses SQL documentation
Let’s see an example of the following using Lenses CLI considering we are using Lenses box, and the configured paths in such environment:
lenses-cli processors | grep -iw 'RUNNING\|FAILED\|PENDING' | \ awk '{print "id=" $1 " cluster-name=" $15}' \ > processors.bk
while read processor; \ do lenses-cli processor stop $(echo "$processor" | \ awk '{print "--"$1 " --"$2}'); done \ < processors.bk
Plugins are added under the plugin.path of each worker. Each plugin should be in its own subdirectory.
As an example, the Connect worker’s configuration may include the line below, which sets the plugin.path to /usr/share/connectors. This path contains a folder for the lenses-sql-streaming plugin.
lenses-sql-streaming
plugin.path=/usr/share/connectors
Remove that /usr/share/connectors/lenses-sql-streaming folder
/usr/share/connectors/lenses-sql-streaming
To install the new SQL connector, download the connector archive (lenses-sql-connect-vX.Y.Z.tar.gz) from the client area, extract it, and copy all files under the connector directory to /usr/share/connectors/lenses-sql-streaming.
Further instructions to install a new Connector plugin.
mkdir -p /usr/share/connectors/lenses-sql-streaming tar -xzf lenses-sql-connect.tar.gz \ -C /usr/share/connectors/lenses-sql-streaming \ --wildcards */connector/* --strip-components=2
Start up the workers.
while read processor; \ do lenses-cli processor start $(echo "$processor" | \ awk '{print "--"$1 " --"$2}'); done \ < processors.bk
On this page