Querying Kafka topics


The CLI allows you to browse and subscribe to topic data with SQL. You can use flags to control the output of the data; the flags are listed in the table below.

FlagsDescription
keysPrint message keys.
keys-onlyPrint the keys only, not the value of the message
metaPrint the message metadata, partition number and offsets
statsPrint query statistics as the last message.

Browsing 

Browse data via the query command, optionally validate and output stats:

lenses-cli query \
    --keys \
    --meta "SELECT * FROM topic LIMIT 50"

Execute with stats and show offsets 

lenses-cli query \
    --stats "SELECT * FROM topic LIMIT 50"

Live 

Live continuous queries update according to the query and never stop until terminated:

lenses-cli \
    query \
    --live-stream "SELECT * FROM topic WHERE payment_type = 2 AND passenger_count > 1"

Interactive shell 

The interactive shell allows you to have an interactive experience and issue Lenses SQL queries from the command line.

To start the interactive shell, issue the following command:

lenses-cli shell

The available options are the following:

FlagsDescription
keysPrint message keys.
keys-onlyPrint the keys only, not the value of the message
metaPrint the message metadata, partition number and offsets
statsPrint query statistics as the last message.
prettyPretty print the JSON output
live-streamrun as a continuous query
!optionsprints the current options

Multiline queries are possible but not available in the history inside the shell. After a restart, the multiline queries are concatenated into one line and can be selected.

Query history is stored in .lenes/history in the home directory of the user.

    __                                 ________    ____
   / /   ___  ____  ________  _____   / ____/ /   /  _/
  / /   / _ \/ __ \/ ___/ _ \/ ___/  / /   / /    / /
 / /___/  __/ / / (__  )  __(__  )  / /___/ /____/ /
/_____/\___/_/ /_/____/\___/____/   \____/_____/___/
Docs at https://docs.lenses.io
Connected to [https://master.lenses.io:443] as [andrew], context [master]
Use "!" to set output options [!keys|!keysOnly|!stats|!meta|!pretty]
Crtl+D to exit

lenses-sql>
--
Last modified: July 26, 2024