Lenses CLI¶

Lenses offers a powerful CLI (command line tool) built in Go that utilizes the REST and WebSocket APIs of Lenses to communicate with Apache Kafka and exposes a straightforward way to perform common data engineering and site reliability engineering tasks such as the following:

Automate your CI/CD (continuous-integration and continuous-delivery)
Create topics/acls/quotas/schemas/processors/connectors/policies/alerts/alert-settings
Deploy and manage SQL processors in all modes, including deployment to Kubernetes
Deploy and manage Connectors
Run SQL queries on Apache Kafka both browsing and continuous live queries
Change or retrieve configurations to store in GitHub

Installation¶

Binaries are available for download:

Linux (2.2) MacOS (2.2) Windows (2.2)

Or you can use Golang’s go get.

$ go get -u github.com/landoop/lenses-go/cmd/lenses-cli

Tip

The new release contains various fixes, table view for a more user-friendly environment, improved end-user messages on CRUD commands and failures and new commands that let you work with the latest Lenses 2.1 version. View all changes from the previous CLI release (for developers)

Getting started¶

The lenses-cli application has a very straightforward way to setup an authenticated user for Lenses communication, including kerberos authentication.

The configure command will guide you through.

$ lenses-cli configure

The result will be saved in the home directory of the current user ~/.lenses/lenses-cli.yml. Depending on the configuration, this file will look similar to the following:

CurrentContext: master
Contexts:
  master:
    Host: http://lenseshost:9991
    Basic:
      Username: user
      Password: password_encoded
  myKerberos:
    Host: https://lenseshost:9991:443
    Insecure: true
    Debug: true
    Kerberos:
      ConfFile: /etc/krb5.conf
      WithPassword:
        Username: user
        Password: password_encoded
        Realm: mydomain.realm

Note

Older configuration formats are still compatible.

Now you are ready to move on with firing some commands like lenses-cli processors to list all available SQL processors.

You can still pass the credentials and host via the command line flags. They take precedence over the data in the configuration file. Alternatively, you can supply a different configuration file with the --config flag as follows:

lenses-cli --config config.yaml processors

The following precedence is used:

Individual flags, host, user and pass
Config file flag
Config file in $HOME/.lenses/

Tip

You can reset existing config contexts through configure --context=existingContext --reset command.

Passing credentials through flags¶

You can always individually configure and log in on the fly for a single command. This is very helpful when you use multi accounts.

$ lenses-cli --host="http://lenseshost:9991" --user="user" --pass="password" $command

All kerberos authentication methods are supported (password, keytab or from ccache).

$ lenses-cli --host="https://lenseshost:9991" --insecure --timeout=30s --kerberos-conf="/etc/krb5.conf" [--kerberos-realm="domain.realm"] [--kerberos-keytab="/tmp/krb5.keytab"] [--kerberos-ccache="/tmp/krb5-ccache.txt"] [--user="kerberosUsername"] [--password="kerberosPassword"] $command

Configuration Contexts¶

Lenses CLI is an application with environmental awareness. It accepts multi-environment configuration structure, each environment is known as “Context”, when no credentials flags are present then the “Current Context” is being used.

Create a new configuration for an environment:

$ lenses-cli configure --context=newContextName

Reset an existing Context:

$ lenses-cli configure --context=existingContext --reset

Switch between Contexts:

$ lenses-cli --context=contextName $command

View and verify the current Context:

$ lenses-cli context

View all available Contexts:

$ lenses-cli contexts

Tip

This command includes validation messages for each context’s configuration between its credentials and its particular lenses box (Host).

Getting help¶

Each command has a --help flag that will show you the available flags and a simple input example for the specific command.

For example, if we do lenses-cli --help, we will see the available (sub)commands and the global flags of the entire lenses-cli application:

Lenses-cli is the command line client for the Lenses REST API.

Usage:
lenses-cli [command]

Examples:
acl -h

Available Commands:
acl         Manage Access Control List
acls        Print the list of the available Kafka Access Control Lists
alert       Manage alerts
alerts      Print the registered alerts
audits      List the last buffered audit entries
configs     Print the whole lenses box configs
configure   Setup your environment for extensive CLI use. Create and save the required CLI configuration and client credentials
connector   Get information about a particular connector based on its name
connectors  List of active connectors' names
context     Print the current context or modify or delete a configuration context using the update and delete subcommands
contexts    Print and validate (through calls to the servers) all the available contexts from the configuration file
export      export a landscape
help        Help about any command
import      import a landscape
init-repo   Initialise a git repo to hold a landscape
license     Print the license information for the connected lenses box
logs        List the info or metrics logs
mode        Print the configuration's execution mode
policies    List all policies
policy      Manage a policy
processor   Manage a processor based on the processor id; pause, resume, update runners, delete or create a new processor
processors  List of all available processors
quota       Manage a quota, create a new quota or update and delete an existing one
quotas      List of all available quotas
schema      Manage particular schema based on its name, get a schema based on the ID or register a new one
schemas     List all available schemas
shell       shell
sql         sql queries, either browsing for continuous (live-stream)
topic       Manage particular topic based on the topic name, retrieve it or create a new one
topics      List all available topics
user        List information about the authenticated logged user such as the given roles given by the lenses administrator or manage the user's profile
version     Print the current version of lenses-cli

Flags:
    --config string            Load or save the host, user, pass and debug fields from or to a configuration file (yaml or json)
    --context string           Load specific environment, embedded configuration based on the configuration's 'Contexts'
    --debug                    Print some information that are necessary for debugging
    --header-bgcolor string    Table headers background gcolor=white
    --header-fgcolor string    Table headers forground gcolor=black
-h, --help                     help for lenses-cli
    --host string              Lenses host
    --insecure                 All insecure http requests
    --kerberos-ccache string   Kerberos keytab file
    --kerberos-conf string     krb5.conf
    --kerberos-keytab string   KeyTab file
    --kerberos-realm string    Kerberos realm
    --output string            TABLE, JSON or YAML results and hide all the info messages (default "table")
    --pass string              Password
    --timeout string           Timeout for the connection establishment
    --token string             Lenses auth token
    --user string              User
    --version                  version for lenses-cli

Use "lenses-cli [command] --help" for more information about a command.


Use "lenses-cli [command] --help" for more information about a command.

Another example is lenses-cli processors --help. Its output looks similar to the following:

List of all available processors

Usage:
lenses-cli processors [flags]
lenses-cli processors [command]

Examples:
processors

Available Commands:
logs        Retrieve SQL Processor logs. Available only in KUBERNETES execution mode

Flags:
    --cluster-name string   Select by cluster name, available only in CONNECT and KUBERNETES mode
-h, --help                 help for processors
    --name string          Select by processor name, available only in CONNECT and KUBERNETES mode
    --namespace string     Select by namespace, available only in KUBERNETES mode
    --pretty               Enable the pretty format for JSON output of commands (default false).
-q, --query string         Jmespath query expression. This allows for querying the JSON output of commands

Global Flags:
    --config string            Load or save the host, user, pass and debug fields from or to a configuration file (yaml or json)
    --context string           Load specific environment, embedded configuration based on the configuration's 'Contexts'
    --debug                    Print some information that are necessary for debugging
    --header-bgcolor string    Table headers background gcolor=white
    --header-fgcolor string    Table headers forground gcolor=black
    --host string              Lenses host
    --insecure                 All insecure http requests
    --kerberos-ccache string   Kerberos keytab file
    --kerberos-conf string     krb5.conf
    --kerberos-keytab string   KeyTab file
    --kerberos-realm string    Kerberos realm
    --output string            TABLE, JSON or YAML results and hide all the info messages (default "table")
    --pass string              Password
    --timeout string           Timeout for the connection establishment
    --token string             Lenses auth token
    --user string              User

Use "lenses-cli processors [command] --help" for more information about a command.

Tip

Use the -h flag on each subcommand to see the available options.

The –output flag¶

The --output flag, which controls the form/view of the output, can be accessed by the majority of the available commands.

The value of output can be Table, YAML or JSON. By default the results of a command are printed as a table.

Additionally, when the --output flag is set to JSON, two more optional flags are available for use named --pretty and --query. These flags can be passed to all commands that fetch and return JSON-formatted results.

`--pretty`	enable the pretty format for JSON output of commands (default false).
`--query`	a jmespath query expression. This allows for querying the JSON output of commands. For more details view the JMESPATH documentation.

Access Control Lists¶

Allows view and create, update and delete operations on Apache Kafka topic ACLs.

View ACLs

View all ACLs

$ lenses-cli acls --output json

[
 {
    "resourceType": "Topic",
    "resourceName": "transactions",
    "principal": "principalType:principalName",
    "permissionType": "Allow",
    "host": "*",
    "operation": "Read"
 }
]

Set/Update ACLs

Modify and create ACLs

$ lenses-cli acl set \
    --resourceType="Topic" \
    --resourceName="transactions" \
    --principal="principalType:principalName" \
    --permissionType="Allow" \
    --acl-host="*" \
    --operation="Read"

ACLs can also be created from a file:

$ lenses-cli acl create ./file.yaml

Delete ACLs

Delete ACLs

$ lenses-cli acl delete \
    --resource-type="Topic" \
    --resource-name="transactions" \
    --principal="principalType:principalName" \
    --permission-type="Allow" \
    --acl-host="*" \
    --operation="Read"

ACLs can also be deleted from a file:

$ lenses-cli acl delete ./file.yaml

file.yaml:

resourceType: Topic
resourceName: transactions
principal: principalType:principalName
permissionType: Allow
host: "*"
operation: Read

Note

Command line flags are ignored when loading from files.

Lenses Configuration¶

This command allows you to view the main configurations that Lenses exposes.

$ lenses-cli configs --output json

Example output:

{
...snip

"lenses.ip": "0.0.0.0",
"lenses.jmx.port": 24027,
"lenses.kafka.brokers": "SASL_PLAINTEXT://kafka1:19094,SASL_PLAINTEXT://kafka2:19094",
"lenses.kubernetes.config.file": "",
"lenses.kubernetes.processor.image.name": "domain.com/lenses-image",
"lenses.kubernetes.processor.image.tag": "2.2",
"lenses.kubernetes.service.account": "default",
"lenses.license.file": "license.json",
"lenses.port": 24017,
"lenses.root.path": "",
"lenses.security.mode": "BASIC",
"lenses.sql.execution.mode": "KUBERNETES",

...snip
}

To see the SQL processors deployment mode, you can inspect the lenses.sql.execution.mode. For example, in CONNECT mode a cluster name is required and for KUBERNETES an additional namespace is required to deploy processors.

$ lenses-cli mode
IN_PROC

$ lenses-cli configs mode
IN_PROC

Get single entry config

Returns an individual config setting.

$ lenses-cli configs lenses.port
24017

Connectors¶

Create, modify, pause, restart and remove Apache Kafka Connect connectors.

View Connectors

List currently deployed connectors, optionally filter by name, cluster and namespace:

# list connectors in the cluster dev
$ lenses-cli connectors --cluster-name="dev" [--output [--query] [--no-pretty]] [--names [--unwrap]]

Note

The --names flag displays only the names of the connectors. When used with --unwrap, it will print each connector name on a separate line in the output.

List all the supported Kafka Connectors instead of the currently deployed:

$ lenses-cli connectors --supported

Plugins

View the available plugins per cluster or filter by cluster:

lenses-cli connectors plugins --cluster-name="dev"

Create

Create connectors via the command line flags or file:

$ lenses-cli connector create --cluster-name="dev" --name="connectorName" --configs="{\"key\": \"value\"}"

From a file:

$ lenses-cli connector create ./connector.yaml

connector.yaml:

clusterName: development
config:
  connector.class: "org.apache.kafka.connect.file.FileStreamSinkConnector"
  file: "/dev/null"
  tasks.max: "1"
  topics: "reddit_posts"
  name: "nullsink"

Note

Command line flags are ignored when loading from files.

Configs

View the configuration of a connector:

$ lenses-cli connector config --cluster-name="dev" --name="connectorName"

Status

View the status of a connector:

$ lenses-cli connector status --cluster-name="dev" --name="connectorName"

Pause

Pause a connector:

$ lenses-cli connector pause --cluster-name="dev" --name="connectorName"

Resume

Resume a connector:

$ lenses-cli connector resume --cluster-name="dev" --name="connectorName"

Restart

Restart a connector:

$ lenses-cli connector restart --cluster-name="dev" --name="connectorName"

Restart a task

Restart a connector task:

$ lenses-cli connector task restart --task=0 --cluster-name="dev" --name="connectorName"

View tasks

View tasks for a connector:

$ lenses-cli connector tasks --cluster-name="dev" --name="connectorName"

Task status

View the status of a task for a connector:

$ lenses-cli connector task status --cluster-name="dev" --name="connectorName" --task=1

Delete

Delete a connector:

$ lenses-cli connector delete --cluster-name="dev" --name="connectorName"

Lenses SQL¶

The CLI allows you to browse and subscribe to topic data with SQL. Please refer to the Lenses SQL documentation for more details.

The data returned can be controlled via extra flags:

keys print message keys
keys-only print the keys only, not the value of the message
meta print the message metadata, parition number and offsets
stats print query statistics as the last message

Browsing

Browse data via the sql command, optionally validate and output stats:

$ lenses-cli query --keys --meta "SELECT score, author FROM reddit_posts LIMIT 50"

# execute with stats every 2 seconds, show offsets at end
$ lenses-cli query --stats "SELECT score, author FROM reddit_posts LIMIT 50"

Live

Live continuous queries update according to the query and never stop until terminated:

$ lenses-cli --live-stream query "SELECT score, author FROM reddit_posts WHERE _sample=2 AND _sampleWindow=200"

Processors¶

The CLI can deploy and manage SQL processors deploy IN_PROC in Lenses or into Apache Kafka Connect or Kubernetes via Lenses.

View processors

View all processors registered, optionally filter by name, cluster and namespace.

$ lenses-cli processors
# filter by namespace
$ lenses-cli processors --namespace lenses-sql

# use the query flag to add further querying
$ lenses-cli processors --output --query="[?ClusterName == 'IN_PROC'].Name | sort(@) | {Processor_Names_IN_PROC: join(', ', @)}"

Create processors

Create processors from the command line flags or from a file. For IN_PROC mode you only need the name, for CONNECT mode the name and cluster, for KUBERNETES mode all three:

The following flags are valid for all processor actions

–id (required if –name) is missing
–name (required if –id is missing)
–cluster-name (should be passed if KUBERNETES mode)
–namespace (should be passed if KUBERNETES mode)

$ lenses-cli processor create \
    --name="processorName" \
    --runners=1 \
    --cluster-name="clusterName" \
    --namespace="ns" \
    --pipeline="prod" \
    --sql="SET autocreate=true;INSERT INTO topicB SELECT * FROM topic1"

From a file:

$ lenses-cli processor create ./processor.yaml

processor.yaml:

Name: processorName
Runners: 1
SQL: >
  SET autocreate=true;INSERT INTO topicB SELECT * FROM topic1

Note

Command line flags are ignored when loading from files

Pause a processor

Pause a running processor. For IN_PROC mode you only need the name, for CONNECT mode the name and cluster, for KUBERNETES mode all three:

$ lenses-cli processor pause --cluster-name="clusterName?" --namespace="namespace?" --name="processorName"

Resume a processor

Resume a paused processor. For IN_PROC mode you only need the name, for CONNECT mode the name and cluster, for KUBERNETES mode all three:

$ lenses-cli processor resume --cluster-name="clusterName?" --namespace="namespace?" --name="processorName"

Update a processor

Update a processor from the command line flags or file. For IN_PROC mode you only need the name, for CONNECT mode the name and cluster, for KUBERNETES mode all three:

$ lenses-cli processor update \
    --cluster-name="clusterName?" \
    --namespace="namespace" \
    --name="processorName" \
    --runners=2

From a file:

$ lenses-cli processor update ./processor.yaml

processor.yaml:

Name: processorName
Runners: 1
SQL: >
  SET autocreate=true;INSERT INTO topicB SELECT * FROM topic1

Warning

You can not update the SQL, only the runners to scale up or down

Delete a processor

Delete a processor by name, cluster and namespace. For IN_PROC mode you only need the name, for CONNECT mode the name and cluster, for KUBERNETES mode all three.

$ lenses-cli processor delete \
    --namespace="namespace" \
    --cluster-name="clusterName" \
    --name="processorName

View processors logs

Available only in KUBERNETES execution mode.

$ lenses-cli processors logs --cluster-name=clusterName --namespace=nameSpace --podName=runnerStateID [--follow --lines=50]

Schemas¶

View, modify and delete schemas and adjust compatibility levels.

View all schemas

View the current schemas registered:

$ lenses-cli schemas --unwrap [--output [--query] [--no-pretty]]

Note

The --unwrap flag is optional, prints only the names separated by new line

View a schema

Schemas can be filtered by name:

$ lenses-cli schema --name="sea_vessel_position_reports-value"

View schema versions

Schemas versions can be viewed filtered by name:

$ lenses-cli schema versions --name="sea_vessel_position_reports-value"
# view by version
$ lenses-cli schema --name="sea_vessel_position_reports-value" --version="latest"

View schema compatibility levels

Schema compatibility levels can be viewed and filtered by name:

$ lenses-cli schema --name="register1" compatibility

Delete schemas

Delete all versions of a schema by name:

$ lenses-cli schema delete --name="coyote_test_02"

Delete schemas by version

Delete a specific version of a schema:

# (defaults to "latest" if not passed)
$ lenses-cli schema delete-version --name="register1" --version=2

Change compatibility of a Schema

Change the compatibility level of a schema:

$ lenses-cli schema --name="register1" compatibility set FULL

Register a schema

Register a new schema for a name/subject by the command line or file:

$ lenses-cli schema register \
    --name="schemaName" \
    --avro="{ \"type\": \"string\" }"

Register by file:

$ lenses-cli schema register ./schema.yaml

schema.yaml:

Name: schemaName
AvroSchema: |-
  {
    "type": "record",
    "name": "evolution",
    "namespace": "com.landoop",
    "doc": "This is a sample AVRO schema to get you started. Please edxt",
    "fields": [
      {
       "name": "name",
       "type": "string"
      },
      {
       "name": "number1",
       "type": "int"
      },
      {
       "name": "number2",
       "type": "float"
      }
    ]
   }

Topics¶

View, create and modify topics:

View topics

View all topics.

$ lenses-cli topics [--names [--unwrap]]

Note

The optional names flag will display only the names of the topics, and if --unwrap then each topic name will be displayed on a new line as a simple text

View a topic

View a specific topic:

$ lenses-cli topic --name=reddit_posts

Update configs

Update a topic configuration:

$ lenses-cli topic update \
    --name="topic2" \
    --configs="{\"max.message.bytes\": \"1000020\"}"

Update from a file:

$ lenses-cli topic update ./topic.yaml

topic.yml:

name: topicName
replication: 1
partitions: 1
configs:
  max.message.bytes: "1000010"

Note

Command line flags are ignored when loading from files

Create topics

Create topics from the command line or files. If no replication, partitions or configuration is specific the broker defaults are used:

$ lenses-cli topic create \
    --name="topicName" \
    --replication=1 \
    --partitions=1 \
    --configs="{\"max.message.bytes\": \"1000010\"}"

For a file:

$ lenses-cli topic create ./topic.yml

topic.yml:

name: topicName
replication: 1
partitions: 1
configs:
  max.message.bytes: "1000010"

Delete topics

Delete topics by name:

$ lenses-cli topic delete --name="topic2"

Quotas¶

View, create, modify and delete quotas.

View quotas

View all quotas:

$ lenses-cli quotas [--output]

Set quotas for users

Create and update quotas for users from command line flags or files:

$ lenses-cli quota users set [--quota-user="user"] [--quota-client=""] \
    --quota-config="{\"producer_byte_rate\": \"100000\",\"consumer_byte_rate\": \"200000\",\"request_percentage\": \"75\"}"

From a file:

$ lenses-cli quotas users set ./quota.yaml

quota.yaml:

# User name for a specific user quota.
# If empty or omitted then create/update the default user quota.
User: user
# All clients, it could be empty as well.
# Change it for a specific client.
ClientID: "*"
Config:
  ProducerByteRate: "100000"
  ConsumerByteRate: "200000"
  RequestPercentage: "75"

Set quotas for clients

Create and update quotas for clients from command line flags or files:

$ lenses-cli quota clients set [--quota-client=""] \
    --quota-config="{\"producer_byte_rate\": \"100000\",\"consumer_byte_rate\": \"200000\",\"request_percentage\": \"75\"}"

From a file:

$ lenses-cli quotas clients set ./quota-clients.yaml

quota-clients.yaml:

# Default client quota, it could be empty or "all" as well.
# Change it for a specific client quota.
ClientID: "*"
Config:
  ProducerByteRate: "100000"
  ConsumerByteRate: "200000"
  RequestPercentage: "75"

Delete for users

Delete quotas for users:

# --quota-user, optional. If empty then default user quota, otherwise specific user quota.
# --quota-client, optional. If empty or * then all clients, otherwise specific client.
# arguments, like producer_byte_rate, are not required.
#   Remove user quota config's specific properties, if empty then all properties will be passed on automatically
#   and the user quota will be removed entirely.
$ lenses-cli quota users delete [--quota-client=""] [--quota-user=""] \
    producer_byte_rate consumer_byte_rate request_percentage

# delete for the default user quota
$ lenses-cli quota users delete

# delete for a specific user quota
$ lenses-cli quota users delete \
    --quota-user="user"

# delete for a specific user and client
$ lenses-cli quota users delete \
    --quota-user="user" \
    --quota-client="clientID"

# delete for a specific user quota's property
$ lenses-cli quota users delete \
    --quota-user="user" \
    request_percentage

Delete for clients

Delete quotas for clients:

# --quota-client, optional. If empty then default client quota, otherwise specific client quota.
# arguments, like producer_byte_rate, are not required.
#   Remove client quota config's specific properties, if empty then all properties will be passed on automatically
#   and the client quota will be removed entirely.
$ lenses-cli quota clients delete [--quota-client=""] \
    producer_byte_rate consumer_byte_rate request_percentage

# delete for the default client quota
$ lenses-cli quota clients delete

# delete for a specific client quota
$ lenses-cli quota clients delete \
    --quota-client="clientID"

# delete for a specific client quota's property
$ lenses-cli quota clients delete \
    --quota-client="clientID" \
    request_percentage

Alerts¶

View all available alerts or modify and delete alert settings.

View all available Alerts

$ lenses-cli alerts [--live]

Note

The --live flag is optional, it displays real-time alert notifications

View all Alert Settings

$ lenses-cli alert settings

View an Alert Setting

$ lenses-cli alert setting --id=2000 --output

{
    "id": 2000,
    "description": "Consumer Lag exceeded",
    "category": "Consumers",
    "enabled": true,
    "docs": "Raises an alert when the consumer lag exceeds the threshold",
    "conditionTemplate": "lag >= $Threshold-Number on group $Consumer-Group and topic $Topic-Name",
    "conditionRegex": "lag >= ([1-9][0-9]*) on group (\\b\\S+\\b) and topic (\\b\\S+\\b)",
    "conditions": {
        "28bbad2b-69bb-4c01-8e37-28e2e7083aa9": "lag >= 100000 on group group and topic topicA",
        "4318a5a7-32e4-43af-9f3f-438e64d14a11": "lag >= 1000000 on group consumerA and topic topicA",
        "92dd89a1-83c0-4251-8610-36fce780a824": "lag >= 1000000 on group minikube.default.london-keyword-count and topic reddit_posts",
        "bdb01792-b79f-44d0-a39e-7f3e0906a89b": "lag >= 100000 on group connect-nullsink and topic topicA",
        "f4dd550f-bcb6-4e02-b28b-18bbd426aaf2": "lag >= 100000 on group groupA and topic topicA"
    },
    "isAvailable": true
}

Enable an Alert Setting

$ lenses-cli alert setting --id=1000 --enable

View all conditions that belong to a specific Alert Setting

$ lenses-cli alert setting conditions --alert=2000 --output

{
    "28bbad2b-69bb-4c01-8e37-28e2e7083aa9": "lag >= 100000 on group group and topic topicA",
    "4318a5a7-32e4-43af-9f3f-438e64d14a11": "lag >= 1000000 on group consumerA and topic topicA",
    "7643d1ec-73f9-4913-95d1-87e62f1ec172": "lag >= 200000 on group groupA and topic topicA",
    "92dd89a1-83c0-4251-8610-36fce780a824": "lag >= 1000000 on group minikube.default.london-keyword-count and topic reddit_posts",
    "bdb01792-b79f-44d0-a39e-7f3e0906a89b": "lag >= 100000 on group connect-nullsink and topic topicA",
    "f4dd550f-bcb6-4e02-b28b-18bbd426aaf2": "lag >= 100000 on group groupA and topic topicA"
}

Create or update an existing condition for a specific Alert Setting

$ lenses-cli alert setting condition set \
--alert=2000 \
--condition="lag >= 200000 on group groupA and topic topicA"

Update condition from a file:

$ lenses-cli alert setting condition set ./alert_cond.yaml

alert_cond.yaml:

Alert: 2000
Condition: >
lag >= 200000 on
group groupA and topic topicA

Note

Command line flags are ignored when loading from files

Delete an existing condition for a specific Alert Setting

$ lenses-cli alert setting condition delete \
--alert=2000 \
--condition="7643d1ec-73f9-4913-95d1-87e62f1ec172"

Note

The --alert flag is the Alert ID that the alert setting belongs to, the --condition flag is the UUID of the condition to delete

Audits¶

View audits

$ lenses-cli audits [--live] [--with-content]

Note

The --live flag is optional, it displays real-time audits. The --with-content flag includes the exact changes per action when table mode

The below command will output all audits fields as JSON, like the rest of the commands that output a structured result:

$ lenses-cli audits --output [--query] [--no-pretty]

Logs¶

View INFO logs

$ lenses-cli logs # or lenses-cli logs info.

View METRICS logs

$ lenses-cli logs metrics

Note

The --output flag will not result in a different output here because the logs are displayed as simple (colorful) text.

User Profile¶

List information about the authenticated logged user, such as the given roles given by the lenses administrator

$ lenses-cli user

List user’s favourites

$ lenses-cli user profile [--silent]

Delete a value from a user profile’s property

$ lenses-cli user profile delete existingProperty existingValueFromProperty [--silent]

Set a value to a user profile’s property

$ lenses-cli user profile set newProperty newValueToTheProperty [--silent]

Data Policies¶

View data policies

$ lenses-cli policies

View a specific policy

$ lenses-cli policy view --name my-policy

Create a policy

Fields are set by suppling a comma separated string to the field flag.

$ lenses-cli policy create --name my-policy --category my-category --impact-type HIGH --redaction First-1 --fields f1,f2,f3

# from file
$ lenses-cli policy create ./policy.yaml

Update a policy

Fields are set by suppling a comma separated string to the field flag.

$ lenses-cli policy update --id my-policy-id --name my-policy --category my-category --impact-type HIGH --redaction First-1 --fields f1,f2,f3

Delete a policy

$ lenses-cli policy delete --id my-policy-id

Exporting¶

The CLI allows you to export all resource in Lenses as requests so that they can be version controlled and imported into another environment.

Export is done per resource, the directory structure defined by the cli, a base directory can be provided by the --dir flag.

Processors, connectors, topic and schemas have an additional prefix flag to restrict resources to export.

export acls --dir my-dir
export alert-settings --dir my-dir
export connectors --dir my-dir
export processors --dir my-dir
export quota --dir my-dir
export schemas --dir my-dir
export topics --dir my-dir
export policies --dir my-dir

The directory structure is:

<directory from flag>
├── alert-settings
│   └── alert-setting.yaml
├── apps
│   ├── connectors
│   │   ├── connector-1.yaml
│   │   └── connector-2.yaml
│   └── sql
├── kafka
│   ├── quotas
│   │   └── quotas.yaml
│   └── topics
│       ├── topic-1.yaml
│       └── topic-2.yaml
├── policies
│   └── policies-city.yaml
└── schemas
    ├── schema-1.yaml
    └── schema-2.yaml

Importing¶

The CLI allows you to import resources from files in Lenses into another environment.

Import is done per resource, the directory structure defined by the cli, a base directory can be provided by the --dir flag.

Processors, connectors, topic and schemas have an additional prefix flag to restrict resources to export.

export acls --dir my-dir
export alert-settings --dir my-dir
export connectors --dir my-dir
export processors --dir my-dir
export quota --dir my-dir
export schemas --dir my-dir
export topics --dir my-dir
export policies --dir my-dir

The directory structure is:

<directory from flag>
├── alert-settings
│   └── alert-setting.yaml
├── apps
│   ├── connectors
│   │   ├── connector-1.yaml
│   │   └── connector-2.yaml
│   └── sql
├── kafka
│   ├── quotas
│   │   └── quotas.yaml
│   └── topics
│       ├── topic-1.yaml
│       └── topic-2.yaml
├── policies
│   └── policies-city.yaml
└── schemas
    ├── schema-1.yaml
    └── schema-2.yaml

Interactive Shell¶

The interactive shell allows you to have an interactive experience and issue Lenses SQL queries from the command line.

The following options are available:

keys print message keys
keys-only print the keys only, not the value of the message
meta print the message metadata, parition number and offsets
stats print query statistics as the last message
pretty pretty print the json output
live-stream run in continuous query mode
!options prints the current options

Options are set using the ! prefix.

Queries are executed by ;, multiline queries are possible but not available in the history inside the shell. After a restart the multiline queries are concatenated into one line and can be selected.

Query history is stored in .lenes/history in the home directory of the user.

    __                                 ________    ____
   / /   ___  ____  ________  _____   / ____/ /   /  _/
  / /   / _ \/ __ \/ ___/ _ \/ ___/  / /   / /    / /
 / /___/  __/ / / (__  )  __(__  )  / /___/ /____/ /
/_____/\___/_/ /_/____/\___/____/   \____/_____/___/
Docs at https://docs.lenses.io
Connected to [https://master.lensesui.dev.landoop.com:443] as [andrew], context [master]
Use "!" to set output options [!keys|!keysOnly|!stats|!meta|!pretty]
Crtl+D to exit

lenses-sql>

Troubleshooting¶

For troubleshooting or additional information join our slack channel.