1 of 3

Applications

Kafka Connectors

This page describes the commands to manage Connectors in Lenses via the CLI.

The CLI can create, modify, pause, restart and remove Apache Kafka Connect connectors.

Kafka Connect cluster permissions will be applied to the operation.

View connectors

List the currently deployed connectors, optionally filter by name, cluster and namespace:

lenses-cli connectors --cluster-name="dev" 

#[[--names [--unwrap]]

The --names flag displays only the names of the connectors. When used with --unwrap, it will print each connector name on a separate line in the output.

List all the supported connectors

lenses-cli connectors --supported

View the available plugins per cluster

lenses-cli connectors plugins --cluster-name="dev"

Create connectors

lenses-cli connector create \
    --cluster-name="dev" \
    --name="connectorName" \
    --configs="{\"key\": \"value\"}"

From a file:

lenses-cli connector create ./connector.yaml

Example file:

clusterName: development
config:
  connector.class: "org.apache.kafka.connect.file.FileStreamSinkConnector"
  file: "/dev/null"
  tasks.max: "1"
  topics: "reddit_posts"
  name: "nullsink"

Update a connector

# Inline
lenses-cli connector update --name=nullsink --cluster-name=dev \
  --configs="{\"connector.class\":\"org.apache.kafka.connect.file.FileStreamSinkConnector\",\"file\":\"/dev/null\",\"name\":\"nullsink\",\"tasks.max\":\"8\",\"topics\":\"reddit_posts\"}"

# or from file
lenses-cli connector update ./connector.yaml

View the configuration of a connector

lenses-cli connector config \
    --cluster-name="dev" \
    --name="connectorName"

View the status of a connector

lenses-cli connector status \
    --cluster-name="dev" \
    --name="connectorName"

Pause a connector

lenses-cli connector pause \
    --cluster-name="dev" \
    --name="connectorName"

Resume a connector

lenses-cli connector resume \
    --cluster-name="dev" \
    --name="connectorName"

SQL Processors

This page describes the commands to manage SQL Processors in Lenses via the CLI.

The CLI can create, modify, pause, restart and remove Lenses SQL Processors.

View processors

View all processors registered. Optionally filter by name, cluster and namespace.

lenses-cli processors
# filter by namespace
lenses-cli processors --namespace lenses-sql

# Please note the following columns in the table: 
# * 'ID' which should be used as the 'id' required for the commands below.
# * 'PROCESSORID' , which the user may have configured in creating the processor, and will be used as the consumer group of the processor.

Create processors

For IN_PROC you only need the name, for KUBERNETES all three. Creating an IN_PROC processor will start it automatically. This is not the case for Kubernetes deployments, a start command (see below) is required once created.

--id (the processor id, optional)
--name (optional)
--cluster-name (should be passed if KUBERNETES)
--namespace (should be passed if KUBERNETES)
--runners (number of times to deploy while in KUBERNETES)

lenses-cli processor create \
    --name="processorName" \
    --runners=1 \
    --cluster-name="clusterName" \
    --namespace="ns" \
    --pipeline="prod" \
    --sql="SET defaults.topic.autocreate=true;INSERT INTO topicB SELECT STREAM * FROM topic1"

From a file:

lenses-cli processor create ./processor.yaml

name: processorName
runners: 1
sql: |-
    SET defaults.topic.autocreate=true;INSERT INTO topicB SELECT STREAM * FROM topic1

Stop a processor

For IN_PROC you only need the name, for KUBERNETES all three:

lenses-cli processor stop \
    --cluster-name="clusterName?" \
    --namespace="namespace?" \
    --name="processorName"

Start a processor

For IN_PROC you only need the name, for KUBERNETES all three:

lenses-cli processor start \
    --cluster-name="clusterName?" \
    --namespace="namespace?" \
    --name="processorName"

Update the number of runners

Update a processor from the command line flags or files. Only for KUBERNETES:

lenses-cli processor update \
    --cluster-name="clusterName?" \
    --namespace="namespace" \
    --name="processorName" \
    --runners=2

From a file.

lenses-cli processor update ./processor.yaml

name: processorName
runners: 1
sql: |-
    SET defaults.topic.autocreate=true;INSERT INTO topicB SELECT STREAM * FROM topic1

Delete a processor

Delete a processor by name, cluster and namespace. For IN_PROC you only need the name, and for KUBERNETES all 3 (name, cluster, namespace).

lenses-cli processor delete \
    --namespace="namespace" \
    --cluster-name="clusterName" \
    --name="processorName

View processors logs

Available only for KUBERNETES.

lenses-cli processors logs \
    --cluster-name=clusterName \
    --namespace=nameSpace \
    --podName=runnerStateID [--follow --lines]