SQL Processors

The CLI can create, modify, pause, restart and remove Lenses SQL Processors.

Note: Kafka Connect cluster permissions will be applied to the operation.

View processors

View all processors registered. Optionally filter by name, cluster and namespace.

lenses-cli processors
# filter by namespace
lenses-cli processors --namespace lenses-sql

# Please note the following columns in the table: 
# * 'ID' which should be used as the 'id' required for the commands below.
# * 'PROCESSORID' , which the user may have configured in creating the processor, and will be used as the consumer group of the processor.

Create processors

For IN_PROC you only need the name, for KUBERNETES all three. Creating an IN_PROC processor, will start it automatically. This is not the case for Kubernetes deployments, a start command (see below) is required once created.

--id (the processor id, optional)
--name (optional)
--cluster-name (should be passed if KUBERNETES)
--namespace (should be passed if KUBERNETES)
--runners (number of times to deploy while in KUBERNETES)

lenses-cli processor create \
    --name="processorName" \
    --runners=1 \
    --cluster-name="clusterName" \
    --namespace="ns" \
    --pipeline="prod" \
    --sql="SET defaults.topic.autocreate=true;INSERT INTO topicB SELECT STREAM * FROM topic1"

From a file:

lenses-cli processor create ./processor.yaml

name: processorName
runners: 1
sql: |-
    SET defaults.topic.autocreate=true;INSERT INTO topicB SELECT STREAM * FROM topic1

Stop a processor

For IN_PROC you only need the name, for KUBERNETES all three:

lenses-cli processor stop \
    --cluster-name="clusterName?" \
    --namespace="namespace?" \
    --name="processorName"

Start a processor

For IN_PROC you only need the name, for KUBERNETES all three:

lenses-cli processor start \
    --cluster-name="clusterName?" \
    --namespace="namespace?" \
    --name="processorName"

Update a processor runners

Update a processor from the command line flags or file. This is not applicable for IN_PROC only for KUBERNETES:

lenses-cli processor update \
    --cluster-name="clusterName?" \
    --namespace="namespace" \
    --name="processorName" \
    --runners=2

From a file.

lenses-cli processor update ./processor.yaml

name: processorName
runners: 1
sql: |-
    SET defaults.topic.autocreate=true;INSERT INTO topicB SELECT STREAM * FROM topic1

Delete a processor

Delete a processor by name, cluster and namespace. For IN_PROC you only need the name, and for KUBERNETES all 3 (name, cluster, namespace).

lenses-cli processor delete \
    --namespace="namespace" \
    --cluster-name="clusterName" \
    --name="processorName

View processors logs

Available only for KUBERNETES.

lenses-cli processors logs \
    --cluster-name=clusterName \
    --namespace=nameSpace \
    --podName=runnerStateID [--follow --lines]

Last modified: April 17, 2024