SQL Processors

The CLI allows you to create, modify, pause, restart and remove Lenses SQL Processors. The commands for these actions are shown in the table below.

Commands	Description
`create`	Create a new processor
`delete`	Delete an existing processor
`start`	Start an existing processor
`stop`	Stop a running processor
`update`	Update the processor runners
`view`	View a processor

Note: Kafka Connect cluster permissions will be applied to the operation.

View processors

View all processors registered. Optionally filter by name, cluster and namespace.

lenses-cli processors
# filter by namespace
lenses-cli processors --namespace lenses-sql

# Please note the following columns in the table: 
# * 'ID' which should be used as the 'id' required for the commands below.
# * 'PROCESSORID' , which the user may have configured in creating the processor, and will be used as the consumer group of the processor.

Create processors

For IN_PROC you only need the name, for KUBERNETES all three. Creating an IN_PROC processor, will start it automatically. This is not the case for Kubernetes deployments, a start command (see below) is required once created.

--id (the processor id, optional)
--name (optional)
--cluster-name (should be passed if KUBERNETES)
--namespace (should be passed if KUBERNETES)
--runners (number of times to deploy while in KUBERNETES)

lenses-cli processor create \
    --name=[PROCESSOR_NAME] \
    --runners=1 \
    --cluster-name=[CLUSTER_NAME] \
    --namespace=[NAMESPACE] \
    --pipeline="prod" \
    --sql="SET defaults.topic.autocreate=true;INSERT INTO topicB SELECT STREAM * FROM topicA"

From a file:

lenses-cli processor create ./processor.yaml

name: processorName
runners: 1
sql: |-
    SET defaults.topic.autocreate=true;INSERT INTO topicB SELECT STREAM * FROM topic1

Stop a processor

For IN_PROC you only need the name, for KUBERNETES all three:

lenses-cli processor stop \
    --cluster-name=[CLUSTER_NAME] \
    --namespace=[NAMESPACE] \
    --name=[PROCESSOR_NAME]

Start a processor

For IN_PROC you only need the name, for KUBERNETES all three:

lenses-cli processor start \
    --cluster-name=[CLUSTER_NAME] \
    --namespace=[NAMESPACE]\
    --name=[PROCESSOR_NAME]

Update a processor runners

Update a processor from the command line flags or file. This is not applicable for IN_PROC only for KUBERNETES:

lenses-cli processor update \
    --cluster-name=[CLUSTER_NAME] \
    --namespace=[NAMESPACE]\
    --name=[PROCESSOR_NAME] \
    --runners=2

Delete a processor

Delete a processor by name, cluster and namespace. For IN_PROC you only need the name, and for KUBERNETES all 3 (name, cluster, namespace).

lenses-cli processor delete \
    --namespace=[NAMESPACE]\
    --cluster-name=[CLUSTER_NAME] \
    --name=[PROCESSOR_NAME]

View processors logs

Available only for KUBERNETES.

lenses-cli processors logs \
    --cluster-name=[CLUSTER_NAME] \
    --namespace=[NAMESPACE] \
    --podName=[RUNNER_STATEID] [--follow --lines]

Last modified: July 26, 2024