SQL Processors


The CLI allows you to create, modify, pause, restart and remove Lenses SQL Processors. The commands for these actions are shown in the table below.

CommandsDescription
createCreate a new processor
deleteDelete an existing processor
startStart an existing processor
stopStop a running processor
updateUpdate the processor runners
viewView a processor

Note: Kafka Connect cluster permissions will be applied to the operation.

View processors 

View all processors registered. Optionally filter by name, cluster and namespace.

lenses-cli processors
# filter by namespace
lenses-cli processors --namespace lenses-sql

# Please note the following columns in the table: 
# * 'ID' which should be used as the 'id' required for the commands below.
# * 'PROCESSORID' , which the user may have configured in creating the processor, and will be used as the consumer group of the processor.

Create processors 

For IN_PROC you only need the name, for KUBERNETES all three. Creating an IN_PROC processor, will start it automatically. This is not the case for Kubernetes deployments, a start command (see below) is required once created.

  • --id (the processor id, optional)
  • --name (optional)
  • --cluster-name (should be passed if KUBERNETES)
  • --namespace (should be passed if KUBERNETES)
  • --runners (number of times to deploy while in KUBERNETES)
lenses-cli processor create \
    --name=[PROCESSOR_NAME] \
    --runners=1 \
    --cluster-name=[CLUSTER_NAME] \
    --namespace=[NAMESPACE] \
    --pipeline="prod" \
    --sql="SET defaults.topic.autocreate=true;INSERT INTO topicB SELECT STREAM * FROM topicA"

From a file:

lenses-cli processor create ./processor.yaml

name: processorName
runners: 1
sql: |-
    SET defaults.topic.autocreate=true;INSERT INTO topicB SELECT STREAM * FROM topic1

Stop a processor 

For IN_PROC you only need the name, for KUBERNETES all three:

lenses-cli processor stop \
    --cluster-name=[CLUSTER_NAME] \
    --namespace=[NAMESPACE] \
    --name=[PROCESSOR_NAME]

Start a processor 

For IN_PROC you only need the name, for KUBERNETES all three:

lenses-cli processor start \
    --cluster-name=[CLUSTER_NAME] \
    --namespace=[NAMESPACE]\
    --name=[PROCESSOR_NAME]

Update a processor runners 

Update a processor from the command line flags or file. This is not applicable for IN_PROC only for KUBERNETES:

lenses-cli processor update \
    --cluster-name=[CLUSTER_NAME] \
    --namespace=[NAMESPACE]\
    --name=[PROCESSOR_NAME] \
    --runners=2

Delete a processor 

Delete a processor by name, cluster and namespace. For IN_PROC you only need the name, and for KUBERNETES all 3 (name, cluster, namespace).

lenses-cli processor delete \
    --namespace=[NAMESPACE]\
    --cluster-name=[CLUSTER_NAME] \
    --name=[PROCESSOR_NAME]

View processors logs 

Available only for KUBERNETES.

lenses-cli processors logs \
    --cluster-name=[CLUSTER_NAME] \
    --namespace=[NAMESPACE] \
    --podName=[RUNNER_STATEID] [--follow --lines]
--
Last modified: June 18, 2024