Kafka Connect


The CLI can create, modify, pause, restart, and remove Apache Kafka Connect connectors. Kafka Connect cluster permissions will be applied to each operation. The commands for these actions are shown in the table below.

CommandsDescription
configGet the connector configuration
createCreate a new connector
deleteDelete a connector
pausePause a connector
restartRestart a connector
resumeResume a paused connector
statusGet the status of a connector
taskManage a specific task from a connector
tasksList the tasks of a connector
updateUpdate the configuration of a connector

View connectors 

List the connectors

lenses-cli connectors 

List currently deployed connectors, optionally filter by name, cluster and namespace:

lenses-cli connectors --cluster-name="dev" 

#[[--names [--unwrap]]

The --names flag displays only the names of the connectors. When used with --unwrap, it will print each connector name on a separate line in the output.

List all the supported connectors 

lenses-cli connectors --supported

View the available plugins per cluster 

lenses-cli connectors plugins --cluster-name="dev"

Create connectors 

lenses-cli connector create \
    --name=example \
    --cluster-name=dev \
    --configs='{"connector.class":"org.apache.kafka.connect.file.FileStreamSinkConnector", "file":"/dev/null", "name":"example", "tasks.max":"8", "topics":"test_topic"}'

You can create a connector using a file:

lenses-cli connector create ./connector.yaml

Example file:

clusterName: dev
config:
  connector.class: "org.apache.kafka.connect.file.FileStreamSinkConnector"
  file: "/dev/null"
  tasks.max: "2"
  topics: "test_topic"
  name: "example"

Update a connector 

# Inline
lenses-cli connector update \
    --name=example \
    --cluster-name=dev \
    --configs='{"connector.class":"org.apache.kafka.connect.file.FileStreamSinkConnector", "file":"/dev/null", "name":"example", "tasks.max":"2", "topics":"test_topic"}'

# or from file
lenses-cli connector update ./connector.yaml

View the configuration of a connector 

lenses-cli connector config \
    --cluster-name="dev" \
    --name="nullsink"

View the status of a connector 

lenses-cli connector status \
    --cluster-name="dev" \
    --name="nullsink"

Pause a connector 

lenses-cli connector pause \
    --cluster-name="dev" \
    --name="nullsink"

Resume a connector 

lenses-cli connector resume \
    --cluster-name="dev" \
    --name="nullsink"

Restart a connector 

lenses-cli connector restart \
    --cluster-name="dev" \
    --name="nullsink"

Restart a connector task 

lenses-cli connector task restart \
    --task=0 \
    --cluster-name="dev" \
    --name="nullsink"

View tasks for a connector 

lenses-cli connector tasks \
    --cluster-name="dev" \
    --name="nullsink"

View the status of a task for a connector 

lenses-cli connector task status \
    --cluster-name="dev" \
    --name="nullsink" --task=1

Delete a connector 

lenses-cli connector delete \
    --cluster-name="dev" \
    --name="nullsink"
--
Last modified: June 18, 2024