Kafka Connect

The CLI can create, modify, pause, restart, and remove Apache Kafka Connect connectors. Kafka Connect cluster permissions will be applied to each operation. The commands for these actions are shown in the table below.

Commands	Description
`config`	Get the connector configuration
`create`	Create a new connector
`delete`	Delete a connector
`pause`	Pause a connector
`restart`	Restart a connector
`resume`	Resume a paused connector
`status`	Get the status of a connector
`task`	Manage a specific task from a connector
`tasks`	List the tasks of a connector
`update`	Update the configuration of a connector

View connectors

List the connectors

lenses-cli connectors

List currently deployed connectors, optionally filter by name, cluster and namespace:

lenses-cli connectors --cluster-name="dev" 

#[[--names [--unwrap]]

The --names flag displays only the names of the connectors. When used with --unwrap, it will print each connector name on a separate line in the output.

List all the supported connectors

lenses-cli connectors --supported

View the available plugins per cluster

lenses-cli connectors plugins --cluster-name="dev"

Create connectors

lenses-cli connector create \
    --name=example \
    --cluster-name=dev \
    --configs='{"connector.class":"org.apache.kafka.connect.file.FileStreamSinkConnector", "file":"/dev/null", "name":"example", "tasks.max":"8", "topics":"test_topic"}'

You can create a connector using a file:

lenses-cli connector create ./connector.yaml

Example file:

clusterName: dev
config:
  connector.class: "org.apache.kafka.connect.file.FileStreamSinkConnector"
  file: "/dev/null"
  tasks.max: "2"
  topics: "test_topic"
  name: "example"

Update a connector

# Inline
lenses-cli connector update \
    --name=example \
    --cluster-name=dev \
    --configs='{"connector.class":"org.apache.kafka.connect.file.FileStreamSinkConnector", "file":"/dev/null", "name":"example", "tasks.max":"2", "topics":"test_topic"}'

# or from file
lenses-cli connector update ./connector.yaml

View the configuration of a connector

lenses-cli connector config \
    --cluster-name="dev" \
    --name="nullsink"

View the status of a connector

lenses-cli connector status \
    --cluster-name="dev" \
    --name="nullsink"

Pause a connector

lenses-cli connector pause \
    --cluster-name="dev" \
    --name="nullsink"

Resume a connector

lenses-cli connector resume \
    --cluster-name="dev" \
    --name="nullsink"

Restart a connector

lenses-cli connector restart \
    --cluster-name="dev" \
    --name="nullsink"

Restart a connector task

lenses-cli connector task restart \
    --task=0 \
    --cluster-name="dev" \
    --name="nullsink"

View tasks for a connector

lenses-cli connector tasks \
    --cluster-name="dev" \
    --name="nullsink"

View the status of a task for a connector

lenses-cli connector task status \
    --cluster-name="dev" \
    --name="nullsink" --task=1

Delete a connector

lenses-cli connector delete \
    --cluster-name="dev" \
    --name="nullsink"

Last modified: July 26, 2024