The CLI allows you to import resources from files in Lenses into another environment.
Import is done on a per resource basis, the directory structure defined by the CLI whereas a base directory can be provided by the –dir flag.
Processors, connectors, topics and schemas have an additional prefix flag to restrict resources to export. The commands to import the configurations are shown in the table below.
acls
alert-channels
alert-settings
connections
connectors
processors
quotas
schemas
topics
policies
groups
serviceaccounts
topic-settings
Commands showing how to import the configurations:
lenses-cli import acls --dir my-dir lenses-cli import alert-channels --dir my-dir lenses-cli import alert-settings --dir my-dir lenses-cli import connections --dir my-dir lenses-cli import connectors --dir my-dir lenses-cli import processors --dir my-dir lenses-cli import quotas --dir my-dir lenses-cli import schemas --dir my-dir lenses-cli import topics --dir my-dir lenses-cli import policies --dir my-dir lenses-cli import groups --dir my-dir lenses-cli import serviceaccounts --dir my-dir lenses-cli import topic-settings --dir my-dir
The directory structure is:
my-dir ├── alert-settings │ └── alert-setting.yaml ├── apps │ ├── connectors │ │ ├── connector-1.yaml │ │ └── connector-2.yaml │ └── sql ├── groups │ └── groups.yaml ├── kafka │ ├── quotas │ │ └── quotas.yaml │ └── topics │ ├── topic-1.yaml │ └── topic-2.yaml ├── policies │ └── policies-city.yaml ├── service-accounts │ └── servic-eaccounts.yaml └── schemas ├── schema-1.yaml └── schema-2.yaml
Lenses is on the journey to GitOps. The initial phase is to standardise the import and export of resources and to provide a way to manage resources in a declarative way. This is where AS-Code comes in. Since Lenses 5.5 the As-code feature is available for Kafka Connectors, the other resource will follow in the upcoming releases.
The general syntax for the AS-Code is:
lenses-cli apply resource.yaml
The resource.yaml file is a YAML file containing the resource definition. Specifically, for Kafka Connectors, the resource definition follows this structure:
apiVersion: lenses.io/v0beta kind: KafkaConnector spec: status: RUNNING cluster: a-cluster-name name: hdfs-sink-connector config: "connector.class": "io.lenses.streamreactor.connect.aws.s3.sink.S3SinkConnector" "tasks.max": "10" "topics": "my-topic" "connect.s3.kcql": "insert into lensesio:demo select * from demo PARTITIONBY _value.metadata_id, _value.customer_id, _header.ts, _header.wallclock STOREAS `JSON` PROPERTIES ('flush.size'=1000000, 'flush.interval'=30, 'flush.count'=5000)"
In this structure:
The status field is optional and can be one of: RUNNING, STOPPED, or PAUSED. The cluster field specifies the name of the Connect cluster where the connector will be deployed. The name field denotes the name of the connector. The config field represents a map of the connector’s configuration.
On this page
