The CLI allows you to import resources from files in Lenses into another environment.
Import is done on a per resource basis, the directory structure defined by the CLI whereas a base directory can be provided by the —dir flag.
Processors, connectors, topics and schemas have an additional prefix flag to restrict resources to export.
lenses-cli import acls --dir my-dir lenses-cli import alert-channels --dir my-dir lenses-cli import alert-settings --dir my-dir lenses-cli import connections --dir my-dir lenses-cli import connectors --dir my-dir lenses-cli import processors --dir my-dir lenses-cli import quota --dir my-dir lenses-cli import schemas --dir my-dir lenses-cli import topics --dir my-dir lenses-cli import policies --dir my-dir lenses-cli import groups --dir my-dir lenses-cli import serviceaccounts --dir my-dir
The directory structure is:
my-dir ├── alert-settings │ └── alert-setting.yaml ├── apps │ ├── connectors │ │ ├── connector-1.yaml │ │ └── connector-2.yaml │ └── sql ├── groups │ └── groups.yaml ├── kafka │ ├── quotas │ │ └── quotas.yaml │ └── topics │ ├── topic-1.yaml │ └── topic-2.yaml ├── policies │ └── policies-city.yaml ├── service-accounts │ └── servic-eaccounts.yaml └── schemas ├── schema-1.yaml └── schema-2.yaml
Lenses is on the journey to GitOps. The initial phase is to standardise the import and export of resources and to provide a way to manage resources in a declarative way. This is where AS-Code comes in. Since Lenses 5.5 the As-code feature is available for Kafka Connectors, the other resource will follow in the upcoming releases.
The general syntax for the AS-Code is:
lenses-cli apply resource.yaml
The resource.yaml file is a YAML file containing the resource definition. Specifically, for Kafka Connectors, the resource definition follows this structure:
apiVersion: lenses.io/v0beta kind: KafkaConnector spec: status: RUNNING cluster: a-cluster-name name: hdfs-sink-connector config: "connector.class": "io.lenses.streamreactor.connect.aws.s3.sink.S3SinkConnector" "tasks.max": "10" "topics": "my-topic" "connect.s3.kcql": "insert into lensesio:demo select * from demo PARTITIONBY _value.metadata_id, _value.customer_id, _header.ts, _header.wallclock STOREAS `JSON` PROPERTIES ('flush.size'=1000000, 'flush.interval'=30, 'flush.count'=5000)"
In this structure:
The status field is optional and can be one of: RUNNING, STOPPED, or PAUSED. The cluster field specifies the name of the Connect cluster where the connector will be deployed. The name field denotes the name of the connector. The config field represents a map of the connector’s configuration.
On this page
