Configuration Reference
This page lists the available configurations in Lenses Agent.
Last updated
This page lists the available configurations in Lenses Agent.
Last updated
2024 © Lenses.io Ltd. Apache, Apache Kafka, Kafka and associated open source project names are trademarks of the Apache Software Foundation.
Set in lenses.conf
Reference documentation of all configuration and authentication options:
Key | Description | Default | Type | Required |
---|---|---|---|---|
System or control topics are created by services for their internal use. Below is the list of built-in configurations to identify them.
_schemas
__consumer_offsets
_kafka_lenses_
lsql_*
lsql-*
__transaction_state
__topology
__topology__metrics
_confluent*
*-KSTREAM-*
*-TableSource-*
*-changelog
__amazon_msk*
Wildcard (*
) is used to match any name in the path to capture a list of topics not just one. When the wildcard is not specified, Lenses matches on the entry name provided.
If the records schema is centralized, the connectivity to Schema Registry nodes is defined by a Lenses Connection.
There are two static config entries to enable/disable the deletion of schemas:
Options for specific deployment targets:
Global options
Kubernetes
Common settings, independently of the underlying deployment target:
Kubernetes connectivity is optional. Minimum supported K8 version 0.11.10. All settings are string.
Optimization settings for SQL queries.
Lenses requires these Kafka topics to be available, otherwise, it will try to create them. The topics can be created manually before Lenses is run, or allow Lenses the correct Kafka ACLs to create the topics:
To allow for fine-grained control over the replication factor of the three topics, the following settings are available:
When configuring the replication factor for your deployment, it's essential to consider the requirements imposed by your cloud provider. Many cloud providers enforce a minimum replication factor to ensure data durability and high availability. For example, IBM Cloud mandates a minimum replication factor of 3. Therefore, it's crucial to set the replication factor for the Lenses internal topics to at least 3 when deploying Lenses on IBM Cloud.
All time configuration options are in milliseconds.
Control how Lenses identifies your connectors in the Topology view. Catalogue your connector types, set their icons, and control how Lenses extracts the topics used by your connectors.
Lenses comes preconfigured for some of the popular connectors as well as the Stream Reactor connectors. If you see that Lenses doesn’t automatically identify your connector type then use the lenses.connectors.info
setting to register it with Lenses.
Add a new HOCON object {}
for every new Connector in your lenses.connectors.info
list :
This configuration allows the connector to work with the topology graph, and also have the RBAC rules applied to it.
To extract the topic information from the connector configuration, source connectors require an extra configuration. The extractor class should be: io.lenses.config.kafka.connect.SimpleTopicsExtractor
. Using this extractor requires an extra property configuration. It specifies the field in the connector configuration which determines the topics data is sent to.
Here is an example for the file source:
An example of a Splunk sink connector and a Debezium SQL server connector
Key | Description | Default |
---|---|---|
Key | Description | Default |
---|---|---|
Key | Description | Default | Type | Required |
---|---|---|---|---|
Key | Description | Default | Type | Required |
---|---|---|---|---|
Key | Description | Default | Type | Required |
---|---|---|---|---|
Key | Description | Type |
---|---|---|
Key | Description | Default |
---|---|---|
Key | Description | Default |
---|---|---|
Key | Description | Type | Default |
---|---|---|---|
Key | Description | Partition | Replication | Default | Compacted | Retention |
---|---|---|---|---|---|---|
Key | Description | Default |
---|---|---|
Key | Description | Type | Default |
---|---|---|---|
Key | Description | Default | Type | Required |
---|---|---|---|---|