Kafka ACLs

This page describes the Kafka ACLs prerequisites for the Lenses Agent if ACLs are enabled on your Kafka clusters.

These ACLs are for the underlying Lenses Agent Kafka client. Lenses has its own set of permissions guarding access.

You can restrict the access of the Lenses Kafka client but this can reduce the functionality on offer in Lenses, e.g. not allow Lenses to create topic at all, even though this can be managed by Lenses own IAM system.

When your Kafka cluster is configured with an authorizer which enforces ACLs, the Agent will need a set of permissions to function correctly.

Common practice is to give teh Agent superuser status or the complete list of available operations for all resources. The IAM model of Lenses can then be used to restrict the access level per user.

kafka-acls \
    --bootstrap-server [broker.url:9092] --command-config [client.properties] \
    --add \
    --allow-principal [User:Lenses] \
    --allow-host [lenses.host] \
    --operation All \
    --topic * \
    --group * \
    --delegation-token * \
    --cluster

Minimal Permissions

The Agent needs permission to manage and access their own internal Kafka topics:

• __topology
• __topology__metrics

kafka-acls \
    --bootstrap-server [broker.url:9092] --command-config [client.properties] \
    --add \
    --allow-principal [User:Lenses] \
    --allow-host [lenses.host] \
    --operation All \
    --topic [topic]

It also needs to read and describe permissions for the consumer offsets and Kafka Connect topics —if enabled:

• __consumer_offsets
• connect-configs
• connect-offsets
• connect-status

kafka-acls \
    --bootstrap-server [broker.url:9092] --command-config [client.properties] \
    --add \
    --allow-principal [User:Lenses] \
    --allow-host [lenses.host] \
    --operation Describe \
    --operation DescribeConfigs \
    --operation Read \
    --topic [topic]

This same set of permissions is required for any topic that the agent must have read access.

kafka-acls \
    --bootstrap-server [broker.url:9092] --command-config [client.properties] \
    --add \
    --allow-principal [User:Lenses] \
    --allow-host [lenses.host] \
    --operation Describe \
    --operation DescribeConfigs \
    --operation Read \
    --topic *

DescribeConfigs was added in Kafka 2.0. It may not be needed for versions before 2.2.

Additional permissions are needed to produce topics or manage them.

Consumer Groups

Permission to at least read and describe consumer groups is required to take advantage of the Consumer Groups' monitoring capabilities.

kafka-acls \
    --bootstrap-server [broker.url:9092] --command-config [client.properties] \
    --add \
    --allow-principal [User:Lenses] \
    --allow-host [lenses.host] \
    --operation Describe \
    --operation Read \
    --group *

Additional permissions are needed to manage groups.

ACLs

To manage ACLs, permission to the cluster is required:

kafka-acls \
    --bootstrap-server [broker.url:9092] --command-config [client.properties] \
    --add \
    --allow-principal [User:Lenses] \
    --allow-host [lenses.host] \
    --operation Describe \
    --operation DescribeConfigs \
    --operation Alter \
    --cluster