User Management

The ability to manage who can access the product, in an efficient and elegant manner, becomes critical for those organizations with a large user volume.

User management is the method by which you create, maintain, and remove user accounts, service accounts, and user-groups. Lenses permission management system makes it easy to manage permissions for groups. The permissions system is not based on hierarchical roles, but rather on groups, reflecting Unix and Windows-based permissions. You can create groups that have pre-defined, fully customizable permissions sets; users added to a group inherit the permissions of that group. Permissions are granted to groups, and users are assigned to groups, as illustrated below:

Each User in the Group inherits the Permissions of the Group. The same applies for Service Accounts.

Permissions

You can assign a pre-defined set of permissions to a group. Once a group is defined, users can be linked to it. Users inherit the permissions from their groups. Any group can be modified to fit your needs, or you can create a new group and assign permissions to it.

Permissions, describing the actions a group member is entitled to perform, are split into three categories:

  • Data Namespace
  • Applications
  • Admin

Data Namespace

The data namespace rule is describing per system (Apache Kafka) which resources are accessible and which actions are allowed. In the case of Apache Kafka, the resources are topics.

A data namespace identifies topics based on a set of wildcard rules. For example payments* identifies all the topics which have payments as a prefix. Similarly, *payments identifies all topics ending with the text payments Last but not least *payments* identifies all the topics containing the text within the wildcards.

Multiple data namespace rules can identify the same topic. For example *_trade and *bonds_trade rules can both identify eu_bond_trades topic. The precedence is determined by the strongest rule - the rule with the closest identifier for the topic. In the scenario above, the *bonds_trade rule provides the permissions for the topic.

Attaching a user to multiple groups can lead to multiple rules identifying a topic. In this scenario, the rule described earlier applies. While wildcards are there to allow identifying multiple topics at a time, we advise using the full topic name when setting data namespace permissions.

For Apache Kafka, these are the action supported for a topic:

Name Description
Show allows viewing the topic name and basic info
Query allows viewing the data in a topic
Create allows creating topics
Drop allows deleting topics
Configure allows changing a topic configuration
Insert Data allows inserting data into the topic
Delete Data allows deleting data from the topic
Update Schema allows configuring the topic storage format and schema
View Schema allows viewing topic schema information

Applications

They control the actions that members of a group can perform when working with applications.

Streaming SQL Processors, or Connect sinks/sources are applications that rely on input and output topics. To manage a SQL processor or Kafka connectors, first, there’s a requirement to have the correct access to the data involved. Hence data namespace permissions provide scope for higher-level functionality actions.

Here is the full permission set:

Name Description
View SQL Processors allows viewing the SQL processors
Manage SQL Processors allows to add/remove/stop/delete SQL processors
View Schemas allows viewing your Schema Registry entries
Manage Schema Registry allows to add/remove/update/delete your Schema Registry entries
View Topology allows viewing the data pipeline topology
Manage Topology allows decommissioning topology applications
View Kafka Connectors allows viewing running Kafka Connectors
Manage Kafka Connectors allows to add/update/delete/stop Kafka Connectors
View Kafka Consumers allows viewing the Kafka Consumers details
Manage Kafka Consumers allows changing the Kafka Consumers offset

View SQL Processors

The permission controls the user access to the SQL processors list. A SQL processor is displayed to the user only if the appropriate permissions are in place for the data involved. To view a processor data namespace rules need to be present, and they need to identify the input and output topics involved.

Manage SQL Processors

To add a SQL processor, the user needs to have the Manage permission, and:

  • for all the input topics the user needs to have View Data permissions, and
  • for all the output topics the user needs to have Insert Data permission for each

To remove/scale a SQL processor it requires the user to have Manage permission and for all the topics involved the user needs to have data namespace rules granting permission to view each one.

View Schema Registry

Grants permission to view the entries present in Schema Registry. A schema entry is visible only if for the corresponding topic the user has, via data namespace rules, View Schema permission.

Manage Schema Registry

Controls the permission to manage your Schema Registry entries regardless of the actual implementation you are using. The actions are constrained by the data namespace rules. The user is allowed to make amendments to a schema only if for the corresponding topic Update Schema action is granted.

View Topology

It allows the user to view the data flow landscape, which involves topics, SQL processors, custom applications, and Kafka Connect sinks/sources. The namespace permission rules constrain the visible nodes. A topic is rendered on the screen if the namespace rules grants permission to the user to view it. A SQL processor, Kafka Connect sink/source or customer application are visible only if the user is allowed to see the topics involved. Otherwise, the topology node won’t be rendered on the screen for the user.

Manage Topology

It allows the user of a group to decommission custom applications. Of course to be able to perform such an action, first the application node needs to be visible.

View Kafka Connectors

It allows the user to view running Kafka Connect sinks or sources. Similar to SQL processors, only those sinks and sources are visible where the data namespaces rules grants permission to see the topics involved.

Manage Kafka Connectors

Grants the user the action to create a new Kafka Connect sink or source. The operation is also restricted by namespace rules. In the case of a Connect source, it requires the user to have Insert Data permission for the target topics. For a Connect sink, it requires the user to have View Data permissions for the source topics.

Updating an existing connector follows the same permission restrictions as seen earlier. To delete an existing connector, all that is required is for it to be visible (see the rules about viewing a connector).

View Kafka Consumers

It allows the user to view Kafka consumer groups. A consumer group is visible if the data namespace rules allow the current user to see all the topics involved. If one of the topics a consumer group uses is not visible given the namespace permissions, then the entire consumer group is not visible.

Manage Kafka Consumers

It allows the user of a group to update the topic-partition offsets for a given consumer group. Of course, it requires the Kafka consumer group to be visible first.

Admin Permission

To manage your infrastructure, Lenses bundles in specific permissions.

Name Description
View Kafka Settings allows viewing Kafka ACLs, Quotas
Manage Kafka Settings allows managing Kafka ACLs, Quotas
View Log allows viewing Lenses logs
View Users allows viewing the users, groups and service accounts
Manage Users allows to add/remove/update/delete users,groups and service accounts
View Alert Rules allows viewing the alert settings rules
Manage Alert Rules allows adding/deleting/updating alert settings rules
View Audit allows viewing the audit records
View Data Policies allows viewing the data policies
Manage Data Policies allows to add/remove/update/delete data policies

Data policies are rules protecting sensitive information your data might contain. These rules are available across all topics and therefore are not subject to data namespace permissions.

Users

Lenses provides out of the box an administrator user<security_administrator>. This is a particular user account that is granted full permissions and does not belong to any group. You would use this account to create the first group(-s) and any users and service accounts you might require.

A user can belong to one or more groups and inherits the permissions defined on all of them.

Users can be managed via the admin section in the web interface, or in the command line via the Lenses CLI tool.

Service accounts

These are special user entries for which the login step is not required. For these users, the authentication token is static and stored securely by Lenses. Similar to a user account, a service account can be part of multiple groups and as a result inherits the permissions defined on all of them.

These accounts are meant to be used by machines. As such, it is advised to keep the token long and complex enough to avoid any brute-force attack attempt.

When using a Service Account, the request made to Lenses needs to have X-Kafka-Lenses-Token header set to this value format: {Service_Account_Name}:${Service_Account_Token}.

To verify a service account, you can use curl. The example below gets the application’s logs via the API:

curl --header "X-Kafka-Lenses-Token: jenkins-user:jenkins-token" --compressed http://lenses.url/api/logs/INFO

The result should be an array of log entries:

[
   {
     "timestamp": 1545091832442,
     "logger": "io.lenses.actors.ConnectConfigActor",
     "stacktrace": "",
     "thread": "default-akka.actor.default-dispatcher-28",
     "message": "Config connector event received. Name logs-broker in cluster dev",
     "time": "2018-12-18 00:10:32.442",
     "level": "INFO"
   },
   ...
]