User Management¶
The ability to manage who can access the product, in an efficient and elegant manner, becomes critical for those organizations with a large user volume.
https://help.lenses.io/using-lenses/user-management how it looks in action
User management is the method by which you create, maintain, and remove user accounts, service accounts, and user-groups. Lenses permission management system makes it easy to manage permissions for groups. The permissions system is not based on hierarchical roles, but rather on groups, reflecting Unix and Windows-based permissions. You can create groups that have pre-defined, fully customizable permissions sets; users added to a group inherit the permissions of that group. Permissions are granted to groups, and users are assigned to groups, as illustrated below:
Permissions¶
You can assign a pre-defined set of permissions to a group. Once a group is defined, users can be linked to it. Users inherit the permissions from their groups. Any group can be modified to fit your needs, or you can create a new group and assign permissions to it.
Permissions, describing the actions a group member is entitled to perform, are split into three categories:
- Data Namespace
- Applications
- Admin
Data Namespace
The data namespace rule is describing per system (Apache Kafka) which resources are accessible and which actions are allowed. In the case of Apache Kafka, the resources are topics.
A data namespace identifies topics based on a set of wildcard rules.
For example payments*
identifies all the topics which have payments as a prefix.
Similarly, *payments
identifies all topics ending with the text payments
Last but not least *payments*
identifies all the topics containing the text within the wildcards.
Multiple data namespace rules can identify the same topic. For example *_trade
and *bonds_trade
rules
can both identify eu_bond_trades topic. The precedence is determined by the strongest
rule - the rule with the closest identifier for the topic. In the scenario above, the *bonds_trade
rule provides
the permissions for the topic.
Attaching a user to multiple groups can lead to multiple rules identifying a topic. In this scenario, the rule described earlier applies. While wildcards are there to allow identifying multiple topics at a time, we advise using the full topic name when setting data namespace permissions.
For Apache Kafka, these are the action supported for a topic:
Name | Description |
---|---|
Show | allows viewing the topic name and basic info |
Query | allows viewing the data in a topic |
Create | allows creating topics |
Drop | allows deleting topics |
Configure | allows changing a topic configuration |
Insert Data | allows inserting data into the topic |
Delete Data | allows deleting data from the topic |
Update Schema | allows configuring the topic storage format and schema |
View Schema | allows viewing topic schema information |
Applications
They control the actions that members of a group can perform when working with applications.
Streaming SQL Processors, or Connect sinks/sources are applications that rely on input and output topics. To manage a SQL processor or Kafka connectors, first, there’s a requirement to have the correct access to the data involved. Hence data namespace permissions provide scope for higher-level functionality actions.
Here is the full permission set:
Name | Description |
---|---|
View SQL Processors | allows viewing the SQL processors |
Manage SQL Processors | allows to add/remove/stop/delete SQL processors |
View Schemas | allows viewing your Schema Registry entries |
Manage Schema Registry | allows to add/remove/update/delete your Schema Registry entries |
View Topology | allows viewing the data pipeline topology |
Manage Topology | allows decommissioning topology applications |
View Kafka Connectors | allows viewing running Kafka Connectors |
Manage Kafka Connectors | allows to add/update/delete/stop Kafka Connectors |
View Kafka Consumers | allows viewing the Kafka Consumers details |
Manage Kafka Consumers | allows changing the Kafka Consumers offset |
View SQL Processors
The permission controls the user access to the SQL processors list. A SQL processor is displayed to the user only if the appropriate permissions are in place for the data involved. To view a processor data namespace rules need to be present, and they need to identify the input and output topics involved.
Manage SQL Processors
To add a SQL processor, the user needs to have the Manage permission, and:
- for all the input topics the user needs to have View Data permissions, and
- for all the output topics the user needs to have Insert Data permission for each
To remove/scale a SQL processor it requires the user to have Manage permission and for all the topics involved the user needs to have data namespace rules granting permission to view each one.
View Schema Registry
Grants permission to view the entries present in Schema Registry. A schema entry is visible only if for the corresponding topic the user has, via data namespace rules, View Schema permission.
Manage Schema Registry
Controls the permission to manage your Schema Registry entries regardless of the actual implementation you are using. The actions are constrained by the data namespace rules. The user is allowed to make amendments to a schema only if for the corresponding topic Update Schema action is granted.
View Topology
It allows the user to view the data flow landscape, which involves topics, SQL processors, custom applications, and Kafka Connect sinks/sources. The namespace permission rules constrain the visible nodes. A topic is rendered on the screen if the namespace rules grants permission to the user to view it. A SQL processor, Kafka Connect sink/source or customer application are visible only if the user is allowed to see the topics involved. Otherwise, the topology node won’t be rendered on the screen for the user.
Manage Topology
It allows the user of a group to decommission custom applications. Of course to be able to perform such an action, first the application node needs to be visible.
View Kafka Connectors
It allows the user to view running Kafka Connect sinks or sources. Similar to SQL processors, only those sinks and sources are visible where the data namespaces rules grants permission to see the topics involved.
Manage Kafka Connectors
Grants the user the action to create a new Kafka Connect sink or source. The operation is also restricted by namespace rules. In the case of a Connect source, it requires the user to have Insert Data permission for the target topics. For a Connect sink, it requires the user to have View Data permissions for the source topics.
Updating an existing connector follows the same permission restrictions as seen earlier. To delete an existing connector, all that is required is for it to be visible (see the rules about viewing a connector).
View Kafka Consumers
It allows the user to view Kafka consumer groups. A consumer group is visible if the data namespace rules allow the current user to see all the topics involved. If one of the topics a consumer group uses is not visible given the namespace permissions, then the entire consumer group is not visible.
Manage Kafka Consumers
It allows the user of a group to update the topic-partition offsets for a given consumer group. Of course, it requires the Kafka consumer group to be visible first.
Admin Permission
To manage your infrastructure, Lenses bundles in specific permissions.
Name | Description | |
---|---|---|
View Kafka Settings | allows viewing Kafka ACLs, Quotas | |
Manage Kafka Settings | allows managing Kafka ACLs, Quotas | |
View Log | allows viewing Lenses logs | |
View Users | allows viewing the users, groups and service accounts | |
Manage Users | allows to add/remove/update/delete users,groups and service accounts | |
View Alert Rules | allows viewing the alert settings rules | |
Manage Alert Rules | allows adding/deleting/updating alert settings rules | |
View Audit | allows viewing the audit records | |
View Data Policies | allows viewing the data policies | |
Manage Data Policies | allows to add/remove/update data policies | |
Manage Connections | allows to add/remove/update connections | as well as viewing their sensitive details |
Data policies are rules protecting sensitive information your data might contain. These rules are available across all topics and therefore are not subject to data namespace permissions.
Connections are resources that contain information to communicate/connect to other systems. We treat these properties as sensitive information, so you need Manage permissions to see them.
Users¶
Lenses provides out of the box an administrator user<security_administrator>. This is a particular user account that is granted full permissions and does not belong to any group. You would use this account to create the first group(-s) and any users and service accounts you might require.
A user can belong to one or more groups and inherits the permissions defined on all of them.
Users can be managed via the admin section in the web interface, or in the command line via the Lenses CLI tool.
Service accounts¶
These are special user entries for which the login step is not required. For these users, the authentication token is static and stored securely by Lenses. Similar to a user account, a service account can be part of multiple groups and as a result inherits the permissions defined on all of them.
These accounts are meant to be used by machines. As such, it is advised to keep the token long and complex enough to avoid any brute-force attack attempt.
When using a Service Account, the request made to Lenses needs to have X-Kafka-Lenses-Token
header set to this value format: {Service_Account_Name}:${Service_Account_Token}
.
To verify a service account, you can use curl
. The example below gets the
application’s logs via the API:
curl --header "X-Kafka-Lenses-Token: jenkins-user:jenkins-token" --compressed http://lenses.url/api/logs/INFO
The result should be an array of log entries:
[
{
"timestamp": 1545091832442,
"logger": "io.lenses.actors.ConnectConfigActor",
"stacktrace": "",
"thread": "default-akka.actor.default-dispatcher-28",
"message": "Config connector event received. Name logs-broker in cluster dev",
"time": "2018-12-18 00:10:32.442",
"level": "INFO"
},
...
]