IAM (Identity and Access Management) in Lenses is controlled by Groups. Users and service accounts belong to groups. Permissions are assigned to groups and apply to the users and service accounts in those groups.

Authentication of users is determined by the configured authentication mechanism.

Authentication is configured in the security configuration file. Lenses Administrator and Basic Auth do not require any configuration.

The following authentication methods are available. Users, regardless of the method need to be mapped to groups.

Account Locking

For BASIC and LDAP authentication types, there is the option to set a policy to temporarily lock the account when successive login attempts fail. Once the lock time window has passed the user can log in again.

These two configuration entries enable the functionality (both of them have to be provided to take effect):

# Number of failed login attempts before an account is locked.
lenses.security.lockout.user.attempts.max = "5"

# The time in seconds to keep the account locked.
lenses.security.lockout.user.period.sec = "600"  #10 minutes

Group Mapping

A Group is a collection of permissions that defines the level of access for users belonging to it. Groups consist of:

• Namespaces

• Application permissions

• Administration permissions

LDAP & Active Directory

When working with LDAP or Active Directory, user and group management is done in LDAP.

Lenses provides fine-grained role-based access (RBAC) for your existing groups of users over data and applications. Create a group in Lenses with the same name (case-sensitive) as in LDAP/AD.

SSO & SAML

When using an SSO solution such as Azure AD, Google, Okta, OneLogin or an open source like KeyCloak user and group management is done in the Identity Provider.

Lenses provides fine-grained role-based access (RBAC) for your existing groups of users over data and applications. Create a group in Lenses with the same name (case-sensitive) as in your SSO group.

Basic Auth

With Basic Authentication, create groups of users and add users to those groups. Authentication and authorization are fully managed, and users can change their passwords.

1. Enable TLS (SSL) for Lenses HTTPS.

2. Create a keystore for SAML.

3. Choose your identity provider (IdP):

Set the following in the security.conf

lenses.security.saml.keystore.location = "/path/to/lenses.p12"
lenses.security.saml.keystore.password = "my_password"
lenses.security.saml.key.password = "my_password"

When you first log in to Lenses, use the default credentials admin/admin

The default account is a super user and can be used to create groups and other accounts with appropriate permissions.

The default account username and password may be adjusted as below.

To create a SHA256 checksum for your password you can use the command line tools available in your Linux server or macOS.

Disabling the Admin Account

To disable the Lenses Administrator user, set an adequately long random password. You can achieve this by using the snippet below:

Groups are case-sensitive and mapped by UUID with Azure

Integrate your user-groups with Lenses using the Azure group IDs. Create a group in Lenses using the UUID as the name.

For example, if the Engineers group has the UUID ae3f363d-f0f1-43e6-8122-afed65147ef8, create a group with the same name.

Set up Microsoft Azure SSO

Learn more about

Add from Azure app-gallery

1. Go to Enterprise applications > + New Application

2. Search for Lenses.io in the gallery directory

3. Choose a name for Lenses e.g. Lenses.io and click Add

4. Select Set up single sign on > SAML

5. Configure the SAML details

1. Download the Federation Metadata XML file with the Azure IdP details. You will reference this file’s path in the Lenses security.conf configuration file.

With custom authentication, you can plug in your own authentication system by using HTTP headers.

In this approach, your own authentication proxy/code sitting in front of Lenses takes care of the authentication and injects appropriate Headers in all HTTP requests for verified users.

Setup a custom authentication layer

Set up a custom authentication layer by introducing in security.conf:

Lenses connects similarly to any other application to the infrastructure You can implement a plugin in a few hours in Java/Scala or other JVM technology by implementing one interface:

The returned object UserAndGroups will contain the username and the groups a person authentication belongs to (or raise an exception if no such user exists).

Example

The best way to get started is to look into a sample open-source implementation of such a plugin in .

Google doesn't expose the groups, or organization unit, of a user to a SAML app. This means we must set up a custom attribute for the Lenses groups that each user belongs to.

Create a custom attribute for Lenses groups

1. Open the from an administrator account.

2. Click the Users button

3. Select the More dropdown and choose Manage custom attributes

4. Click the Add custom attribute button

5. Fill the form to add a Text, Multi-value field for Lenses Groups, then click Add

Assign Lenses groups attributes to Google users

1. Open the from an administrator account.

2. Click the Users button

3. Select the user to update

4. Click User information

5. Click the Lenses Groups attribute

6. Enter one or more groups and click Save

Add Google custom SAML app

2. Click the Apps button

3. Click the SAML apps button

4. Select the Add App dropdown and choose Add custom SAML app

App Details

1. Enter a descriptive name for the Lenses installation

Download IdPXML file

Configure in security.conf.

With Basic Auth, user accounts are managed by Lenses and a unique username and a password are used to log in.

Account locking

For BASIC and LDAP authentication type, there is the option to set a policy to temporarily lock the account when successive login attempts fail. Once the lock time window has passed the user can log in again.

.

Accounts storage

The internal database that stores user/group information is stored on disk, under the lenses.storage.directory or an external Postgres database.

If using the embedded H2 database keep this directory intact between updates and upgrades.

Password rules

To enforce specific password rules the following configurations need to be set:

Password history

To not allow previous passwords to be reused, use the following configuration:

Azure AD supports the LDAP protocol. You can use it as an authentication provider with users, passwords, and groups stored in Azure AD. When a user is authenticated successfully, Lenses queries Azure AD to get the user’s groups and authorizes the user with the selected permissions.

Here is a sample Lenses configuration:

Create Azure AD Domain Instance

1. In the Azure portal create a resource. Search for Domain service and select Azure AD Domain Services from the options.

2. Set the DNS Domain Name as the same one you have with for your existing Azure AD tenant

1. In the Administration tab, you can manage the group membership for the AAD DC Administrator and control the members with access rights on Azure AD.

Azure AD Domain Services provides one-way synchronization from Azure Active Directory to the managed domain. Only certain attributes are synchronized to the managed domain, along with groups, group memberships and passwords.

The Synchronization tab provides two options. The first one is All, where everything will be synchronized to Azure AD DS managed domain. The second one is Scoped, which allows the selection of specific groups to be synced.

Configure DNS server settings

Once the managed domain is ready to be used, configure the DNS server settings for the Azure Virtual Network. Click the button configure:

For the DNS changes to be applied, all the VMs are required to be restarted.

Azure AD DS needs password hashes in a format that’s suitable for NT LAN Manager (NTLM) and Kerberos authentication. Azure AD does not generate or store password hashes in the format that’s required for NTLM or Kerberos authentication until you enable Azure AD DS for your tenant.

For security reasons, Azure AD doesn’t store any password credentials in clear-text form. Therefore, Azure AD can’t automatically generate these NTLM or Kerberos password hashes based on users’ existing credentials.

Virtual network peering

The Virtual Network to deploy Lenses, requires enabling Virtual Network Peering. This allows it to communicate with Azure AD DS. You should add the IPs that have been generated in the previous step as DNS Servers.

Enable Secure LDAP

To enable the LDAP(S) protocol on Azure AD DS, use the following PowerShell to generate the self-signed certificate:

In case PowerShell is not available, you can use the openssl command. This following script generates a certificate for Azure AD DS.

Under Secure LDAP, upload the PFX certificate and make sure the options Allow secure LDAP and access over the Internet are enabled.

After the secure LDAP is enabled to allow secure LDAP access, use the Azure AD DS properties to review the external IP address that is used to expose the LDAP service.

Finally, you need to allow inbound traffic to the Azure AD DS network security group for the LDAPS port 636 and limit the access only to the the virtual machine or the range of the IPs to which they should have inbound access.

Lenses can be configured via LDAP handle the user authentication.

All the user’s groups are then matched by name (case sensitive) with the groups stored in Lenses. All the matching groups' permissions are combined. If a user has been assigned manually a set of Lenses groups, then the groups coming from LDAP are ignored.

Due to the LDAP standard ambiguity, it is impossible to support all the configurations in the wild. The most common pain point is LDAP group mapping. If the default class that extracts and maps LDAP groups to Lenses groups does not work, it is possible to implement your own.

Before setting up an LDAP connection, we advise you to familiarize yourself with LDAP and/or have access to your LDAP and/or Active Directory administrators.

An LDAP setup example with LDAP group mapping is shown below:

In the example above you can distinguish three key sections for LDAP:

• the connection settings,

• the user search settings,

• and the group search settings.

Lenses uses the connection settings to connect to your LDAP server. The provided account should be able to list users under the base path and their groups. The default group plugin only needs access to the memberOf attributes for each user, but your custom implementation may need different permissions.

When a user tries to log in, a query is sent to the LDAP server for all accounts that are under the lenses.security.ldap.base and match the lenses.security.ldap.filter. The result needs to be unique; a distinguished name (DN) —the user that will log in to Lenses.

In the example, the application would query the LDAP server for all entities under ou=Users,dc=example,dc=com that satisfy the LDAP filter (&(objectClass=person)(sAMAccountName=)) which would be replaced by the username that tries to login to Lenses. A more simple filter could be cn=, which for user Mark would return the DN cn=Mark,ou=Users,dc=example,dc=com.

Once the user has been verified, Lenses queries the user groups and maps them to Lenses groups. For every LDAP group that matches a Lenses group, the user is granted the selected permissions.

Depending on the LDAP setup, only one of the users or the Lenses service user may be able to retrieve the group memberships. This can be controlled by the option lenses.security.ldap.use.service.user.search.

The default value (false) uses the user itself to query for groups. Groups be can be created in the admin section of the web interface, or in the command line via the lenses-cli application.

Set lenses.security.ldap.use.service.user.search to true to use lenses.security.ldap.user account to list a logged user groups when your LDAP setup restricts most of the user's action to list their groups.

Group mapping

When working with LDAP or Active Directory, user and group management is done in LDAP.

Lenses provides fine-grained role-based access (RBAC) for your existing groups of users over data and applications.

If mapping LDAP groups to Lenses groups is not desired. Manually map LDAP users to Lenses groups, using the web interface, or the lenses-cli.

LDAP still provides the authentication, but all LDAP groups for this user are ignored.

When you create an LDAP user in Lenses, the username will be used in the search expression set in lenses.security.ldap.filter to authenticate them. If no user should be allowed to use the groups coming from LDAP, then this functionality should be disabled.

Set lenses.security.ldap.plugin.memberof.key or lenses.security.ldap.plugin.group.extract.regex to a bogus entry, rendering it unusable.

An example would be:

Group extract plugin

The group extract plugin is a class that implements an LDAP query that retrieves a user’s groups and makes any necessary transformation to match the LDAP group to a Lenses group name.

The default class implementation that comes with Lenses is io.lenses.security.ldap.LdapMemberOfUserGroupPlugin.

If your LDAP server supports the memberOf functionality, where each user has his/her group memberships added as attributes to his/her entity, you can use it by setting the lenses.security.ldap.plugin.class option to this class:

Below you will see a brief example of its setup.

As an example, the memberOf search may return two attributes for user Mark:

The regular expression (?i)cn=(\w+),ou=Groups.* will return these two regex group matches:

If any of these groups exist in Lenses, Mark will be granted the permissions of the matching groups.

The lenses.security.ldap.plugin.group.extract.regex should contain exactly one regular expression capturing group.

If you need to apply more groups for your matching purposes, you should use non-capturing groups (e.g (?:groupRegex).

As an example, the regular expression (?i)cn=((?:Kafka|Apps)Admin),ou=Groups,dc=example,dc=com applied to memberOf attributes:

will return these two regex group matches:

Custom LDAP plugin

Do not forget to grant to the account any permissions it may need for your plugin to work.

LDAP Configuration Options

The following configuration entries are specific to the default group plugin. A custom LDAP plugin might require different entries under lenses.security.ldap.plugin:

Groups are case-sensitive and mapped by name with Okta

Integrate your user-groups with Lenses using the Okta group names. Create a group in Lenses using the same case-sensitive group name as in Okta.

For example, if the Engineers group is available in Okta, create a group with the same name.

Set up Okta IdP

Lenses is available directly in Okta’s Application catalog.

Add application in the Catalog

1. Go to Applications > Applications

2. Click Add Application

3. Search for Lenses

4. Select by pressing Add

Set General Settings

1. App label: Lenses

2. Set the base url of your lenses installation e.g. https://lenses-dev.example.com

3. Click Done

Download IdP XML file

Download the Metadata XML file with the Okta IdP details.

1. Go to Sign On > Settings > SIGN ON METHODS

2. Click on Identity Provider metadata and download the XML data to a file.

3. You will reference this file’s path in the security.conf configuration file.

lenses.security.saml.idp.metadata.file="/path/to/OktaIDPMetadata.xml"

A Group is a collection of permissions that defines the level of access for users belonging to it. Groups consist of:

• Namespaces

• Application permissions

• Administration permissions

Groups must be pre-created, and the group's names in Lenses must match (case sensitive) those in the SSO provider.

Creating groups

To create a new Group, go to Admin->Groups->New Group.

For every Group, you must set the data namespaces for Kafka or other available connections to data sources.

Groups must be given names, optionally a description.

Adding namespaces to groups

Namespace permissions define the access to datasets. and

Each group must have a namespace. A namespace is a set of permissions that apply to topics or a set of topics, for example, prod*. This allows you to define virtual multi-tenancy.

Adding application permissions

Application permissions define how a user can interact with applications and linked resources associated with those datasets.

Application permissions cover:

1. Viewing or resetting Consumer group offsets linked to a group's namespaces

2. Deploying or viewing connectors linked to a group's namespaces

3. Deploying or viewing SQL Processors linked to a group's namespaces

Additionally, application permissions define whether a group can access a specified Connect cluster.

Adding administration permissions

Admin permissions refer to activities that are in the global scope of Lenses and affect all the related entities.

Service accounts require an authentication token to be authenticated and must belong to at least one group for authorization.

Service accounts are commonly used for automation, for example, when using Lenses CLI or APIs, or any other application or service to interact with Lenses.

Creating a service account

To create a new Service Account, navigate to the Admin and select Users and New Service Account.

Authentication token

You can manually enter the authentication token or autogenerate it. If you select to auto-generate tokens, then you will receive a one-time token for this service account. Follow the instructions and copy and store this token. You can now use this token to authenticate via API and CLI.

Editing a service account

You can only change the groups and owner of services accounts. Go to the service account and select Edit Info, from the Actions menu.

Revoking a service account

To change the token, go to the service account and select Revoke Token from the Actions menu.

Using a service account

To use the service account you need to prefix the token with its name separated by a colon. You then include that in the corresponding header.

Example

For a service account named myservice and a token da6bad50-55c8-4ed4-8cad-5ebd54a18e26 then the combination looks like this:

myservice:28ab4195-18cf-426a-abda-c41a451e001a

To use the CLI with a service account for CI/CD you need to pass these options:

lenses-cli topics \
  --token=<service-account-name>:<service-account-token> \
  --host=<lenses-url-host>

# Real Example
lenses-cli topics \
  --token=ci:58d86476-bcc6-47e2-a57e-0c6bbd9c88b9 \
  --host=http://<your-lenses-url>:9991

Groups are case-sensitive and mapped to roles, by name, with OneLogin

Integrate your user roles with Lenses using the Keycloak role names. Create a group in Lenses using the same case-sensitive role name as in OneLogin.

For example, if the Engineers role is available in OneLogin, create a group with the same name.

Set up OneLogin IdP

Lenses is available in the OneLogin Application catalog.

Visit OneLogin’s Administration console. Select Applications > Applications > Add App

1. Search and select Lenses

2. Optionally add a description and click save

Add Lenses via the Application Catalog

1. In the Configuration section set the base path from the url of the Lenses installation e.g. lenses-dev.example.com ( without the https://)

2. Click Save

Download the IdP XML file

1. Use the More Actions button

2. Click and download the SAML Metadata

3. You will reference this file’s path in the security.conf configuration file.

lenses.security.saml.idp.metadata.file="/path/to/OneLoginIDPMetadata.xml"

Authentication instances too old or in the future

The error that you see

Authentication issue instant is too old or in the future

What causes it

1. The user’s session in the SSO provider is too old.

2. The system clocks of the SSO provider and the Lenses instance are out of sync.

For security purposes, Lenses prevents authenticating SSO users that have remained logged in SSO for a very long time.

Example: You use Okta SSO and, you logged in to Okta a year ago. Okta might allow you to remain logged in along that year without having to re-authenticate. Lenses has a limit of 100 days. In that case, Lenses will receive an authenticated user that originally logged in before the 100 days mark.

How to solve it

1. Ensure that the SSO and Lenses system clocks are in sync.

2. If the SSO provider supports very long sessions either:

   1. Log out of the SSO and log back in. This explicitly renews the SSO session.

   2. Increase the Lenses limit to more than 100 days.

Example:

lenses.security.saml.idp.session.lifetime.max = 365days

Kerberos uses SPNEGO (Simple and Protected GSSAPI Negotiation Mechanism) for authentication.

Kerberos will automatically log in authorized users when using the /api/auth REST endpoint. If using Microsoft Windows, logging into your Windows domain is usually sufficient to issue your Kerberos credentials.

On Linux, if you use Kerberos with PAM, your Kerberos credentials should be already available to Kerberos-enabled browsers. Otherwise, you will need to authenticate to the KDC manually using kinit at the command line and start your browser from the same terminal.

Configuration

In order to use Kerberos authentication in Lenses, both a static configuration and Kerberos Connection is required.

• Static configuration To set up Kerberos you need a Kerberos principal and a password-less keytab. Add them in security.conf before starting Lenses:

lenses.security.kerberos.service.principal="HTTP/lenses.url[@REALM]"
lenses.security.kerberos.keytab=/path/to/lenses.keytab

• Kerberos Connection A Kerberos Connection should be defined in order to use a proper krb5.conf

Integrate your user groups with Lenses using the Keycloak group names. Create a group in Lenses using the same case-sensitive group name as in Keycloak.

For example, if the Engineers group is available in Keycloak, with Lenses assigned to it, create a group with the same name.

Create a new SAML application client in Keycloak

1. Go to Clients

2. Click Create

3. Fill in the details: see the table below.

4. Click Save

Change the settings on client you just created to:

Map user groups

Configure Keycloak to communicate groups to Lenses. Head to the Mappers section.

1. Click Create

2. Fill in the details: see table below.

3. Click Save

Download IdP XML file

Configure in the security.conf file.

lenses.security.saml.idp.metadata.file="/path/to/KeycloakIDPMetadata.xml"

Users must be assigned to a group. SSO and LDAP users are mapped to a group matching the group name provided by the Idp.

Multiple types of users can be supported at the same time.

Adding a user with Basic Authentication

Select Admin->Users->New User->Basic Auth.

Mapping SSO users to groups

By default, users are mapped to the group provided by the SSO provider. If you wish to override the group mapping from your SSO, users can be created directly in Lenses and you can manually map the user to a group.

Mapping LDAP users to groups

By default, users are mapped to the group provided by the LDAP server. If you wish to override the group mapping you manually map the user to a group.

See LDAP user group mapping.

Listing Users

Lenses allows you to view users and:

1. Authentication type

2. Groups they belong to

3. Last login

Go to Admin -> Users.

Admin permissions

This matrix shows both display name (first column) and code name (second column) for permissions. Knowing code name may be helpful while using API / CLI.

Data permissions

Application permissions

This matrix shows both display name (first column) and code name (second column) for permissions. Knowing code name may be helpful while using API / CLI.