AWS is an advanced Amazon MSK integration offering data observability, productivity, monitoring, security and governance for Apache Kafka and event/streaming applications.

Learn more 

Read bellow to learn more about


Familiarity with AWS MSK (Managed Apache Kafka) is assumed.

The AWS Marketplace offering requires AWS MSK (Managed Apache Kafka) to be available. Optionally, AWS RDS (or any other PostgreSQL compatible database) can be configured for Lenses to store its state.

The following AWS resources are created:

  • An EC2 instance that runs Lenses;
  • A SecurityGroup to allow network access to the Lenses UI;
  • A SecurityGroupIngress for Lenses to connect to MSK;
  • A CloudWatch LogGroup where Lenses stores its logs;
  • An IAM Role to allow the EC2 instance to store logs;
  • An IAM InstanceProfile to pass the role to the EC2 instance;
  • Optionally if enabled during deployment: an IAM Policy to allow the EC2 instance to emit CloudWatch metrics.

Deployment takes approximately three minutes.

AWS Marketplace Installation 

Select CloudFormation Template, Lenses EC2 and your region

MSK CloudFormation template

2) Choose Launch CloudFormation

MSK launch Lenses action

3) Continue with the default options for creating the stack in the AWS wizard.

Fill in the parameters at Specify stack details.

  • Deployment Here the EC2 instance size and password for the Lenses admin user are set. A t2.large instance size is recommended;
  • Network Configuration This section controls the network settings of the Lenses EC2 instance. The ingress allows access to the Lenses UI only from particular IP addresses;
  • MSK Set the Security Group ID to that of your MSK cluster. A rule will be added to it so that Lenses can communicate with your cluster. You can find the ID by navigating in the AWS console to your MSK cluster and then under Properties -> Networking settings;
  • Monitoring Optionally produce the Lenses logs to CloudWatch;
  • Storage Lenses can stores its state in a database locally on the EC2 instance’s disk or in a PostgreSQL database. Local storage is a development/quickstart option and not suitable for production use. It is advised to use a postgres database for smoother upgrades.

4) Review the stack:

MSK Lenses installation review

5) Accept the terms and conditions and create the stack:

MSK Lenses ack

6) Once the stack has deployed, go to the Output tab and click on the FQDN link. If there are no outputs listed you might need to press the refresh button.

MSK stack FQDN link

7) Login to Lenses with admin and the password value you have submitted on parameter LensesAdminPassword.

Template policies 

The template creates policies that allow the EC2 instance to:

  • logs:CreateLogGroup;
  • logs:CreateLogStream;
  • logs:PutLogEvents;
  • logs:PutLogEvents;
  • cloudwatch:PutMetricData, optionally if enabled.

Persistence Storage 

There are two options for Lenses to persist its state: local or PostgreSQL. With the local option, Lenses creates a database locally on the EC2 instance' filesystem. With the PostgreSQL option, an external database is used.

In-order to do in-place Lenses version upgrades, by downloading Linux binary from Lenses Archive, and you are using storage mode local it is suggested first to do an H2 database backup as in the following article.

Note that the storage may contain sensitive data, treat it accordingly.

Backup and Recovery 

When using PostgreSQL-compatible storage (recommended over local storage), use the backup and recovery mechanisms suitable for your database. When using AWS RDS, follow the instructions here



Find the instructions for upgrading to the latest version of Lenses here.

Managing Licences 

Connect with SSH to EC2 Instance 

In version 5.0.0 and onwards port 22 is no longer allowed as an Inbound rule, on the Lenses security group. This is in sync with AWS Marketplace security recommendations, since most users would leave the default allow rule of, which is a known security risk.

However, to be able to connect with SSH to the instance, you can apply the following workaround step-by-step guide, that utilizes EC2 Instance Connect browser feature:

1) Find your cloudformation stack and open the stack resources menu

Cloudformation Stack
Stack Resources

2) From there you can find the Lenses Instance details and on the Security tab, go to the Lenses Security group to Edit inbound rules.

EC2 Details
Lenses Security Group

3) Add a temporary SSH allow rule with type SSH, for All Ipv4 addresses and Save. This is a pre-requisite for EC2 Instance Connect feature.

Add 22 Rule All
Rule Applied

4) Go back to instance details and hit the Connect button. It will take you to the EC2 Instance Connect screen, to SSH via the browser. Leave parameters user root as is and hit Connect

Use the browser terminal to edit ~/.ssh/authorized_keys file.

EC2 Instance Connect
EC2 Instance Connect Terminal

5) Assuming you have an SSH keypair, copy the public key content to ~/.ssh/authorized_keys. EC2 Instance Connect terminal, uses Ctrl+V to paste copied content from outside the browser screen (such as clipboard). Save the file when done.

You can generate a new secure keypair using ssh-keygen -t rsa -b 4096 command.

Public Key setup

6) Now, after adding a public key, you no longer need EC2 Instance Connect so you can restrict SSH either to a specific IP address, e.g. a VPN server address or only your own public IP, using option My IP on the previous SSH rule.

Restrict SSH Rule
Restrict SSH Applied

7) You can now connect from your local environment to the Lenses Instance, using the private key as root user. Remember to revoke SSH access when done.

SSH Login

IAM Support 

Lenses supports connection to MSK brokers via IAM. If Lenses is deployed on an EC2 instance it will use the default credential chain loader to authenticate and connect to MSK.

Supported Regions 

The following Regions are supported:

  • us-east-1;
  • us-east-2;
  • us-west-1;
  • us-west-2;
  • ca-central-1;
  • eu-central-1;
  • eu-west-1;
  • eu-west-2;
  • eu-west-3;
  • ap-southeast-1;
  • ap-southeast-2;
  • ap-south-1;
  • ap-northeast-1;
  • ap-northeast-2;
  • sa-east-1.

Security Recommendations 


  • Do not use your AWS root user for deployment or operations;
  • Follow the least privileges principle when granting access to the individual IAM user accounts;
  • Avoid allowing traffic to the Lenses UI from a broad CIDR block where a more specific block could be used.


AWS billing applies for the EC2 instance, CloudWatch logs and optionally CloudWatch metrics.

For the hourly billed version additional hourly charges apply, which depend on the instance size. For the Bring Your Own License (BYOL) you can get a free trial license here.


In case you run into problems, e.g. you cannot connect to Lenses, then the logs could provide more information. The easiest route to do this is to go to CloudWatch in the AWS console. Here, find the log group corresponding to your deployment (it has the same name as the deployment) and pick a log stream. The stream with the /lenses.log suffix contains all log lines regardless of the log level; the stream with the /lenses-warn.log suffix only contains warning level logs.

If the above fails, for example because the logs integration is broken, you can SSH into the EC2 instance. First, set up an ssh connection. Lenses is installed into /opt/lenses, the logs can be found under /opt/lenses/logs for further inspection.


Lenses adds to Amazon MSK a secure User Interface with DataOps capabilities for:

  • Data discovery
  • Data security
  • Data governance
  • Data monitoring
  • Data observability
  • Data alerts

Installation methods 

1. Secure AWS installation
2. AWS marketplace (Hourly usage)
3. AWS marketplace (BYOL - Bring Your own license) - get a trial license
4. AWS EDP Private Offer contact us for an AWS EDP offer.

Help video