AWS MSK

Lenses.io is a featured integration of Amazon MSK and can be installed on a EC2 instance with a Cloud Formation template.

Lenses is also available as an AMI and you can install on an EC2 instance and configure as a standard Linux application.

MSK prerequisites 

  • AWS subscription
  • MSK cluster with with Open Monitoring enabled

Follow the MSK instructions to enable Open Monitoring during the cluster creation.

Note down the Cluster ARN and Security group:

MSK ARN

Installing 

1) Select CloudFormation Template and Lenses - EC2 MSK as the delivery method.

MSK CloudFormation template

Make sure you select the region your cluster is in.


2) Choose Launch CloudFormation as the action:

MSK launch Lenses action

3) Continue with the default options for creating the stack in the AWS wizard.

At the Specify stack details, enter the following:

  • Name for your Lenses deployment at the top
  • InstanceType - Recommended t2.large
  • SSHLocation - 0.0.0.0/0
  • LensesLocation - 0.0.0.0/0
  • Lenses license - the contents of the JSON license file
  • AWS MSK ARN - From AWS MSK Console
  • AWS Cluster Security Group - From AWS MSK Console
  • Enable CloudWatch metrics

Optionally set the Connect and Schema Registry configurations if you have them installed.

MSK CloudFormation details

4) Review the stack:

MSK Lenses installation review

5) Accept the terms and conditions and create the stack:

MSK Lenses ack

6) Once the stack has deployed, go to the Output tab and click on the FQDN link:

MSK stack FQDN link

7) Login to Lenses with admin/ec-instance-name:

You can find the instance name in the resources tab of the Cloud Formation stack:

MSK Lenses instance name

Template polices 

The template enables the following policies:

  • logs:CreateLogGroup
  • logs:CreateLogStream
  • logs:PutLogEvents
  • kafka:Describe*
  • kafka:List*
  • logs:PutLogEvents
  • acm-pca:IssueCertificate
  • acm-pca:GetCertificate

Template options 

FieldDescriptionTypeRequired
LensesLicenseThe Lenses licensestringyes
MSKClusterARNThe generated MSK Cluster ARNstringyes
MSKSecurityGroupThe MSK security group in order to allow all traffic as inbound traffic to Lensesstringno
SchemaRegistryURLsA list of Schema Registry nodesarrayno
ConnectURLsA list of all the Kafka Connect clustersarrayno

AWS MSK UI 

Lenses adds to Amazon MSK:

  • Data discovery
  • Data security
  • Data governance
  • Data monitoring
  • Data observability
  • Data alerts

Installation methods 

1. BYOL (Bring Your own license) or use a trial license
2. Hourly usage via the AWS Marketplace
3. Private Offer. Contact us to receive a private offer that works with AWS EDP.