You are viewing documentation for an older version of Lenses.io platform. For the latest, click here.

Features

Lenses is a Streaming Data Platform for Apache Kafka

Kafka Data Management Platform

A secure User Interface and vital enterprise capabilities that improves engineering teams productivity, and gives visibility and secure access to business users, data scientists and administrators using Apache Kafka.

  • Flexibility View, use and process any type of data via interactive SQL
  • Integration Integrate with popular data stores using open source Kafka connectors
  • Security Handle sensitive data, LDAP, and Kerberos support
  • Scalability Run and scale Streaming SQL
  • High Availability Perform real-time operational business tasks with confidence
  • Monitoring Infrastructure and application topology KPIs
  • Compatibility Leverage your existing IT infrastructure
  • Auditing Role-based access and auditing

Lenses SQL Engine 

Lenses Kafka SQL is a multi-purpose engine, it fully supports Avro and Json payload types and can execute both batch and real-time streaming SQL.

KStreams with SQL

Explore Kafka data 

Lenses provides a secure web UI to explore Kafka topics in real time or simply browse the topics content while controlling the data returned by filtering on the Kafka message content and its partition / offset / timestamp information.

Kafka data visibility

SQL Processors 

Create boundless data queries with SQL processors to aggregate, join and/or transform streams. Via the UI you can visualize, monitor the topologies or even scale out processing. Furthermore, the execution plan viewer allows you to optimize for performance.

Kafka data visibility

Kafka Streaming SQL topologies 

With Lenses you can now build and operate with confidence complex streaming topologies, such as ETL data pipelines, and stream processing and analytics combining multiple Connectors, Processors and Topics with full data lineage.

Lenses is able to visualize not only the topologies of LSQL processors but is also able to display a global, high level view of your landscape, including topics, connectors and processors that comprise your enterprise pipelines.

Kafka data visibility

Avro Messages 

Full support for Avro messages, including decimal type (for financial institutions), is available in Lenses SQL Engine. Lenses platform integrates with your schema registry and provides a rich user interface to create, edit and track schemas.

Kafka AVRO message viewer

Kafka Connectors 

With Lenses you can manage multiple connect clusters, build streaming ETL data pipelines with ease while monitoring connectors and their tasks. Lenses contains the largest collection of Apache Kafka connectors with Lenses SQL support, for all major data sources and sinks including Cassandra, Elastic, InfluxDB, Azure CosmosDB, MQTT, JMS and more.

.. image:: lenses-kafka-connectors.png :align: center :width: 600px

See how British Gas streams 4+ billion <https://www.landoop.com/blog/2017/05/how-british-gas-is-streaming-4-billion-messages-with-our-connectors-to-enable-connected-homes/>__ entries every day into Elastic-Search with a simple SQL statement:

# Upsert into elastic with auto create and index suffix
UPSERT INTO index SELECT id, string_field FROM topic PK id AUTOCREATE WITHINDEXSUFFIX=_{YYYY-MM-dd} WITHDOCTYPE=lenses

Kafka Consumers Lag 

Monitor Kafka consumer lag in real time and manage consumer offsets. Set up alerting to make sure you can scale or take the appropriate action.

Kafka Consumer Lag Tool

Kafka Monitoring 

Monitor the core services and infrastructure via JMX for Kafka Brokers, Zookeeper, Schema Registry and Connect. Prevent and react to potential issues and get insights into your clusters’ performance.

Kafka monitoring
Kafka offline partitions

Kafka Alerting 

Set up alerting and prevent issues early.

Kafka Alerting service

Kafka Auditing 

Track all changes to your Kafka cluster: topic creation, deletion, configuration, modification for schemas,topics, connectors and processors. A single point of reference for Who did what and when on your streaming data platform.

Kafka Alerting service

Kafka Security 

Lenses supports basic authentication and LDAP. Additionally, different operator roles are provided to restrict user actions, giving operators a way to provide security on Kafka. Users can manage topic ACLs (Access Control Lists) to restrict readers and writers of data.

Kafka ACLs

Latest features 

For the latest features check https://features.lenses.io