1.0
Features
Lenses is a Streaming Data Platform for Apache Kafka
A secure User Interface and vital enterprise capabilities that improves engineering teams productivity, and gives visibility and secure access to business users, data scientists and administrators using Apache Kafka.
- Flexibility View, use and process any type of data via interactive SQL
- Integration Integrate with popular data stores using open source Kafka connectors
- Security Handle sensitive data, LDAP, and Kerberos support
- Scalability Run and scale Streaming SQL
- High Availability Perform real-time operational business tasks with confidence
- Monitoring Infrastructure and application topology KPIs
- Compatibility Leverage your existing IT infrastructure
- Auditing Role-based access and auditing
Lenses SQL Engine
Lenses Kafka SQL is a multi-purpose engine, it fully supports Avro and Json payload types and can execute both batch and real-time streaming SQL.
Explore Kafka data
Lenses provides a secure web UI to explore Kafka topics in real time or simply browse the topics content while controlling the data returned by filtering on the Kafka message content and its partition / offset / timestamp information.
Kafka Streaming SQL topologies
Continuous data queries with streaming SQL to filter, aggregate, join and/or transform streams. Via the UI you can visualise, monitor the topologies or even scale out the processing. Furthermore the execution plan viewer allows your to optimize for performance.
Avro Messages
Full support for Avro messages, including decimal type (for financial institutions), is available in Lenses SQL Engine. Lenses platform integrates with your schema registry and provides a rich user interface to create, edit and track schemas.
Kafka Connectors
Manage multiple connect clusters as well as manage and monitor connectors workers and their tasks. We have built 20+ Kafka connectors with Lenses SQL support.
Through our Cloudera integration, the Cloudera Custom Service Descriptor, you can manage your Kafka Connect clusters. It allows spinning up new clusters, controlling the number of Connect workers and of course leverages Cloudera Manager integration for monitoring.
Use simple configuration and only SQL to create end-to-end pipelines from all major data sources and sinks including Cassandra, Elastic, InfluxDB, Azure CosmosDB, MQTT, JMS and more. See how British Gas loads 4 billion messages per day into Elastic with our Connector and simple SQL statement.
# Upsert into elastic with auto create and index suffix
UPSERT INTO index SELECT id, string_field FROM topic PK id AUTOCREATE WITHINDEXSUFFIX=_{YYYY-MM-dd} WITHDOCTYPE=lenses
Kafka Consumers Lag
Monitor Kafka consumer lag in real time and manage consumer offsets. Set up alerting to make sure you can scale or take the appropriate action.
Monitor Kafka Services
Monitor the core services and infrastructure via JMX for Kafka Brokers, Zookeeper, Schema Registry and Connect. Prevent and react to potential issues and get insights into your clusters’ performance.
Kafka Alerting
Set up alerting and prevent issues early.
Kafka Auditing
Track all changes to your Kafka cluster: topic creation, deletion, configuration, modification for schemas,topics, connectors and processors. A single point of reference for Who did what and when on your data streaming platform.
Kafka Security
Lenses supports basic authentication and LDAP. Additionally, different operator roles are provided to restrict user actions, giving operators a way to provide security on Kafka. Users can manage topic ACLs (Access Control Lists) to restrict readers and writers of data.
Latest features
For the latest features check https://features.lenses.io