Alerting & monitoring


Lenses monitors your topics and provide visuals to help you ensure health status on data production/consumption and data distribution.

Data produced 

Lenses continuously monitors all Kafka events produced in a Kafka cluster. It uses the native Kafka client to calculate in real-time metrics around events produced in a Kafka topic per time window.

Data produced

The above highlights the messages produced on the Kafka topic over the period of the last 30 days. The minimum, average and maximum number of events produced per day are visualised in a way that enables you to easily identify the real behavior of your producers.

The metrics are stored in Lenses for 30 days.

You can add Alerts on data produced metrics to ensure your SLAs.

Data produced alerts

Adding a new Data SLA requires setting up a threshold, a time window and also who (which channel) to notify when the alert is triggered.

Data produced add alert

JMX On the bottom of the metrics page, a data table is available. This provides a summary of key topic metrics across your cluster sourced by JMX if enabled.

Jmx

Partitions and replicas 

Lenses adds observability and health monitoring of Kafka partitions. The visual map displays the number of messages that are currently stored per topic-partition. Using this map, you can quickly identify if a partition has been over-utilized and requires to rearrange the partitioning strategy.

Topic partitions

The technical information of the partitions is also available, including the replication status. Partition information include:

  • Leader broker per partition
  • Records per partition
  • Data size per partition
  • Max and min offset per partition
  • Whether replicas are in-sync

Monitor topic consumers 

For each topic you can view its consumers. The lag is visualized in a bar, by showing the minimum and maximum consumer lag for each partition. Drill down to the consumer details to see each partition.

Topic consumers

For each consumer group you can add Alerts for the lag threshold. If you want to monitor all your cluster consumers navigate to Consumers.

--
Last modified: December 1, 2023