Alert Rules

Understand how to set up consumer, producer and infrastructure alerting rules in Lenses.

You can set up Alert rules on:

1. Consumers

2. Producers

3. Infrastructure

Alerts are always stored in Lenses even if you don't send them out to an external alerting system.

Consumer Alert Rules

Consumer alert rules are rules set on a consumer groups for specific topics. They allow you defined a consumer lag per partition for the topic overall and optionally send them to an channel for routing to your external alerting system.

Select the Environments & Topics option from the left sidebar, expand the environment node you are interested in and select the Alerts node, this will open the listing of alerts.

Select consumer rules to see a list of existing consumer groups rules, you can then either edit one or create a new rule.

Producer Alert Rules

Producer alert rules allow you set alerts when the number of messages produced to a topic meets certain criteria:

1. Less, equals or greater than a specified amount

2. Over a time period, e.g every 1 minute, hour etc.

To create an producer alert for an environment, select the topics and environment on the left main panel, then expand the environment node you are interested in and select Alerts. You can also acces this via the context menus on the environment dashboard view.

Select consumer rules to see a list of existing producer rules, you can then either edit one or create a new rule.

Infrastructure Alert Rules

Infrastructure alerts are built in to Lenses. You can toggle them on or off and send them to alert channels.

Alert	Description
Lenses License	The Lenses license is invalid.
Kafka broker is down	A Kafka broker in the cluster is not healthy.
Zookeeper node is down	A Zookeeper node is not healthy.
Connect Worker is down	A Kafka Connect worker node is not healthy.
Schema Registry is down	A Schema Registry instance is not healthy.
Under replicated partitions	The Kafka cluster has one or more under-replicated partitions.
Partitions offline	The Kafka cluster has one or more partitions offline, with no active leader.
Active Controller	The Kafka cluster has zero, or more than one, active controller.
Multiple Broker versions	The Kafka cluster is under a version upgrade and not all brokers have been upgraded.
File-open descriptors on Brokers	A Kafka broker has an alarming number of open file descriptors. This alert triggers when the operating system exceeds 90% of the available open file descriptors.
Average % the request handler is idle	The average fraction of time the request handler threads are idle is dangerously low. The alert is HIGH when the value is below 10%, and CRITICAL when it is below 2%.
Fetch requests failure	Fetch requests are failing. If the failure rate per second is greater than 10%, the alert level is CRITICAL. Otherwise, it is HIGH.
Produce requests failure	Produce requests are failing. If the failure rate per second is greater than 10%, the alert level is CRITICAL. Otherwise, it is HIGH.
Broker disk usage	A Kafka broker’s disk usage is greater than the cluster average. The built-in threshold is 1 GByte.
Leader imbalance	A Kafka broker has more leader replicas than the average broker in the cluster.