Kafka Streams Metrics

For more advanced applications, you may wish to display metrics in addition to the topology overlay. In the case of Kafka Streams this is as simple as providing an implementation of the KafkaClientSupplier interface when creating the KafkaStreams object. This interface is used by the streams library to instantiate instances of producers and consumers. By using a custom supplier, the topology client is able to access the metrics of each producer and consumer and dispatch these to Lenses.

First, you will require this maven module:

<dependency>
    <groupId>com.landoop</groupId>
    <artifactId>lenses-topology-client-kafka</artifactId>
    <version>1.0.0</version>
</dependency>

Then, you must instantiate an instance of TopologyKafkaStreamsClientSupplier, which is created by passing in the topology object that describes the application, as well as the client from earlier. This instance is then provided to the Kafka Streams constructor.

For example:

KafkaStreams streams = new KafkaStreams(builder.build(), new StreamsConfig(streamProps), new TopologyKafkaStreamsClientSupplier(client, topology));

And that is all that is required.

Note: When you create the topology instance, set the app type to AppType.KafkaStreams so that the UI renders with the correct visualization.

Simple consumer/producer Metrics

With the growing adoption of Streaming Microservices, specifically on Apache Kafka, you can easily make your application and its metrics available in the Lenses topology.

A consumer only application can be registered like this:

Similarly, for a producer only application, you have the following:

Last, for a service that both consumes and produces records, the following example can be used to register it with the topology: