Services metrics

A common instruction for Services’ metrics. All core services (Kafka, Schema Registry, Kafka Connect, Zookeeper) use the same set of properties for services’ monitoring.

JMX 

Simple 

The same port used for all brokers/workers/nodes. No SSL, no authentication.

- metricsPort: 9585
- metricsType: JMX
- metricsSsl: false

with SSL 

- metricsPort: 9585
- metricsType: JMX
- metricsSsl: true

with Basic Auth 

- metricsPort: 9585
- metricsType: JMX
- metricsSsl: false
- metricsUsername: user
- metricsPassword: pass

Example 

  kafka:
    tags: []
    templateName: Kafka
    configurationObject:
      kafkaBootstrapServers:
        - PLAINTEXT://my-kafka-host-0:9092
      protocol: PLAINTEXT
      metricsPort: 9581
      metricsType: JMX
      metricsSsl: false
      metricsUsername: user
      metricsPassword: pass

Such a configuration means that Lenses will try to connect using JMX with every pair of kafkaBootstrapServers.host:metricsPort, so following the example: my-kafka-host-0:9581.

Jolokia 

For jolokia Lenses supports two types of requests: GET (JOLOKIAG) and POST (JOLOKIAP).

Simple 

The same port used for all brokers/workers/nodes. No SSL, no authentication.

- metricsPort: 9585
- metricsType: JOLOKIAG
- metricsSsl: false
- metricsHttpSuffix: /jolokia/

with SSL 

- metricsPort: 9585
- metricsType: JOLOKIAG
- metricsSsl: true
- metricsHttpSuffix: /jolokia/

with Basic Auth 

- metricsPort: 9585
- metricsType: JOLOKIAG
- metricsSsl: false
- metricsUsername: user
- metricsPassword: pass
- metricsHttpSuffix: /jolokia/

with custom Http Request Timeout 

JOLOKIA monitoring works on the top of HTTP protocol. To fetch metrics Lenses has to perform either GET or POST request. There is a way of configuring http request timeout using httpRequestTimeout property (ms value). Its default value is 20 seconds.

- metricsPort: 9585
- metricsType: JOLOKIAG
- metricsSsl: false
- httpRequestTimeout: 30000
- metricsHttpSuffix: /jolokia/

with custom Metrics Http Suffix 

Default suffix for Jolokia endpoints is /jolokia/, so that should be provided value. Sometimes that suffix can be different, so there is a way of customizing it by using metricsHttpSuffix field.

- metricsPort: 9585
- metricsType: JOLOKIAG
- metricsSsl: false
- metricsHttpSuffix: /custom/

Example 

  kafka:
    tags: []
    templateName: Kafka
    configurationObject:
      kafkaBootstrapServers:
        - PLAINTEXT://my-kafka-host-0:9092
      protocol: PLAINTEXT
      metricsPort: 9581
      metricsType: JOLOKIAP
      metricsSsl: false
      metricsUsername: user
      metricsPassword: pass
      httpRequestTimeout: 30000
      metricsHttpSuffix: /custom/

Such a configuration means that Lenses will try to connect using JOLOKIA for each broker following this pattern: protocol://kafkaBootstrapServers.host:metricsPort/metricsHttpSuffix/. Protocol part is http or https - it depends on metricsSsl value. Taking the values from the example, Lenses will try to fetch metrics using http://my-kafka-host-0:9581/custom/.

AWS 

AWS has predefined metrics configuration. Lenses hits Prometheus using port 11001 for each broker. There is an option of customizing AWS metrics connection in Lenses by using metricsUsername, metricsPassword, httpRequestTimeout, metricsHttpSuffix, metricsCustomUrlMappings, metricsSsl properties, but most likely no one will need to do that - AWS has its own standard and most probably it won’t change. Customization can be achieved only by API or CLI - UI does not support it.

Custom url mapping 

There is also a way to configure custom mapping for each broker (Kafka) / node (Schema Registry, Zookeeper) / worker (Kafka Connect).

Example 

  kafka:
    tags: []
    templateName: Kafka
    configurationObject:
      kafkaBootstrapServers:
        - PLAINTEXT://my-kafka-host-0:9092
        - PLAINTEXT://my-kafka-host-1:9092
      protocol: PLAINTEXT
      metricsPort: 9581
      metricsType: JMX
      metricsSsl: false
      metricsCustomUrlMappings:
        "my-kafka-host-0:9092": my-kafka-host-0:9582

Such a configuration means that Lenses will try to connect using JMX for:

• my-kafka-host-0:9582 - because of metricsCustomUrlMappings
• my-kafka-host-1:9581 - because of metricsPort and no entry in metricsCustomUrlMappings

Note: For JOLOKIA each entry value in metricsCustomUrlMappings must contain protocol.

Note: Kafka connection requires Bootstrap Servers, which means that to establish connection and discover all the brokers in cluster there is no need to provide whole list of brokers in kafkaBootstrapServers. In such a case Lenses will discover all the brokers by itself and will try to fetch metrics using metricsPort, metricsCustomUrlMappings and other properties (if specified).