# Infrastructure JMX Metrics

{% hint style="success" %}
All core services (Kafka, Schema Registry, Kafka Connect, Zookeeper) use the same set of properties for services’ monitoring.

The Agent will discover all the brokers by itself and will try to fetch metrics using **metricsPort, metricsCustomUrlMappings** and other properties (if specified).

See [JSON schema](https://docs.lenses.io/latest/deployment/configuration/overview#json-schema-support) for support.

Environment variables are supported; escape the dollar sign

```
sslKeystorePassword:
  value: "${ENV_VAR_NAME}"
```

{% endhint %}

```yaml
[connnection]
  configuration:
    metricsPort: 
      value: 9581
    metricsType: 
      value: JMX
    metricsSsl: 
      value: false
    metricsUsername: 
      value: user
    metricsPassword: 
      value: pass
```

## JMX <a href="#jmx" id="jmx"></a>

### Simple <a href="#simple" id="simple"></a>

The same port used for all brokers/workers/nodes. No SSL, no authentication.

```yaml
kafka:
  tags: []
  name: kafka
  configuration:
    kafkaBootstrapServers:
      - PLAINTEXT://my-kafka-host-0:9092
    protocol:
      value: PLAINTEXT
    metricsPort: 
      value: 9585
    metricsType: 
      value: JMX
```

### SSL <a href="#with-ssl" id="with-ssl"></a>

```yaml
kafka:
  tags: []
  name: kafka
  configuration:
    kafkaBootstrapServers:
      - PLAINTEXT://my-kafka-host-0:9092
    protocol: 
      value: PLAINTEXT
    metricsPort: 
      value: 9585
    metricsType: 
      value: JMX
    metricsSsl: 
      value: true
```

### Basic Auth <a href="#with-basic-auth" id="with-basic-auth"></a>

```yaml
kafka:
  tags: []
  name: kafka
  configuration:
    kafkaBootstrapServers:
      - PLAINTEXT://my-kafka-host-0:9092
    protocol: 
       value: PLAINTEXT
    metricsPort: 
      value: 9581
    metricsType: 
      value: JMX
    metricsSsl: 
      value: false
    metricsUsername: 
      value: user
    metricsPassword: 
      value: pass
```

Such a configuration means that the Agent will try to connect using JMX with every pair of **kafkaBootstrapServers.host:metricsPort**, so following the example: **my-kafka-host-0:9581**.

## Jolokia <a href="#jolokia" id="jolokia"></a>

For Jolokia the Agent supports two types of requests: GET (**JOLOKIAG**) and POST (**JOLOKIAP**).

{% hint style="info" %}
For JOLOKIA each entry value in **metricsCustomUrlMappings** must contain protocol.
{% endhint %}

### Simple <a href="#simple-1" id="simple-1"></a>

The same port used for all brokers/workers/nodes. No SSL, no authentication.

```yaml
kafka:
  tags: []
  name: kafka
  configuration:
    kafkaBootstrapServers:
      - PLAINTEXT://my-kafka-host-0:9092
    protocol:
      value: PLAINTEXT
    metricsPort: 
      value: 9585
    # For GET method: JOLOKIAG
    # For POST method: JOLOKIAP 
    metricsType:
      value: JOLOKIAG
    metricsSsl: 
      value: false
    metricsHttpSuffix: 
      value: /jolokia/
```

### Custom Http Request Timeout <a href="#with-custom-http-request-timeout" id="with-custom-http-request-timeout"></a>

JOLOKIA monitoring works on the top of HTTP protocol. To fetch metrics the Agent has to perform either GET or POST request. There is a way of configuring http request timeout using **httpRequestTimeout** property (ms value). Its default value is 20 seconds.

```yaml
httpRequestTimeout: 
  value: 30000
```

### Custom Metrics Http Suffix <a href="#with-custom-metrics-http-suffix" id="with-custom-metrics-http-suffix"></a>

Default suffix for Jolokia endpoints is **/jolokia/**, so that should be provided value. Sometimes that suffix can be different, so there is a way of customizing it by using **metricsHttpSuffix** field.

```yaml
metricsHttpSuffix: 
  value: /custom/
```

## AWS <a href="#aws" id="aws"></a>

{% hint style="warning" %}
Before enabling collection of metrics within Agents provision configuration, make sure in your MSK Provisioned cluster you have enabled open monitoring with Prometheus.
{% endhint %}

AWS has a predefined metrics configuration.

The Agent hits the Prometheus endpoint using port **11001** for each broker.

There is an option of customizing AWS metrics connection in Lenses by using:

* **metricsUsername**
* **metricsPassword**
* **metricsHttpTimeout**
* **metricsHttpSuffix**
* **metricsCustomUrlMappings** and
* **metricsSsl** properties.
* However, except for **metricsHttpTimeout**, the other settings will seldom be needed - AWS has its standard, which is unlikely to change. Customization can be achieved only by API or CLI.

```yaml
kafka:
  tags: [ "dev", "dev-2", "eu"]
  configuration:
    kafkaBootstrapServers:
      value:
        - SASL_SSL://my-broker-0:9098
        - SASL_SSL://my-broker-1:9098
        - SASL_SSL://my-broker-2:9098
    protocol:
      value: SASL_SSL
    saslMechanism:
      value: AWS_MSK_IAM
    saslJaasConfig:
      value: software.amazon.msk.auth.iam.IAMLoginModule required;
    additionalProperties:
      value:
        sasl.client.callback.handler.class: "software.amazon.msk.auth.iam.IAMClientCallbackHandler"
    metricsType:
      value: AWS
    metricsHttpTimeout: # optional, milliseconds
      value: 20000
```

In some cases, the **metricsHttpTimeout** option may be required. Typically, this occurs when the OpenMetrics instance is undersized for the size of the MSK cluster, resulting in longer-than-usual metric retrieval times.

Each Kafka partition adds a large number of metrics, so the OpenMetrics instance should ideally be sized to accommodate the number of partitions that the MSK will host.

Another common pitfall with MSK OpenMetrics is that there exists a global rate limit for each instance. If more than one service hits the OpenMetrics endpoint, the rate limit may be triggered, and the clients will receive an HTTP error code 429. To overcome this, you can set the `lenses.interval.metrics.refresh.broker` option in Lenses Agent. As an example, to make Lenses request metrics every minute, set the value to `60000` (milliseconds).

### Custom URL mapping <a href="#custom-url-mapping" id="custom-url-mapping"></a>

There is also a way to configure custom mapping for each broker (Kafka) / node (Schema Registry, Zookeeper) / worker (Kafka Connect).

Such a configuration means that the Agent will try to connect using JMX for:

* my-kafka-host-0:9582 - because of **metricsCustomUrlMappings**
* my-kafka-host-1:9581 - because of **metricsPort** and no entry in **metricsCustomUrlMappings**

```yaml
kafka:
  tags: ["optional-tag"]
  name: kafka
  configuration:
    kafkaBootstrapServers:
      - PLAINTEXT://my-kafka-host-0:9092
    protocol:
      value: PLAINTEXT
    metricsPort: 
      value: 9581
    metricsType: 
      value: JMX
    metricsSsl: 
      value: false
    metricsCustomUrlMappings:
      value:
        "my-kafka-host-0:9092": my-kafka-host-0:9582
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.lenses.io/latest/deployment/configuration/agent/automation/jmx-metrics.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.