Zookeepers

Connectivity to Zookeeper is optional for Lenses and may be deprecated in the future. Its main purpose is to provide quotas management until quotas can be managed via the Brokers API. Furthermore Zookeeper may be used to autodetect the JMX connectivity settings to the brokers.

Authenticated access to Zookeeper is not officially supported. Whilst Lenses may use information from the jaas.conf file to authenticate to Zookeeper, it will not use these credentials to write into Zookeeper by default since only the Brokers should create ACL protected entries in Zookeeper.

Some managed Kafka services, such as those provided by Aiven, Confluent, and IBM Event Streams do not provide access to Zookeeper.

lenses.zookeeper.hosts = [
  { url: "ZK_HOST_1:2181" },
  { url: "ZK_HOST_2:2181" }
]
# lenses.zookeeper.chroot = ""       # To set a ZK chroot path

If Zookeeper JMX is secured via basic authentication or/and SSL certificates use:

lenses.zookeeper.hosts = [
  {
    url: "ZK_HOST_1:2181",
    metrics: {
      ssl: true,            # Optional, please make the remote JMX/HTTP
                            # certificate is accepted by the Lenses truststore
      user: "admin",        # Optional, the remote JMX/HTTP user
      password: "admin",    # Optional, the remote JMX/HTTP password
      type: "JMX",          # One of 'JMX', 'JOLOKIAP' (POST), 'JOLOKIAG' (GET)
      url: "ZK_HOST_1:9585" # When using jolokia, include the scheme (e.g http://)
    }
  },
  {
    url: "ZK_HOST_2:2181",
    metrics: {
      ssl: true,            # Optional, please make the remote JMX/HTTP
                            # certificate is accepted by the Lenses truststore
      user: "admin",        # Optional, the remote JMX/HTTP user
      password: "admin",    # Optional, the remote JMX/HTTP password
      type: "JMX",          # One of 'JMX', 'JOLOKIAP' (POST), 'JOLOKIAG' (GET)
      url: "ZK_HOST_2:9585" # When using jolokia, include the scheme (e.g http://)
    }
  }
]

If Lenses is deployed through the HDInsight Marketplace the Zookeepers are automatically configured but JMX is not enabled.

If you are deploying on outside of the HDInsight marketplace or via the Azure marketplace , log on to your HDInsights cluster Ambari dashboard and retrieve the Zookeeper urls and ports and update in the lenses.conf file.

Zookeeper JMX ports are not enabled by default.

Go to the Zoookeper
Select Advanced Zookeeper tab
Paste the following at the top and perform a rolling restart of your Zookeepers

JMXPORT=<YOUR-PREFERRED-JMX-PORT>

Update the lenses.conf file:

lenses.zookeeper.hosts = [
  {
    url: "ZK_HOST_1:2181",
    metrics: {
      type: "JMX",       # One of 'JMX', 'JOLOKIAP' (POST), 'JOLOKIAG' (GET)
      url: "ZK_HOST_1:9585"
    }
  },
  {
    url: "ZK_HOST_2:2181",
    metrics: {
      type: "JMX",       # One of 'JMX', 'JOLOKIAP' (POST), 'JOLOKIAG' (GET)
      url: "ZK_HOST_2:9585"
    }
  }
]

lenses.zookeeper.security.enabled

By default, the connection to Zookeeper remains unauthenticated. This only affects the Quota entries, which are written without any Zookeeper ACLs to protect them.

The option lenses.zookeeper.security.enabled is used to enable authentication.

Use the brokers’ principal for Lenses. If Lenses is configured with a different principal, then the brokers will not be able to manipulate the Quota entries, and will fail to start. Please contact our support if you need help with this feature.

See configuration settings.