Zookeepers

Connectivity to Zookeeper is optional for Lenses and may be deprecated in the future. Its main purpose is to provide quotas management until quotas can be managed via the Brokers API. Furthermore Zookeeper may be used to autodetect the JMX connectivity settings to the brokers.

Authenticated access to Zookeeper is not officially supported. Whilst Lenses may use information from the jaas.conf file to authenticate to Zookeeper, it will not use these credentials to write into Zookeeper by default since only the Brokers should create ACL protected entries in Zookeeper.

Some managed Kafka services, such as those provided by Aiven, Confluent, and IBM Event Streams do not provide access to Zookeeper.

lenses.zookeeper.hosts = [
  { url: "ZK_HOST_1:2181" },
  { url: "ZK_HOST_2:2181" }
]
# lenses.zookeeper.chroot = ""       # To set a ZK chroot path

lenses.zookeeper.hosts = [
  {
    url: "ZK_HOST_1:2181",
    metrics: {
      ssl: true,            # Optional (default: false); if used - please make sure the remote 
                            # certificate is accepted by Lenses truststore
      user: "admin",        # Optional, the remote JMX user
      password: "admin",    # Optional, the remote JMX password
      type: "JMX",          
      url: "ZK_HOST_1:9585" # no scheme should be included, just host and port
    }
  },
  {
    url: "ZK_HOST_2:2181",
    metrics: {
      ssl: true,            # Optional (default: false); if used - please make sure the remote 
                            # certificate is accepted by Lenses truststore
      user: "admin",        # Optional, the remote JMX user
      password: "admin",    # Optional, the remote JMX password
      type: "JMX",          
      url: "ZK_HOST_2:9585" # no scheme should be included, just host and port
    }
  }
]

lenses.zookeeper.hosts = [
  {
    url: "ZK_HOST_1:2181",
    metrics: {
      ssl: true,           # Optional (default: false); if used - please make sure the remote 
                            # certificate is accepted by Lenses truststore
      user: "admin",        # Optional, the remote HTTP user
      password: "admin",    # Optional, the remote HTTP password
      type: "JOLOKIAP",     # One of 'JOLOKIAG' (for GET requests), 'JOLOKIAP' (for POST requests)
      url: "http://host1:9585/jolokia/" # notice the scheme and trailing slash - they are both required for standard Jolokia configuration
    }
  },
  {
    url: "ZK_HOST_2:2181",
    metrics: {
      ssl: true,            # Optional (default: false); if used - please make sure the remote 
                            # certificate is accepted by Lenses truststore
      user: "admin",        # Optional, the remote HTTP user
      password: "admin",    # Optional, the remote HTTP password
      type: "JOLOKIAP",     # One of 'JOLOKIAG' (for GET requests), 'JOLOKIAP' (for POST requests)
      url: "https://host2:9585/jolokia/" # notice the scheme and trailing slash - they are both required for standard Jolokia configuration
    }
  }
]

If Lenses is deployed through the HDInsight Marketplace the Zookeepers are automatically configured but JMX is not enabled.

If you are deploying on outside of the HDInsight marketplace or via the Azure marketplace , log on to your HDInsights cluster Ambari dashboard and retrieve the Zookeeper urls and ports and update in the lenses.conf file.

Zookeeper JMX ports are not enabled by default.

Go to the Zookeeper
Select Advanced Zookeeper tab
Paste the following at the top and perform a rolling restart of your Zookeepers

JMXPORT=<YOUR-PREFERRED-JMX-PORT>

Update the lenses.conf file:

lenses.zookeeper.hosts = [
  {
    url: "ZK_HOST_1:2181",
    metrics: {
      type: "JMX",       # One of 'JMX', 'JOLOKIAP' (POST), 'JOLOKIAG' (GET)
      url: "ZK_HOST_1:9585"
    }
  },
  {
    url: "ZK_HOST_2:2181",
    metrics: {
      type: "JMX",       # One of 'JMX', 'JOLOKIAP' (POST), 'JOLOKIAG' (GET)
      url: "ZK_HOST_2:9585"
    }
  }
]

lenses.zookeeper.security.enabled

By default, the connection to Zookeeper remains unauthenticated. This only affects the Quota entries, which are written without any Zookeeper ACLs to protect them.

The option lenses.zookeeper.security.enabled is used to enable authentication.

Use the brokers’ principal for Lenses. If Lenses is configured with a different principal, then the brokers will not be able to manipulate the Quota entries, and will fail to start. Please contact our support if you need help with this feature.

See configuration settings.