Rate Limiting

Rate limit the calls Lenses makes to Schema Registries and Connect Clusters.

Careful monitoring of the data managed by the configured Schema Registry is paramount in order for Lenses to provide its users with the most up-to-date data.

For most cases, this monitoring doesn't cause any issues, but it might happen that, in some cases, Lenses is forced to access the Schema Registry too often.

If this happens, or if you want to make sure Lenses does not go over a rate limit imposed by the Schema Registry, it is possible to throttle Lenses usage of the Schema Registry's API.

In order to do so, it is possible to set the following Lenses configuration:

lenses.conf

# schema registry
lenses.schema.registry.client.http.rate.type="sliding" 
lenses.schema.registry.client.http.rate.maxRequests= 200
lenses.schema.registry.client.http.rate.window="2 seconds"

# connect clusters
lenses.connect.client.http.rate.type="sliding"                 
lenses.connect.client.http.rate.maxRequests=200        
lenses.connect.client.http.rate.window="2 seconds"  

Doing so will make sure Lenses does not issue more than maxRequests over any window period.

The exact values provided will depend on things like the resources of the machine hosting the Schema Registry, the number of schemas, how often are new schemas added, so some trial and error is required. These values should however define a rate smaller than the one allowed by the Schema Registry.