> For the complete documentation index, see [llms.txt](https://docs.lenses.io/latest/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.lenses.io/latest/changelog/lenses-6.3/agent.md). # Agent Change log for Lenses Agent 6.3. {% updates format="full" %} {% update date="2026-10-09" tags="new,improvements,fixed" %} ## 6.3.0 {% hint style="warning" %} **Before you upgrade** * **Java 17 or later is required.** The Docker images include it. An Agent started on Java 11 fails at startup. * **Kafka brokers must run Apache Kafka 2.1 or later (Confluent Platform 5.1 or later).** * **SASL/OAUTHBEARER with a token endpoint needs an allow-list.** The Kafka 4.2 client only calls token endpoints listed in the JVM system property `org.apache.kafka.sasl.oauthbearer.allowed.urls` (comma-separated). Set it with `LENSES_OPTS`, for example `-Dorg.apache.kafka.sasl.oauthbearer.allowed.urls=https://idp.example.com/oauth2/token`, or with `lensesAgent.lensesOpts` in the Helm chart. Setting it in the Kafka connection has no effect. Schema Registry connections that use OAuth bearer authentication need the same property. There is no setting yet to pass it to SQL Processors on Kubernetes, so keep processors that use such a connection on their current image. The callback handler `org.apache.kafka.common.security.oauthbearer.secured.OAuthBearerLoginCallbackHandler` was removed; use `org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginCallbackHandler`. * **SQL Processors must use `exactly_once_v2`.** `processing.guarantee` values `exactly_once` and `exactly_once_beta` are no longer accepted. Processors that run inside the Agent (`IN_PROC` mode) and use them fail to start after the upgrade, and must be recreated with `exactly_once_v2`. Processors on Kubernetes keep the image they were deployed with; recreate them the same way before moving them to the 6.3 image. {% endhint %} ### New * **Data policies move to Lenses HQ.** Data policies can now be managed centrally in Lenses HQ, as part of IAM roles, instead of on each Agent. They become global: the Agent applies, for each user, the data policies of that user's roles, so the same masking applies in every environment. Set `lenses.data.policies.legacy.enabled` to `false` to switch an Agent over. The default, `true`, keeps the Agent's own data policies, which work as in 6.2. Switch only after upgrading Lenses HQ to 6.3 and re-creating your data policies on roles: with an older HQ, nothing is masked. Once switched, the Agent's data policy API is read-only, and its data policy list no longer reports which topics the inactive policies would mask. Agent-managed data policies will be replaced by IAM role data policies in a future release. * **Read records by offset from Lenses HQ.** With Agent 6.3, Lenses HQ can fetch specific records from a topic partition by offset. HQ uses this for its new API endpoint for reading records, for the record preview shown when you click an offset in the consumer group and topic **Consumers** tables, and for the raw-bytes view of records that cannot be decoded. With older Agents these features are unavailable or fall back to SQL. The Agent checks that the user may read the topic, and applies data policies before records leave it. `lenses.kafka.read.topic.records.max.concurrent` (default 16) limits how many of these reads run at once. Each read holds its own Kafka connection, and reads beyond the limit are refused rather than queued. ### Improvements * **Kafka 4.2 and Confluent 8.2.** The Agent and SQL Processors use the Apache Kafka 4.2 and Confluent Schema Registry 8.2 client libraries. Kafka clusters that still run on ZooKeeper remain supported, including ZooKeeper connections for broker discovery and monitoring. * **SQL Processors on Kafka Streams 4.2.** Processors that repartition streams, such as joins between topics with different partition counts, keep their original topology, so redeploying or scaling them does not change their repartition topics. The RocksDB settings `rocksdb.table.block.cache.compressed.size` and `rocksdb.table.block.cache.compressed.num.shard.bits` are now ignored, with a warning, because RocksDB removed the compressed block cache. * **Consumer group reads use their own Kafka admin clients.** Listing and describing consumer groups and reading their offsets no longer wait behind other admin operations. Set the pool size with `lenses.kafka.admin.pool.max.size` (default 1). Idle clients beyond the first close after `lenses.kafka.admin.pool.idle.ttl` (default 5m). * **Faster topic format detection.** Detection keeps a pool of up to 8 Kafka consumers open, samples several partitions so an empty first partition no longer matters, skips tombstones, and honors the sampling poll timeout and attempts settings. A Kafka consumer leak when sampling shuts down is fixed. * **Consumer groups open in Lenses HQ from the legacy Agent UI.** The legacy Agent UI no longer has its own consumer group screens. Its consumer group pages, and the consumer group links in its menu, dashboard, topic **Consumers** tab, alert rules and SQL Processors, now open the environment's consumer groups in Lenses HQ Studio. ### Fixed * **Permissions and data policies the Agent cannot interpret.** A role permission whose resources the Agent cannot interpret, for example a resource type introduced by a newer Lenses HQ, now grants nothing. A request carrying a data policy the Agent cannot interpret used to have that policy ignored, leaving its fields unmasked. Such requests are now rejected. To drop only the unknown data policy during an upgrade instead, set the JVM system property `lenses.datapolicies.strict-conversion=false` or the environment variable `LENSES_DATAPOLICIES_STRICT_CONVERSION=false`. * **Data policy list permissions.** A data policy's affected resources in the data policy list no longer include topics, SQL processors, connectors or applications that you are not allowed to list, matching the details view. * **Kafka Connect worker addresses stay hidden.** With `lenses.api.services.endpoints.hide` enabled, Kafka Connect error messages no longer include worker URLs or addresses, and connector and task worker IDs are hidden. The health reason shown for a Kafka Connect cluster now reports only the HTTP status the worker returned. * **Roles with a `kafka:cluster` resource.** Users whose role names a `kafka:cluster` resource can use the environment again; every request they made through Lenses HQ used to fail. The resource has no effect, as in Lenses HQ. * **Connector restarts on Kafka Connect older than 3.0.** Restarting a connector now restarts its tasks too, and reports any task that could not be restarted. * **Connector-to-topic lineage.** Lineage for connectors that do not name their topics in their configuration no longer disappears until the Agent restarts after a single 404 or 403 from a Kafka Connect worker, for example during a rolling restart. * **Connector monitoring.** Monitoring of a Kafka Connect cluster no longer stops silently on an error whose chain of causes loops back on itself. * **Schema inference.** Inference no longer hangs on topics with null keys or values, or whose last offset is a transaction marker, and reports an error if it stops making progress. * **Kafka quota audit entries.** Audit entries for adding or removing Kafka quotas record the same entity names as the quota listing, with `` for default quotas and `clients` for the client part of user-and-client quotas. {% endupdate %} {% endupdates %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.lenses.io/latest/changelog/lenses-6.3/agent.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.