A Java SDK is available, for example, to add Kafka Streams, Akka Streams, and Spark Structured Streaming. You can find full working examples on GitHub.

Each app can be registered with a set of endpoints that return the status of each app’s running instances. These endpoints are called runners and allow Lenses to get the running instances’ status by pinging these endpoints. The app state is consolidated based on the individual runner’s state, which is periodically checked.

Each runner can be only:

• RUNNING - When the health-check endpoints return 200 HTTP Status Code

• UNKNOWN - When the health-check endpoints return anything other than 200 HTTP Status Code

Lenses consolidates the individual runners’ statuses into a single app state:

• RUNNING - All Runners are RUNNING

• WARNING - At least 1 Runner is UNKWOWN

• UNKNOWN - No information about Runners, either they are unreachable

Registering a new application

To add your external apps to Lenses, you can currently use the API by doing a POST to api/v1/apps/external endpoint.

Viewing your application in the Lenses

Apps are another type of built-in application. Go to Workspace->Apps to view your application.

In the application details page, you can see additional information about each application, and also information about each runner, along with metadata information, such as Description, and auditing information such as Created At, Created By and Modified At, Modified By.

Scaling your application runners

You can add runners with a POST request to api/v1/apps/external. The body of the new request should be the same as the first one except for the runners key. What needs to be changed is only the array of the runners key.

You can also remove runners with a DELETE request.

Remove your Application

You can remove the app by making a DELETE request to /api/v1/apps/external/{name}. By Removing an App from Lenses, you just drop Lenses’ visibility for that specific App.

Alternatively, you can use the Lenses UI to remove the App:

The Topology is a rich UI that enables the visualization of detailed Data Flow Graphs, ETL Pipelines Stream Processing and Analytics workflows and their relation to native Lenses/Kafka resources, such as Kafka Topics, SQL Processors, and Kafka Connectors.

The Graph in Lenses Topology is automatically generated based on the available applications running against your cluster.

Lenses Topology is comprised of several different Nodes Types. From Topics, SQL Processors, and Connectors, to External Application. Each Node on the UI is clickable. That way, we can take a close view and look for more information regarding each Node.

You can link any custom microservice or other streaming application running and producing, consuming, or processing data in Kafka by self-registering the application.