ProductsDownload Community Edition

Datasets

get

Retrieves a list of datasets

Path parameters
environmentstringRequired
Query parameters
pageintegerOptional

The page number to be returned, must be greater than zero. Defaults to 1.

Example: 1
pageSizeintegerRequired

The elements amount on a single page, must be greater than zero.

Example: 25
querystringOptional

A search keyword to match dataset, fields and description against.

Example: name
connectionsstring[]Optional

A list of connection names to filter by. All connections will be included when no value is supplied.

tagsstring[]Optional

A list of tag names to filter by. All tags will be included when no value is supplied.

sortBystring · enumOptional

The field to sort results by

Possible values:
sortingOrderstring · enumOptional

Sorting order. Defaults to ascending

Possible values:
includeSystemEntitiesbooleanOptional

A flag to include in the search also system entities (e.g. Kafka's __consumer_offsets topic).

includeMetadatabooleanOptional

Whether to search only by table name, or also to include field names/documentation (defaults to true)

formatstring[]Optional

Schema format. Relevant only when sourceType is SchemaRegistrySubject

hasRecordsbooleanOptional

Filter based on whether the dataset has records

compactedbooleanOptional

Filter based on compacted. Relevant only when sourceType is Kafka

Responses
200

A response wrapping a paginated list of datasets and the set of relative source types

application/json
get
/api/v1/environments/{environment}/proxy/api/v1/datasets
GET /api/v1/environments/{environment}/proxy/api/v1/datasets?pageSize=1 HTTP/1.1
Host: 
Accept: */*

{
  "datasets": {
    "values": [
      {
        "name": "telecom_italia",
        "highlights": [],
        "records": 80000000,
        "recordsPerSecond": 1813,
        "keyType": "AVRO",
        "valueType": "AVRO",
        "connectionName": "Kafka",
        "replication": 1,
        "consumers": 0,
        "partitions": 6,
        "fields": {
          "key": [],
          "value": [
            {
              "type": "STRING",
              "name": "name",
              "typeDetails": {
                "lensesDataType": "STRING",
                "kind": "Lenses"
              },
              "ancestors": [],
              "highlights": [
                {
                  "fieldName": "name",
                  "startIndex": 1,
                  "endIndex": 5,
                  "arrayIndex": 0
                }
              ],
              "isNullable": false
            }
          ]
        },
        "isSystemEntity": false,
        "isMarkedForDeletion": false,
        "isCompacted": false,
        "sizeBytes": 25,
        "policies": [],
        "permissions": [
          "ShowTopic"
        ],
        "tags": [],
        "retentionMs": 604800000,
        "retentionBytes": 1073741824,
        "backupRestoreState": "backup-restore",
        "lrn": "kafka:topic:my-env/kafka/telecom_italia",
        "sourceType": "Kafka"
      }
    ],
    "pagesAmount": 1,
    "totalCount": 2
  },
  "sourceTypes": [
    "Kafka"
  ]
}
post

Deletes multiple datasets at once

Path parameters
environmentstringRequired
Body
Responses
200Success
application/json
post
/api/v1/environments/{environment}/proxy/api/v1/bulk/datasets/delete
POST /api/v1/environments/{environment}/proxy/api/v1/bulk/datasets/delete HTTP/1.1
Host: 
Content-Type: application/json
Accept: */*
Content-Length: 65

{
  "items": [
    {
      "id": "kafka://telecom_italia"
    },
    {
      "id": "kafka://bank"
    }
  ]
}
{
  "items": [
    {
      "id": "telecom_italia",
      "success": true
    },
    {
      "id": "bank",
      "success": false,
      "message": "Dataset not found"
    }
  ]
}
put

Sets a dataset description. Will respond with a bad request if a blank description is supplied

Path parameters
environmentstringRequired
connectionstringRequiredExample: kafka
datasetNamestringRequiredExample: user_details
Body
descriptionstringOptional
Responses
200Success
No content
put
/api/v1/environments/{environment}/proxy/api/v1/datasets/{connection}/{datasetName}/description
PUT /api/v1/environments/{environment}/proxy/api/v1/datasets/{connection}/{datasetName}/description HTTP/1.1
Host: 
Content-Type: application/json
Accept: */*
Content-Length: 55

{
  "description": "Details about user's purchase history"
}
No content
put

Add one or more tags to a dataset

Path parameters
environmentstringRequired
connectionstringRequiredExample: kafka
datasetNamestringRequiredExample: user_details
Body
Responses
200Success
No content
put
/api/v1/environments/{environment}/proxy/api/v1/datasets/{connection}/{datasetName}/tags
PUT /api/v1/environments/{environment}/proxy/api/v1/datasets/{connection}/{datasetName}/tags HTTP/1.1
Host: 
Content-Type: application/json
Accept: */*
Content-Length: 58

{
  "tags": [
    {
      "name": "crm"
    },
    {
      "name": "PII"
    },
    {
      "name": "legacy"
    }
  ]
}
No content
get

Get a single dataset by connection/name. While information mastered externally might be a few second out of sync with their respective sources (e.g. JMX metadata, etc), information mastered in Lenses's db is guaranteed to be up to date (e.g. tags, descriptions).

Path parameters
environmentstringRequired
connectionstringRequiredExample: kafka
datasetstringRequiredExample: customer-positions
Responses
200Success
application/json
Responseone of
or
get
/api/v1/environments/{environment}/proxy/api/v1/datasets/{connection}/{dataset}
GET /api/v1/environments/{environment}/proxy/api/v1/datasets/{connection}/{dataset} HTTP/1.1
Host: 
Accept: */*

{
  "name": "text",
  "highlights": [
    {
      "fieldName": "text",
      "startIndex": 1,
      "endIndex": 1,
      "arrayIndex": 1
    }
  ],
  "records": 1,
  "recordsPerSecond": 1,
  "keyType": "text",
  "valueType": "text",
  "connectionName": "text",
  "replication": 1,
  "consumers": 1,
  "partitions": 1,
  "fields": {
    "key": [
      {
        "name": "text",
        "typeDetails": {
          "lensesDataType": "text"
        },
        "description": "text",
        "ancestors": [
          "text"
        ],
        "highlights": [
          {
            "fieldName": "text",
            "startIndex": 1,
            "endIndex": 1,
            "arrayIndex": 1
          }
        ],
        "isNullable": true,
        "default": "text"
      }
    ],
    "value": [
      {
        "name": "text",
        "typeDetails": {
          "lensesDataType": "text"
        },
        "description": "text",
        "ancestors": [
          "text"
        ],
        "highlights": [
          {
            "fieldName": "text",
            "startIndex": 1,
            "endIndex": 1,
            "arrayIndex": 1
          }
        ],
        "isNullable": true,
        "default": "text"
      }
    ]
  },
  "isSystemEntity": true,
  "isMarkedForDeletion": true,
  "isCompacted": true,
  "sizeBytes": 1,
  "policies": [
    {
      "policyId": "text",
      "policyName": "text",
      "policyCategory": "text",
      "obfuscation": "text",
      "matchingKeyFields": [
        {
          "name": "text",
          "parents": [
            "text"
          ]
        }
      ],
      "matchingValueFields": [
        {
          "name": "text",
          "parents": [
            "text"
          ]
        }
      ],
      "lrn": "text"
    }
  ],
  "permissions": [
    "text"
  ],
  "description": "text",
  "tags": [
    {
      "name": "text"
    }
  ],
  "retentionMs": 1,
  "retentionBytes": 1,
  "backupRestoreState": {},
  "lrn": "text",
  "sourceType": "text"
}
get

Get tags sorted by dataset count

Path parameters
environmentstringRequired
Query parameters
querystringOptional

Match tag by keyword

Example: user
Responses
200Success
application/json
get
/api/v1/environments/{environment}/proxy/api/v1/datasets/tags
GET /api/v1/environments/{environment}/proxy/api/v1/datasets/tags HTTP/1.1
Host: 
Accept: */*

{
  "tags": [
    {
      "name": "user-generated-content",
      "count": 5
    },
    {
      "name": "user-details",
      "count": 2
    },
    {
      "name": "testuser",
      "count": 1
    }
  ]
}
get

Get a list of available dataset connections

Path parameters
environmentstringRequired
Query parameters
querystringOptional

Match connection by keyword

Example: kafka
perPageintegerOptional

Number of results to be returned. Defaults to 25 if not supplied

Example: 50
Responses
200Success
application/json
get
/api/v1/environments/{environment}/proxy/api/v1/datasets/connections
GET /api/v1/environments/{environment}/proxy/api/v1/datasets/connections HTTP/1.1
Host: 
Accept: */*

{
  "connections": [
    {
      "name": "kafka",
      "sourceType": "Kafka",
      "connectionStatus": "Green"
    },
    {
      "name": "kafka-reconcilation.dev",
      "sourceType": "Kafka",
      "connectionStatus": "Yellow"
    }
  ]
}
get

get ranged metrics for requested dataset

Path parameters
environmentstringRequired
entityNamestringRequired

Dataset's entity name

Example: topic
Responses
200Success
application/json
get
/api/v1/environments/{environment}/proxy/api/v1/datasets/kafka/{entityName}/messages/metrics
GET /api/v1/environments/{environment}/proxy/api/v1/datasets/kafka/{entityName}/messages/metrics HTTP/1.1
Host: 
Accept: */*

[
  {
    "date": "2025-10-24T15:52:30.010Z",
    "messagesCount": 1
  }
]
PreviousWebsocketNextKafka

Last updated

Was this helpful?