Adding metadata to datasets


For every dataset you can add custom metadata: tags and descriptions. Metadata serve several purposes in order to describe the dataset owners, producers or other supported functions. Use tags to create taxonomies and reports among your data sources.

Required permission 

PermissionTypeDescription
Update MetadataNamespaceAdd/Remove dataset tags Add/Edit dataset description

Every user can view the dataset metadata and use it in catalog filters. To manage metadata you need to be authorized with the Update Metadata namespaced permission. That means you can only add and update the metadata within your data namespaces.

Access Management & permissions

Add metadata 

To add tags or description go to the dataset page:

  • Navigate to Explore and select the dataset (ie. a Kafka topic)
  • On the dataset page Summary click the edit buttons to add metadata:
metadata permission

Description 

Description is plain text and accepts for 0 to 512 characters long excluding whitespace characters. Markdown is a valid input for description but currently is not get rendered.

Tags 

Tags are alphanumeric strings, without spaces that can be from 1 to 255 characters long. Special characters are allowed.

Filtering datasets by tags 

Use the tags to filter datasets from the data catalog, for any data source. While the data sources filter is accumulative (logical OR), the tags are aggregating (logical AND).

metadata permission

To enable the tags, select on the display settings:

metadata permission

API clients 

Metadata management is also supported by the CLI to enable automation scenarios.

CLI - API

FAQ 

Can I use the description in catalog search?

No currently we only support tags for filtering datasets.

Are the tags instantly available in the catalog?

Lenses updates periodically the catalog cache. By default, it is 30 secs, but you can override it, in the configuration.

Are dataset tags related to the app tags?

Currently there is no correlation between the two. App tags are available to the register apps api.

Are there any limitations for the Description?

Description accepts for 0 to 512 characters long and we do not count whitespace characters. Bear in mind, that while Markdown is a valid input for description, we do not render any markdown elements. We treat it as plain string.

Are there any limitations for the Tags?

Tags are alphanumeric strings, that can be from 1 to 255 characters long. Special characters are allowed. The only limitation is that Tags, cannot have spaces. You can create as many Tags as you want.

More resources