Backup & Restore

Backup & Restore

Lenses seamlessly integrates with AWS S3 using Kafka Connect to facilitate both the creation of topic backups and the restoration of data from them. In this guide, we will walk you through the process of creating backups and restoring your Kafka topics, ensuring data resilience and availability.


Lenses version 5.3 introduces an expanded feature set, empowering users with the ability to perform Kafka topic backups and restorations through AWS S3. This advancement not only facilitates Kafka cluster replication but also enhances data resilience and redundancy, strengthening your data infrastructure.

Backup required permission 

Kafka Connectors / ManageApplicationAllows add/update/delete/stop Kafka Connectors based on your namespace
Kafka Namespace Permission: View Data (QueryTopic)NamespacesAllows viewing the data in a topic

Restore required permission 

Kafka Connectors / ManageApplicationAllows add/update/delete/stop Kafka Connectors based on the topics involved and the namespace permissions
Kafka Namespace Permission: Insert Data (InsertData)NamespacesAllows writing data to the target topic

Additional restrictions 

License limitations apply, the functionality is available as a Lenses addon.

How to create a backup or restore 

To initiate either a topic backup to S3 or topic restoration from S3, follow these steps:

  • Navigate to the Actions menu within the Kafka topic details screen.
  • Choose your desired action: “Backup Topic to S3” or “Restore Topic from S3.”
  • A modal window will open, providing step-by-step guidance to configure your backup or restoration entity.

It’s important to note that a single topic can be backed up or restored to/from multiple locations. You can easily manage these backup and restoration information in the “Summary” section, conveniently positioned on the right-hand side of the topic details screen. Furthermore, you have the flexibility to remove specific backup and restoration information — please be aware that this action does not delete the actual data stored in S3.

Additionally, bear in mind that connectors, once created, may require separate deletion, and you should consider any associated connector-related permissions that might be in effect.

Backup topic Restore topic

Data topology 

Your backup and restore connector instances are seamlessly integrated into Lenses’ topology view. In this view, you can gain valuable insights into the configuration and interconnections of these connectors within your broader streaming pipeline. Furthermore, topics that are undergoing backup, restoration, or both, are clearly labeled for quick and easy identification.

To enhance clarity further, connectors responsible for backup or restoration tasks are distinguished by unique indicators, such as an S3 icon. This makes it easy to differentiate them from other connectors in your topology view. It’s important to note that your access to the topological view is subject to your permissions, ensuring secure and controlled management of these components.

Topology screen

More resources