Lenses seamlessly integrates with AWS S3 using Kafka Connect to facilitate both the creation of topic backups and the restoration of data from them. In this guide, we will walk you through the process of creating backups and restoring your Kafka topics, ensuring data resilience and availability.
Lenses version 5.3 introduces an expanded feature set, empowering users with the ability to perform Kafka topic backups and restorations through AWS S3. This advancement not only facilitates Kafka cluster replication but also enhances data resilience and redundancy, strengthening your data infrastructure.
QueryTopic
InsertData
License limitations apply, the functionality is available as a Lenses addon.
To initiate either a topic backup to S3 or topic restoration from S3, follow these steps:
It’s important to note that a single topic can be backed up or restored to/from multiple locations. You can easily manage these backup and restoration information in the “Summary” section, conveniently positioned on the right-hand side of the topic details screen. Furthermore, you have the flexibility to remove specific backup and restoration information — please be aware that this action does not delete the actual data stored in S3.
Additionally, bear in mind that connectors, once created, may require separate deletion, and you should consider any associated connector-related permissions that might be in effect.
Your backup and restore connector instances are seamlessly integrated into Lenses’ topology view. In this view, you can gain valuable insights into the configuration and interconnections of these connectors within your broader streaming pipeline. Furthermore, topics that are undergoing backup, restoration, or both, are clearly labeled for quick and easy identification.
To enhance clarity further, connectors responsible for backup or restoration tasks are distinguished by unique indicators, such as an S3 icon. This makes it easy to differentiate them from other connectors in your topology view. It’s important to note that your access to the topological view is subject to your permissions, ensuring secure and controlled management of these components.
On this page
