Cluster Config Backup
Kafka is a highly distributed system and can be configured to provide a high level of resilience on its own. Kafka’s replication factor can help survive the loss of several brokers and when it is stretched across multiple datacenters, we can survive the loss of data centers as well.
In addition, we backup important Kafka configurations so that it can be restored to a new Kafka cluster in the case of total Zookeeper failure. The backup will be a tar file of the Zookeeper snapshot and transaction directories which is all that is required to restore a Zookeeper ensemble. We backup the entire Zookeeper data directory which contains topics, their configuration, users, acls and passwords. The backup runs every 24 hours and stores it on Amazon S3 storage.
If and when you want to restore the configuration to a new Kafka cluster, please send an email to our support team.