Feature Releases Technical — Kafka Monday 12th October 2020

Dedicated ZooKeeper for Apache Kafka

By Paul Aubrey

Instaclustr is pleased to announce the release of dedicated ZooKeeper nodes as an additional optional feature of our managed Apache Kafka offering. ZooKeeper is used for the management and coordination of nodes in Kafka.

Dedicated ZooKeeper nodes provide customer benefits in situations that require high performance and reliability:

  • Improvements to reliability in failure scenarios (primarily due to less chance of ZooKeeper nodes and Kafka brokers failing simultaneously)
  • ZooKeeper nodes can be scaled independently of Kafka
  • Reduction in I/O conflict between ZooKeeper and Kafka processes.

Key features of the dedicated open source ZooKeeper solution include:

  • Only available for Kafka version 2.5.1 and above
  • Provides customers the option to select dedicated ZooKeeper nodes in developer and production ready node sizes. The support page for this new feature outlines this process
  • Available on AWS, GCP, and Azure cloud service providers
  • Supports provisioning via the Instaclustr Console, Provisioning API, or via the Instaclustr Terraform provider
  • Access to ZooKeeper specific monitoring metrics via our API or console
  • Continuous monitoring and alerting
  • 24×7 comprehensive support from our Technical Operations team

“In the event of a node failure this architecture is great as we’re not faced with the potential loss of a Kafka Broker and ZooKeeper node at the same time. It’s far more robust, reducing recovery time, and reducing the risk of edge case issues with cluster state inconsistencies. More generally, having dedicated ZooKeeper eliminates contention for resources such as IO, allowing ZooKeeper to remain performant even as load increases on Kafka, reducing bottlenecks and latency.”—Joe Clay, Instaclustr Technical Operations.

With the introduction of dedicated ZooKeeper nodes we have revised our SLAs. Deploying dedicated ZooKeeper moves Enterprise tier availability SLA from 99.99% to 99.999%. Instaclustr continues to offer industry leading availability SLAs. Full details can be found on our SLA Page.

For existing customers interested in migrating to Dedicated ZooKeeper nodes, please contact our Support team to discuss your requirements.
If you want to try our Kafka, login to our Management Console and create an Instaclustr managed Kafka cluster. If you haven’t created a Kafka cluster on our platform, create a test Kafka cluster and use the 30-day free trial. If you’d also like a free trial of a Kafka Connect cluster please contact our Sales team.