What is KRaft?

KRaft is short for “Kafka Raft”, with Raft referring to a consensus algorithm. KRaft offers a replacement for the use of ZooKeeper for Kafka’s metadata storage and leader elections. Instead of requiring running a ZooKeeper ensemble alongside your Kafka cluster, Kafka nodes can be configured as KRaft Controllers which manage the cluster’s metadata and leader elections. When using KRaft mode, nodes can be configured as Kafka Brokers, KRaft Controllers, or both.

KRaft is generally available on the Instaclustr managed platform for Kafka 3.6.1 and will be enabled by default for all newly created clusters using Kafka 3.6.1. ZooKeeper has been officially deprecated by the Kafka project since Kafka 3.5.0 but will still be available on the Instaclustr managed platform with Kafka 3.6.1 if explicitly chosen at cluster creation time. The Kafka project intends to remove support for ZooKeeper entirely in Kafka version 4.0 (https://kafka.apache.org/documentation/#zk_depr).

Why use KRaft instead of ZooKeeper?

ZooKeeper has been deprecated by the Kafka project since version 3.5.0, and support for ZooKeeper is expected to be removed completely in Kafka 4.0.

KRaft also brings with it performance improvements over the use of ZooKeeper, allowing for the same hardware running Kafka to support more partitions and with lower request latencies than is currently possible with ZooKeeper. Benchmarks performed by Instaclustr comparing KRaft and ZooKeeper showed the 90th percentile message latency (producer and consumer) was lower by 3-18% depending on cluster configuration with KRaft. These latency improvements were present for both collocated and dedicated KRaft, with collocated seeing slightly greater improvement compared to dedicated.

Considerations

Please be aware of the following considerations when provisioning clusters with KRaft.

• Apache Kafka is not capable of using both KRaft and ZooKeeper at the same time. This restriction is imposed by the Kafka project.
• KRaft is compatible with the Karapace Schema Registry and Karapace REST proxy add-ons, and not Kafka Schema Registry and Kafka REST Proxy.