NetApp is announcing the general availability of Instaclustr for Apache Kafka and Kafka Connect 4.0.0 on the NetApp Instaclustr Managed Platform.

Kafka 4.0.0, announced by the Kafka project in March 2025, was the first major release in over 3 years and introduces a few significant changes. A full list of inclusions and changes can be found in the release notes.

Some of the significant changes are:

  • KAFKA-17611: This is the first major release of Kafka that works exclusively with KRaft, not requiring Apache ZooKeeper. This significant change comes some 10 years after Kafka started making use of ZooKeeper for metadata management. What this means for our Instaclustr for Kafka customers:
    • New Kafka clusters being created on Kafka 4.0.0 and later versions will by default run in KRaft mode, and there will be no option to use ZooKeeper. This will happen by default; Customers do not need to take any particular action.
    • Existing Kafka clusters on Kafka versions 3.x must first be upgraded to Kafka 3.9.x and can then be migrated from ZooKeeper to KRaft. Once that’s been successfully completed, these clusters can be upgraded to Kafka 4.x and later. Note: KRaft mode clusters on our managed platform are only supported with current Instaclustr API versions (APIv2 and later). If you’re using the deprecated APIv1, a migration to a current API version will need to be undertaken before planning a move to KRaft. Please reach out to our support team to get started working out the details for the Kafka version upgrade and any required migrations.
  • KIP-848: The next-generation consumer rebalance protocol has been released to general availability. This new protocol improves rebalance performance by reducing downtime and latency, especially for large deployments. It is enabled by default on Kafka 4.0 servers, but consumers need to explicitly add the setting group.protocol=consumer.
  • KIP-932: With the early access release of queue support, the Kafka project has added support for share groups, making it more suitable for additional use-cases. For example, ones where messages are independent work items that have non sequential ordering. Share groups allow multiple member consumers to process messages from the same partition and allow more active members than there are partitions, enabling increased parallelism. Instaclustr for Kafka customers wanting to try this feature out are advised to get in touch with our support team, noting this feature is not yet recommended for production use and isn’t yet covered with production SLAs.

There are several other changes we advise customers should evaluate the impacts of in a non-production environment prior to planning a move to Kafka 4.0:

  • Removal of previously deprecated APIs: As in the past, Kafka’s major releases including this one remove APIs that have been deprecated for over 12 months. We advise all Kafka customers wanting to deploy Kafka 4.0 clusters to ensure they’ve removed dependency on any impacted APIs. Some of those removed are as part of: KAFKA-18262, KAFKA-18264, KAFKA-18289, KAFKA-18290, KAFKA-18291, KAFKA-18292, KAFKA-18293, KAFKA-18294, KAFKA-18295, KAFKA-18296, KAFKA-18348, KAFKA-12822, KAFKA-12690, KAFKA-15387, KAFKA-15907, KAFKA-16188, KAFKA-16769. For an exhaustive list, please refer to the Kafka 4.0 release notes.
  • KIP-750, KIP-1013 and KIP-1032: Kafka brokers, tools, Kafka Connect and dependent modules (like MirrorMaker 2) now require Java 17, having dropped support for older Java versions. But this change does not impact the remaining modules (for example, Kafka clients, Kafka streams), which will continue to support Java 11.
  • KIP-896: To reduce the cost of maintaining support for older protocol API versions, this release updates the minimum supported broker and client version. The new baseline for protocol API versions is Kafka 2.1. Customers should ensure brokers are at least version 2.1 before upgrading clients to 4.0. They should also ensure that their Java client version is at least 2.1 before upgrading brokers to 4.0. More details on upgrading Kafka clients are published in KIP-1124. Alternatively, our customers can open a support ticket to discuss this further with our Kafka experts.
  • KIP-724: Writes with message formats v0 or v1 in Kafka 4.0 is no longer supported.
  • KIP-1030: Some configs’ default values or constraints have been changed. Please refer to the KIP for all the changes, keeping in mind that some have already been changed in Kafka 4.0, while others are planned for changes with Kafka 5.0. Where the defaults do not suit, please let us know, and we can change them to something more appropriate for your workload.
  • For other deprecations and changes, please review, respectively:

With this new release, we have reviewed and updated the lifecycle states for our older supported Kafka versions (as per our lifecycle policy). To ensure you get the full benefit of our support and SLAs, please reach out to us and request an upgrade to a GA version. To stay on a supported version of Kafka, we advise customers upgrade their Kafka versions at least once a year to the latest GA version. More details on all Kafka versions supported on our managed platform are available here.

Prior to deploying a new Kafka version in production, we recommend trying your preferred Kafka version in a non-production environment to confirm compatibility with your Kafka clients. If you need any help spinning up a new Kafka cluster or upgrading existing managed Kafka clusters, please feel free to get in touch with us via our support website.