Instaclustr is pleased to announce the immediate availability of Instaclustr for Apache Kafka® and Apache Kafka® Connect 3.4.1 on our managed platform. 

Kafka 3.4.1 follows the release pattern we introduced with Kafka 3.3.1: it is generally available when used in ZooKeeper™ mode, but as a public preview when used in KRaft mode. The Kafka project here provides an overview of the main new features with Kafka 3.4. 

Some features that we feel are important to highlight to our customers are: 

• ZooKeeper to KRaft migration support has been released as an Early Access feature. KIP-866 explains the overall feature implementation planned. Of note is that since this feature hasn’t been fully implemented, it is not recommended for production use yet. 
• Rack aware partition assignment support is now included and allows consumers using this feature to be able to avoid costly cross-AZ traffic by consuming from the local replica. Read more about it on KIP-881. 
• ZooKeeper upgraded to version 3.6.4. 

More details on our managed Kafka 3.4.1 release: 

• Clusters being upgraded from Kafka versions older than Kafka 3.4.1 will be upgraded to Kafka 3.4.1 with ZooKeeper mode enabled. Note that clusters running the Public Preview version of Kafka 3.3.1 with KRaft mode cannot be upgraded. 
• For new Kafka clusters being spun up with Kafka 3.4.1, at time of cluster creation you will have the option of selecting whether you’d like the cluster to be created with ZooKeeper mode or preview KRaft mode. If you don’t actively make a choice, ZooKeeper mode will be the default for a newly created cluster. 

Kafka 3.4.1 with KRaft mode is available only as a Public Preview due to current shortcomings, most notably a lack of full support for a mechanism for migrating ZooKeeper clusters to KRaft, and a lack of support for SCRAM. Support for the former is expected to be present in Kafka 3.6, while for the latter it has already been released in Kafka 3.5 (and will be supported by Instaclustr in the next 3 months with the next Kafka version update on our managed platform). 

With the release of Kafka 3.3.1 on our managed platform, we will be updating the lifecycle states for our older supported Kafka versions to more restrictive lifecycle states as per our lifecycle policy shown in the table below. In summary, the typical lifecycle for Kafka versions is GA -> Closed -> Legacy Support (if needed) -> EOL.  

We encourage customers on any non-GA Kafka versions to get in touch with us via our support portal and schedule a version upgrade. 

* Kafka versions which are currently in more restricted states than GA or closed are not listed. Dates when they will be transitioning to EOL are communicated directly to the few customers still using them. 

We’re also excited to announce that alongside the new Kafka release, we have made Karapace version 3.6.2 available on our platform. Between versions 3.4.3 and 3.6.2, numerous fixes, performance, and feature improvements have been introduced. Some of the notable changes we’d like to specifically mention are: 

• Support for schema references has now been added for Protobuf schemas. See #560 for more specifics. Simply stated, schema references allow schemas to reference other schemas, thereby ensuring schemas being referenced are retrieved by the Schema Registry when needed. 
• Support for Python 3.7 was dropped, as part of change #569 given the EOL status of Python 3.7 w.e.f. end June 2023. As part of this change, the default Python version has now been changed to Python 3.8. 

With the release of Karapace 3.6.2, we will be updating the lifecycle state of Karapace 3.4.3 to Closed on August 1, 2023. 

We encourage you to try your preferred Kafka version in your non-production environments first to confirm compatibility with your Kafka clients before using it in production environments.  

If you need any help spinning up new Kafka cluster, or upgrading existing managed Kafka clusters, please feel free to get in touch with us via our support website.