-
- Technical
Apache Kafka KRaft Abandons the Zoo(Keeper): Part 3 — Maximum Partitions and Conclusions
In this final part of the Kafka KRaft blog series we try to answer the final question (to Kafka, KRaft, Everything!) that has eluded us so far: Is there a limit to the number of partitions for a cluster? How many partitions can we create? And can we reach 1 Million or more partitions?! 1....
Learn MorePaul Brebner
January 26, 2023 -
- Technical
Exploring Karapace—the Open Source Apache Kafka® Schema Registry: Part 1—Apache Avro Introduction with Platonic Solids
Despite Kafka being a schemaless open-source platform, Karapace offers support for schemas. The main scheme type offered by Karapace is Avro so In part 1 we are going to walk through Apache Avro to serialize and deserialize a Platonic Solids example.
Learn MorePaul BrebnerJanuary 18, 2023 -
- News
Apache Kafka® KRaft Abandons the Zoo(Keeper): Part 2—Partitions and Meta-Data Performance
In part 1 of this blog series we explored the background to Kafka® and meta-data management, including the use of ZooKeeperTM and the recent move to KRaft to replace it. We revealed the results of our first experiment with Kafka ZooKeeper vs. KRaft, to measure the impact of increasing partitions on a data workload. Interestingly, there...
Learn MorePaul BrebnerDecember 08, 2022 -
- Technical
Apache Kafka® KRaft Abandons the Zoo(Keeper): Part 1— Partitions and Data Performance
In order to provide a fair comparison of ZooKeeper and KRaft, we decided to start with the managed service using ZooKeeper, and then modify it for KRaft until the general availability of KRaft is released.
Learn MorePaul BrebnerNovember 08, 2022 -
- Technical
Change Data Capture (CDC) With Kafka® Connect and the Debezium PostgreSQL® Source Connector
After learning how to capture change data from an Apache Cassandra® database and produce event streams in Apache Kafka®, But now we need to speed up PostgreSQL® to be as fast as a Kafka.
Learn MorePaul BrebnerAugust 09, 2022 -
- Technical
Spinning Your Drones With Cadence and Apache Kafka®: How Many Drones Can We Fly?
In our Cadence Drone demonstration application blog series we have come up with we have a “New and Improved” Cadence+Kafka integration pattern, and push the boundaries of Cadence scalability to discover how many drones we can fly at once.
Learn MorePaul BrebnerJuly 22, 2022 -
- Technical
Spinning Your Drones With Cadence and Apache Kafka – Integration Patterns and New Cadence Features
Understand how the Drone and Order workflows interact, discover some extra Cadence+Kafka integration patterns, and explore some new Cadence features
Learn MorePaul BrebnerMay 17, 2022 -
- Technical
Spinning Your Drones With Cadence and Apache Kafka® – Architecture, Order and Delivery Workflows
In Part 3 of my Cadence blog series, we introduced the Drone Delivery Demo application, focusing on the Drone Workflow. In this blogpart we’ll look at the Drone Delivery from the perspective of the Order Workflows, understand how the Drone and Order workflows interact, discover some extra Cadence+Kafka integration patterns, and explore some new Cadence features
Learn MorePaul BrebnerApril 28, 2022 -
Spinning Your Drones with Cadence – Introduction
Let's build a new and more complex Cadence application for Drone Deliveries! This is the first part of a multi-part Cadence Drone series, and introduces the Drone Delivery problem, the main Drone Cadence Workflow, and an appendix on Drone movement.
Learn MorePaul BrebnerMarch 22, 2022