# Updated Apache Kafka® Benchmarking

[Blog](/blog/)&gt;[Technology](/blog/category/technical/)&gt;Updated Apache Kafka® Benchmarking 

Updated Apache Kafka® Benchmarking
==================================

January 14, 2019 | By [ Ben Slater](https://www.instaclustr.com/blog/author/bens/)

 

 

 

 



   [ ](https://x.com/intent/tweet?text=Updated%20Apache%20Kafka%C2%AE%20Benchmarking&url=https://www.instaclustr.com/blog/updated-apache-kafka-benchmarking/) [ ](https://www.linkedin.com/shareArticle?mini=true&url=https://www.instaclustr.com/blog/updated-apache-kafka-benchmarking/&title=&summary=Updated%20Apache%20Kafka%C2%AE%20Benchmarking&source=) 

Late last year we published our first round of benchmarking for Apache Kafka running on our managed service. At the time, we flagged an issue that the approach we adopted did not balance reads and writes to a cluster. We’ve now completed another round of benchmarking with a balanced read and write load and Kafka 2.0 and 2.1.

The approach we followed was similar to our previous benchmarking with a change to step 4 to ensure a balanced load:

1. Provisioned the required Apache Kafka cluster using the [Instaclustr Managed Platform](https://www.instaclustr.com/platform/). All clusters had TLS enabled for broker to broker and client to broker communication and used Kafka 1.1.0.
2. Create a test topic with RF=3 and partitions equal to the number of nodes in the cluster.
3. No compression was used and batch.size was the default (16384)
4. We used the [rdkafka\_performance tool](https://github.com/edenhill/librdkafka/blob/master/examples/rdkafka_performance.c) to generate load with 512 byte messages and ran the load for 1 hour.
5. The number of stress clients was determined by increasing the number of clients machines in use until maximum throughput was reached (the number of clients therefore varied depending on the cluster under test and readers and writers were adjusted to produce a close to equal number of reads and writes for the test).

Following this approach we achieved the following results:

 ![](https://www.instaclustr.com/wp-content/uploads/2021/10/yJfswZ8K4D7udS5VjKRh-8jOi4rBJYP32m2BsKB9GyHP9ULP5AulWLWPgsjW8FamH8-iZ5UkbYfClp2K8UpxN-gf94ii-fFdtvDmV-KFtrlTBQIQ7rAQcsUdxsSqemCfZNfh6TIi.png)Overall, the 3 node clusters processed just over 800k messages / second for the r4.xlarge and just over 1 million messages/sec for the r4.2xlarge. There are a couple of interesting observations here:

- Performance was roughly equal between 2.0 and 2.1 although there was a very small degradation in both tests.
- The r4.2xlarge produce a relatively small increase in throughput for hardware costing 2 times the cost of the r4.xlarge. This was a lower scale-up than experienced with write-only tests (which did approximately double). We will do some more investigation on this point.

As always, these benchmarks can provide an initial estimate for sizing but testing should be undertaken for your particular application scenario. If you have any question regarding this benchmarks then please feel free to drop them in the comments section below or email <info@instaclustr.com>.

 

### About the author

**[Ben Slater](https://www.instaclustr.com/blog/author/bens/)** | Chief Product Officer

Chief Product Officer, charged with steering Instaclustr’s development roadmap and overseeing the product engineering, production support, open source, and consulting teams.

 

 [ Add Instaclustr as a preferred source on Google ](https://google.com/preferences/source?q=instaclustr.com)



 

 ![mail icon]()#### Get the latest articles for open sourceIn your inbox

 <a class="btn btn-primary btn-popup text-dark" href="">Sign up now</a> 

 

 

 

  ### Related content

 [ Zero Downtime Migration to Instaclustr 

 

 Yes, we can migrate existing Cassandra clusters to Instaclustr without any downtime. Here's what to expect from the process... 

 

 

 

 

 

 

 ](https://www.instaclustr.com/blog/zero-downtime-migration-to-instaclustr/) 

 [ Workflow Comparison: Uber Cadence vs Netflix Conductor 

 

 When choosing what’s right for your company’s opensource workflow needs it is important to know the difference and similarities ... 

 

 

 

 

 

 

 ](https://www.instaclustr.com/blog/workflow-comparison-uber-cadence-vs-netflix-conductor/) 

 [ Will Your Cassandra Database Project Succeed?: The New Stack 

 

 Open source Apache Cassandra® continues to stand out as an enterprise-proven solution for organizations seeking high availability... 

 

 

 

 

 

 

 ](https://www.instaclustr.com/blog/will-your-cassandra-database-project-succeed-the-new-stack/) 

 

  <a class="close-modal" href="">×</a>Sign upto ourNewsletter
-----------------------
