Kubernetes is one of the emerging platforms that enables companies to run and manage containerized applications globally. Apache Kafka is a very popular choice for messaging and event streaming in modern Cloud Native applications. Kafka is traditionally well suited to run on statically defined clusters but running it on container orchestrated clusters like Kubernetes is becoming more common. While Kubernetes is a great choice for running stateless applications, however running stateful applications such as Kafka on Kubernetes can have its own challenges.
This talk will review different approaches for running stateful applications like Kafka on Kubernetes. We will then review the popular approach of deploying Kafka on Kubernetes using operators. In the second half, we will run a demo on how to deploy a production-ready Kafka cluster in Kubernetes. If time permits, we will also deploy Prometheus in Kubernetes to monitor our cluster.
By the end of the talk we are confident that you will have enough knowledge to deploy Kafka on Kubernetes after this Meetup.
Mussa Shirazi has a vast experience with open source technologies such as Elasticsearch and Kafka and has been involved in designing and deployment of data solutions for big financial organisations. Mussa is a big fan of open source technologies and likes getting involved in all kinds of data projects that involve open source technologies.