-
- Popular
- Technical
Anomalia Machina 10: Final Results—Massively Scalable Anomaly Detection with Apache Kafka® and Apache Cassandra®
In this tenth and final blog of the Anomalia Machina series we tune the anomaly detection system and succeed in scaling the application out from 3 to 48 Cassandra nodes, resulting in some impressive numbers: 574 CPU cores (across Cassandra, Kafka, and Kubernetes clusters), 2.3 million writes/s into Kafka (peak), 220,000 anomaly checks per second...
Learn MorePaul BrebnerApril 30, 2019 -
- Popular
- Technical
Anomalia Machina 9: Anomaly Detection at Scale—Massively Scalable Anomaly Detection with Apache Kafka® and Apache Cassandra®
In the previous blog, we deployed the Anomalia Machina application in a production-like Kubernetes environment. In this blog, we test it out and see many anomalies it can detect at scale on small Kafka and Cassandra Instaclustr production clusters. Coming towards the end of our epic odyssey we now have a final challenge to overcome....
Learn MorePaul BrebnerMarch 13, 2019 -
- Popular
- Technical
Anomalia Machina 8: Production Application Deployment With Kubernetes
In the previous blog we explored deploying the Anomalia Machina application on Kubernetes with the help of AWS EKS. In the recent blogs (Anomalia Machina 5 and Anomalia Machina 6), we enhanced the observability of the Anomalia Machina Application using two Open Source technologies: Prometheus for distributed monitoring of metrics such as throughput and latency;...
Learn MorePaul BrebnerMarch 05, 2019 -
- Popular
- Technical
Anomalia Machina 7: Kubernetes Cluster Creation and Application Deployment
Kubernetes—Greek: κυβερνήτης = Helmsman If you are Greek hero about to embark on an epic aquatic quest (encountering one-eyed rock-throwing monsters, unpleasant weather, a detour to the underworld, tempting sirens, angry gods, etc.) then having a trusty helmsman is mandatory (even though the helmsman survived the Cyclops, like all of Odysseus’s companions, he eventually came...
Learn MorePaul BrebnerFebruary 11, 2019 -
- Popular
- Technical
Anomalia Machina 6: Application Tracing with OpenTracing—Massively Scalable Anomaly Detection with Apache Kafka® and Apache Cassandra®
In the previous blog (Anomalia Machina 5 – Application Monitoring with Prometheus) we explored how to better understand an Open Source system using Prometheus for distributed metrics monitoring. In this blog we have a look at another way of increasing visibility into a system using OpenTracing for distributed tracing. 1. A History of Tracing Over...
Learn MorePaul BrebnerJanuary 15, 2019 -
- Popular
- Technical
Anomalia Machina 5: Application Monitoring with Prometheus—Massively Scalable Anomaly Detection with Apache Kafka® and Apache Cassandra®
1. Introduction In order to scale Anomalia Machina we plan to run the application (load generator and detector pipeline) on multiple EC2 instances. We are working on using Kubernetes (AWS EKS) to automate this, and progress so far is described in this webinar. However, before we can easily run a Kubernetes deployed application at scale...
Learn MorePaul BrebnerDecember 19, 2018 -
- Popular
- Technical
Anomalia Machina 4: Prototype—Massively Scalable Anomaly Detection with Apache Kafka® and Apache Cassandra®
“Nothing is so painful to the human mind as a great and sudden change.” —Mary Wollstonecraft Shelley, Frankenstein 1. Initial Prototype “It was on a dreary night of November that I beheld the accomplishment of my toils. With an anxiety that almost amounted to agony, I collected the instruments of life around me, that I...
Learn MorePaul BrebnerOctober 29, 2018 -
- Popular
Anomalia Machina 1: Massively Scalable Anomaly Detection With Apache Kafka® and Apache Cassandra®
anomalia—Latin (1) irregularity, anomaly machina—Latin (1) machine, tool, (2) scheme, plan, machination What do you get if you combine Anomalia and Machina? Machine Anomaly—A broken machine (Machina Anomalia) Irregular Machinations—Too political (Anomalia Machina, 2nd definition) Anomaly Machine! (Anomalia Machina, 1st definition) Let’s Build the Anomalia Machina! A Steampunk Anomalia Machina—possibly, I actually have no clue...
Learn MorePaul BrebnerSeptember 28, 2018 -
- Technical
Apache Kafka “Kongo” 6.3: Production Apache Kafka® Application Scaling on Instaclustr
The goal of this blog is to scale the Kongo IoT application on Production Instaclustr Kafka clusters. We’ll compare various approaches including scale-out, scale-up, and multiple clusters. There are two versions to the story. In the Blue Pill version scaling everything goes according to plan and scaling is easy. If you are interested in the...
Learn MorePaul BrebnerAugust 02, 2018