illustration of network nodes connected by light

An AI visualisation of Agent Systems: Enterprise control room showing agent systems coordinating via Agent2Agent across a Kafka event backbone in distributed systems.

What does it take to scale agent systems with Apache Kafka and Agent2Agent (A2A)? That’s the question I’ll be chasing across this new series, and this is part 1.

Agent systems are networks of AI-powered agents that can reason, plan, call tools, retrieve data, and coordinate with other agents or systems to complete complex tasks. As they scale, agent systems start to look less like standalone AI applications and more like distributed systems, requiring orchestration, observability, guardrails, and reliable communication patterns.

Why does this matter? Because as AI systems move from prototypes to production, teams run into the same challenges we’ve seen for years in distributed systems: scaling, coordination, fault handling, and governance. Agent systems are where those worlds collide. And that’s where platforms like Kafka start to become essential — not only for “doing AI”, but for making AI systems work reliably at scale

But first, a confession: there’s almost no Kafka or A2A in this post. Before we get to the “meat”, I want to map the background to agent systems, where they came from, what makes them tick, and why they look so much like the distributed systems some of us have been building for years. Think of this as setting the table. The Kafka and A2A feast come later.

Is Kafka useful for AI agent systems?

At an online Kafka event last year, a panelist gave that answer (roughly) when someone asked how relevant Apache Kafka really is to AI. It didn’t spark much controversy at the time. But it stuck with me. And the more I thought about the Kafka-and-AI question, the more it pointed me towards two observations.

  1. Streaming AI, meaning real-time AI. You use AI alongside streaming data and Kafka to build real-time, AI-enhanced systems.
  2. Scaling AI, meaning enterprise distributed AI systems. You use Kafka to help build scalable AI systems, also known as agent systems or agentic systems.

Both are valid perspectives. They’re also subtly different, and the difference matters more than it first appears. So, let’s take them in turn.

What is streaming AI?

Streaming AI applies AI to data in motion. Instead of waiting for a batch to land, the AI learns, checks, predicts, and acts as events flow through. Why does that matter? Because the moment you stop waiting for batches, you stop reacting late.

There are plenty of precedents here, including several of my own Instaclustr projects and blogs.

Taken together, these projects provide strong precedents that AI, whether old-school or new-school (LLMs included), works with streaming data to build scalable, real-time, AI-enhanced applications. That’s the streaming AI perspective.

Here, AI does the hard work. It learns from streaming data, checks the streams, produces predictions and actionable insights, and even repairs malformed data, all invisibly. But streaming AI is only half the story, so let’s chase the other half.

What do scalable AI systems actually need?

Let’s go back to the original quote, “Kafka is the last thing you need for AI”, and run with it for a moment. Accept it, and an obvious question follows.

What do you actually need for AI? First, second, third, and so on?

By AI here, I mean the second perspective above: scalable AI systems. So where do we start?

First, you need AI

The first thing you need for AI systems is, yes, you guessed it, AI.

AI has been around in some form since at least the “old-school” days (mine, and earlier). But to keep up with the moment, let’s assume we mean the large language models (LLMs) that have appeared over the past few years.

There’s more to AI than LLMs, of course. For now, though, we’ll focus on them and tick the box. So, what’s next on the list?

Second, you need agents

Agents? What are they?

This is where things get more interesting, and more complex. Like AI, agents have been around for decades. I remember learning about Carl Hewitt’s Actor Model in the early 1980s. Actors were more or less agents, but minus the AI: independent entities that processed messages and ran concurrently.

Fast-forward to today, and agents usually mean LLM-powered systems that autonomously carry out tasks on behalf of users or other systems. Give one instructions and context, and it can:

  • plan
  • reason
  • execute multi-step actions
  • call tools and retrieve data
  • produce results and take actions
  • retain a memory of past interactions

But a single agent rarely works alone, which brings us to the third thing.

Third, you need agent systems

This definition of agents hints at the third thing AI needs: systems of agents.

Agents don’t exist on their own. They’re part of (and can themselves be) orchestrated systems that communicate and collaborate to solve increasingly complex problems. So why bother with more than one agent at all?

Why use a multi-agent architecture?

There are plenty of reasons to reach for a multi-agent architecture.

  1. LLMs aren’t equal. Different providers offer models with different strengths, weaknesses, and costs. Some are even open source, or at least open weights.
  2. Privacy and security constraints matter. Some LLMs won’t be usable at all in certain environments.
  3. Deployment characteristics vary. Different LLMs run in different locations with different latency, performance, and scalability profiles. Some can run locally.
  4. Specialisation matters. Some agents can be specialised through training, domain-specific data, or access to private data via RAG, vector search, and similar techniques.
  5. Concurrency matters. Running several agents at once, on independent or scalable infrastructure, gives you real concurrency.
  6. Supervision matters. Some agents can be “hardcoded” to handle quality control or supervisory roles.
  7. Accuracy and governance matter. Some agents can be reinforced with code and structured data to improve accuracy, reduce hallucinations, and prevent violations of physical or business rules.
  8. Context management matters. Multiple agents let you tackle smaller, focused problems with only the relevant context, which helps with context overload and “context rot”.

That list probably isn’t exhaustive. More broadly, the motivation for agent systems traces back to distributed systems in general, including:

  • conquering complexity through decentralization
  • improving scalability and performance
  • fault tolerance and resilience
  • modularity and flexibility through encapsulation
  • security and privacy
  • architectures built to handle geographical separation, latency, asynchronous communication, and failure

All of that adds up to robust global systems. These were the premises behind the early internet, and plenty more besides, landing with modern cloud platforms to run distributed systems reliably at scale. But, if agent systems lean so heavily on distributed systems, what’s genuinely new about them?

What makes agent systems different from traditional distributed systems?

Honestly, I suspect agent systems share more with distributed systems than they differ from them. But how do you find out for sure?

I recently heard about the O’Reilly Agent Orchestration Superstream and decided to attend, to get up to speed on agent systems in general. Many of the talks emphasized the commonalities with distributed systems. A few differences stood out.

  • A shift from deterministic to probabilistic systems. Though I think this is sometimes overstated. Earlier distributed systems already had non-deterministic components, including ML and simulation workloads.
  • Run-time behaviour and checks or bounds are critical. This isn’t entirely new, but it matters more as agent-based systems become harder to test meaningfully before production. Real data and real usage tend to change how the system behaves. LLMs also bring different failure modes, including arbitration issues and violation detection. Errors compound, too. Drift accumulates, failures cascade, and amplification effects can turn catastrophic. Errors inevitably propagate.
  • That isn’t a new idea. Years ago, I experimented with feeding model outputs back into model inputs, measuring how many iterations held up before the signal-to-noise ratio degraded past a threshold. About three iterations was my best result for accurate software performance modeling scenarios. (Three. That’s it. Surprising, but the noise builds fast.) And the most recent article from Professor Sir Anthony Finkelstein is a timely reminder, which points out that real systems are non-linear, and disturbances are often amplified!
  • Guardrails are essential. Some agents exist mainly to check other agents and block harmful or invalid actions. They needn’t be LLM-based at all. Circuit breakers and observability matter just as much.
  • Orchestration and architectural flexibility are essential. Architectures won’t stay limited to simple pipelines or hierarchies. They can grow arbitrarily complex and dynamic, support agent interplay, and even become autonomous and self-correcting, while still allowing governance, self-improvement, and termination from outside the system itself (whether by agents or the orchestration framework).

I’ve written some of this up already elsewhere. But knowing what’s peculiar is one thing. How do you get agents to actually work together?

How do AI agents communicate and coordinate?

So how do you get agents to work with other systems, or orchestrate several at once?

Some of the streaming AI examples above sit right on the edge of agent orchestration. In the Kafka Streams DLQ repair example, the agents are essentially Java Kafka clients with specific prompts, the ability to read Kafka records, and the ability to call local LLMs. The orchestration is deterministic, driven by the type of error that needs fixing. Why deterministic? Because the error type tells you exactly which repair to run, so there’s no need to guess.

But what happens when orchestration becomes more complex? There are more and more agent frameworks to choose from for orchestration, usually language-specific.

Here are some agent toolkits and frameworks I’m familiar with:

The last talk in the Superstream explored an open-source framework I hadn’t met before: the NeMo Agent Toolkit from NVIDIA, which looks promising. It supports multiple agent patterns (sequential, parallel, loops, hierarchies, and more), along with telemetry, observability, and A2A support.

And that gives us a nice segue into Agent2Agent itself, which is where we’ve been heading all along. So next, let’s see what A2A really offers. Look out for the next blog in this series! There will be some A2A – I promise!

So what’s next?

So yes, there will be A2A next. But there’s another piece of the agent-system puzzle worth exploring now: how agents safely and reliably get to the data and tools they need.

That’s the job of Instaclustr for MCP Gateway. It connects AI agents to trusted open source data infrastructure through the Model Context Protocol, helping turn clever agent experiments into something closer to production-ready systems.

If you’re building with agent systems, streaming AI, Kafka, or Agent2Agent, take a look at Instaclustr for MCP Gateway and start giving your agents the governed, scalable data access they need.