# Will Your Cassandra Database Project Succeed?: The New Stack [Blog](/blog/)>[Technology](/blog/category/technical/)>Scaling agent systems with Apache Kafka and A2A (Part 7): An experimental Kafka transport and the Semaphore Agent Scaling agent systems with Apache Kafka and A2A (Part 7): An experimental Kafka transport and the Semaphore Agent ================================================================================================================= October 29, 2026 | By [ Paul Brebner](https://www.instaclustr.com/blog/author/paul-brebner/) [ ](https://x.com/intent/tweet?text=Scaling%20agent%20systems%20with%20Apache%20Kafka%20and%20A2A%20(Part%207):%20An%20experimental%20Kafka%20transport%20and%20the%20Semaphore%20Agent&url=https://www.instaclustr.com/blog/scaling-agent-systems-with-apache-kafka-and-a2a-part-7-an-experimental-kafka-transport-and-the-semaphore-agent/) [ ](https://www.linkedin.com/shareArticle?mini=true&url=https://www.instaclustr.com/blog/scaling-agent-systems-with-apache-kafka-and-a2a-part-7-an-experimental-kafka-transport-and-the-semaphore-agent/&title=&summary=Scaling%20agent%20systems%20with%20Apache%20Kafka%20and%20A2A%20(Part%207):%20An%20experimental%20Kafka%20transport%20and%20the%20Semaphore%20Agent&source=) What changes when agent requests and task updates travel through Kafka topics instead of HTTP connections? In this installment of our series on scaling agent systems, we arrive at one of the planned destinations on our A2A + Kafka “campaign” map – an experimental Kafka A2A transport. We still use the same A2A semantics but transport them over Kafka. In the Napoleonic era, this would have been like replacing dispatch riders with the faster semaphore telegraph system introduced in France during the 1790s. The system used tall poles on towers with rotating wooden semaphores. Operators in nearby towers could retransmit the signals along the line of towers, enabling messages to be transmitted hundreds of kilometres per hour. The French semaphore system shares some interesting conceptual similarities with Apache Kafka, moving large volumes of information reliably across a geographically distributed network; multiple networks extending 100’s of km over the whole of France, and beyond. Relay towers functioned as brokers, producers and consumers, and operators reacted quickly to changes of state in the observed semaphores, relaying them to the next tower and so on. They also shared a major design feature with Kafka – the operators couldn’t understand the messages – only the people sending (encoding) and receiving (decoding) the messages could understand them using code books. *Kafka brokers also don’t understand messages – only the producers and consumers do.* ![Chappe’s telegraph photo]() Chappe’s telegraph (Public domain, [https://commons.wikimedia.org/wiki/File:Chappe\_semaphore.jpg](https://commons.wikimedia.org/wiki/File:Chappe_semaphore.jpg)) **About this series:** This is Part 7 of *Scaling agent systems with Apache* *Kafka and A2A*. New to the topic? Start with [Part 1: The evolution of agent systems](https://www.instaclustr.com/blog/scaling-agent-systems-with-apache-kafka-and-agent2agent-part-1-the-evolution-of-agent-systems/). For the Java examples this experiment builds on, see [Part 5: The Clockwork Agent](https://www.instaclustr.com/blog/scaling-agent-systems-with-apache-kafka-and-a2a-part-5-the-clockwork-agent-a-java-a2a-example/) and [Part 6: The Quartz Chronometer Agent.](https://www.instaclustr.com/blog/scaling-agent-systems-with-apache-kafka-and-a2a-part-6-the-quartz-chronometer-agent-an-example-using-the-default-a2a-protocols/) Why investigate A2A over Kafka? ------------------------------- The primary motivation for this experiment was to explore whether A2A could be implemented on top of Apache Kafka rather than its default transports of JSON-RPC over HTTP, SSE, and webhooks. Kafka is a widely used platform for integration, event streaming, and large-scale distributed systems. Given its scalability, durability, fan-out capabilities, and replayable event log, it seems like a natural candidate for scaling Agent2Agent systems. Does that mean standard A2A doesn’t scale? ------------------------------------------ No. The default A2A HTTP binding, based on JSON-RPC and Agent Card discovery, scales well for traditional point-to-point interactions between clients and agents. A client sends a request, receives either an immediate response or a long-running task, and tracks progress through polling, SSE, or webhooks. The challenges begin to appear when systems grow beyond simple client-agent communication. Polling generates repeated requests even when nothing has changed. SSE requires long-lived connections for every observer. Webhooks replace inbound traffic with outbound traffic and introduce delivery, retry, and idempotency concerns. At the platform level, A2A is designed as a point-to-point protocol rather than a distributed event fabric. Agent task state is often held in-process, meaning horizontally scaled agent pools may require sticky routing or shared storage. Likewise, task lifecycle events are not automatically broadcast to multiple consumers for auditing, monitoring, analytics, or orchestration. As workloads increase, with many agents, many concurrent long-running tasks, and many observers, additional infrastructure may be needed to provide fan-out, replay, durability, and centralized visibility. These capabilities sit outside the A2A protocol itself. In short, the default A2A protocols scale normal client-agent interactions very well. However, they do not automatically provide large-scale event distribution, replay, auditing, or multi-observer task telemetry. Those concerns require supporting architecture beyond polling, streaming, and webhooks. Different implementation choices introduce different scaling challenges: **Polling (GetTask):** - Generates extra RPCs even when task state is unchanged - May return the same state multiple times - Can become expensive with many clients and many tasks **SSE (Streaming)** - Requires one long-lived connection per observer - Works well for individual clients but is less efficient for large numbers of subscribers - Replay and reconnect behaviour depend on the agent implementation rather than a durable event log **Push (Webhooks)** - Requires the agent to POST updates to every registered endpoint - Often still requires a follow-up GetTask request to retrieve full state - Retry, delivery guarantees, and idempotency must be implemented separately **Running Many Agent Instances** - Task state frequently resides within a single process - Load balancing can be complicated without sticky sessions or shared storage - Requests related to the same task may reach different agent instances **Fan-Out and Orchestration** - No built-in one-to-many event distribution - Audit pipelines, dashboards, and orchestration systems typically require additional infrastructure - Replay capability is not provided by the protocol itself **The bottom line:** Kafka is a candidate for that supporting event infrastructure because it provides a retained log and consumer groups which means A2A scales point-to-point agent communication effectively. The question is not whether Kafka replaces A2A, but how an event-streaming architecture might carry selected A2A interactions. *A2A is optimized for point-to-point communication. Kafka potentially complements it by adding durable event storage, replay, fan-out, and other distributed-systems capabilities that become increasingly valuable at scale.* How does the experimental A2A over Kafka design work? ----------------------------------------------------- For the new A2A Kafka design, we need two components: 1. A Kafka implementation of the A2A patterns 2. New A2A Kafka clients Let’s see what this looks like in more detail, starting with some diagrams. First up, here’s a conceptual diagram showing how standard A2A modes were implemented in Kafka: ![conceptual diagram showing how standard A2A modes are implemented in Kafka]() This diagram shows how three standard A2A interaction shapes map onto the same Kafka transport, using one Client, one broker, and one agent throughout. They are comparison blocks, not concurrent flows; just pick the one required pattern per task. Read top to bottom through the three shaded sections to compare event count, timing, and delivery semantics on identical infrastructure. All three start the same way: the client produces message/send on a topic `a2a.requests`, the agent consumes the request, and replies go to topic `a2a.updates`. - **Pattern 1** (request/response) is the short path — artifact, COMPLETED, task snapshot, then the client consumes until terminal. - **Pattern 2** (streaming) adds WORKING, then a loop of artifact chunks consumed in order, ending with COMPLETED; on this binding the Kafka consumer replaces HTTP SSE. - **Pattern 3** (notifications) publishes the same kind of lifecycle events, but the client pulls them asynchronously from the log when ready — it does not receive an HTTP POST to a webhook URL. In the A2A Kafka POC, standard A2A semantics are preserved in JSON-RPC-style envelopes on Kafka topics — but the three HTTP delivery paths (poll, SSE, push) are not implemented as three separate mechanisms. They collapse into produce to topic `a2a.requests` → consume from topic `a2a.updates` (or, potentially, a per-client reply topic). Mapping interactions to Kafka topics ------------------------------------ A2A conceptKafka topicEnvelope **method** Client → agent (SendMessage)`a2a.requests`message/sendAgent → client (status)a2a.updates (or reply topic)status-updateAgent → client (artifact)sameartifact-updateAgent → client (full task)sametaskDiscovery`a2a.agent-cards`agent-cardThe Record `key = taskId` (ordering per task). The client can set `metadata.replyTopic` (ReplyRouting) so the agent publishes updates to a dedicated topic (used in future benchmarks). Semaphore Agent: Adapting the Clockwork Agent Implementation ------------------------------------------------------------ The Semaphore Agent adapts the asynchronous countdown from the earlier Clockwork and Quartz examples to the Kafka transport. It’s the same 10-second tick story as Clockwork/Quartz, but no HTTP, SSE, or webhooks. The main components are: ClassRoleCountdownKafkaAgentConsumes message/send from a2a.requests, publishes status-update / artifact-update to `a2a.updates`KafkaCountdownClientProduces the countdown request, consumes updates until terminal stateAuditEventConsumerExtra consumer group — fan-out / audit logReplayEventConsumerReplays retained events from the log (earliest offset)Following a countdown from request to completion ------------------------------------------------ The countdown sequence demonstrates the transport end to end with A2A Kafka. Here’s a diagram that shows concretely how our countdown example works. This sequence diagram (as shown below) illustrates the countdown demo when Kafka is the entire A2A wire. 1. **The client publishes** a message/send envelope to the topic `a2a.requests` 2. **The agent consumes** it and publishes lifecycle events to the topic `a2a.updates` (status-update, artifact-update, final task). 3. **The client consumes** those updates until the task completes. 4. **An optional AuditEventConsumer uses a separate consumer group** on the same updates topic to illustrate fan-out (same events, multiple readers). 5. **Discovery via `a2a.agent-cards` is shown** as optional. 6. **The point: poll, SSE, and push on HTTP are all collapsed** into produce/consume on Kafka topics with the same JSON-RPC-shaped envelopes. ![diagram that shows concretely how the countdown example works]() ### Explore the implementation on GitHub The Kafka A2A implementation and demo contain approximately 2,114 lines of code, around 3.7 times larger than the earlier Quartz Chronometer example (566 Java LOC). The additional complexity comes from Kafka client plumbing, transport models, agents, and supporting infrastructure. Explore the code and demo [here](https://github.com/instaclustr/code-samples/tree/main/AI/A2A/a2a-kafka). What the proof of concept does not yet implement ------------------------------------------------ For a transport POC demo, the code is solid; for a complete, spec-advertisable Kafka binding, several important pieces are still missing or deliberately skipped. Critical gaps (if you want “real A2A on Kafka”) are as follows: **1. Discovery and binding are unfinished**: - Countdown hard-codes `a2a.requests` / `a2a.updates` (there’s currently no card-driven lookup) - Agent Cards use placeholder `url: "kafka://a2a"` with no actionable topic metadata - No `KafkaA2AClient.discover(skill)` → resolve topics → send flow Without that, peers can’t interoperate the way HTTP A2A does via `supportedInterfaces`. **2. No interoperability with standard a2a-java/ HTTP clients** This is by design, so not really a bug — but it’s **critical** if you expect a drop-in replacement for the Quartz Chronometer Demo. Generic clients speak HTTP JSON-RPC + SSE/push; this POC needs Kafka producers/consumers on both sides. **3. Several A2A operations aren’t on the wire** OperationStatusmessage/sendImplementedstatus-update / artifact-update / taskImplementedGetTask (poll)Not implemented — replaced by consuming updatestasks/cancelDocumented in config comment, not dispatched in A2AKafkaConsumerPush webhook registrationNot applicable — Kafka consume instead**4. Countdown ≠ full Clockwork mechanisms** Kafka countdown is async countdown only. Not on Kafka yet: - Ex 1 — sync time → immediate Message (no task) - Ex 3 — input-required + confirm continuation (`TaskState.INPUT_REQUIRED` exists in the model; countdown agent doesn’t use it.) **5. Multi-agent routing – not yet.** My initial design docs describe topic-per-type, tenant isolation, routers; but the code is limited to shared topics and a homogeneous pool of consumers. What are the real limitations? ------------------------------ This Kafka transport remains an experimental proof of concept; it’s not fully tested or usable as a drop-in replacement for the default A2A protocol. The main limitations are: GapWhy it mattersNo registered protocolBindingCustom binding sketch only — not an official A2A bindingPoll / SSE / push not separate pathsAll collapsed into consuming `a2a.updates` — fine for transport demo, not a literal map of HTTP update strategiesNo SubscribeToTask-style reconnect contractKafka offsets handle replay differently; not spelled out as A2A re-attachProduction opsNo auth/TLS story in code, idempotency, dedup, schema registry, DLQ, exactly-once semanticsThin testsBenchmark + manual runs; no full binding compliance suiteLike a temporary Napoleonic-era encampment before the next advance, this proof of concept provides a useful staging point for demos, experimentation, and benchmarking. That’s where we’ll go next. Bringing the series together: agent interactions and an event backbone ---------------------------------------------------------------------- Across this series, we have moved from the foundations of agent systems to the Agent2Agent object model, runtime interactions, visual explanations, and working Java examples. The Clockwork and Quartz agents made the interaction patterns concrete. The Semaphore Agent adds a different architectural perspective: selected A2A-style interactions carried through a shared Kafka event log, with independent consumers and replay of retained updates. The takeaway is not that Kafka replaces A2A. **A2A defines how agents exchange work and results; Kafka offers an event-streaming foundation for distributing and retaining the resulting events.** This experiment shows one way to connect those ideas, while making the remaining compatibility, operational, and testing work explicit. For developers exploring agent systems, that is the larger lesson of the series: separate the interaction model from the infrastructure carrying it, then evaluate each against the behavior your application actually needs. Start with the Semaphore Agent code to follow a request through its lifecycle, observe the same updates from another consumer group, and explore replay. Completing discovery, operation coverage, routing, security, and validation would be potential next steps—not capabilities this proof of concept already provides. --- Want to explore the entire series? ***Scaling agent systems with Apache Kafka and A2A*** - [**Part 1:** The evolution of agent systems](https://www.instaclustr.com/blog/scaling-agent-systems-with-apache-kafka-and-agent2agent-part-1-the-evolution-of-agent-systems/) - [**Part 2:** Understanding the A2A object model](https://www.instaclustr.com/blog/scaling-agent-systems-with-apache-kafka-and-agent2agent-part-2-understanding-the-a2a-object-model/) - [**Part 3:** Agent2Agent protocol explained](https://www.instaclustr.com/blog/scaling-agent-systems-with-kafka-and-a2a-part-3-agent2agent-protocol-explained/) - [**Part 4:** Visualizing the Agent2Agent protocol](https://www.instaclustr.com/blog/scaling-agent-systems-with-apache-kafka-and-a2a-part-4-visualizing-the-agent2agent-protocol/) - [**Part 5:** The Clockwork Agent—a Java A2A example](https://www.instaclustr.com/blog/scaling-agent-systems-with-apache-kafka-and-a2a-part-5-the-clockwork-agent-a-java-a2a-example/) - [**Part 6:** The Quartz Chronometer Agent—an example using the default A2A protocols](https://www.instaclustr.com/blog/scaling-agent-systems-with-apache-kafka-and-a2a-part-6-the-quartz-chronometer-agent-an-example-using-the-default-a2a-protocols/) - **Part 7:** An experimental Kafka transport and the Semaphore Agent *(you’re here)* For the Kafka infrastructure layer, try [NetApp Instaclustr for Apache Kafka](https://console2.instaclustr.com/signup?_ga=2.200991470.1505602066.1726152168-1815699945.1710961315&_gl=1*i7t8m9*_gcl_aw*R0NMLjE3ODg4OTM1ODQuQ2owS0NRandrT3FaQmhETkFSSXNBQUNzYmZMZndTU3YwbXpETXA1dHcwY2kzSlMwcVhtSERDTXlsTFViNVFBa1o2VjZJeGZ5NUZVb3JCWWFBdGlhRUFMd193Y0I.*_gcl_au*MTk2NzE2NTg4NS4xNzkwMzYzMzky*_ga*MTk1NjAzOTk0MS4xNzc0NjU4OTY2*_ga_4NBQSJMP6D*czE3OTI2OTk0NzEkbzIzMSRnMSR0MTc5MjcwMDY4OSRqNjAkbDAkaDA.) free. A managed Kafka service addresses cluster operations; it does not complete the missing A2A functionality or make this experimental transport production-ready. ### About the author **[Paul Brebner](https://www.instaclustr.com/blog/author/paul-brebner/)** | Technology Evangelist at Instaclustr Paul has extensive R&D and consulting experience in distributed systems, technology innovation, software architecture, and engineering, software performance and scalability, grid and cloud computing, and data analytics and machine learning. [ Add Instaclustr as a preferred source on Google ](https://google.com/preferences/source?q=instaclustr.com) ![mail icon]()#### Get the latest articles for open sourceIn your inbox Sign up now ### Related content [ Zero Downtime Migration to Instaclustr Yes, we can migrate existing Cassandra clusters to Instaclustr without any downtime. Here's what to expect from the process... ](https://www.instaclustr.com/blog/zero-downtime-migration-to-instaclustr/) [ Workflow Comparison: Uber Cadence vs Netflix Conductor When choosing what’s right for your company’s opensource workflow needs it is important to know the difference and similarities ... ](https://www.instaclustr.com/blog/workflow-comparison-uber-cadence-vs-netflix-conductor/) [ Will Your Cassandra Database Project Succeed?: The New Stack Open source Apache Cassandra® continues to stand out as an enterprise-proven solution for organizations seeking high availability... ](https://www.instaclustr.com/blog/will-your-cassandra-database-project-succeed-the-new-stack/) ×Sign upto ourNewsletter -----------------------