Hey everyone, welcome back to another edition of Coding with Alex! If you’ve been building in the AI space lately, you’ve probably noticed a massive shift in the wind. A year ago, we were all obsessed with prompt engineering, RAG (Retrieval-Augmented Generation), and how to get a single LLM to output clean JSON. Today, the conversation has moved entirely. We are no longer just building chat interfaces; we are building systems of autonomous agents that collaborate, write code, run analyses, and make decisions on our behalf.
But here is the dirty secret of the AI boom: building a single-agent demo is easy, but orchestrating a multi-agent system in production is an absolute software engineering nightmare. When you have multiple LLMs talking to each other, calling tools, and managing their own state, things get chaotic fast. Race conditions, infinite loops, state drift, and runaway API costs are the new normal.
Today, we are going to dive deep into the architectural patterns that are emerging to tame this chaos, and the critical pitfalls you need to avoid if you want to ship a multi-agent system that actually works (and doesn't bankrupt you on your OpenAI bill).
What is a Multi-Agent System anyway?
Before we look at the architecture, let’s define what we mean by a "multi-agent system" (MAS). In a traditional software system, you have deterministic modules that talk to each other via defined APIs. In a single-agent system, you have one LLM backed by tools (like a calculator or database search) running in a loop (often called the ReAct framework).
In a multi-agent system, you break a complex task down into specialized personas. For example, if you are building an automated software development pipeline, you might have:
- The Product Owner Agent: Clarifies requirements and writes user stories.
- The Architect Agent: Designs the database schema and system architecture.
- The Coder Agent: Writes the actual Python or TypeScript code.
- The QA Agent: Writes tests, runs them, and feeds the error logs back to the Coder Agent.
By segregating duties, you can use smaller, more specialized prompts (and even smaller, cheaper models) for each task, resulting in much higher accuracy than asking one giant LLM to do everything at once.
Architectural Pattern 1: Orchestrated vs. Choreographed (Peer-to-Peer)
Just like in microservices architecture, one of the first decisions you must make is how your agents communicate and coordinate. This split generally falls into two patterns: Orchestrated (Hub-and-Spoke) and Choreographed (Peer-to-Peer).
The Orchestrated (Hub-and-Spoke) Pattern
In this pattern, there is a central "Manager" or "Router" agent. The worker agents do not talk to each other directly; they only talk to the manager. The manager receives the high-level goal, breaks it into sub-tasks, assigns them to specialized agents, collects the results, and decides on the next step.
[ User Input ]
│
▼
┌──────────────┐
│ Manager Agent│◄─── (Orchestrates State & Routine)
└──────┬───────┘
│
┌───────┼───────┐
▼ ▼ ▼
┌──────┐┌──────┐┌──────┐
│Agent ││Agent ││Agent │ (Specialized Workers)
│ A ││ B ││ C │
└──────┘└──────┘└──────┘
When to use: This is highly recommended for structured business workflows where you need deterministic control over the steps. If you are building a system to generate financial reports, you want a manager to ensure Phase A (data ingestion) is completed before Phase B (analysis) begins.
The Choreographed (Peer-to-Peer) Pattern
In a choreographed system, agents publish messages to a shared event bus or state space, and other agents react to those messages based on their internal logic. There is no central manager dictating who does what and when.
For example, if the Developer Agent commits code to a shared repository, the QA Agent automatically triggers its test suite, writes a bug report, and posts it back to the channel. The Developer Agent sees the bug report and gets back to work.
When to use: Use this for highly creative, exploratory, or open-ended tasks (like game development simulations or brainstorming tools) where rigid step-by-step execution stifles the outcome.
Architectural Pattern 2: Shared State vs. Message Passing
How do agents share information? This is the state management problem of the AI era, and developers usually lean toward one of two approaches.
1. Blackboard (Shared State) Architecture
In this pattern, there is a single, centralized data store (the "Blackboard") that represents the current state of the world. All agents have read/write access to this blackboard. They can inspect the current state, modify it, or append new insights.
Here is a basic conceptual implementation in Python of how you might model a shared blackboard for a research-writing multi-agent system:
class Blackboard:
def __init__(self):
self.state = {
"research_topic": "Quantum Computing Security",
"outline": None,
"draft_sections": {},
"review_notes": []
}
def update_state(self, key, value):
# In production, you'd want thread-safety locks here!
self.state[key] = value
print(f"[State Updated] {key} updated.")
class OutlineAgent:
def execute(self, blackboard):
topic = blackboard.state["research_topic"]
# Imagine LLM call here to generate outline
outline = ["Intro to Qubits", "Shor's Algorithm", "PQC Mitigations"]
blackboard.update_state("outline", outline)
class WriterAgent:
def execute(self, blackboard):
outline = blackboard.state["outline"]
if not outline:
return "Waiting for outline..."
# Imagine LLM writes intro
draft = "This is a draft about " + outline[0]
blackboard.update_state("draft_sections", {"Intro": draft})
2. Isolated Message Passing
Instead of a shared database, agents act as isolated actors (mimicking the Actor Model in languages like Erlang or Akka). They send immutable messages to each other's mailboxes. This is much cleaner for distributed systems because it avoids concurrency write conflicts, but it can make tracking the "global truth" of a long-running process incredibly difficult.
The Pitfalls: Where Multi-Agent Systems Fail
While these patterns sound clean on paper, the non-deterministic nature of LLMs introduces unique engineering failures that you won't find in traditional software engineering.
1. The Infinite Loop (The "After You, Alphonse" Problem)
This is the most common failure mode in peer-to-peer agent systems. Agent A generates a piece of code. Agent B reviews the code, finds a minor formatting issue, and sends it back to Agent A. Agent A "fixes" it but introduces a typo. Agent B catches the typo, sends it back. This can loop infinitely, draining your API limits while you sleep.
How to fix: You must implement hard limits at the framework level. Every multi-agent graph should have a max_iterations counter or a budget guardrail. If an agent-to-agent loop exceeds 5 turns, pause execution and escalate to a human-in-the-loop (HITL) prompt.
2. Context Window Bloat and "Prompt Taxes"
When agents pass messages back and forth, the history of those messages must be fed into the LLM context window with every new turn. If you have 4 agents talking to each other, the context grows exponentially. By step 10, you are paying for 30,000 tokens of input history just to get a 200-token response. This is called the "Prompt Tax."
How to fix: Implement aggressive state summarization. Instead of passing the entire chat history between agents, use a specialized summarizer tool that distills the conversation down to key facts, decisions made, and outstanding questions before passing the payload to the next agent.
3. Tool Execution Race Conditions
If Agent A and Agent B both have access to your database or filesystem and are running concurrently, they can easily step on each other's toes. For instance, Agent A might read a file to refactor it, while Agent B deletes the directory to clean up space.
How to fix: treat agent tool executions like database transactions. Implement file locks, state locks, or routing queues (using tools like Redis or RabbitMQ) to guarantee that tool execution is serialized where concurrency could cause destructive states.
Conclusion and Next Steps
Multi-agent AI systems represent a massive leap forward in what software can accomplish. By moving away from single prompt-and-response paradigms toward systems of collaborative, specialized agents, we can build software that solves genuinely complex, multi-step problems.
However, as developers, we must resist the temptation to treat these systems as magical black boxes. Under the hood, they are still distributed systems. They require the same architectural discipline we apply to microservices: state management, rate limiting, error handling, and robust logging.
If you're looking to build your first multi-agent system, I highly recommend checking out frameworks like LangGraph (by LangChain), AutoGen (by Microsoft), or CrewAI. They provide the basic building blocks for state management and agent routing out of the box, saving you from writing thousands of lines of boilerplate orchestration code.
What are your thoughts? Have you shipped a multi-agent system to production yet? What frameworks are you using, and what has been your biggest headache? Let’s chat in the comments below!