Phase 4: AI Agents & Autonomous Systems

Multi-agent architectures with specialized collaborating agents

Advanced ~16 min read
Think of it this way A friendly analogy. Read this if the technical version feels dense. Show Hide

Imagine you want to create a magnificent feast for your whole family. You need appetizers, a main course with side dishes, and a fancy dessert. If you tried to do absolutely everything yourself – planning the menu, chopping all the vegetables, cooking all the dishes at the right time, and then baking a perfect cake – it would be incredibly difficult! You'd probably get overwhelmed, forget an ingredient, or even burn something because you're trying to do too many different things at once. It's just too much for one person to handle perfectly.

That's exactly why professional kitchens have a team of chefs. Think of it like this: There's a Head Chef who plans the whole meal, decides what needs to happen when, and keeps everyone on track. There's a Prep Chef who focuses only on getting ingredients ready – washing vegetables, chopping onions, measuring spices. They have their own special knives and cutting boards. Then there's the Line Cook who expertly handles the main dishes, making sure everything is cooked perfectly and at the right temperature. They have all the pots and pans. And a Pastry Chef who is a wizard with desserts, using special mixers and ovens for cakes and cookies. Each chef has their own super-important, but narrow, job. They're like specialists.

These chefs don't work alone; they collaborate! The Prep Chef hands ingredients to the Line Cook, who then might pass a finished dish to the Head Chef for a final check before it goes out. They all talk to each other to make sure the whole meal comes together perfectly. When we build smart computer programs, especially ones that can help us solve complex problems or create things, it's very similar. Instead of trying to make one giant "super-chef" program do absolutely everything, which would get confused and make many mistakes, we design a team of smaller, specialized programs.

So, when you want your computer to help you do something really tricky, like write a complicated story, research a big school project, or even design a new game level, this idea means you can build a whole "kitchen crew" of computer helpers. One computer helper might be really good at brainstorming ideas (the Head Chef), another at finding facts on the internet (the Prep Chef), another at writing different parts of the story (the Line Cook), and another at checking if the story makes sense (the final taste-tester). This way, each computer helper stays focused, does its job really well, and together they can create something amazing that no single helper could do alone!

The mental model: specialized agents as microservices

Think of each agent as a microservice with a language interface instead of an HTTP interface. A researcher agent exposes one operation: given a topic, return a structured summary. A code agent exposes one operation: given a spec, return working code. An orchestrator agent composes these operations in sequence or in parallel, passing outputs as inputs. The critical insight is that the boundary between agents is a message, not a function call, which means you can checkpoint, retry, audit, and replace individual agents without touching the rest of the system.

Under the hood, most multi-agent frameworks model the system as a directed graph. Nodes are agents; edges are message flows. LangGraph makes this explicit with a state machine. CrewAI uses a sequential or hierarchical process model. AutoGen uses an actor model where agents respond to each other's messages. The orchestrator pattern is the safest for production: one agent holds the plan and delegates, so you have a single place to enforce budget limits, timeouts, and safety checks. Fully decentralized agent meshes where agents call each other freely look elegant on a whiteboard but are hard to debug when something goes wrong at step 17 of 23.

A real-world scenario: competitive intelligence pipeline

Imagine you're building a weekly competitive intelligence report. A single-agent approach fails here because the task requires web search, financial data lookup, natural language summarization, structured data extraction, and final synthesis. That's too many tools and too much context for one agent to handle reliably.

A pro would decompose it like this: a Planner agent takes the target company name and outputs a list of sub-tasks with dependencies. A Search agent runs web searches for each sub-task and returns raw snippets. An Extraction agent parses those snippets into structured JSON (company name, metric, date, source URL). An Analysis agent compares this cycle's data against last cycle's stored data. A Synthesis agent writes the final Markdown report. An optional Critic agent runs a quick factual consistency check before delivery.

You'd wire these with a message-passing layer (Redis streams or a simple in-memory queue for local dev), give the Orchestrator a budget of N total LLM calls, and log every agent's input/output pair for debugging. If the Search agent returns no results for a sub-task, the Orchestrator retries with a rephrased query or marks that section as "data unavailable" rather than hallucinating.

Tradeoffs vs. alternatives

The main alternative to multi-agent decomposition is a single powerful agent with a large context window. GPT-4o and Claude 3.5 Sonnet both support 128k+ tokens, which is tempting. The problem is that long contexts increase latency, cost, and the probability of the model losing track of details buried in the middle of the prompt (the "lost in the middle" effect). A single agent also serializes everything; you cannot parallelize independent sub-tasks.

The other alternative is rigid prompt chaining without agents at all. This works fine for linear pipelines with deterministic steps, but falls apart when you need conditional branching, error recovery, or dynamic task generation. Agents add autonomy: they can decide what tool to call next based on intermediate results. The tradeoff is observability and control. Pure prompt chains are easier to trace; agent systems require explicit logging and tracing infrastructure from day one.

What changes at scale

At 10 users, you can run a multi-agent crew synchronously in a single process. At 10k users, each crew run needs to be async, queued, and isolated. You need per-run state management (a database row per run, not in-memory dicts), and you need to think about agent timeout budgets so one slow search agent doesn't block the whole pipeline. At 10M users, the economics of model selection become critical: the researcher agent running gpt-4o on every request at scale will bankrupt you. You'd route simple extraction tasks to gpt-4o-mini or a fine-tuned smaller model, and reserve gpt-4o for the synthesis step that actually requires reasoning depth. You'd also add caching at the tool layer so repeated lookups hit a cache before the LLM even sees them.

Cost, latency, and reliability

Latency is the hardest problem. A 5-agent sequential pipeline where each agent averages 3 seconds adds up to 15 seconds minimum, which is unacceptable for synchronous UI. The solution is to identify which tasks have no dependencies and run those agents in parallel using asyncio. An Orchestrator that fans out to 3 parallel agents and waits for all three reduces wall-clock time to the longest individual task. Build your graph before you build your agents; the dependency structure determines your latency floor. For reliability, treat agent failures as expected events. Every agent call should have a timeout, a retry budget, and a fallback behavior (return empty result, use cached data, or escalate to human review).

Key Takeaways

  • Assign each agent exactly one job and a minimal tool set to prevent context bloat.
  • Use an orchestrator agent to route tasks; never let agents call each other directly without governance.
  • Pick the cheapest model that meets the quality bar for each specialized agent role.
  • Design explicit message contracts between agents so failures are observable and retryable.

Pro tips

  • Give each agent a role budget constraint in its system prompt: 'You have at most 3 tool calls to answer this.' Without a budget, agents loop on ambiguous tasks and burn tokens. An explicit cap forces the agent to commit to an answer with available information.
  • Log every inter-agent message as a structured event with a shared trace ID. When a multi-agent run produces a bad result, you need to replay the exact message sequence that caused it. Without trace IDs threading through all agent calls, debugging a 6-agent pipeline is close to impossible.
  • Use different models for different agent roles based on what the role actually demands. A routing or extraction agent rarely needs GPT-4o reasoning ability; gpt-4o-mini at one-tenth the cost does the job. Reserve your expensive model budget for the synthesis or critic agent where reasoning depth matters.
  • Design your orchestrator to accept a 'partial results' mode. If the search agent times out, the pipeline should synthesize from whatever data it has and flag the output as incomplete rather than failing the whole run. Users can tolerate incomplete answers; they cannot tolerate errors.

Common pitfalls

  • Mistake: Letting agents pass raw unvalidated output directly to the next agent. Fix: Define a Pydantic schema for every inter-agent message and validate before passing. One malformed output cascades into garbage downstream.
  • Mistake: Building a circular dependency between agents where agent A waits on B and B waits on A. Fix: Draw the dependency graph before writing code. Every edge must point in one direction; cycles require an explicit arbitration mechanism.
  • Mistake: Using the same large context window LLM for every agent role because it's easier. Fix: Profile the token usage per agent role after your first prototype. Reassign cheap roles to smaller models before costs compound.
  • Mistake: Treating multi-agent pipelines as synchronous request-response from the start. Fix: Design for async from day one with asyncio or a task queue (Celery, ARQ). Retrofitting async into a synchronous pipeline is significantly more painful than building it right initially.

When to use multi-agent vs single-agent vs prompt chaining

Option Use when Avoid when
Single agent with tools Task fits in one context window, requires fewer than ~5 tool calls, and has no parallel sub-tasks. Task needs multiple distinct skill sets or the prompt becomes longer than ~4k tokens of instructions.
Sequential prompt chaining Steps are deterministic and linear; no branching, no error recovery, no dynamic task generation needed. Steps have dependencies on intermediate results that require conditional routing or retries.
Multi-agent with orchestrator Task decomposes into parallel sub-tasks, requires different model capabilities per step, or needs independent failure boundaries. You need sub-second latency or the overhead of agent coordination outweighs the complexity of the task.
Fully decentralized agent mesh Each agent is fully autonomous and can self-organize; research or simulation contexts with relaxed correctness requirements. Production systems where observability, cost control, and deterministic routing are required.

Code Example

python
# crewai==0.28.0
from crewai import Agent, Task, Crew
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

researcher = Agent(
    role="Researcher",
    goal="Find factual information about the given topic",
    backstory="Expert at distilling web information into concise summaries",
    llm=llm,
    verbose=False,
)

writer = Agent(
    role="Writer",
    goal="Turn research notes into a clear, structured report",
    backstory="Technical writer with experience in AI topics",
    llm=llm,
    verbose=False,
)

research_task = Task(
    description="Research the current state of LLM context windows in 2024",
    expected_output="Bullet-point summary of key facts and numbers",
    agent=researcher,
)

write_task = Task(
    description="Write a 200-word report using the research notes",
    expected_output="A structured 200-word report",
    agent=writer,
    context=[research_task],
)

crew = Crew(agents=[researcher, writer], tasks=[research_task, write_task])
result = crew.kickoff()
print(result)

How this code works

This code demonstrates how to build a collaborative multi-agent system using crewai, where specialized AI agents work together to produce a structured report. It sets up a small team to research a topic and then write about it, showcasing a foundational pattern for dividing complex problems among AI assistants.

The process begins by defining an llm (a language model) that all agents will use as their intelligence. Then, two Agents are created: a researcher and a writer. Each agent is given a specific role, a clear goal (what it needs to achieve), and a backstory to define its persona and capabilities. Setting verbose=False keeps the output clean by hiding the agents' internal thought processes. Next, Tasks are defined. The research_task is assigned to the researcher to gather information, specifying its description and expected_output. The write_task is then assigned to the writer, but critically, it includes context=[research_task]. This subtle detail ensures the writer waits for and uses the output of the research_task before starting its own work, orchestrating the collaboration and execution order. Finally, a Crew is assembled with both agents and tasks, and crew.kickoff() starts the entire workflow, producing the final report.

Production-grade example

Adds per-agent retries with backoff, timeouts, token logging, graceful degradation on agent failure, and async parallel execution.

python
# langgraph==0.1.13, langchain-openai==0.1.8
import asyncio
import logging
import os
import time
from typing import TypedDict, Annotated
import operator

from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage
from langchain_core.exceptions import OutputParserException
from tenacity import retry, stop_after_attempt, wait_exponential, retry_if_exception_type
from openai import RateLimitError, APITimeoutError

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(name)s %(levelname)s %(message)s")
logger = logging.getLogger("multi_agent")

researcher_llm = ChatOpenAI(
    model="gpt-4o-mini",
    temperature=0,
    timeout=20,
    api_key=os.environ["OPENAI_API_KEY"],
)
analyst_llm = ChatOpenAI(
    model="gpt-4o",
    temperature=0.2,
    timeout=30,
    api_key=os.environ["OPENAI_API_KEY"],
)

@retry(
    retry=retry_if_exception_type((RateLimitError, APITimeoutError)),
    wait=wait_exponential(multiplier=1, min=2, max=30),
    stop=stop_after_attempt(4),
)
async def call_agent(llm, system_prompt: str, user_message: str, agent_name: str) -> str:
    start = time.monotonic()
    try:
        response = await llm.ainvoke([
            SystemMessage(content=system_prompt),
            HumanMessage(content=user_message),
        ])
        elapsed = time.monotonic() - start
        usage = response.response_metadata.get("token_usage", {})
        logger.info(
            "agent_call",
            extra={
                "agent": agent_name,
                "input_tokens": usage.get("prompt_tokens", 0),
                "output_tokens": usage.get("completion_tokens", 0),
                "latency_s": round(elapsed, 3),
            },
        )
        return response.content
    except (RateLimitError, APITimeoutError):
        logger.warning("Retryable error in agent %s, backing off", agent_name)
        raise
    except Exception as exc:
        logger.error("Non-retryable failure in agent %s: %s", agent_name, exc)
        return "[agent_error: no output available]"

async def run_pipeline(topic: str) -> dict:
    RESEARCHER_PROMPT = "You are a research assistant. Return 5 concise bullet points of key facts."
    ANALYST_PROMPT = "You are a senior analyst. Given research notes, produce a 150-word executive summary with a risk assessment."

    research_notes, _ = await asyncio.gather(
        call_agent(researcher_llm, RESEARCHER_PROMPT, topic, "researcher"),
        asyncio.sleep(0),
    )

    if "agent_error" in research_notes:
        logger.warning("Researcher failed, falling back to minimal context for analyst")
        research_notes = f"Limited data available for: {topic}"

    summary = await call_agent(
        analyst_llm,
        ANALYST_PROMPT,
        f"Research notes:\n{research_notes}",
        "analyst",
    )

    return {"topic": topic, "research": research_notes, "summary": summary}

if __name__ == "__main__":
    result = asyncio.run(run_pipeline("Open-source LLM adoption in enterprise 2024"))
    print(result["summary"])

How this code works

This code showcases a multi-agent architecture where specialized AI agents collaborate to process information. It defines two distinct roles: a researcher_llm and an analyst_llm. The researcher_llm is designed to gather key facts about a given topic, while the analyst_llm then synthesizes these facts into an executive summary, including a risk assessment. This sequential workflow demonstrates how different AI capabilities can be chained to accomplish a more complex task than any single agent could perform alone.

The process begins by setting up two ChatOpenAI instances with different models (gpt-4o-mini for the researcher, gpt-4o for the analyst) and temperature settings to reflect their specialized functions. The call_agent function is a robust wrapper that handles communication with the LLMs, injecting SystemMessage and HumanMessage prompts. A key feature here is the @retry decorator, which automatically retries the call if transient network issues like RateLimitError or APITimeoutError occur, making the system more resilient. The run_pipeline function orchestrates the interaction: it first calls the researcher agent, and then uses the research_notes output to inform the analyst agent. A subtle but important detail is the if "agent_error" in research_notes: check. If the researcher agent fails to produce output, the system doesn't crash but instead provides minimal context to the analyst, ensuring the pipeline can still deliver a result, albeit a degraded one.

Practice & master

Try the exercise, check your understanding, then mark this lesson mastered to track your path to pro.

Exercise

Build a two-agent research pipeline using CrewAI or direct LLM calls. Agent 1 (Researcher) takes a company name and returns 3 bullet points of recent facts. Agent 2 (Critic) reviews those facts and flags any that appear vague or unverifiable. Wire them sequentially so the Critic always sees the Researcher's output.

python
# crewai==0.28.0 or use direct ChatOpenAI calls
import os
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0,
                 api_key=os.environ["OPENAI_API_KEY"])

RESEARCHER_SYSTEM = "You are a research assistant. Return exactly 3 bullet points of recent, specific facts."
CRITIC_SYSTEM = "You are a fact-checking critic. For each bullet point, label it VERIFIED, VAGUE, or UNVERIFIABLE and explain why in one sentence."

def researcher_agent(company: str) -> str:
    # TODO: call llm with RESEARCHER_SYSTEM and the company name
    pass

def critic_agent(research_notes: str) -> str:
    # TODO: call llm with CRITIC_SYSTEM and pass research_notes as user message
    pass

def run_pipeline(company: str) -> None:
    # TODO: call researcher_agent, print output, then call critic_agent with result, print output
    pass

run_pipeline("Anthropic")

Quick check

  1. Why does assigning each agent a minimal, focused tool set improve multi-agent system reliability?

  2. An orchestrator agent delegates to 3 parallel sub-agents. Sub-agent 2 times out. What is the correct production behavior?

  3. You have a 6-agent sequential pipeline where each agent averages 4 seconds. What is the most effective way to reduce total latency to under 10 seconds?

Self-check: Describe how you would decompose a 'research and publish a weekly industry newsletter' task into specialized agents. Name each agent, its model choice, its tool set, and the message it passes to the next agent. Then identify which agents can run in parallel.