Multi-Agent Orchestration Patterns: Supervisor, Pipeline, Swarm and Hierarchy

When multi-agent is actually justified

Splitting one agent into five multiplies tokens, latency and failure surface. It is worth it for exactly three reasons, and "it seems more modular" is not one of them:

• Context isolation. Each specialist needs a different, large, mostly disjoint context — a 40-tool prompt degrades tool selection badly.
• Different capability or trust levels. One agent has write access to production, another does not. Separation is a security boundary, not a style choice.
• Genuine parallelism. Independent subtasks that can run concurrently and cut wall-clock time.

The four topologies

Supervisor

One coordinator holds the plan and calls specialists as if they were tools. Specialists never talk to each other; every result flows back through the supervisor.

const NEXT = z.enum(["researcher", "analyst", "writer", "FINISH"]);

async function supervisor(state: State) {
  if (state.turns >= state.maxTurns) return { next: "FINISH", reason: "budget" };
  const { next } = await model.withStructuredOutput(
    z.object({ next: NEXT, reason: z.string() })
  ).invoke([system(SUPERVISOR_PROMPT), ...state.messages]);
  return { next, turns: state.turns + 1 };
}

• Strengths: one place to enforce budget, policy and stop conditions; easy to trace.
• Weaknesses: the supervisor's context grows with every result — it becomes the bottleneck and the most expensive node.
• Fix for the bottleneck: specialists return summaries plus artefact ids, not full payloads. The supervisor routes on summaries; consumers fetch artefacts directly.

Pipeline

A fixed sequence of specialists, each transforming the output of the last. The least glamorous topology and usually the right first choice, because it is deterministic, testable stage by stage, and cheap.

• Use when the stages are genuinely ordered: extract → enrich → verify → write.
• Add a validation gate between stages with a typed schema. Most pipeline failures are stage N emitting something stage N+1 cannot parse.
• Retry per stage with the validation error appended, not the whole pipeline.

Hierarchy

Supervisors of supervisors. A top-level orchestrator delegates to team leads, each of which owns a pool of workers. This is what you reach for past roughly ten specialists, when a single supervisor's routing prompt stops fitting or stops being accurate.

• Each level compresses: workers return findings, leads return conclusions, the orchestrator sees only conclusions.
• Budgets cascade — the orchestrator allocates a token budget per team and teams sub-allocate.
• Cost grows fast. Instrument per level before scaling the tree.

Swarm / handoff

No central coordinator: agents hand control to each other directly, and the active agent owns the conversation until it hands off. This is the customer-support model — triage passes to billing, billing passes to refunds.

type Handoff = {
  to: "billing" | "technical" | "refunds" | "human";
  reason: string;
  contextSummary: string;   // what the next agent must know
  unresolved: string[];     // what it must still do
};

• Strengths: low latency, natural for conversational products, no supervisor bottleneck.
• Weaknesses: handoff ping-pong. Two agents can bounce a ticket forever, each convinced it belongs to the other.
• Mandatory guard: a handoff counter in shared state with a cap, and an escalation to a human when it trips.

Handoff contracts and shared memory

The interface between agents matters more than the agents. Free-form prose handoffs are where multi-agent systems rot.

1. Type every handoff. A schema with task, inputs, constraints, definition of done, and a budget. Validate on both sides.
2. Separate shared state from private scratchpad. Shared state is small, structured and durable; scratchpads are per-agent and discarded.
3. Reference, do not copy. Pass artefact ids into a store rather than pasting 8k tokens of document into the next agent's prompt.
4. Preserve provenance. Every fact in shared state carries which agent produced it and from what source, or debugging becomes archaeology.
5. One writer per field. Two agents writing the same state key is a race with a nondeterministic winner.

Cost and latency budgets

Those multipliers are the honest reason to start small. Budget enforcement belongs in state and in edge conditions — a per-run token ceiling, a per-agent call ceiling, and a graceful degradation path that returns partial work instead of an exception.

Failure modes and containment

How to choose

1. Default to one agent. Measure where it fails before splitting anything.
2. Ordered work → pipeline. Deterministic and testable; do not add a supervisor to a sequence.
3. Dynamic routing over a few specialists → supervisor. Keep the specialist count under about eight.
4. Conversational domain routing → swarm, with a hard handoff cap.
5. Ten-plus specialists or multi-team research → hierarchy, with cascading budgets.
6. Whatever you pick, make the topology explicit as a graph — typed state, coded edges, checkpoints — so the system stays debuggable when it grows.

Worth reading: Anthropic on building a multi-agent research system for real cost numbers, and LangGraph's multi-agent concepts for supervisor and swarm implementations.