Why a graph and not a chain
Chains are fine until the first real requirement arrives: "if the draft is weak, research again", "pause here for a human to approve", "resume this run tomorrow from where it crashed". Each of those is a control-flow feature, and expressing control flow inside prompt text is how you get an agent that loops forever and bills you for it.
Graph engineering in the agentic sense means making control flow explicit and inspectable: nodes are steps, edges are transitions, and the transitions are code you can read, test and put a limit on. The LLM decides content; the graph decides what runs next.
The mental model: state, nodes, edges
| Concept | What it is | Practical rule |
|---|---|---|
| State | A single typed object threaded through every step | Keep it small and serialisable; it is written to a checkpoint on every hop |
| Node | A pure-ish function: state in, partial state out | One responsibility per node, so retries are cheap and traces are readable |
| Edge | A transition, fixed or conditional | Conditional edges return a node name — plain code, no LLM required |
| Cycle | An edge back to an earlier node | Every cycle needs a counter and a hard cap |
| Checkpoint | Persisted state after each node | Turns a run into something you can pause, resume and audit |
Designing the state object
State design decides whether your agent stays debuggable at step 30. Two rules cover most of it: keep it flat and typed, and make each field's update semantics explicit — replace or append.
type AgentState = {
question: string; // replace
messages: Message[]; // append (reducer)
facts: Fact[]; // append, deduped by source id
draft?: string; // replace
critique?: { score: number; gaps: string[] };
attempts: number; // incremented by the critique node
budgetUsdSpent: number; // guardrail, checked by edges
};- Do not stuff raw documents into state. Store ids and fetch on demand, or your checkpoints grow into megabytes and every hop pays serialisation cost.
- Never let a node mutate state in place. Return a partial update; the framework merges it. In-place mutation makes replay produce a different result than the original run.
- Put counters in state, not in closures. A resumed run must know it is already on attempt three.
Routing patterns
1. Conditional edge (the workhorse)
function afterCritique(s: AgentState): "END" | "retrieve" | "escalate" {
if (s.critique && s.critique.score >= 8) return "END";
if (s.attempts >= 3 || s.budgetUsdSpent > 0.5) return "escalate";
return "retrieve";
}
graph.addConditionalEdges("critique", afterCritique, {
END: END, retrieve: "retrieve", escalate: "escalate",
});2. LLM-as-router
Sometimes the branch genuinely depends on semantics ("is this a billing question or a technical one?"). Constrain the router to a closed enum, give it a default branch, and log the decision. Never let a router return a free-form string that you then match on.
3. Fan-out / fan-in
Independent subtasks — summarise five documents, check three data sources — should run as parallel branches that converge on a join node. The join node is where you handle partial failure: one branch timing out should degrade the answer, not kill the run.
4. Supervisor
A supervisor node owns the decision of which specialist runs next and how many turns remain. Useful once you have four or more specialists; overkill for three, where explicit edges are clearer and cheaper.
Cycles and termination
The cycle is the whole reason to use a graph, and it is also the thing that will page you at 2am. Every loop needs three defences, all in state:
- Iteration cap. A hard maximum on total node executions per run, enforced by the framework, not by a prompt.
- Progress check. If the critique score has not improved between attempts, exit. Loops that spin without improving are the common runaway.
- Budget guard. Track tokens and dollars in state; route to a graceful degradation node when the cap is hit, and return the best draft so far rather than an error.
Checkpoints, resumability, human-in-the-loop
Persist state after every node against a thread id. That single decision buys you four features: crash recovery, pause-and-resume, time-travel debugging (replay from checkpoint N with a tweaked prompt), and human approval gates.
const app = graph.compile({
checkpointer, // Postgres / Redis in production
interruptBefore: ["sendEmail"], // pause before side effects
});
// run until the gate
await app.invoke({ question }, { configurable: { thread_id: runId } });
// later, after a human clicks approve
await app.invoke(null, { configurable: { thread_id: runId } });Put the interrupt before every irreversible side effect — sending mail, moving money, writing to a customer record. Reversible side effects can stay inline.
A worked example: research assistant
- plan — decompose the question into 3–5 sub-questions; write them to state.
- retrieve — run sub-questions in parallel against search and your knowledge graph; append deduped facts.
- draft — write the answer using only facts in state; every claim cites a fact id.
- critique — score the draft against a rubric, list gaps, increment
attempts. - route — score ≥ 8 → END; gaps and attempts < 3 → back to retrieve with the gaps as new sub-questions; otherwise escalate.
Notice that the critique node returns structured output, not prose. Routing on structured fields is deterministic; routing on "the critic said it looks good" is not.
Observability and cost control
- Trace per run, span per node. Record inputs, outputs, tokens, latency and the routing decision at every hop.
- Track cost per node, not per run. One expensive node usually dominates; you cannot find it from a run total.
- Alert on loop depth distribution. A rise in average attempts is a quality regression showing up before user complaints do.
- Cache retrieval by normalised sub-question. Loops re-ask the same things; a cache hit is a free iteration.
- Right-size models per node. Routing and critique often run fine on a small fast model; only drafting needs the expensive one.
Failure modes
| Symptom | Root cause | Fix |
|---|---|---|
| Runs never terminate | Loop with no progress check | Compare scores between attempts; exit on no improvement |
| Checkpoints huge, hops slow | Documents stored in state | Store ids; fetch content inside nodes |
| Replay gives different results | Nodes mutate state or read wall-clock/random | Pure nodes; inject time and seeds through state |
| Routing feels random | LLM router with open-ended output | Closed enum, low temperature, default branch, log decisions |
| Cost spikes on some runs | No per-run budget in state | Track spend, route to degraded output at the cap |
| Parallel branch failure kills run | No partial-failure handling at the join | Join node treats missing branches as degraded input |
Design checklist
- Draw the graph before writing code; label every edge with its condition.
- Type the state; declare replace-vs-append per field.
- Give every cycle a cap, a progress check and a budget guard.
- Add an escalate branch that returns partial work.
- Enable checkpointing on day one; interrupt before side effects.
- Instrument per-node cost and latency before you optimise anything.
Primary sources worth reading: the LangGraph low-level concepts guide for state and edges, and Anthropic's "Building effective agents" for when a workflow beats an autonomous agent entirely.