All articles
Design PatternsSystem DesignDistributed Systems13 min read

Saga Pattern: Transactions That Span Multiple Services

When Order, Payment, Inventory and Shipping each own a database, there is no ROLLBACK. The saga pattern replaces it with local transactions and compensating actions — here is how to design, orchestrate and observe one.

The problem: no shared transaction

Inside one database, a failed step is trivial: ROLLBACK. Once Order, Payment, Inventory and Shipping are separate services with separate databases, that word stops existing. Each service commits on its own, immediately and independently.

So the interesting question is not the happy path. It is: payment captured ₹4,999, inventory says the last unit sold ninety milliseconds ago. The order row exists, the customer is charged, and there is nothing to ship.

Why not two-phase commit?
2PC gives you atomicity by making every participant hold locks until the coordinator decides. A coordinator crash leaves resources blocked, availability drops to the weakest participant, and most managed services do not offer XA at all. Sagas trade isolation for availability.

The core idea

A saga is a sequence of local transactions. Each step commits on its own, and each step has a compensating action that semantically undoes it. If step N fails, you run the compensations for steps N−1 … 1 in reverse order.

  • Forward path: T1, T2, T3, T4 — each atomic within its own service.
  • Backward path: C3, C2, C1 — business-level reversals, not rollbacks.
  • The system is never “locked”; it is temporarily inconsistent and then repaired.

A checkout saga, step by step

OrderCancel orderPaymentRefund paymentInventoryShippingInventory reservation fails → run compensations backwards
Compensations are new transactions that run forward in time to undo prior effects.
1Create order2Capture payment3Reserve stock4Book shipment5Confirm
The same flow as a pipeline: every stage is retryable and every stage is reversible.

Designing compensating actions

The hardest part of a saga is not the framework, it is admitting that some things cannot be undone — only offset.

Forward actionCompensationNote
Capture paymentIssue refundFees may not be recoverable
Reserve stockRelease reservationClean; prefer reservations over decrements
Send confirmation emailSend correction emailCannot be unsent — order it last
Create shipment labelVoid labelTime-bounded by the carrier
Award loyalty pointsDeduct pointsGuard against negative balances

Rules that save you

  1. Put irreversible steps last, so fewer compensations exist.
  2. Prefer reservations (pending state) over destructive writes.
  3. Compensations must be idempotent — they will be retried.
  4. Compensations must not fail permanently; if they can, they need a dead-letter and an alert.

Choreography vs orchestration

ChoreographyOrchestration
ControlEach service reacts to eventsA coordinator issues commands
CouplingLow, but implicitHigher, but explicit
VisibilityPoor — flow lives nowhereGood — one state machine
Best for3 steps or fewerAnything with money in it
Failure handlingScatteredCentralised, testable

Rule of thumb: start choreographed, switch to an orchestrator the first time someone asks “where is this order stuck?” and nobody can answer in under a minute.

Orchestrator sketch

type Step = {
  name: string;
  run: (ctx: Ctx) => Promise<void>;
  compensate: (ctx: Ctx) => Promise<void>;
};

async function runSaga(sagaId: string, steps: Step[], ctx: Ctx) {
  const done: Step[] = [];
  for (const step of steps) {
    try {
      await saveState(sagaId, { at: step.name, status: "running" });
      await step.run(ctx);            // idempotent, keyed by sagaId
      done.push(step);
    } catch (err) {
      await saveState(sagaId, { at: step.name, status: "compensating" });
      for (const s of done.reverse()) {
        await retryForever(() => s.compensate(ctx));
      }
      await saveState(sagaId, { status: "compensated" });
      throw err;
    }
  }
  await saveState(sagaId, { status: "completed" });
}
Persist saga state before each step so a crash resumes instead of restarting.

Every run and compensate call carries the saga id as an idempotency key, because the orchestrator can crash after the call and before the state write.

Pitfalls and how they bite

  • No isolation. Other transactions can read the intermediate state. Model it explicitly: PENDING, CONFIRMED, CANCELLED, and never show pending as final.
  • Lost compensations. A refund that silently fails is worse than the original bug. Compensations need their own retry queue and alerting.
  • Unbounded sagas. Add timeouts per step; a saga stuck for hours is an outage nobody paged for.
  • Non-idempotent steps. At-least-once delivery plus a non-idempotent capture equals double charges.

Interview framing

Say this: “Because each service owns its data, I will use a saga with an orchestrator. Payment capture is compensated by a refund, stock is reserved rather than decremented, and the order stays PENDING until all steps confirm. Every call carries the saga id as an idempotency key, and compensation failures go to a dead-letter queue with an operator runbook.” That paragraph covers consistency, isolation, retries and operations in four sentences.

Keep reading

Suggested next articles based on this one.

Design it, don't just read it.

Practise LLD and system design problems with structured rubrics and AI feedback.

Start practising free