Circuit Breaker: Stop One Slow Service From Taking Down Everything

Slow is worse than down

A dependency returning connection-refused in two milliseconds is a rounding error. A dependency taking nine seconds to answer is an outage generator, because every caller holds a thread, a connection and a chunk of memory while it waits.

Recommendation service degrades (p99: 180ms -> 9s)
        |
Checkout calls it inline for "you may also like"
        |
Each checkout request now occupies a thread for 9s
        |
Tomcat pool (200 threads) saturates in ~12s of normal traffic
        |
Checkout returns 503 -- for a widget nobody needed

The three states

• Closed — traffic flows; the breaker counts outcomes in a rolling window.
• Open — calls are rejected instantly without touching the network, for a cooldown period.
• Half-open — a small number of probe calls are allowed. All succeed → close. Any fail → open again with a longer cooldown.

Choosing thresholds

Count slow calls as failures. A breaker that only counts exceptions will happily let every request sit at the timeout limit forever.

Fallbacks: what to serve when open

1. Omit the feature entirely — hide the recommendations block.
2. Serve stale cache — yesterday's top sellers beat a spinner.
3. Serve a static default — a curated list, computed offline.
4. Queue it — for writes, accept and process asynchronously.
5. Fail loudly — for payments, a wrong answer is worse than an error.

The fallback decision is a product decision. Ask which is worse: a missing section or a wrong number. Only the second one needs to propagate the error.

Implementation sketch

class CircuitBreaker {
  private state: "closed" | "open" | "half" = "closed";
  private failures = 0; private calls = 0; private openedAt = 0;

  constructor(private opts = { rate: 0.5, minCalls: 20, cooldownMs: 10_000, timeoutMs: 800 }) {}

  async call<T>(fn: () => Promise<T>, fallback: () => T): Promise<T> {
    if (this.state === "open") {
      if (Date.now() - this.openedAt < this.opts.cooldownMs) return fallback();
      this.state = "half";
    }
    try {
      const result = await withTimeout(fn(), this.opts.timeoutMs);
      this.onSuccess();
      return result;
    } catch {
      this.onFailure();
      return fallback();
    }
  }

  private onSuccess() { if (this.state === "half") this.reset(); this.calls++; }
  private onFailure() {
    this.calls++; this.failures++;
    const tripped = this.calls >= this.opts.minCalls && this.failures / this.calls >= this.opts.rate;
    if (this.state === "half" || tripped) { this.state = "open"; this.openedAt = Date.now(); }
  }
}

Breakers need timeouts, retries need budgets

A circuit breaker is one of three controls that only work together:

• Timeouts bound a single call. Without them, nothing is ever “a failure”.
• Retries with jitter handle transient blips — but cap them at one or two, and never retry inside a retry, or one user request becomes eight backend calls.
• Breakers handle sustained failure, which retries make worse.

What to measure

• Breaker state transitions, as events with the dependency name.
• Rejected-while-open count — this is user impact, and it is invisible in error logs.
• p99 latency of the wrapped call, separately from your own handler.
• Fallback usage rate, so silent degradation does not last for weeks.

Interview framing

“The recommendation call is non-critical, so I wrap it in a breaker with an 800ms timeout, a 50% failure threshold over a 20-request window, and a 10-second cooldown. When open, we render the page without the block and emit a degradation metric. Payments get no fallback — they fail closed and surface an explicit error.” That answer shows you distinguish critical from optional dependencies, which is the actual signal.