Shared resources are shared failure
Most services start with one HTTP thread pool, one database connection pool and one queue consumer group. That is fine until a single downstream call gets slow. Then every in-flight request for that call holds a thread, and the pool becomes a global lock on your service.
The users who notice are not the ones using the broken feature. They are the ones trying to pay, because payment requests are now queued behind three hundred stuck recommendation calls.
The idea, borrowed from ships
A ship's hull is divided into watertight compartments. A breach floods one section; the vessel stays afloat. In software, you partition the finite resources — threads, connections, memory, concurrency permits — so that exhaustion is local.
Where to place bulkheads
| Level | Partition by | Mechanism |
|---|---|---|
| Thread / concurrency | Downstream dependency | Separate executors or semaphores |
| Connection pool | Workload class | Distinct DB pools for OLTP vs reporting |
| Queue | Message type or tenant | Separate topics and consumer groups |
| Process | Criticality | Deploy payments as its own service |
| Cluster | Tenant tier | Dedicated node pools for enterprise customers |
The cheapest useful version is a semaphore per dependency inside one service. The strongest is physical separation. Most teams need the first and reach for the last only for the money path.
Sizing the compartments
- Start from Little's Law:
concurrency = throughput × latency. Twenty requests per second at 200ms needs about four permits, not two hundred. - Add headroom for the p99, not the mean — that is where saturation happens.
- Deliberately under-provision optional features. Shedding recommendations is fine.
- Never let the sum of pools exceed what the process can actually run.
Implementation sketch
class Bulkhead {
private inFlight = 0;
constructor(private name: string, private limit: number) {}
async run<T>(fn: () => Promise<T>): Promise<T> {
if (this.inFlight >= this.limit) {
metrics.increment("bulkhead.rejected", { pool: this.name });
throw new BulkheadFullError(this.name);
}
this.inFlight++;
metrics.gauge("bulkhead.inflight", this.inFlight, { pool: this.name });
try {
return await fn();
} finally {
this.inFlight--;
}
}
}
const pools = {
payments: new Bulkhead("payments", 60),
search: new Bulkhead("search", 30),
recommendations: new Bulkhead("recommendations", 8),
};
// recommendations can never consume more than 8 concurrent slots
await pools.recommendations.run(() => recoClient.fetch(userId));Combine this with a circuit breaker: the bulkhead caps concurrency, the breaker stops calling altogether once failure is sustained. They solve adjacent problems and are usually deployed as a pair.
Bulkheads for multi-tenant systems
The noisy neighbour is the same failure wearing a different hat. One customer submits a hundred-thousand-row import and every other tenant's dashboards stall. Partition by tenant:
- Per-tenant concurrency caps on expensive operations.
- Separate queues (or at least fair scheduling) so one tenant cannot monopolise consumers.
- Dedicated infrastructure for the largest accounts, priced accordingly.
The cost: utilisation
Partitioned resources cannot be borrowed. Three pools of fifty are strictly less efficient than one pool of a hundred and fifty, because idle capacity in one compartment cannot rescue another. You are buying predictability with hardware.
That trade is almost always correct for anything touching revenue, and often wrong for internal batch systems where throughput matters more than isolation.
Interview framing
“Checkout, search and recommendations get separate connection pools and concurrency limits, sized from Little's Law with p99 latency. Recommendations gets eight permits and fails open to a cached list. When a pool is full we reject in microseconds rather than queueing, and we alert on rejection rate — that is the signal that a compartment is under-sized rather than broken.”