Event-Driven Architecture: Loose Coupling and the Bill That Comes With It

The orchestration chain

placeOrder()
  -> inventory.reserve()      // 120ms, must succeed
  -> shipping.schedule()      // 200ms, must succeed
  -> notification.send()      // 400ms, third-party SMTP
  -> analytics.track()        // 60ms, nobody would fail an order for this

total latency: ~780ms
availability:  0.999^4 ≈ 0.996  (≈ 3.5 hours downtime/month)

The order service now knows about four other services, waits for all of them, and inherits the product of their availabilities. Worse, the analytics team cannot add a field without a pull request against checkout.

Publish facts, not commands

Invert the direction. The order service commits its own state and publishes a statement about the past: OrderPlaced. Who reacts, and how, is not its concern.

Designing good events

1. Name in the past tense. OrderPlaced, not CreateOrder. It already happened; consumers cannot veto it.
2. Carry enough context. A bare id forces every consumer to call back to the producer, recreating the coupling you removed. Include the fields consumers need.
3. Do not carry the entire aggregate either — huge events become an implicit shared schema that nothing can change.
4. Version from day one and evolve additively: new optional fields yes, renamed or removed fields no.
5. Include event id, timestamp, aggregate id and a correlation id. These four make debugging possible.

{
  "eventId": "01J8X...",              // dedup key
  "eventType": "OrderPlaced",
  "version": 2,
  "occurredAt": "2026-08-20T09:14:22Z",
  "aggregateId": "order_8812",        // partition key -> per-order ordering
  "correlationId": "req_5f2a",        // ties back to the originating request
  "data": { "customerId": "c_41", "total": 4999, "currency": "INR", "items": [...] }
}

Delivery, ordering, duplicates

These three properties are what people actually mean when they say event-driven systems are hard.

• At-least-once is the realistic guarantee. Exactly-once exists only within a broker's own transactional boundary and rarely survives contact with an external side effect. Build idempotent consumers instead of chasing it.
• Ordering is per partition. Key by aggregate id and you get ordering per order, which is what the domain needs. Global ordering means one partition, which means no scaling.
• Publishing must be atomic with the state change. Use the transactional outbox; a bare publish() after a commit will silently lose events.
• Poison messages need a dead-letter queue plus an alert, or one malformed payload halts a partition indefinitely.

Eventual consistency in the product

The technical part is easy; the product conversation is the real work. After checkout the order exists but the shipping estimate does not yet. What does the screen say?

Whatever you choose, put a number on it: “95% of orders reach CONFIRMED within 2 seconds” is an SLO you can alert on. “Eventually” is not.

Consumer sketch

async function handleOrderPlaced(msg: Message) {
  const e = parse(msg.value);

  await db.transaction(async (tx) => {
    const claimed = await tx.query(
      "INSERT INTO processed_events(event_id) VALUES ($1) ON CONFLICT DO NOTHING RETURNING 1",
      [e.eventId],
    );
    if (claimed.rowCount === 0) return;          // duplicate delivery, already applied

    await tx.reservations.insert({
      orderId: e.aggregateId,
      items: e.data.items,
      expiresAt: addMinutes(new Date(), 30),
    });
  });

  await msg.ack();
}

// wiring: bounded retries, then dead-letter with the full envelope
consumer.on("error", async (err, msg) => {
  if (msg.attempts < 5) return retryWithBackoff(msg);
  await deadLetter.publish(msg, { reason: err.message });
  metrics.increment("consumer.dead_lettered", { type: msg.type });
});

Debugging what you cannot follow

There is no stack trace across a broker. Replace it deliberately:

• Propagate a correlation id through every event and log line.
• Use distributed tracing with producer and consumer spans linked by that id.
• Alert on consumer lag per group — it is the earliest signal of nearly every problem.
• Keep a searchable event archive; being able to answer “what did we publish at 09:14?” ends most incidents.
• Make replay a supported operation, not a heroic one-off script.

When synchronous is the right answer

Events are not a universal upgrade. Stay synchronous when the caller genuinely needs the result to continue: authentication, payment authorisation, inventory checks that block the purchase, anything where the user must see a definitive yes or no on this screen.

The pragmatic architecture is usually both: a short synchronous critical path that establishes the fact, and asynchronous fan-out for everything that reacts to it.

Interview framing

“Checkout commits the order and writes OrderPlaced to an outbox in the same transaction; a relay publishes to a topic keyed by order id, so ordering holds per order. Inventory, shipping, notifications and analytics consume independently with their own offsets and dedup on event id. Delivery is at-least-once, so consumers are idempotent and failures dead-letter after five attempts. Checkout latency is now just the local commit, and the SLO is 95% of orders CONFIRMED within two seconds, alerted on consumer lag.”