Idempotency: Why the Same Request Twice Must Charge Once

Timeouts are unknowns

When a request times out, the client learns exactly one thing: it did not get a response. It does not learn whether the server received the request, whether it processed it, or whether the response was lost on the way back. Those three cases are indistinguishable from outside.

Clients — browsers, mobile SDKs, load balancers, message brokers, your own retry wrapper — respond to unknowns by retrying. That is correct behaviour. The server is what has to change.

What idempotent actually means

An operation is idempotent if performing it N times has the same effect as performing it once. Note what this does not say: it does not say the operation is read-only, and it does not say retries are free.

Prefer absolute writes over relative ones wherever the domain allows it. Where it does not — payments, order creation, seat booking — you need an explicit key.

Idempotency keys, done properly

1. The client generates the key, once, before the first attempt — a UUID, or a deterministic value like the cart id. A server-generated key defeats the purpose.
2. The key travels in a header: Idempotency-Key: 6f2c…. Keep it out of the body so proxies and logs can see it.
3. The same key must be reused for every retry of that logical action, and never reused for a different one.
4. The server stores the outcome against the key and replays it verbatim, including the status code.

The concurrent duplicate case

The naive implementation — check if the key exists, then process — has a race. A double-tap sends two requests 30ms apart; both check, both find nothing, both charge. The check and the claim must be a single atomic operation.

CREATE TABLE idempotency_keys (
  key            TEXT PRIMARY KEY,
  request_hash   TEXT NOT NULL,
  status         TEXT NOT NULL,       -- 'in_progress' | 'completed'
  response_code  INT,
  response_body  JSONB,
  created_at     TIMESTAMPTZ NOT NULL DEFAULT now(),
  expires_at     TIMESTAMPTZ NOT NULL
);

• Insert with status in_progress. A duplicate insert fails on the primary key.
• If the existing row is completed → replay the stored response.
• If it is in_progress → return 409 Conflict and let the client retry shortly. Do not block a worker waiting.

Implementation sketch

async function withIdempotency(req, handler) {
  const key = req.headers["idempotency-key"];
  if (!key) return handler(req);                    // or reject, for money endpoints

  const hash = sha256(canonicalise(req.body));

  const claimed = await db.query(`
    INSERT INTO idempotency_keys (key, request_hash, status, expires_at)
    VALUES ($1, $2, 'in_progress', now() + interval '24 hours')
    ON CONFLICT (key) DO NOTHING
    RETURNING key`, [key, hash]);

  if (claimed.rowCount === 0) {
    const row = await db.one("SELECT * FROM idempotency_keys WHERE key = $1", [key]);
    if (row.request_hash !== hash) return json(422, { error: "key_reused_with_different_payload" });
    if (row.status === "in_progress") return json(409, { error: "request_in_progress" });
    return json(row.response_code, row.response_body);   // exact replay
  }

  const res = await handler(req);
  await db.query(`UPDATE idempotency_keys
     SET status='completed', response_code=$2, response_body=$3 WHERE key=$1`,
    [key, res.status, res.body]);
  return res;
}

Where possible, write the key row and the business effect in the same transaction. Otherwise a crash between the charge and the key update leaves a key marked in-progress forever — which is why the reconciliation job below matters.

Scope, lifetime and fingerprinting

• Scope keys per account and endpoint. A global namespace lets one tenant's key collide with another's.
• Store a hash of the request body. Same key with a different amount is a client bug — reject it with 422 rather than silently replaying the wrong result.
• Expire keys. Twenty-four hours is the industry norm; long enough for every realistic retry, short enough that the table stays small.
• Reconcile stuck rows. A job that finds in_progress rows older than a few minutes and checks the downstream provider for the real outcome.

Beyond HTTP: consumers and jobs

Every message broker worth using delivers at-least-once, which means your consumers face the identical problem. The same mechanism applies: insert the message id into aprocessed_events table inside the transaction that performs the side effect. If the insert conflicts, you have already handled that message — acknowledge and move on.

Scheduled jobs need it too. A cron that fires twice because of a leader election should not send the invoice twice; key it on the period it is processing.

Interview framing

“Payment creation requires an Idempotency-Key header generated client-side. We claim the key with an INSERT … ON CONFLICT DO NOTHING, so concurrent duplicates are resolved by the unique constraint rather than a read-then-write race. Completed keys replay the stored response verbatim; in-progress ones return 409. Keys are scoped per merchant, hashed against the request body to catch misuse, and expire after 24 hours.” Mention the concurrent case unprompted — that is what separates a real answer from a memorised one.