Skip to content

feat(api): idempotency-Key support for booking and payment mutations - #162

Open
DevRushd wants to merge 1 commit into
DocForum:mainfrom
DevRushd:feat/idempotency-key-support-for-booking-and-payment
Open

DevRushd wants to merge 1 commit into
DocForum:mainfrom
DevRushd:feat/idempotency-key-support-for-booking-and-payment

Conversation

@DevRushd

Copy link
Copy Markdown

Closes #48

What Changed

Database schema

  • backend/prisma/schema.prisma — new IdempotencyKey model, mapping (key, user, route, request hash) → (status, response body):

    column purpose
    userId, key the scope and the concurrency mechanism — @@unique([userId, key])
    method, route the route pattern (/payments/intents/:id/fund), so a row is self-describing when debugging
    requestHash SHA-256 over method + concrete path + canonical JSON body
    responseStatus, responseBody, responseHeaders nullable — NULL while the attempt is in flight, which is the gap a concurrent duplicate sees
    createdAt, expiresAt 24 h replay window by default; @@index([expiresAt]) backs the prune job

    Deliberately no foreign key on userId: these are short-lived transport bookkeeping rows, and an FK would only make account deletion a slower, more coupled operation for a cache.

  • backend/prisma/migrations/20261019113906_add_idempotency_key/migration.sql — generated by node scripts/gen-migration.mjs add_idempotency_key (a real prisma migrate diff against a throwaway embedded Postgres, not hand-written). The unique index is created in the same migration as the table, so there is never a window in which rows can be written without it. node scripts/check-migration-drift.mjs reports no drift between the committed migrations and the schema.

Middleware

  • backend/src/middleware/idempotency.ts (new) — cross-cutting, like csrf.ts / request-id.ts, because two modules opt in and no module owns it.

    exported purpose
    idempotency() Express middleware factory, mounted per route
    canonicalJson / hashRequest order-independent body hashing (pure, unit-tested)
    isValidIdempotencyKey 1–255 printable-ASCII, bounded
    pruneIdempotencyKeys() TTL cleanup, called on an interval

    Request flow: INSERT … ON CONFLICT DO NOTHING → count 1 means this request owns the attempt, count 0 means read the row and decide (replay / 409 IN_PROGRESS / 422). res.json is hooked so the outcome is stored before the bytes go out — the alternative loses the only durable record of what the client was told whenever the process dies in between, which is exactly the retry storm this exists for.

  • backend/src/config/index.ts — config.idempotency.*: recordTtlHours (24), inProgressTimeoutSeconds (300), maxCachedBodyBytes (256 KiB), pruneIntervalSeconds.

  • backend/src/server.ts — unref()'d prune interval for expired rows, same shape as the login-throttle prune (plain intervals, no queue — per ARCHITECTURE_ESSENTIALS.md).

Route opt-in (header stays optional — a request without one passes through untouched, so no existing client breaks)

  • backend/src/modules/appointments/routes/index.ts — idempotency() on POST /appointments, after authenticate (the key is namespaced per user, so there must be a user).
  • backend/src/modules/payments/routes/index.ts — idempotency() on POST /payments/intents and POST /payments/intents/:id/{fund,release,refund}.
Situation Response
Same key, same request, first attempt normal 201/200, stored for 24 h
Same key, same request, retried within 24 h the stored status + body, plus Idempotency-Replayed: true; the handler does not run again
Same key, different body / route / target :id 422 { code: 'IDEMPOTENCY_KEY_REUSED' }, nothing executed
Same key while the first is still running 409 { code: 'IN_PROGRESS' } + Retry-After: 1
Attempt failed with 5xx row deleted, nothing cached — retry the same key and it runs again
Malformed key 400 { code: 'INVALID_IDEMPOTENCY_KEY' }, before anything runs

Documentation

  • ARCHITECTURE.md — new §4.12 (model + semantics + why), a §7 cross-cutting bullet, and a new §6.2.6 hard question: the cached body is a temporary copy of a response, non-PHI today, so a route returning clinical free text must not opt in until the §3.1 export registry can account for it.
  • ARCHITECTURE_ESSENTIALS.md — data-model map line + a known-gap line mirroring §6.2.6.
  • docs/api/README.md — new "Idempotency-Key" section (client requirements, full status table, env knobs), the header noted on the POST /appointments row, and a new Payments section — which also corrects the stale "Not yet implemented" list that still claimed payments had no routes.
  • ROADMAP.md — Phase 8 status line, the issue-48 item, and a changelog entry. backend/.env.example — the four new knobs.

Why: the lock is the constraint, not a mutex

The failure this addresses is a race, and the two obvious fixes are both wrong:

  • In-memory lock (Map of pending keys): correct on one instance, silently broken on two — the second instance would run the duplicate — and emptied by exactly the restart a retry storm follows. The deployment is a single instance today, but LoginAttempt (issue [Phase 8] Login brute-force protection: per-account and per-IP throttling with backoff #35) was built for the same reason: state that must survive a restart and behave correctly on ten instances belongs in Postgres.
  • Check-then-write (SELECT the key, then insert): two retries arriving 20 ms apart both see nothing and both run. This is precisely the class of bug ARCHITECTURE.md §5.1 calls "the [Phase 2] IntakeForm model + structured symptom entry #1 predicted break" for slot booking, where the fix is a conditional UPDATE … WHERE status = 'open' and exactly one caller can ever see count === 1.

So the ownership decision is pushed all the way down to INSERT … ON CONFLICT DO NOTHING against @@unique([userId, key]). Postgres decides, atomically, under concurrency; the application only reads what the constraint decided. This is the same division of labour as §5.1's claimOpenSlot, one layer up — and it is why the payments half matters: without it, a retried fund pushes a second Stellar transaction, which is the mess issue #24's reconcile job would have to clean up.

Secondary consequences of that choice, all deliberate:

  • Per-user scoping. @@unique([userId, key]) + every read filtered by userId means an Idempotency-Key is never a bearer credential for someone else's cached body. A key is a guessable string, so treating it as one would be a data leak.
  • 5xx is never cached. A server error is not a result — the request may or may not have had side effects. Caching one would turn a transient failure into a permanent answer for that key, so the row is deleted and the client's retry actually re-runs.
  • Takeover is hash-matched. An attempt left unfinished by a crashed process is taken over only by an identical request, via a conditional UPDATE … WHERE responseStatus IS NULL AND createdAt = <read>. A stale row is never re-pointed at a different operation — "I reused my key" must not become "I refunded the wrong payment". Hash mismatch is checked before staleness for exactly this reason.
  • The concrete path is in the hash, not just the route pattern, so one key aimed at two different :ids is a 422, not a replay of the first record's response.
  • Only Location is stored in responseHeaders, never Set-Cookie — a replay must not re-issue a credential.
  • This complements rather than replaces the conditional status write in payments' commitTransition: two different keys racing one transition still resolve to a single winner and a 409.

How it Was Tested

backend/tests/integration/idempotency.test.mts — 18 tests, self-contained embedded Postgres on its own port (5443), applied with prisma migrate deploy. Nothing here is provable against a mock, because the thing under test is the unique index deciding ownership between two concurrent writers. Run 4× consecutively with no flake.

  • Replay — POST /appointments: same key + same body returns the stored status/body byte-for-byte with Idempotency-Replayed: true. Asserted that one request ran, not two: 1 Appointment, 1 audit event, slot claimed once. The stored row is checked directly (responseStatus, responseBody, route: '/appointments', 64-hex requestHash, future expiresAt).
  • Replay — POST /payments/intents: exactly 1 PaymentIntent row for the order after the retry.
  • 422 on a different body: the refused request never reached the handler — the target slot is still status: 'open' and holds 0 appointments.
  • 422 on a different target :id: the same key aimed at fund on intent B after funding intent A is refused, and B is still created with sdkCalls.createEscrow unchanged — refused before any chain call.
  • Deterministic 409 IN_PROGRESS: a handler held open by a gate makes "one request is running" a fact the test controls rather than a race it hopes to win. While the winner is inside the handler: the duplicate gets 409 { code: 'IN_PROGRESS' } + Retry-After, and a different body under the same key gets 422 — a colliding key can never redirect the running request. After the gate opens, the retry replays and the handler call count is still 1.
  • Simultaneous race: 8 genuinely concurrent requests (one Promise.all, one key) on both POST /appointments and POST /payments/intents. Asserted: exactly 1 IdempotencyKey row, exactly 1 Appointment / 1 PaymentIntent (an intent has no natural uniqueness — without the key these eight would create eight intents, which is the duplicate charge [Phase 8] Idempotency-Key support for booking and payment mutations #48 exists to stop), every loser either 409 IN_PROGRESS or a replay of the winner's exact body, and all successful bodies collapse to one id.
  • Per-user scoping: two patients pick the same key string → 2 rows (the index is on (userId, key), not key), each retrying into their own response, with each replayed body asserted not to contain the other's appointment id. Plus a cross-role case: a patient books under a key, an admin reuses that exact key string on a different route, gets their own 201, and the patient's stored row is still intact and still theirs.
  • Payments: no second chain call — the fake escrow SDK counts createEscrow / release / refund. Retried fund → still 1 call; retried release → still 1; retried refund → still 1; replaying a cached 4xx → still 1. This is the strongest available proof of the payments half.
  • 5xx is not cached: a route that throws reaches the shared error handler, returns 500, and the row is gone (findUnique → null) — the client's retry with the same key runs the handler again.
  • Oversized responses are executed but not cached (row deleted, retry re-executes) rather than replaying a truncated body.
  • Crash recovery: a pre-seeded row with responseStatus: null aged an hour is taken over by an identical retry and completed — but the same key with a different body is still 422, proving a stale row is never re-pointed.
  • TTL expiry: a 48-hour-old stored response is not replayed; the request executes fresh.
  • Prune removes only rows past their expiresAt; a just-created row survives.
  • Unopted-in routes untouched: GET /appointments/mine with an Idempotency-Key header returns 200 and creates no row.

backend/tests/unit/idempotency.test.ts — 3 unit tests over the pure helpers: key acceptance/refusal (including control characters and non-ASCII), canonical JSON being key-order independent but array-order significant and undefined ≠ null, and the hash separating method / path / body.

Local verification, all green: npm run audit:prod, npm run prisma:generate, npm run typecheck, npm test (96 unit), npm run test:integration (175 across 9 files, including the pre-existing booking-concurrency and audit suites — unchanged), npm run build, plus node scripts/check-migration-drift.mjs.

A patient on a flaky mobile connection times out mid-request and taps retry.
The backend cannot tell that retry from a deliberate second attempt, so the
same tap books twice or funds twice. "Check then write" cannot fix it either —
two retries 20 ms apart both pass the check. This is the same problem
ARCHITECTURE.md §5.1 already solves for slots, one layer up.

The mechanism is the boring one: the unique constraint on (userId, key) IS
the lock. Whoever wins the INSERT ... ON CONFLICT DO NOTHING owns the attempt;
everyone else reads the row and either replays the stored response, is told
one is in flight, or is refused for reusing the key on a different request.
No in-memory lock anywhere — a Map of pending keys is correct on one instance,
silently broken on two, and emptied by exactly the restart a retry storm
follows.

Schema: IdempotencyKey (userId, key, method, route, requestHash,
responseStatus/body/headers, createdAt, expiresAt), unique (userId, key) plus
an expiresAt index. Generated with scripts/gen-migration.mjs (real diff, not
hand-written); scripts/check-migration-drift.mjs reports no drift.

Middleware (src/middleware/idempotency.ts), opt-in per route:
  - same key + same request within 24h -> stored status/body replayed, with
    Idempotency-Replayed: true; the handler does not run again
  - same key + different body/route/target :id -> 422 IDEMPOTENCY_KEY_REUSED
  - collides with a request still in flight -> 409 IN_PROGRESS + Retry-After
  - malformed key -> 400 INVALID_IDEMPOTENCY_KEY, before anything runs
  - 5xx is NEVER cached and the row is deleted: a server error is not a
    result, so the client retries the same key and the request runs again
  - an attempt left unfinished by a crashed process is taken over, but only by
    an identical request — a stale row is never re-pointed at a new operation
  - keys are namespaced per user: one account can never replay or read
    another's stored body

Opted in: POST /appointments, POST /payments/intents, and
POST /payments/intents/:id/{fund,release,refund}. The header stays optional,
so existing clients are unaffected. This complements rather than replaces the
conditional status write in commitTransition — two different keys racing one
transition still resolve to a single winner and a 409.

Tests: 18 integration tests against a real Postgres (own embedded cluster,
port 5443) plus 3 unit tests. A gated handler makes "in progress" a fact the
test controls rather than a race it hopes to win; a genuine 8-way concurrent
burst proves the unique index decides ownership; the fake escrow SDK counts
calls so a retried fund/release/refund is proven to make exactly one chain
call; per-user scoping is proven by two accounts reusing one key string.

Docs: ARCHITECTURE.md §4.12 + §7, new §6.2 hard question (a route returning
clinical free text must not opt in until the export registry can account for
the cached copy), ARCHITECTURE_ESSENTIALS.md data-model map + known gaps,
docs/api/README.md (client requirements, status table, env knobs), ROADMAP.md
item + changelog, backend/.env.example.

Closes DocForum#48
@drips-wave

drips-wave Bot commented Oct 19, 2026

Copy link
Copy Markdown

@DevRushd Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@afeezorobsco-cyber

Copy link
Copy Markdown
Contributor

@DevRushd Thanks for the PR. It can't be merged yet. To unblock it:

Merge conflicts: this branch conflicts with main. Please rebase onto the latest main (git fetch upstream && git rebase upstream/main), resolve the conflicts (often ROADMAP.md, ARCHITECTURE*.md or prisma/schema.prisma), and force-push.

Push to this same branch and CI will re-run. Once it's green and conflict-free, it'll be reviewed for merge.

@github-actions github-actions Bot added the needs-rebase Has merge conflicts with main; the author needs to rebase label Oct 20, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-rebase Has merge conflicts with main; the author needs to rebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Phase 8] Idempotency-Key support for booking and payment mutations

2 participants