Repository navigation
Conversation
A patient on a flaky mobile connection times out mid-request and taps retry.
The backend cannot tell that retry from a deliberate second attempt, so the
same tap books twice or funds twice. "Check then write" cannot fix it either —
two retries 20 ms apart both pass the check. This is the same problem
ARCHITECTURE.md §5.1 already solves for slots, one layer up.
The mechanism is the boring one: the unique constraint on (userId, key) IS
the lock. Whoever wins the INSERT ... ON CONFLICT DO NOTHING owns the attempt;
everyone else reads the row and either replays the stored response, is told
one is in flight, or is refused for reusing the key on a different request.
No in-memory lock anywhere — a Map of pending keys is correct on one instance,
silently broken on two, and emptied by exactly the restart a retry storm
follows.
Schema: IdempotencyKey (userId, key, method, route, requestHash,
responseStatus/body/headers, createdAt, expiresAt), unique (userId, key) plus
an expiresAt index. Generated with scripts/gen-migration.mjs (real diff, not
hand-written); scripts/check-migration-drift.mjs reports no drift.
Middleware (src/middleware/idempotency.ts), opt-in per route:
- same key + same request within 24h -> stored status/body replayed, with
Idempotency-Replayed: true; the handler does not run again
- same key + different body/route/target :id -> 422 IDEMPOTENCY_KEY_REUSED
- collides with a request still in flight -> 409 IN_PROGRESS + Retry-After
- malformed key -> 400 INVALID_IDEMPOTENCY_KEY, before anything runs
- 5xx is NEVER cached and the row is deleted: a server error is not a
result, so the client retries the same key and the request runs again
- an attempt left unfinished by a crashed process is taken over, but only by
an identical request — a stale row is never re-pointed at a new operation
- keys are namespaced per user: one account can never replay or read
another's stored body
Opted in: POST /appointments, POST /payments/intents, and
POST /payments/intents/:id/{fund,release,refund}. The header stays optional,
so existing clients are unaffected. This complements rather than replaces the
conditional status write in commitTransition — two different keys racing one
transition still resolve to a single winner and a 409.
Tests: 18 integration tests against a real Postgres (own embedded cluster,
port 5443) plus 3 unit tests. A gated handler makes "in progress" a fact the
test controls rather than a race it hopes to win; a genuine 8-way concurrent
burst proves the unique index decides ownership; the fake escrow SDK counts
calls so a retried fund/release/refund is proven to make exactly one chain
call; per-user scoping is proven by two accounts reusing one key string.
Docs: ARCHITECTURE.md §4.12 + §7, new §6.2 hard question (a route returning
clinical free text must not opt in until the export registry can account for
the cached copy), ARCHITECTURE_ESSENTIALS.md data-model map + known gaps,
docs/api/README.md (client requirements, status table, env knobs), ROADMAP.md
item + changelog, backend/.env.example.
Closes DocForum#48
|
@DevRushd Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
Contributor
|
@DevRushd Thanks for the PR. It can't be merged yet. To unblock it: Merge conflicts: this branch conflicts with Push to this same branch and CI will re-run. Once it's green and conflict-free, it'll be reviewed for merge. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #48
What Changed
Database schema
backend/prisma/schema.prisma— newIdempotencyKeymodel, mapping(key, user, route, request hash) → (status, response body):userId,key@@unique([userId, key])method,route/payments/intents/:id/fund), so a row is self-describing when debuggingrequestHashresponseStatus,responseBody,responseHeaderscreatedAt,expiresAt@@index([expiresAt])backs the prune jobDeliberately no foreign key on
userId: these are short-lived transport bookkeeping rows, and an FK would only make account deletion a slower, more coupled operation for a cache.backend/prisma/migrations/20261019113906_add_idempotency_key/migration.sql— generated bynode scripts/gen-migration.mjs add_idempotency_key(a realprisma migrate diffagainst a throwaway embedded Postgres, not hand-written). The unique index is created in the same migration as the table, so there is never a window in which rows can be written without it.node scripts/check-migration-drift.mjsreports no drift between the committed migrations and the schema.Middleware
backend/src/middleware/idempotency.ts(new) — cross-cutting, likecsrf.ts/request-id.ts, because two modules opt in and no module owns it.idempotency()canonicalJson/hashRequestisValidIdempotencyKeypruneIdempotencyKeys()Request flow:
INSERT … ON CONFLICT DO NOTHING→ count 1 means this request owns the attempt, count 0 means read the row and decide (replay /409 IN_PROGRESS/422).res.jsonis hooked so the outcome is stored before the bytes go out — the alternative loses the only durable record of what the client was told whenever the process dies in between, which is exactly the retry storm this exists for.backend/src/config/index.ts—config.idempotency.*:recordTtlHours(24),inProgressTimeoutSeconds(300),maxCachedBodyBytes(256 KiB),pruneIntervalSeconds.backend/src/server.ts—unref()'d prune interval for expired rows, same shape as the login-throttle prune (plain intervals, no queue — perARCHITECTURE_ESSENTIALS.md).Route opt-in (header stays optional — a request without one passes through untouched, so no existing client breaks)
backend/src/modules/appointments/routes/index.ts—idempotency()onPOST /appointments, afterauthenticate(the key is namespaced per user, so there must be a user).backend/src/modules/payments/routes/index.ts—idempotency()onPOST /payments/intentsandPOST /payments/intents/:id/{fund,release,refund}.201/200, stored for 24 hIdempotency-Replayed: true; the handler does not run again:id422{ code: 'IDEMPOTENCY_KEY_REUSED' }, nothing executed409{ code: 'IN_PROGRESS' }+Retry-After: 1400{ code: 'INVALID_IDEMPOTENCY_KEY' }, before anything runsDocumentation
ARCHITECTURE.md— new §4.12 (model + semantics + why), a §7 cross-cutting bullet, and a new §6.2.6 hard question: the cached body is a temporary copy of a response, non-PHI today, so a route returning clinical free text must not opt in until the §3.1 export registry can account for it.ARCHITECTURE_ESSENTIALS.md— data-model map line + a known-gap line mirroring §6.2.6.docs/api/README.md— new "Idempotency-Key" section (client requirements, full status table, env knobs), the header noted on thePOST /appointmentsrow, and a new Payments section — which also corrects the stale "Not yet implemented" list that still claimedpaymentshad no routes.ROADMAP.md— Phase 8 status line, the issue-48 item, and a changelog entry.backend/.env.example— the four new knobs.Why: the lock is the constraint, not a mutex
The failure this addresses is a race, and the two obvious fixes are both wrong:
Mapof pending keys): correct on one instance, silently broken on two — the second instance would run the duplicate — and emptied by exactly the restart a retry storm follows. The deployment is a single instance today, butLoginAttempt(issue [Phase 8] Login brute-force protection: per-account and per-IP throttling with backoff #35) was built for the same reason: state that must survive a restart and behave correctly on ten instances belongs in Postgres.SELECTthe key, then insert): two retries arriving 20 ms apart both see nothing and both run. This is precisely the class of bugARCHITECTURE.md§5.1 calls "the [Phase 2] IntakeForm model + structured symptom entry #1 predicted break" for slot booking, where the fix is a conditionalUPDATE … WHERE status = 'open'and exactly one caller can ever seecount === 1.So the ownership decision is pushed all the way down to
INSERT … ON CONFLICT DO NOTHINGagainst@@unique([userId, key]). Postgres decides, atomically, under concurrency; the application only reads what the constraint decided. This is the same division of labour as §5.1'sclaimOpenSlot, one layer up — and it is why the payments half matters: without it, a retriedfundpushes a second Stellar transaction, which is the mess issue #24's reconcile job would have to clean up.Secondary consequences of that choice, all deliberate:
@@unique([userId, key])+ every read filtered byuserIdmeans anIdempotency-Keyis never a bearer credential for someone else's cached body. A key is a guessable string, so treating it as one would be a data leak.UPDATE … WHERE responseStatus IS NULL AND createdAt = <read>. A stale row is never re-pointed at a different operation — "I reused my key" must not become "I refunded the wrong payment". Hash mismatch is checked before staleness for exactly this reason.:ids is a422, not a replay of the first record's response.Locationis stored inresponseHeaders, neverSet-Cookie— a replay must not re-issue a credential.payments'commitTransition: two different keys racing one transition still resolve to a single winner and a409.How it Was Tested
backend/tests/integration/idempotency.test.mts— 18 tests, self-contained embedded Postgres on its own port (5443), applied withprisma migrate deploy. Nothing here is provable against a mock, because the thing under test is the unique index deciding ownership between two concurrent writers. Run 4× consecutively with no flake.POST /appointments: same key + same body returns the stored status/body byte-for-byte withIdempotency-Replayed: true. Asserted that one request ran, not two: 1Appointment, 1 audit event, slot claimed once. The stored row is checked directly (responseStatus,responseBody,route: '/appointments', 64-hexrequestHash, futureexpiresAt).POST /payments/intents: exactly 1PaymentIntentrow for the order after the retry.status: 'open'and holds 0 appointments.:id: the same key aimed atfundon intent B after funding intent A is refused, and B is stillcreatedwithsdkCalls.createEscrowunchanged — refused before any chain call.409 IN_PROGRESS: a handler held open by a gate makes "one request is running" a fact the test controls rather than a race it hopes to win. While the winner is inside the handler: the duplicate gets409 { code: 'IN_PROGRESS' }+Retry-After, and a different body under the same key gets422— a colliding key can never redirect the running request. After the gate opens, the retry replays and the handler call count is still 1.Promise.all, one key) on bothPOST /appointmentsandPOST /payments/intents. Asserted: exactly 1IdempotencyKeyrow, exactly 1Appointment/ 1PaymentIntent(an intent has no natural uniqueness — without the key these eight would create eight intents, which is the duplicate charge [Phase 8] Idempotency-Key support for booking and payment mutations #48 exists to stop), every loser either409 IN_PROGRESSor a replay of the winner's exact body, and all successful bodies collapse to one id.(userId, key), notkey), each retrying into their own response, with each replayed body asserted not to contain the other's appointment id. Plus a cross-role case: a patient books under a key, an admin reuses that exact key string on a different route, gets their own201, and the patient's stored row is still intact and still theirs.createEscrow/release/refund. Retriedfund→ still 1 call; retriedrelease→ still 1; retriedrefund→ still 1; replaying a cached4xx→ still 1. This is the strongest available proof of the payments half.500, and the row is gone (findUnique→null) — the client's retry with the same key runs the handler again.responseStatus: nullaged an hour is taken over by an identical retry and completed — but the same key with a different body is still422, proving a stale row is never re-pointed.expiresAt; a just-created row survives.GET /appointments/minewith anIdempotency-Keyheader returns200and creates no row.backend/tests/unit/idempotency.test.ts— 3 unit tests over the pure helpers: key acceptance/refusal (including control characters and non-ASCII), canonical JSON being key-order independent but array-order significant andundefined≠null, and the hash separating method / path / body.Local verification, all green:
npm run audit:prod,npm run prisma:generate,npm run typecheck,npm test(96 unit),npm run test:integration(175 across 9 files, including the pre-existing booking-concurrency and audit suites — unchanged),npm run build, plusnode scripts/check-migration-drift.mjs.