Skip to content

fix(idempotency): complete and delete the record acquired at operation start - #5717

Merged
svozza merged 9 commits into
aws-powertools:mainfrom
vishwakt:fix/5715-idempotency-bind-record-identity
Sep 17, 2026
Merged

svozza merged 9 commits into
aws-powertools:mainfrom
vishwakt:fix/5715-idempotency-bind-record-identity

Conversation

@vishwakt

@vishwakt vishwakt commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Changes

IdempotencyHandler keeps a reference to the payload it hashes, and BasePersistenceLayer recomputed the idempotency key and validation hash from that payload on every call. If the wrapped function mutates its input (normalising a field, adding a processed flag), saveInProgress() hashes the original payload while saveSuccess() and deleteRecord() hash the mutated one, so they target a different key. The in-progress record stays INPROGRESS, an identical retry throws IdempotencyAlreadyInProgressError until it expires and can then re-run the side effect, failure cleanup deletes the wrong key, and with payload validation enabled an identical retry can fail validation. This applies to makeIdempotent, the decorator, and the Middy middleware, since all three pass the event by reference.

This binds record identity to the operation rather than the payload, the first approach suggested in the issue:

  • BasePersistenceLayer.getRecordIdentity() resolves the idempotency key and payload hash for a payload. IdempotencyHandler calls it before attempting acquisition, so the identity is available both when a record is acquired and when the function runs without acquiring one (durable replay on an existing INPROGRESS record).
  • saveInProgress(), saveSuccess(), and deleteRecord() accept an optional trailing options object, { identity }, and use the identity instead of hashing data again. The options shape leaves room for future settings without more positional arguments. Without it they behave exactly as before, so direct callers and custom persistence layers keep working. Public signatures are otherwise unchanged: saveInProgress() still returns Promise<void>, and existing overrides of all three methods still compile. An override of saveInProgress() that forwards only the original arguments still lands on the right record, because the payload has not been mutated at that point. An override of saveSuccess(data, result) or deleteRecord(data) that forwards only the original arguments discards the identity and can still hash a mutated payload, so the docs now recommend accepting and forwarding the whole options object unchanged in public-method overrides, and using the protected _putRecord / _updateRecord / _deleteRecord hooks for custom storage, which need no changes. BasePersistenceLayerInterface is updated to match, and the IdempotencyRecordIdentity and PersistenceOperationOptions types are exported.

Only the post-acquisition calls need the identity: getRecord() and processExistingRecord() run in the replay branch before the wrapped function executes, so the payload cannot have mutated yet.

Regression tests assert that completion (_updateRecord) and cleanup (_deleteRecord) carry the same idempotencyKey as acquisition (_putRecord) when the wrapped function mutates its input, that the stored payloadHash matches when a validated field is mutated, that the same holds during a durable replay where _putRecord rejects with IdempotencyItemAlreadyExistsError, and that a subclass overriding saveInProgress() as Promise<void> compiles and completes the right record. The mutation and replay tests fail on main with a different key or hash, and the override test fails to compile against the first revision of this PR. Unit tests for getRecordIdentity() and for the identity argument on each of the three methods are included. The idempotency unit suite passes (151 tests) with the 100% coverage gate.

Note on #5708

This is the direction the issue describes as replacing the prefix re-application in #5708: with the fully resolved key captured before acquisition, saveSuccess() and deleteRecord() no longer depend on idempotencyKeyPrefix being re-applied before each call. Both PRs touch IdempotencyHandler, so whichever lands second needs a small rebase. Happy to rebase either way, or to fold the configure() prefix reset from #5708 in here if that is preferred.

Issue number: closes #5715


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…n start

IdempotencyHandler kept a reference to the payload it hashes and
BasePersistenceLayer recomputed the idempotency key and validation hash from
it on every call. If the wrapped function mutated its input, saveInProgress()
hashed the original payload while saveSuccess() and deleteRecord() hashed the
mutated one, so completion and cleanup targeted a different key: the acquired
record stayed INPROGRESS, an identical retry threw
IdempotencyAlreadyInProgressError until it expired and could then re-run the
side effect, failure cleanup deleted the wrong key, and with payload
validation enabled an identical retry could fail validation. This affected
makeIdempotent, the decorator, and the Middy middleware.

Bind record identity to the operation instead of the payload. saveInProgress()
now returns the record it acquired, whose key and payload hash IdempotencyHandler
passes to saveSuccess() and deleteRecord(), which use that identity instead of
hashing the payload again. The identity parameters are optional so direct
callers and custom persistence layers are unaffected.

Closes aws-powertools#5715
Comment thread packages/idempotency/src/persistence/BasePersistenceLayer.ts Outdated
Comment thread packages/idempotency/src/IdempotencyHandler.ts Outdated
vishwakt and others added 5 commits September 15, 2026 08:46
…cord

Address review feedback on the first version of the fix.

Changing saveInProgress() to return the acquired record broke subclasses
that override it with the previous Promise<void> signature, and compiled
subclasses that discarded the return value silently handed the handler
no identity. Capturing the identity from a successful acquisition also
left the durable replay path uncovered: when saveInProgress() rejects
with IdempotencyItemAlreadyExistsError during a replay the function still
runs, so completion and cleanup fell back to hashing the mutated payload.

Restore saveInProgress() to Promise<void> and add getRecordIdentity(),
which resolves the idempotency key and payload hash from the payload.
IdempotencyHandler resolves the identity before attempting acquisition
and passes it to saveInProgress(), saveSuccess(), and deleteRecord(), so
the replay path targets the same record and overrides that drop the
optional identity argument keep hashing the not-yet-mutated payload.

Add regressions for replay completion and cleanup with a mutated payload
and for a subclass overriding saveInProgress() with a void return.
…tence overrides

Add guidance to the "Bring your own persistent store" section: prefer the
protected _getRecord, _putRecord, _updateRecord, and _deleteRecord hooks for
custom storage, and when overriding saveInProgress(), saveSuccess(), or
deleteRecord() accept and forward the optional identity argument to super so
the base class guarantees do not depend on the caller leaving its payload
unchanged.
Address review feedback: saveInProgress(), saveSuccess(), and deleteRecord()
now take an optional trailing options object with the identity as a
property instead of the identity itself, so future settings can be added
without more positional arguments. The PersistenceOperationOptions type is
exported.

Update the docs guidance to recommend forwarding the whole options object
unchanged to super, and add a test for an override that does so.
…dentity' into fix/5715-idempotency-bind-record-identity
Comment thread packages/idempotency/tests/unit/makeIdempotent.test.ts Outdated
vishwakt and others added 3 commits September 16, 2026 11:47
…sting

Derive the durable context used by the replay regressions from DurableContext with the replay mode literal, so the tests no longer cast through unknown.
Extend the structural durable context type with an optional execution mode and use it in the registration and isReplay passthrough tests, so no test in the file casts through unknown.
@svozza

svozza commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Great work @vishwakt, this turned out to be trickier than expect!

@svozza
svozza merged commit daa6e7c into aws-powertools:main Sep 17, 2026
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L PRs between 100-499 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Idempotency completes or deletes the wrong record when the wrapped function mutates its input

2 participants