Conversation
…n start IdempotencyHandler kept a reference to the payload it hashes and BasePersistenceLayer recomputed the idempotency key and validation hash from it on every call. If the wrapped function mutated its input, saveInProgress() hashed the original payload while saveSuccess() and deleteRecord() hashed the mutated one, so completion and cleanup targeted a different key: the acquired record stayed INPROGRESS, an identical retry threw IdempotencyAlreadyInProgressError until it expired and could then re-run the side effect, failure cleanup deleted the wrong key, and with payload validation enabled an identical retry could fail validation. This affected makeIdempotent, the decorator, and the Middy middleware. Bind record identity to the operation instead of the payload. saveInProgress() now returns the record it acquired, whose key and payload hash IdempotencyHandler passes to saveSuccess() and deleteRecord(), which use that identity instead of hashing the payload again. The identity parameters are optional so direct callers and custom persistence layers are unaffected. Closes aws-powertools#5715
svozza
reviewed
Sep 15, 2026
…cord Address review feedback on the first version of the fix. Changing saveInProgress() to return the acquired record broke subclasses that override it with the previous Promise<void> signature, and compiled subclasses that discarded the return value silently handed the handler no identity. Capturing the identity from a successful acquisition also left the durable replay path uncovered: when saveInProgress() rejects with IdempotencyItemAlreadyExistsError during a replay the function still runs, so completion and cleanup fell back to hashing the mutated payload. Restore saveInProgress() to Promise<void> and add getRecordIdentity(), which resolves the idempotency key and payload hash from the payload. IdempotencyHandler resolves the identity before attempting acquisition and passes it to saveInProgress(), saveSuccess(), and deleteRecord(), so the replay path targets the same record and overrides that drop the optional identity argument keep hashing the not-yet-mutated payload. Add regressions for replay completion and cleanup with a mutated payload and for a subclass overriding saveInProgress() with a void return.
…tence overrides Add guidance to the "Bring your own persistent store" section: prefer the protected _getRecord, _putRecord, _updateRecord, and _deleteRecord hooks for custom storage, and when overriding saveInProgress(), saveSuccess(), or deleteRecord() accept and forward the optional identity argument to super so the base class guarantees do not depend on the caller leaving its payload unchanged.
Address review feedback: saveInProgress(), saveSuccess(), and deleteRecord() now take an optional trailing options object with the identity as a property instead of the identity itself, so future settings can be added without more positional arguments. The PersistenceOperationOptions type is exported. Update the docs guidance to recommend forwarding the whole options object unchanged to super, and add a test for an override that does so.
…dentity' into fix/5715-idempotency-bind-record-identity
svozza
reviewed
Sep 16, 2026
…sting Derive the durable context used by the replay regressions from DurableContext with the replay mode literal, so the tests no longer cast through unknown.
Extend the structural durable context type with an optional execution mode and use it in the registration and isReplay passthrough tests, so no test in the file casts through unknown.
svozza
approved these changes
Sep 17, 2026
Contributor
|
Great work @vishwakt, this turned out to be trickier than expect! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Changes
IdempotencyHandlerkeeps a reference to the payload it hashes, andBasePersistenceLayerrecomputed the idempotency key and validation hash from that payload on every call. If the wrapped function mutates its input (normalising a field, adding aprocessedflag),saveInProgress()hashes the original payload whilesaveSuccess()anddeleteRecord()hash the mutated one, so they target a different key. The in-progress record staysINPROGRESS, an identical retry throwsIdempotencyAlreadyInProgressErroruntil it expires and can then re-run the side effect, failure cleanup deletes the wrong key, and with payload validation enabled an identical retry can fail validation. This applies tomakeIdempotent, the decorator, and the Middy middleware, since all three pass the event by reference.This binds record identity to the operation rather than the payload, the first approach suggested in the issue:
BasePersistenceLayer.getRecordIdentity()resolves the idempotency key and payload hash for a payload.IdempotencyHandlercalls it before attempting acquisition, so the identity is available both when a record is acquired and when the function runs without acquiring one (durable replay on an existingINPROGRESSrecord).saveInProgress(),saveSuccess(), anddeleteRecord()accept an optional trailing options object,{ identity }, and use the identity instead of hashingdataagain. The options shape leaves room for future settings without more positional arguments. Without it they behave exactly as before, so direct callers and custom persistence layers keep working. Public signatures are otherwise unchanged:saveInProgress()still returnsPromise<void>, and existing overrides of all three methods still compile. An override ofsaveInProgress()that forwards only the original arguments still lands on the right record, because the payload has not been mutated at that point. An override ofsaveSuccess(data, result)ordeleteRecord(data)that forwards only the original arguments discards the identity and can still hash a mutated payload, so the docs now recommend accepting and forwarding the whole options object unchanged in public-method overrides, and using the protected_putRecord/_updateRecord/_deleteRecordhooks for custom storage, which need no changes.BasePersistenceLayerInterfaceis updated to match, and theIdempotencyRecordIdentityandPersistenceOperationOptionstypes are exported.Only the post-acquisition calls need the identity:
getRecord()andprocessExistingRecord()run in the replay branch before the wrapped function executes, so the payload cannot have mutated yet.Regression tests assert that completion (
_updateRecord) and cleanup (_deleteRecord) carry the sameidempotencyKeyas acquisition (_putRecord) when the wrapped function mutates its input, that the storedpayloadHashmatches when a validated field is mutated, that the same holds during a durable replay where_putRecordrejects withIdempotencyItemAlreadyExistsError, and that a subclass overridingsaveInProgress()asPromise<void>compiles and completes the right record. The mutation and replay tests fail onmainwith a different key or hash, and the override test fails to compile against the first revision of this PR. Unit tests forgetRecordIdentity()and for the identity argument on each of the three methods are included. The idempotency unit suite passes (151 tests) with the 100% coverage gate.Note on #5708
This is the direction the issue describes as replacing the prefix re-application in #5708: with the fully resolved key captured before acquisition,
saveSuccess()anddeleteRecord()no longer depend onidempotencyKeyPrefixbeing re-applied before each call. Both PRs touchIdempotencyHandler, so whichever lands second needs a small rebase. Happy to rebase either way, or to fold theconfigure()prefix reset from #5708 in here if that is preferred.Issue number: closes #5715
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.