fix(redis): reserve idempotency keys atomically with SET NX - #906
Merged
Merged
Conversation
… reserve keys through it IdempotencyStore.TryReserveAsync now delegates to the repository, passing the current timestamp and the logical-expiry cutoff. All first-party providers implement the new member by composing ExistsAsync and StoreAsync, which keeps their current non-atomic behaviour. Refs #816
…re-and-set RedisIdempotencyKeyRepository.TryStoreAsync now returns the SET NX result instead of composing ExistsAsync and StoreAsync. A logically expired key that is still physically present is replaced through a WATCH/MULTI transaction conditioned on the observed value, so exactly one concurrent reservation wins and the refreshed timestamp rejects later duplicates. Without a TimeToLive an existing key is never reserved again. Closes #816
…he shared base Reservation and TTL tests now live in IdempotencyTestsBase. The concurrency tests and the refresh assertion only run for providers that reserve keys atomically (currently Redis); other providers keep the non-atomic behaviour tracked in #814.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #906 +/- ##
==========================================
+ Coverage 95.81% 95.84% +0.02%
==========================================
Files 275 275
Lines 11879 11879
Branches 1122 1122
==========================================
+ Hits 11382 11385 +3
+ Misses 278 275 -3
Partials 219 219 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Resolve the conflicts with #909 in favour of its IIdempotencyKeyRepository.TryReserveAsync: drop TryStoreAsync and the WATCH/MULTI compare-and-set, since the Redis provider on main already reserves atomically with SET NX (no TTL) or a single Lua script that refreshes expired keys. Keep the concurrency tests, drop the reservation test duplicated by #909, and point the interceptor remarks at IIdempotencyKeyRepository.TryReserveAsync.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AddRedisIdempotencyStorewas documented as atomicSET NX, but reservation used to go through the non-atomicExistsAsync+StoreAsyncdefault. Concurrent submissions of the same key all gottruefromTryReserveAsync, so the handler ran once per submission.The production fix for this is now on
main: #909 addedIIdempotencyKeyRepository.TryReserveAsync, and the Redis implementation of it is a single atomic server operation. It usesSET NXwhen noTimeToLiveis set, which keeps the #790 behavior (nullTTL means never expire). When aTimeToLiveis set, it uses one Lua script that reserves an absent key or refreshes an expired one. After the merge withmain, this PR adds the concurrency regression tests that prove #816 is fixed, plus one doc correction.Closes #816
Changes
main(feat(idempotency): refresh expired idempotency keys atomically on reserve and store #909) and resolved the conflicts in favor of feat(idempotency): refresh expired idempotency keys atomically on reserve and store #909's design:IIdempotencyKeyRepository.TryStoreAsync. It had the same signature and meaning as feat(idempotency): refresh expired idempotency keys atomically on reserve and store #909'sTryReserveAsync, and two abstract members doing the same job would break the pre-1.0 interface evolution ADR.WATCH/MULTIcompare-and-set path inRedisIdempotencyKeyRepository. The Lua script already refreshes expired keys atomically, without aGETplus transaction.IdempotencyStore, the Redis README and the builder XML docs are exactly as onmain.IdempotencyTestsBase: new shared integration tests, gated onSupportsAtomicReservation(currentlytrueonly for Redis):Should_Reserve_Once_When_Reserving_Same_Key_Concurrently: 20 parallel reservations, exactly one returnstrue.Should_Run_Handler_Once_When_Sending_Same_Command_Concurrently: 20 parallelSendAsynccalls, the handler runs once and the other 19 getIdempotencyConflictException.Should_Reserve_Expired_Key_Once_When_Reserving_Concurrently: after the TTL, 20 parallel re-reservations of the expired key, exactly one wins.Should_Reserve_New_Key_OnceandShould_Never_Reserve_Existing_Key_Again_When_TimeToLive_Is_Null(fix: Honor nullTimeToLive("never expire") instead of a 24h physical expiry inNetEvolve.Pulse.Redis#790).Should_Reserve_Expired_Key_That_Is_Still_Physically_Present, because feat(idempotency): refresh expired idempotency keys atomically on reserve and store #909'sShould_Reject_Duplicate_After_Expired_Reservecovers it.IdempotencyCommandInterceptorremarks: they said that "the non-atomic default leaves a small window" onIIdempotencyStore.TryReserveAsync. They now say that atomicity depends onIIdempotencyKeyRepository.TryReserveAsync, to whichIdempotencyStoredelegates.Bug hunt
TryReserveAsynccalls for the same key all returntrueon RedistrueRedisIdempotencyTests.Should_Reserve_Once_When_Reserving_Same_Key_ConcurrentlySendAsynccalls of the same idempotent command run the handler more than onceRedisIdempotencyTests.Should_Run_Handler_Once_When_Sending_Same_Command_ConcurrentlyRedisIdempotencyTests.Should_Reserve_Expired_Key_Once_When_Reserving_ConcurrentlyTimeToLive = null, an existing key becomes reservable again*IdempotencyTests.Should_Never_Reserve_Existing_Key_Again_When_TimeToLive_Is_NullImpact
main. This PR changes only tests and one XML doc remark.TryStoreAsyncin all SQL/EF providers #907 (atomic reservation for SQL/EF) may already be covered by feat(idempotency): refresh expired idempotency keys atomically on reserve and store #909, which added atomicTryReserveAsyncimplementations for all providers. WideningSupportsAtomicReservationto those providers can happen there.Test evidence
dotnet build Pulse.slnx -c Release: 0 errors, 0 warnings.csharpier check .is clean.SQLite*Idempotency*: 30 passed, 6 skipped. The skipped ones are the gated concurrency tests.InMemory*Idempotency*: 15 passed, 3 skipped.