feat(idempotency): refresh expired idempotency keys atomically on reserve and store

[samtrion]

Summary

With IdempotencyKeyOptions.TimeToLive set, a key that had expired and was then reserved again kept its old CreatedAt. ExistsAsync kept reporting it as absent, so every later duplicate of that command ran the handler again. This happened in every provider.

Reserving and storing now go through one atomic repository operation. It inserts an absent key or refreshes the timestamp of an expired one, and it leaves a key that has not expired untouched.

Closes #814

Changes

• Extensibility: new IIdempotencyKeyRepository.TryReserveAsync(key, createdAt, validFrom, ct), with no default implementation. It returns true when the key was inserted or an expired key was refreshed.
• IdempotencyStore: overrides TryReserveAsync and routes StoreAsync through it with the TTL cutoff. When TimeToLive is null, the cutoff is null and an existing key is never modified.
• SQL Server: new usp_ReserveIdempotencyKey, using MERGE ... WITH (HOLDLOCK) with WHEN MATCHED AND CreatedAt < @validFrom THEN UPDATE and returning @@ROWCOUNT > 0. It has a documented header, a NULL guard for @idempotencyKey/@createdAt (THROW 50000) and a TRY/CATCH that rethrows.
• PostgreSQL: new fn_reserve_idempotency_key(...) RETURNS BOOLEAN, using ON CONFLICT ("idempotency_key") DO UPDATE ... WHERE created_at < p_valid_from. It is a new function because CREATE OR REPLACE cannot change the VOID return type of fn_insert_idempotency_key.
• SQLite: ON CONFLICT ... DO UPDATE ... WHERE. The result is the affected-row count.
• MySQL: INSERT IGNORE, then a conditional UPDATE ... WHERE CreatedAt < @validFromTicks. ON DUPLICATE KEY UPDATE was rejected on purpose: with MySql.Data's default found-rows mode it reports 1 for both an insert and an untouched duplicate, so the result cannot be told apart. When the UPDATE matches no row, INSERT IGNORE runs once more, because a cleanup job (for example usp_DeleteExpiredIdempotencyKeys-style maintenance) can delete the expired row between the two statements.
• Entity Framework Core:
  • The expired row is removed with ExecuteDeleteAsync, then the existing insert path runs. A primary key conflict returns false.
  • ExecuteUpdateAsync is not used, because the Oracle MySQL provider cannot bind converted DateTimeOffset setters (see MySqlOutboxRepositoryExecutor).
  • InMemory uses change tracking instead.
  • A tracked expired entity is detached first, so a re-reserve in the same scope works.
• Redis:
  • Without a cutoff, SET NX as before, so fix: Honor null TimeToLive ("never expire") instead of a 24h physical expiry in NetEvolve.Pulse.Redis #790 behaviour is kept: a null TTL means no expiry.
  • With a cutoff, a Lua script overwrites an expired value and resets PX. It compares UTC round-trip timestamps as text. Values that are not timestamps, or that carry a non-UTC offset (only possible from earlier versions through direct StoreAsync calls), are treated as present, so a live key is never overwritten.
  • README, builder XML docs and repository docs describe the SET NX / Lua split and the scripting requirement.
• Scripts and READMEs: SQL Server and PostgreSQL IdempotencyKey.sql stay re-runnable. Upgrade notes are in each provider README.
• Docs: IdempotencyKeyOptions.TimeToLive remarks describe the refresh on store/reserve; IIdempotencyKeyRepository.StoreAsync remarks point implementers to TryReserveAsync.
• ADR: decisions/2026-09-28-idempotency-refresh-expired-keys-on-reserve.md, state proposed.

Bug hunt

Suspicion	Confirmed?	Test
An expired key that is reserved again keeps its old CreatedAt, so a second reservation is not rejected (#814)	Yes: failed for SQLite ADO.NET, SQLite EF and EF InMemory before the fix	IdempotencyTestsBase.Should_Reject_Duplicate_After_Expired_Reserve
Same, across fresh scopes/DbContexts (the EF path without the change tracker)	Yes: failed before the fix	Should_Reject_Expired_Reserve_Dup_Across_Scopes
StoreAsync on an expired key leaves ExistsAsync false, which breaks the StoreAsync contract	Yes: failed before the fix	Should_Report_Key_Present_After_Expired_Store
A reservation of a key that has not expired could refresh its CreatedAt and extend its window	No: stayed green, kept as a guard	Should_Keep_CreatedAt_When_Reserving_Live_Key
MySQL: a cleanup job deleting the expired row between INSERT IGNORE and the refresh UPDATE makes TryReserveAsync return false for an absent key (false duplicate)	Confirmed by code inspection; not deterministically reproducible (no seam between the two statements), so no red test. The retry branch runs on every live-duplicate reservation with a TTL, covered by the shared integration tests	Existing IdempotencyTestsBase TTL tests (MySQL, CI)
Redis: the Lua script compares timestamps as text, so a live value with a negative offset (e.g. 08:00-05:00 = 13:00Z against a 12:00Z cutoff) is overwritten and a real duplicate is accepted	Yes by analysis ("08" < "12"); the test was committed before the fix but could only run in CI (no local Docker)	RedisIdempotencyTests.Should_Keep_Live_Non_Utc_Value_On_Reserve (negative and positive offset cases)
SQL Server usp_ReserveIdempotencyKey fails with a raw constraint error for NULL parameters	Not testable through the provider: the C# ArgumentException guards stop NULL keys before the procedure; hardened per tsql.instructions.md	None (covered by the existing SQL Server integration tests in CI)
IdempotencyStore composes ExistsAsync + StoreAsync (not atomic, no refresh) instead of an atomic repository call	Yes: failed before the fix	IdempotencyStoreTests.TryReserveAsync_*, StoreAsync_WithTtl_RefreshesExpiredKeyThroughReserve

The integration test names are at most 51 characters, because the test table name is the method name and MySQL limits IX_<table>_CreatedAt to 64 characters.

Impact

• Interface addition for external implementers: IIdempotencyKeyRepository.TryReserveAsync(string, DateTimeOffset, DateTimeOffset?, CancellationToken) is a new member without a default implementation, per the accepted pre-1.0 interface evolution ADR. Custom repositories must implement it atomically.
• IIdempotencyKeyRepository.StoreAsync stays on the interface but is no longer called by IdempotencyStore.
• Redis: once a TimeToLive is set, reservation runs EVAL, so the Redis user needs the scripting commands (+@scripting). Values with a non-UTC offset written by earlier versions through direct StoreAsync calls stay unreservable until their physical expiry (keys stored without a TTL have none and must be deleted manually).
• Deployment: SQL Server and PostgreSQL users must re-run IdempotencyKey.sql together with the package upgrade, because the new stored procedure/function is required. The scripts are idempotent and keep existing data. MySQL, SQLite, EF Core and Redis need no schema change.
• Behaviour with TimeToLive = null is unchanged.

Test evidence

• dotnet build Pulse.slnx -c Release: 0 errors, 0 warnings.
• Unit tests, all TFMs (net8.0/net9.0/net10.0): 7284 passed, 0 failed (after merging main).
• Integration tests run locally, net10.0, --treenode-filter "/*/*/*Idempotency*/*":
  • SQLite ADO.NET, SQLite EF and EF InMemory: 39/39 passed, including the 4 new tests.
  • The same 3 re-reserve/re-store tests per provider failed on the test-only commit.
  • Not run locally: SQL Server, PostgreSQL, MySQL and Redis. The Docker daemon was unavailable (the Docker Desktop engine returned HTTP 500), so these are left to CI. The new tests are in the shared IdempotencyTestsBase and cover them; the fix: Refresh expired idempotency keys on re-store so duplicates are rejected again in all SQL/EF providers and NetEvolve.Pulse.Redis #814 acceptance criterion for these providers is met only once the CI integration run is green.
• csharpier check .: clean.

[codecov]

Codecov Report

❌ Patch coverage is 97.14286% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.77%. Comparing base (972fb55) to head (3ac9011).
⚠️ Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
...r/Idempotency/SqlServerIdempotencyKeyRepository.cs	91.66%	1 Missing and 1 partial ⚠️
.../Idempotency/PostgreSqlIdempotencyKeyRepository.cs	94.44%	0 Missing and 1 partial ⚠️
...Redis/Idempotency/RedisIdempotencyKeyRepository.cs	92.85%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #909      +/-   ##
==========================================
- Coverage   96.09%   95.77%   -0.33%     
==========================================
  Files         274      274              
  Lines       11511    11700     +189     
  Branches     1069     1097      +28     
==========================================
+ Hits        11062    11206     +144     
- Misses        232      274      +42     
- Partials      217      220       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.