fix(idempotency): store and compare idempotency keys exactly on SQL Server and MySQL

[samtrion]

Summary

Idempotency keys are now stored and compared exactly on SQL Server and MySQL.

• The maximum key length is 450 characters for every provider.
• A key longer than that is rejected. It is never truncated.
• The SQL Server and MySQL key columns use binary collations, so keys that differ only by case are different keys.

Closes #850
Closes #858

Changes

• Core (IdempotencyKeySchema, IdempotencyStore)
  • MaxLengths.IdempotencyKey drops from 500 to 450.
    • CREATE INDEX, Index key size: "The maximum size for an index key is 900 bytes for a clustered index and 1,700 bytes for a nonclustered index."
    • NVARCHAR needs 2 bytes per character, so 450 × 2 = 900 bytes.
  • ExistsAsync, StoreAsync and TryReserveAsync reject a longer key with ArgumentOutOfRangeException (an ArgumentException) before any database call.
• SQL Server
  • The column and all three procedure parameters (Exists, Insert and the Reserve procedure from feat(idempotency): refresh expired idempotency keys atomically on reserve and store #909) are NVARCHAR(450) COLLATE Latin1_General_100_BIN2.
    • Collation and Unicode support: BIN2 collations "implement a pure code-point comparison".
  • All three SqlParameters are bound with the central constant instead of the literal 500. SqlClient used to cut longer values silently.
  • The repository has the same length guard as the store.
  • Re-running IdempotencyKey.sql upgrades an existing table. A guarded sys.columns check runs one XACT_ABORT transaction that drops PK_<Table>, runs ALTER COLUMN and re-creates the clustered PK.
    • Shrinking to 450 cannot truncate data, because keys over 450 characters could never be inserted (Msg 1946).
    • The upgrade runs in BEGIN TRY / BEGIN CATCH. On failure it rolls back and throws error 50001 with the table name and the original message, then resets XACT_ABORT.
  • The EF Core SQL Server configuration maps nvarchar(450), derived from the constant, with UseCollation("Latin1_General_100_BIN2").
• MySQL
  • The key column is utf8mb4_bin.
    • MySQL 8.0 manual, 12.8.5: for _bin collations "ordering is based on numeric character code values".
  • Re-running the script switches an existing _ci column. This uses the information_schema.columns + PREPARE guard from the accepted re-runnable MySQL scripts ADR. There is no DELIMITER and no ; inside literals.
  • INSERT IGNORE is replaced by a plain INSERT, and ER_DUP_ENTRY (1062, MySqlErrorCode.DuplicateKeyEntry) counts as an existing key.
    • MySQL 8.0, INSERT: "With IGNORE, invalid values are adjusted to the closest values and inserted".
    • ON DUPLICATE KEY UPDATE was rejected on purpose. MySql.Data reports found rows by default, so a duplicate also returns 1 affected row, and TryReserveAsync from feat(idempotency): refresh expired idempotency keys atomically on reserve and store #909 would then report every duplicate as reserved.
  • The column stays VARCHAR(500). Existing rows may be longer than 450 characters, and the core guard enforces the limit anyway.
  • The EF Core MySQL configuration sets UseCollation("utf8mb4_bin") and the Oracle provider's own MySQL:Collation annotation. MySql.EntityFrameworkCore ignores the relational collation, so without the annotation the column got the case-insensitive database default (found in CI).
• Entity Framework
  • The repository has the same length guard.
  • The PostgreSQL, SQLite and MySQL configurations keep HasMaxLength(500) in the model, so their existing migrations stay in sync. Only SQL Server (type and PK) and MySQL (collation) need a migration.
  • The IdempotencyKey entity docs are updated.
• Docs
  • Each of the SqlServer, MySql and EntityFramework READMEs has a new, self-contained subsection on key length and case sensitivity, with the upgrade SQL.
  • The core Pulse README has a new Idempotency Keys subsection (450-character limit, ArgumentOutOfRangeException, case-sensitive comparison). The Redis and PostgreSql READMEs link to it.
  • The docs on IIdempotentCommand<TResponse>.IdempotencyKey and the IIdempotencyStore members now state the limit and the case-sensitive comparison.
  • New ADR decisions/2026-09-29-exact-idempotency-key-storage.md (state: proposed).
• Tests
  • Unit tests for the guard on the store and on the SQL Server, MySQL and EF repositories.
  • EF model metadata tests for column type and collation.
  • Three new tests in the shared IdempotencyTestsBase, which run for every provider:
    • keys differing only by case are distinct;
    • a key of the maximum length round-trips;
    • a key over the maximum is rejected with ArgumentOutOfRangeException.
  • A SQL Server legacy-table upgrade test, plus a failed-upgrade test (custom PK name: error 50001, column unchanged).
  • A MySQL legacy-collation upgrade test.
  • MySqlScriptRunner now also rewrites ALTER TABLE `X`.

Bug hunt

Suspicion	Confirmed?	Test
Keys of 451 to 500 characters fail on SQL Server with Msg 1946 (clustered key > 900 bytes)	Confirmed by analysis. The red run is in CI (Docker)	IdempotencyTestsBase.Should_Store_And_Find_Key_Of_Max_Length (SqlServer ADO + EF), IdempotencyKeyConfigurationMetadataTests.Configure_WithSqlServerConfiguration_* (red locally)
SqlClient truncates keys over 500 characters, so keys sharing a 500-character prefix collide	Confirmed locally: over-length keys reached the database instead of being rejected	SqlServerIdempotencyKeyRepositoryTests.*_WithKeyLongerThanMaxLength_*
MySQL INSERT IGNORE stores a truncated key and hides data errors	Confirmed locally: the repository accepted over-length keys	MySqlIdempotencyKeyRepositoryTests.*_WithKeyLongerThanMaxLength_*
The store has no length guard, so every provider behaves differently	Confirmed locally (SQLite ADO + EF accepted a 451+ character key)	IdempotencyStoreTests.*_WithKeyLongerThanMaxLength_*, IdempotencyTestsBase.Should_Reject_Key_Longer_Than_Max_Length
The EF repository accepts over-length keys	Confirmed locally	EntityFrameworkIdempotencyKeyRepositoryTests.*_WithKeyLongerThanMaxLength_*
MySQL utf8mb4_unicode_ci and the SQL Server default collation treat aBc123 = ABC123 and cause false conflicts	Confirmed by analysis. The red run is in CI (Docker)	IdempotencyTestsBase.Should_Treat_Keys_Differing_Only_By_Case_As_Distinct (MySql/SqlServer, ADO + EF), Configure_WithMySqlConfiguration_UsesBinaryCollation (red locally)
Existing SQL Server and MySQL tables would keep the broken column, because the scripts only create missing tables	Confirmed by analysis. Runs in CI (Docker)	SqlServerAdoNetIdempotencyTests.Should_Upgrade_Legacy_Key_Column_When_Script_Is_Rerun, MySqlSchemaScriptTests.IdempotencyKeyScript_WhenKeyColumnIsCaseInsensitive_SwitchesToBinaryCollation
Replacing INSERT IGNORE with ON DUPLICATE KEY UPDATE (as the issue suggested) would break TryReserveAsync, because found-rows reports 1 for a duplicate	Confirmed by analysis. A plain INSERT + 1062 is used instead	The existing Should_Reserve_New_Key_Once / Should_Reserve_Once_When_Reserving_Same_Key_Concurrently (MySQL, CI)
Lowering the shared EF HasMaxLength to 450 changes the model snapshot of PostgreSQL, SQLite and MySQL too, so Migrate() on EF 9+ reports pending model changes although their column types are unchanged	Confirmed locally (review finding). The model now keeps 500 for those providers	IdempotencyKeyConfigurationMetadataTests.Configure_With{PostgreSql,Sqlite,MySql}Configuration_KeepsModelMaxLengthOfExistingMigrations (3 red, then green)
The second assertion of Should_Reject_Key_Longer_Than_Max_Length (450-character prefix not stored) could never fail	Confirmed (review finding). The assertion is removed, and the test now expects ArgumentOutOfRangeException	IdempotencyTestsBase.Should_Reject_Key_Longer_Than_Max_Length
A failed SQL Server column upgrade (for example a hand-managed table with another PK name) aborts with an unclear error and no TRY/CATCH	Guideline hardening, not a data bug: XACT_ABORT already rolled back. Before, the batch failed with Msg 3728; now it rolls back explicitly and throws 50001 with context. Runs in CI (Docker)	SqlServerAdoNetIdempotencyTests.Should_Roll_Back_And_Report_Failed_Upgrade_When_Primary_Key_Name_Differs
The Oracle MySql.EntityFrameworkCore provider does not emit COLLATE from UseCollation	Confirmed in CI (MySqlEntityFrameworkIdempotencyTests.Should_Treat_Keys_Differing_Only_By_Case_As_Distinct failed on all TFMs) and locally: GenerateCreateScript produced varchar(500) NOT NULL. Fixed with the MySQL:Collation annotation	EntityFrameworkIdempotencyKeyCreateScriptTests.GenerateCreateScript_WithMySqlProvider_DeclaresBinaryKeyCollation (red, then green; no Docker needed), plus a SQL Server counterpart (green)
SQLite, PostgreSQL, Redis and InMemory compare case-insensitively	Not confirmed: they already compare exactly. The tests are kept	The shared case test, green locally on SQLite and InMemory

Impact

• Behavior change, all providers. A key of 451 to 500 characters now throws ArgumentOutOfRangeException. Before, it worked on PostgreSQL, SQLite, Redis and InMemory, and failed or truncated on SQL Server and MySQL.
• Public constant. IdempotencyKeySchema.MaxLengths.IdempotencyKey changes from 500 to 450. It is a const, so code compiled against the old package keeps 500 until it is recompiled.
• Case sensitivity. On SQL Server and MySQL, keys that differed only by case used to be treated as duplicates. They now run as distinct commands.
• Schema upgrade.
  • ADO.NET users of SQL Server and MySQL must re-run IdempotencyKey.sql. It upgrades existing tables in place.
  • EF Core users need a new migration for the SQL Server and MySQL key column type and collation. On EF 9+, Migrate() reports pending model changes until they add it.
  • PostgreSQL and SQLite EF models are unchanged: they keep their column types and MaxLength 500, so they need no migration.
• Known ceiling. Trailing spaces stay insignificant on SQL Server and MySQL.
  • SQL Server pads strings before every = comparison, whatever the collation (= (String comparison)).
  • utf8mb4_bin is a PAD SPACE collation. utf8mb4_0900_bin (NO PAD) would still differ from SQL Server, and MariaDB (Pomelo) lacks it.
  • The ADR records this.
• Warning 1945. It no longer applies, because the column is exactly 900 bytes. CI covers this with the max-length round-trip test on SQL Server.
• Oracle MySQL EF provider. The collation is now set through the provider's MySQL:Collation annotation, and the generated DDL is checked without Docker.
• No interface changes for external IIdempotencyKeyRepository or IIdempotencyStore implementers. Only the XML docs now state the 450-character limit and the case-sensitive comparison.

Test evidence

• dotnet build Pulse.slnx -c Release: 0 errors, 0 warnings.
• Red first, from commit test(idempotency): ... before the fix:
  • 12 guard tests failed (store + SQL Server + MySQL + EF repositories);
  • 2 EF collation tests failed;
  • Should_Reject_Key_Longer_Than_Max_Length failed on SQLite ADO and EF.
• After the fix:
  • dotnet test --project tests/NetEvolve.Pulse.Tests.Unit (net8.0, net9.0, net10.0): 7542 passed, 0 failed.
  • Integration, treenode filter SQLite*|InMemory*|EntityFrameworkIdempotencyKeyCreateScriptTests (net8.0, net9.0, net10.0): 915 passed, 27 skipped, 0 failed.
• Review round: the 3 new KeepsModelMaxLengthOfExistingMigrations tests failed before fix(entityframework): keep the idempotency key model max length at 500 ... and pass after it.
• SQL Server, MySQL, PostgreSQL and the other Docker-backed integration tests run in CI.
• csharpier check . is clean.

[codecov]

Codecov Report

❌ Patch coverage is 90.42553% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.96%. Comparing base (facdf36) to head (48cf502).

Files with missing lines	Patch %	Lines
...r/Idempotency/SqlServerIdempotencyKeyRepository.cs	77.77%	8 Missing ⚠️
...MySql/Idempotency/MySqlIdempotencyKeyRepository.cs	96.15%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #923      +/-   ##
==========================================
+ Coverage   95.93%   95.96%   +0.02%     
==========================================
  Files         276      276              
  Lines       12063    12133      +70     
  Branches     1150     1150              
==========================================
+ Hits        11573    11643      +70     
  Misses        267      267              
  Partials      223      223              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.