Skip to content

testing.md

Latest commit

 

History

History
456 lines (362 loc) · 56.7 KB

testing.md

File metadata and controls

456 lines (362 loc) · 56.7 KB
Raw

Testing

Running Tests

dotnet test                          # All tests (method-level parallel)
dotnet test --filter "ClassName=RangeContainsTests"   # Single class
dotnet test --filter "FullyQualifiedName~Contains_FiniteRange"  # Single method

Tests run in parallel at the method level (MSTestSettings.cs). No shared state between tests.

Organization

One test file per operation, named Range[Operation]Tests.cs:

File Covers
RangeContainsTests.cs Point and range containment
RangeOverlapsTests.cs Overlap detection
RangeIsAdjacentTests.cs Adjacency (discrete step-aware, continuous complementary inclusiveness)
RangeIntersectTests.cs Intersection across all shape combinations
RangeUnionTests.cs Union (merge overlapping, keep disjoint)
RangeExceptTests.cs Set difference (0/1/2 element results)
RangeContainedByTests.cs Symmetric containment alias
RangeDoesNotExtendLeftOfTests.cs / RightOfTests.cs PostgreSQL &</&>
RangeStrictlyLeftOrRightOfTests.cs PostgreSQL <</>>
RangeBoundAccessorTests.cs lower/upper/lower_inc/upper_inc equivalents on ranges and sets
RangeMergeSpanTests.cs Merge convex hull (range_merge), spanning gaps
RangeAggregateTests.cs RangeAgg/RangeIntersectAgg (range_agg, range_intersect_agg)
RangeSetComparisonTests.cs Multirange operator parity: state checks, set-operand ops, positional comparisons, ==/!=
RangeParseFormatTests.cs PostgreSQL range literal round-trips
RangeSetTests.cs RangeSet construction, normalization, bulk ops
RangeSetOptimizationTests.cs Performance-critical invariants
RangeJsonConverterTests.cs JSON serialization round-trips; null tokens rejected on read, null references written as null; variant-typed and object-typed declarations producing the same literal, with wrong-shape reads rejected
RangeShapePredicateTests.cs The five IRange<T> shape predicates as a one-hot matrix over all five variants, plus degenerate-bound normalization
NamedJsonConverterTests.cs The 24 pre-built named converters (ranges, multiranges and sets): the full inventory, each bound to the family it is named for, and each matching the factory's payload
ParserResilienceTests.cs Hostile input in the request path: megabyte-scale malformed literals and 200k-element sets parse or are rejected within a [Timeout], TryParse never throws, and rejection messages carry an excerpt of the input, never the payload
TimeRangeTests.cs What is new with TimeRange: TimeOnly parse/format, half-open default, midnight-wrap sets (per-type file; the engines are covered by the per-operation files)

Value set tests (v6) follow the same one-file-per-concern pattern with a Set prefix:

File Covers
SetCanonicalFormTests.cs Dedupe/sort per family, the ordinal-vs-culture pin (umlauts prove IComparable is ignored on string families), numeric 10-vs-"10" order, decimal-scale and instant deduplication, null rejection, empty singletons, collection expressions
SetAlgebraTests.cs Contains/Overlaps/IsSubsetOf/IsSupersetOf/Union/Intersect/Except/Add/Remove, including instance-preservation on no-op results
SetEqualityTests.cs Structural equality, operators, hash codes, null handling
SetParseFormatTests.cs PostgreSQL array-literal round-trips, quoting/escaping, unquoted-NULL rejection, invariant-culture pins
SetJsonConverterTests.cs JSON array round-trips via the shared converter factory, null-element rejection, wrapper-element delegation, and the ElementJsonConverter split — null for the primitive-backed families, defined for every wrapper arity, integer-backed wrappers writing numbers
SetElementBridgeTests.cs The validated-wrapper arities: text bridge round-trips for string/Guid/int/long-backed test keys (TestKeys.cs), backing-value order pins

SetElementBridgeTests.cs and SetJsonConverterTests.cs cover all ten core arities, one generator-shaped test key each (TestKeys.cs). The temporal assertions are all on sub-second components, because those are the digits a default-form bridge drops. Two element types there are deliberately non-conforming — TestLossyStamp swallows the format argument — and pin where that is caught: silently truncating in the pure model's array literal, loudly at the EF boundary.

Coverage of the arities as a family lives in the conventions suite, because it is discovery-driven: ValueSetContractTests closes every one over a representative element from WrapperElements.cs and runs the same three contract assertions it runs on the closed types, so an arity added without a probe or without honouring NormalizeElement/CanonicalOrder/ElementJsonConverter fails there.

JsonConverterPrecedenceTests.cs pins the 9.0.0 type-level [JsonConverter] attributes from the outside: the unregistered wire form, a converter on the options or on a property still winning — a union converter on the options reaching values declared as a variant, through object, in an object collection or dictionary and as variant-typed keys, and following first match — and a source-generated context. InterfaceDeclaredJsonTests.cs covers values declared as IRange<T>, the shape interfaces and IValueSet<T>: written through the runtime type, reads refused naming the remedy, null, a registered converter, a source-generated context. JsonDictionaryKeyTests.cs covers ranges, variants and multiranges as dictionary keys. The per-type sweep of both — every range shape and set family, unregistered against registered, every range shape under a custom union converter on the options, and every shape and set family declared as each package interface it implements — is JsonDefaultContractTests in the conventions project.

JsonColumnTests (integration) puts every range, multirange and set family into one jsonb column via ComplexProperty(…).ToJson(), round-trips it, queries each property server-side (proving the stored text casts to its store type), and pins the stored text, including the YearMonth types' month-aligned form. Its two tracked-change tests take the path an application does — load, change, SaveChanges — once on the complex document, which EF rewrites whole, and once on an owned one, which EF updates with jsonb_set for a lone changed property: a range and a value set are replaced and a nullable range, temporal range and value set go from null to a value and back, one change per save so the change tracker has to see each. Once every value is set, and again once the nullable ones are back to null, the document is read through a new context and queried on the server, and every property no change touched must still hold its original value.

TemporalStoreParityTests (integration) is the authority on temporal values written as text. It holds one property per registered store conversion — every temporal range type, closed set and wrapper set family except the three over YearMonth, which have nothing below a day to truncate, plus two multiranges standing for the rest, since a multirange takes its range type's definition — in one row twice, a column and a JSON document, built from probes chosen by rounding case (remainders above, below and at 500 ns, before 2000, the extremes, sub-microsecond ranges). Per property it asserts that the server reads the document as the column, that a parameter and a constant each match both copies, that the copies materialize equal, and that Contains agrees with memory one microsecond either side of every bound. The element operands are held to an expected answer rather than to each other: every element a set was built from must match its own row as a parameter and as a constant, and a range's Contains on its original sub-microsecond bounds must answer as the materialized range does for the probe as stored — two paths that agree would also agree on a miss. InfinityConversionsDisabledTests, in its own project …InfinityDisabled.IntegrationTests because the switch is read once per process, holds the same rule at the extremes with Npgsql.DisableDateTimeInfinityConversions set: MinValue / MaxValue written finite and truncated, as Npgsql then sends them, in the document, the literal and the element operands. Its first test checks the switch actually reached the process — without it, every other assertion there would pass for the default configuration. TemporalStoreTextTests (unit) pins the text itself. ExecuteUpdateJsonTests (integration) covers what ExecuteUpdate may do inside a document, ExecuteUpdateGuardTests (unit) the refusal and the shape of EF Core's lowering it matches.

TemporalLiteralGrammarTests (unit) pins the strict ISO 8601 temporal grammar, and TemporalLiteralStrictnessContractTests (conventions) refuses a non-ISO element through every entry point of every temporal range and set type, discovered — and accepts HH:mm uniformly wherever a time of day appears, NodaTime included.

The literal grammars themselves follow PostgreSQL's range_in and array_in. RangeLiteralGrammarContractTests (conventions) sweeps every range type: only an absent bound is infinite (a quoted "" or a whitespace-only bound is refused), quotes and backslash escapes work anywhere in a bound, and only ASCII whitespace is skipped. SetParseFormatTests pins the array grammar case by case (escapes in unquoted elements, nested braces and stray quotes refused) plus a round-trip sweep over every structural character. LiteralGrammarParityTests (integration) hands the same text to PostgreSQL and requires the same answer — refused by both, or the same value. WrapperGrammarContractTests (conventions) holds every wrapper arity, closed over a deliberately lenient element and over its representative one, to its primitive sibling's element grammar through every entry point.

LiteralCorpusProbeTests (integration, route B's third probe) widens that parity sweep to a corpus and to every entry point. Per element kind it generates the structural families of each grammar (bounds, equal and inverted bounds per bracket pair, empty in every case, quoting, escapes, trailing junk, Unicode whitespace, array dimensions and NULLs, multirange canonicalization) and the texts the element input function reads differently (sign, leading zeros, exponents, NaN, non-decimal integers, more than 28 digits, infinity spellings, epoch/now/today, non-ISO dates, 24:00:00, offsets, sub-microsecond digits, uuid spellings, NUL), about 100 literals for each of the 48 range, multirange and value set types. The server answers each with pg_input_is_valid and its canonical text in one batched statement per type; Parse(string), Parse(span), both TryParse overloads, JSON, the TypeConverter and the EF Core JSON document reader must all agree with each other and with it, and a value the library reads is written back through EF Core as a SQL constant and as a JSON document and compared with the original on the server as a typed value — which is what sees a precision the parser dropped. A refusal of a value outside the CLR domain, or of a form docs/serialization.md names (strict ISO 8601, J07; the numeric grammar, J06), is counted rather than reported. The whole sweep takes about five seconds; its known disagreements are candidates keyed {type} {family} .

test/CodoMetis.ValueRanges.AspNetCore.Tests/ runs the host-level behaviour with no ValueRanges registration at all: configuration binding through the TypeConverter, a TestServer host with MVC and minimal-API endpoints (query binding, JSON bodies both ways), and the OpenAPI document generated with and without the documented ValueRangesSchemaTransformer — whose patterns are checked to accept every literal form the parsers accept, and whose source is held verbatim against docs/serialization.md.

HostCultureProbeTests there is the host half of route B's second probe sweep (#124): one TestServer with a minimal-API route group and a closed ProbeController<T> per declared type (every union, each of its five variants, every RangeSet and every set type, 107 in all), each bound from a query, a nullable query, a route value, a form field and a JSON body, under four request cultures set by UseRequestLocalization from Accept-Language. A valid value must come back with 200, the right runtime type and the same JSON value; a malformed literal, or a literal of another shape for a variant, must get 400. Each handler echoes the culture it ran under, so a row whose culture never arrived fails rather than passing vacuously. It links RangeProbes, SetProbes, WrapperElements and Reflect from the conventions project instead of copying them, and runs about 12,400 requests in about three seconds.

test/CodoMetis.ValueRanges.NodaTime.Tests/ covers what is new in the NodaTime satellite — type wiring, the one-day and one-month discrete steps, ISO calendar normalization (and YearMonthRange's non-ISO rejection), wire-form parsing, Interval/DateInterval/month-alignment interop — with representative shape coverage; the engines themselves are exhaustively covered by the core tests. NodaTimeSetTests.cs/NodaTimeSetJsonTests.cs do the same for the five set types (calendar normalization, wire-form parsing, JSON with ConfigureForNodaTime). NodaTimeSetElementBridgeTests.cs covers the five NodaTime arities — ISO literal round-trips, ordering, algebra, JSON, the refusal of an element backed by a non-ISO value (construction, collection expressions, a registered element converter, and the Contains/Add/Remove probe against the empty set, where nothing is compared), and CultureBoundDay, an element that ignores the format argument and so leaks NodaTime's culture long form into the literal. NodaTimeConverterRegistrationTests.cs covers both JSON entry points: the ElementJsonConverter hook (all five sets round-tripping under plain AddRangeConverters() and under a bare factory, a registered converter still winning, identical output either way), and AddNodaTimeRangeConverters() (ISO 8601 element forms, all four ranges and all five sets, bare elements alongside a set, idempotence and order-independence against AddRangeConverters() and ConfigureForNodaTime — measured through GetConverter and a non-ISO date, since a round trip of ISO values passes whichever converter wins — and the refusal of bare non-ISO values on write).

EF Core translation tests live in test/CodoMetis.ValueRanges.EFCore.PostgreSQL.Tests/ — they assert generated SQL via ToQueryString() without a database. SetModelMappingTests.cs/SetQueryTranslationTests.cs cover the value set columns: every operator in constant, parameter, and column form, the string[]-coexistence and store-name negatives, the without-UseValueRanges model-build failure, and the wrapper element binding. WrapperArityIntegrationTests.cs closes the live-database half of the same gap: twelve of the fifteen arities had never touched PostgreSQL, so everything known about them was a claim about SQL text. It round-trips each one, asserts the stored column text (a bridge that coarsens consistently on both legs round-trips perfectly and stores the wrong thing, which is how the format-pinning defect hides), and runs Contains server-side both for a stored element and for a near miss one unit of precision away. Seeding the unpinned-TimeSet<> defect fails the round trip, the stored text and the near miss — but not the positive Contains, since probe and column truncate alike. Its element types are linked from WrapperElements.cs in the conventions project rather than copied, so the values that round-trip are the ones whose semantics ValueSetContractTests checks. SqlLiteralTests.cs pins what the wrapper bridge actually emits — the sub-second digits of a DateTimeSet<T> element, its Kind normalization, and a DecimalSet<T> element's scale — which is the only place that conversion is observable without a database. WrapperSetTranslationTests.cs does for the ten core arities what NodaWrapperSetTranslationTests.cs (below) does for the five NodaTime ones: the Contains literal per arity, the two set-operand forms, equality, cardinality, the Union refusal, and the array_remove shape — the only operand position where the element carries no array cast to type it. Its precision sweep is stated as survival of a value, not absence of a lossy shape, because an arity that stops pinning its format still emits a well-formed ISO literal: unpinning TimeSet<> yields ARRAY['09:30:00.0000000'], seven fraction digits and fifteen seconds short. DateTime and DateOnly cannot fail this way — their default form is rejected by the bridge's ParseExact — so TimeOnly and decimal are the two cases with no exception to catch. QueryTranslationTests.cs also pins the shape predicates on a single range column, including the two PostgreSQL has no function for — IsInfinity as lower_inf AND upper_inf, IsFinite as the negation of both plus NOT isempty. test/CodoMetis.ValueRanges.EFCore.PostgreSQL.NodaTime.Tests/ does the same for the NodaTime types, proving every translation path (operators, functions, factories, aggregates, multiranges, sets, BCL coexistence). NodaWrapperSetTranslationTests.cs covers the NodaTime arities specifically — the highest-risk corner, since they carry the most custom bridge code and reach the registry through RegisterFamily rather than as closed definitions. Its operands go through EF.Constant on purpose: a captured variable is parameterized, and a parameter binds the converted primitive natively, which hides the entire literal path. That is not hypothetical — writing these tests is what caught LocalDateSet<T> rendering ARRAY['Saturday, 15 June 2024']::date[], because its family registration had left literalText at the default. MigrationScaffoldTests.cs builds the design-time services dotnet ef migrations add builds (DesignTimeServicesBuilder, through the project's test-only Microsoft.EntityFrameworkCore.Design reference) and asserts that the scaffolded migration and its SQL create timerange ahead of the table that declares the TimeRange and timemultirange columns.

RegistryConcurrencyTests.cs is the odd one out in that project — it asserts nothing about SQL. The two registries are the only mutable process-wide state in the packages, and registration is a read-modify-write over an immutable snapshot (Current = BuildSnapshot([.. Current.Definitions, definition])), so two interleaving calls lose one. The volatile field does not prevent that; the lock does, and nothing said so. Each registry is hammered through both of its mutating entry points at once — they share a lock and each rebuilds the snapshot from fields the other writes, so one clobbering the other is a distinct failure from either losing its own update — with readers alongside checking the built-in registrations stay visible, over four rounds. The registered types are private markers no model can name and whose type mappings throw, so they stay in the process-wide registries for the rest of the run without being reachable. Four rounds rather than one because a lost update is a race: with both locks removed a single round let the set registry through about one time in three, and four rounds fails 5 runs of 5.

test/CodoMetis.ValueRanges.EFCore.PostgreSQL.IntegrationTests/ executes against live PostgreSQL via Testcontainers (Docker required; tests report Inconclusive without it — except under CI=true, where a missing database fails the tests so the build badge can never be green with the live layer silently skipped): round-trips for every range/multirange type, timestamp normalization rules, and SQL-vs-in-memory parity assertions for the v4 operations. TimeAndYearMonthIntegrationTests.cs covers the v5 types: the custom timerange created via HasPostgresRange + EnableUnmappedTypes, and YearMonthRange's month-aligned daterange storage form. CustomRangeCatalogueTests.cs reads pg_type after EnsureCreated of the integration model in a database of its own (ContainerLifecycle.RequireServer), so a defect that breaks the shared schema fails it with PostgreSQL's error rather than leaving it inconclusive. ComplexIndexesCrossTests.cs holds the pairing docs/efcore.md recommends with EFCore.ComplexIndexes.PostgreSQL (a test-only reference): its temporal constraints on PostgreSQL 18, its exclusion constraints on 16 and 18, and GiST indexes, over a DateRange, a RangeSet<DateRange, DateOnly> and a TimeRange column, each in a database of its own with UseNpgsqlComplexIndexes() registering that package's differ for EnsureCreated, plus the silent case without it. Its servers are containers of their own, started on first use. SetIntegrationTests.cs (v6, Id block 8xxx) covers the value sets: canonical round-trips with stored-array-text pins, empty-vs-NULL, non-canonical foreign rows (operators match, = does not, reads normalize without rewriting the row), corrupt rows throwing on read, change detection, GIN-served @> (EXPLAIN pin), the StringSet<TestKey> wrapper hinge, and server-vs-in-memory parity for the whole set algebra. These tests are the authority on PostgreSQL semantics — they caught the discrete upper() canonicalization offset and the directional multirange adjacency rule.

ShapeMatrixParityTests.cs generalizes that authority: it asks PostgreSQL for all eight binary predicates over every ordered pair of range shapes — and, in a second sweep, the four value-producing operations (Intersect, Merge, Union, Except) against *, range_merge, + and - — int4range, numrange and daterange, some 2,400 comparisons — and requires the in-memory answer to match. The per-predicate tests elsewhere pick the shapes a human thought to try, which is how IsAdjacentTo (6.2.1), IsStrictlyLeftOf and Except (both 7.0.0) all shipped answering wrongly for a whole family of receivers: each was a switch on the receiver's shape whose inner switch handled the operand's, so the two directions drifted apart without any one test noticing. The matrix has no such blind spot. Since 7.0.0 the model agrees with the server on every pair, the empty-subset rule included, so the matrix carries no exclusions.

DifferentialFuzzParityTests.cs (with Fuzzing/) does what the matrix cannot: it picks the rows. A seeded generator draws 200 rows per type (VALUERANGES_FUZZ_ROWS raises it, VALUERANGES_FUZZ_SEED moves it) for every range, multirange and value set type it discovers by reflection, BCL and NodaTime. The bounds are drawn next to the element type's extremes and next to each other, and every operation EF Core translates is answered three ways: in memory, by EF Core against its own fuzz database, and by PostgreSQL's operator on the literals EF writes. See The Differential Fuzzing Sweep.

MultirangeCoverageTests.cs (Id block 91xx) closes the two coverage holes the 7.0.0 audit found. Five of the eleven RangeSet<,> instantiations — over Int64Range, DecimalRange, DateTimeRange, DateTimeOffsetRange and LocalDateTimeRange — had never been mapped to a column, so their store type, literal form and normalization on read were untested; they now round-trip with the stored column text pinned beside CLR equality, because a bridge that coarsens consistently on both legs round-trips perfectly and stores the wrong thing. The file also executes the two operand positions that carry no type of their own: a constant element against a polymorphic range operator (anyrange @> anyelement resolves without implicit coercion, which is why a bare 25 failed against int8range), and the server-side range constructors, whose bounds string '[]' and bare NULL bound are resolved by PostgreSQL's function-overload resolution — every previous use of CreateFinite in this suite had constant arguments and was folded client-side, so the generated SQL was a claim about the server that nothing ran.

Three further parity sweeps close the gaps the 8.0.0 audit left, each the structural analogue of a check that had already earned its keep elsewhere:

  • SetOperatorParityTests.cs (Id blocks 82xx–84xx) is the shape matrix for value sets. The sets had an exhaustive in-memory oracle on one side and hand-picked integration examples on the other, and nothing asked PostgreSQL to confirm @>/<@/&&/cardinality over the whole value space — the exact check that found three range bugs. Every ordered pair of subsets of a four-element universe is swept for StringSet (text[], where ordinal and collation order differ), Int64Set (the element literals PostgreSQL will not coerce) and StringSet<TestKey> (the text-bridged wrapper arities), together with the three compositions on Union that stay legal because they are order- and multiplicity-insensitive. It runs through EF, so a correct operator reached by a wrong translation still fails — and because EF answers an untranslatable projection by evaluating it client-side, which would silently reduce the whole sweep to the model agreeing with itself, one query's SQL is asserted to carry the actual operators. Seeding a Union that refuses to translate is what that assertion catches; seeding IsProperSupersetOf as a bare @> is what the sweep catches.
  • RangeAggregateParityTests.cs (group blocks 30000/40000) sweeps RangeAgg/RangeIntersectAgg against range_agg/range_intersect_agg over every subset of an eight-range universe that includes Empty and Infinite — inputs neither aggregate had ever been fed, on either side. The empty and infinity results are also stated as literals rather than compared, because parity alone cannot say the two implementations agree on the right answer; the zero-row divergence (NULL on the server, the empty set in memory) is executed rather than only documented, via raw SQL, since LINQ cannot express a group with no rows. The expected values come from the shipped aggregates, handed in as delegates because the overloads are per range type: the first draft re-derived the fold in the test, and a seeded defect in RangeIntersectAgg walked through it untouched.
  • ContinuousMultirangeParityTests.cs sweeps the continuous multirange algebra and predicates against nummultirange — every subset of six fragments and every ordered pair through +, -, * and the complement, and through @>, &&, <<, >>, &<, &> and -|-, plus every subset against a single range whose bound sits on a fragment's with the other inclusivity. This is the half of the multirange surface no in-memory oracle can reach (see The Multirange Oracle below). The predicate half is what RangeSet.Overlaps slipped through until 10.0.0: it chose its candidate element by the operand's upper bound value alone, which on a continuous domain is an element that starts exactly where the operand stops, and never asked the element before it (#106). The fragments differ only in inclusivity at each meeting point, which is the one thing that decides a merge without a step, and one of them is a single-value range whose only job is to be absorbed or left standing. Worth knowing about the seeds: a blunt break of continuous adjacency fails nine worked examples too, so it proves only that the sweep runs. The seed that shows what it adds is an inclusivity-blind lower-bound comparer — the whole core suite notices that only through the comparer's own unit test, while its consequence, unions that stop merging, is caught nowhere at the value level.

UnionCanonicalityTests.cs (Id block 12xxx) and the second half of ExecuteUpdateJsonTests.cs (96xx) hold the two refusals ValueSetSqlGuard makes on the final SQL. Every refusal is asserted by executing the query — the 8.0.0 projection test only called ToQueryString() and asserted nothing, which is how a comparison that answered false against a live server was documented as working. The union data is the probe that found the gaps ({a,c} and {b,c}, each unioned with {a,b}: equal in memory, different concatenations on the server), and the positive half executes every legal composition and compares with memory. Several shapes compose over a UNION ALL (Concatenated) on purpose: EF inlines a projected union into a later predicate wherever it can, so only a derived table EF cannot see through proves that the guard follows a column back to its array_cat — disabling that tracking fails exactly those tests. The JSON half uses UpdateGuardRow, whose complex type is mapped both inside a document and as plain columns, so a refusal keyed off the CLR type rather than the translated setter would fail TheSameComplexTypeOutsideAJsonColumn_IsStillUpdatable.

test/CodoMetis.ValueRanges.Conventions.Tests/ covers what the compiler cannot: defects that build, test and pack clean and only surface once a consumer installs the package. Five groups — changelog consistency (the shipped Version is documented at the root and in every package's changelog, no file runs ahead of the version property, per-package versions are a subset of the root's, sections newest-first), packaging (every package declares a PackageReadmeFile and packs a README from its own directory rather than a shared one, carries a description and tags; the shared props ship symbols and Source Link with ContinuousIntegrationBuild gated on CI), value set contracts (the NormalizeElement, CanonicalOrder and ElementJsonConverter rules from AGENTS.md, asserted behaviourally: a set that breaks any of them fails to find an element it was built from, or writes objects where scalars belong), and mapping parity (every range and set type in both shipping assemblies resolves through IRelationalTypeMappingSource — including each wrapper arity closed over its representative element, which is a second way to be missing from the registry, since the families are registered by open generic definition rather than as closed definitions; plain arrays keep their native provider mapping), and the default JSON contract (JsonDefaultContractTests: every range shape — through the union, through object, inside a RangeSet, as a dictionary key — and every set family serializes unregistered exactly as under AddRangeConverters(), which is what fails when a new type or variant lacks its own [JsonConverter]). The probe sweeps (*ProbeTests, one [DynamicData] row per discovered type) hold the same promises over the product of consumer settings listed by hand in test/oracles/axes.md: JsonOptionsProbeTests runs every type through six declarations (union, sealed variant, interface, object, record property, dictionary value) under six JsonSerializerOptions (default, camelCase, snake_case, ReferenceHandler.Preserve, a source-generated context, NumberHandling.AllowReadingFromString) in under a second, and hands its disagreements to Candidates.Verdict scoped to the row. CultureEntryPointProbeTests runs every type, through the union and the sealed variant, through every entry point that needs no host — Parse and TryParse (string and span, a null and the current-culture provider), TypeDescriptor both ways with a null and the current culture, configuration binding, JSON (value and dictionary key) and interpolation — under de-DE, sv-SE, fa-IR and th-TH as the current culture, against the reference read and written under the invariant culture. LiteralProviderContractTests varies only the provider and LiteralCultureContractTests only ToString, so a parser that reached for CultureInfo.CurrentCulture itself passed both on a machine set to the invariant culture; its seed, parser-reads-the-current-culture, is that defect.

Everything is discovered — projects by globbing src/, types by reflection — so a new package or type is covered without editing the tests. Two consequences worth knowing:

  • Both discovery-driven classes assert a floor on how many types they find. This is not padding: the mapping parity tests were passing while iterating an empty list, because the range unions are abstract records and the predicate filtered abstract types out. The floor caught it on the first run. Range discovery goes one step further since 10.0.0: every range sweep asserts that it reached every union of RangeCensus, which finds unions by structure (a nested sealed variant implementing IEmptyRange<T>) rather than by the interface the sweep reads, so a stale count cannot hide a missing type (#222).
  • The probe values in ValueSetContractTests are load-bearing. Probes that ordinal and culture comparison order identically let the CanonicalOrder rule pass while broken — "Zebra"/"apple" split the two orders, and TextKey compares culture-sensitively on purpose, because an ordinal IComparable made the element agree with canonical order by accident.

Every test in this project has been verified by seeding the defect it claims to catch and confirming it — and only it — fails.

Patterns

The Oracle Catalogue

The sweeps below are the oracles; test/oracles/ is what makes them a system. claims.json lists every promise the consumer-facing documents make, quoting the sentence verbatim and naming the test that holds it — or nothing, which is the worklist. ApiSurface enumerates every public member and OracleSurfaceCoverageTests requires each to be named by a claim or waived with a reason. Every registered oracle (oracles.json) ships a seed under seeds/, a patch to src/ that plants a defect it was shown to catch: OracleSeedTests keeps the seeds present and applicable in CI, and dotnet run tools/oracles.cs seeds re-proves locally that each is caught. A disagreement an oracle finds is registered in candidates.json against its issue through Candidates.Verdict, so the suite stays green without hiding it and the entry is forced out when the fix lands. docs/oracles.md has the whole of it.

DomainParityOracleTests (conventions) is the first sweep the catalogue produced: the small-model oracle generalized from Int32Range and DecimalRange to every discovered range type through an order-preserving embedding (a day, a month, a half hour per grid point). The engines are generic; what each type supplies itself — the step, the canonicalizing factories, the element's comparison, the half-open default — had worked examples only on nine of the eleven types before it.

The Two Oracles

Two tests check range algebra against something other than itself, and they answer different questions. Neither replaces the other.

  • ShapeMatrixParityTests (integration, needs Docker) asks PostgreSQL for every binary predicate and value operation over every ordered shape pair and diffs. This is the only check that the library agrees with the database rather than with its own idea of what a range means. It uses about ten hand-picked representatives per domain.
  • SmallModelOracleTests (unit, no Docker) asks set theory.
  • ShapeMatrixCoverageTests (conventions, no Docker) keeps the matrix honest: adding a binary operation to RangeExtensions without a row in the matrix fails there. Operations are discovered by reflection, coverage is parsed out of the matrix source, and the two forms are checked separately by return type — a predicate must appear in the (name, operator) table, a value-producing operation must be invoked against its SQL counterpart. Checking a predicate against invocations alone would not work: the matrix's own dispatch switch calls all eight, so a deleted table row would still look covered. It enumerates every representable range over a tiny universe from its specification — around 110 per domain, all five shapes and all four inclusiveness combinations at every bound — derives the expected value set arithmetically from that specification, and checks all ~12,100 ordered pairs. About 460,000 assertions across the discrete and continuous domains, in under 200 ms.

The small-model oracle is the stronger bug detector of the two, because hand-picked representatives are how the first version of the 7.0.0 Except sweep came back with zero disagreements on the exact defect it was written to catch — its operands happened to be disjoint, so the failing pair never arose. It also covers the RangeSet arities, which is where two of the five known bugs of this family lived: an operation lifted through RangeSet.From([r]) must give the lifted answer.

Two details make the small domain faithful, and a change to either breaks the model silently:

  • Bounds are only ever drawn from the interior of the universe, so the outermost grid points are values no bound names. An unbounded side then reaches a point a finite side cannot, which is what lets the model tell [1,6] from [1,+∞) — a distinction &< and &> turn on.
  • The continuous domain uses a half-step grid, so an exclusive bound is one grid point away from an inclusive one at the same value. [1,2) and [2,3] abut; [1,2) and (2,3] leave the point 2 between them.

It also sweeps the surfaces no shape-pair matrix reaches: the bound accessors and Clamp, and equality. The equality law is that two ranges are equal exactly when they hold the same values, and equal ranges hash alike — which only holds because canonicalization gives one representation per value set, so over the integers (1,5), [2,5), (1,4] and [2,4] must all compare equal. Equals(object), the IEquatable<T> path and the ==/!= operators are checked separately, since they can diverge; the operators are passed in as delegates because a generic method cannot invoke them.

The model has one axiom — that Contains(T) is correct — since results are read back through it. That is pinned by its own test per domain rather than assumed, and Contains(T) is itself verified against PostgreSQL's element @> in the integration suite.

The Differential Fuzzing Sweep

DifferentialFuzzParityTests (integration, needs Docker) is the third way to check range algebra, next to the shape matrix and the small model. Those two choose their operands: about ten hand-picked representatives per domain, or every range over a tiny universe. Each past value-level defect between memory and the database hid until someone chose the row that showed it. This sweep draws the rows.

  • What it draws. A seeded System.Random per type, so CI draws the same rows on every run and the first rows of a seed never change when the budget grows. Bounds are drawn near an extreme, near a shared edge, near the centre or near another bound of the same row, then stepped a few units, so equal, adjacent, touching and off-by-one bounds come up constantly. It draws all five shapes, empties, RangeSets of 0 to 5 members, and value sets from a small pool per row, so duplicates and shared elements are the norm. Strings come from a list built to disagree under some rule: case and culture folds, composed and decomposed forms, whitespace, and everything an array literal quotes or escapes. The types are discovered like MappingParityTests discovers them. An element type with no domain in FuzzDomains fails loudly.
  • Three legs. Each operation is one expression. Compiled, it is the memory leg. Spliced into a query, it is the EF leg: once with the right operand as a column (written through Npgsql's binary parameters), and once per probe as an inlined constant and as a parameter, against every row. The literal leg sends PostgreSQL's own operator over the literals the mapping writes (@>, -|-, range_merge, array_cat, …). A boolean is compared on the server. A range, multirange or set result is read back through the mapping's converter, as EF Core reads a column. A bound is compared as lower([v,)) / upper((,v]), so the discrete step lands on both sides.
  • What it leaves out on purpose. Temporal values below the store's microsecond, which TemporalStoreParityTests owns. Literals PostgreSQL refuses with a data exception (22003, 22008, 22009), such as [x, int.MaxValue] or a year PostgreSQL cannot hold. That is the documented domain boundary, so those rows are counted and set aside. Reading a result back also absorbs the (,-2147483648) row of DomainBoundaryDivergenceTests: what the server holds beyond the model's domain reads back as the value the model sees. The other documented row — a date range bounded at an extreme Npgsql writes as ±infinity is adjacent to nothing on the server — is set aside only when it disagrees in exactly that shape (memory true, EF and PostgreSQL false) and counted; an adjacency row at the edge that disagrees any other way still fails.
  • A failure line is Type Operation [at the domain edge] LeftShape×RightShape [leg]: operands → memory=…, EF=…, PostgreSQL=… (seed=…, row=…). The part before the leg is what a candidate row matches. "At the domain edge" marks a row whose operand touches an exact extreme, so a candidate for an edge defect cannot also swallow the same operation in the middle of the domain. Under another seed the verdict does not require every candidate to fail. Those rows came from the default seed, so missing one elsewhere does not mean it was fixed.

It runs in about 12 seconds at the default budget and about 40 at 2,000 rows. Its first run found three families, none of which a hand-picked sweep had shown: IsUnboundedStart/IsUnboundedEnd on (,) (#109), RangeSet.Overlaps on every continuous type (#106), and adjacency at the date extremes (#110). Its seeds plant a tied-end Intersect and a value set IsSubsetOf that ignores the tail.

The fuzz sweep binds an operand as a column, a constant or a single parameter, never as a collection. CapturedCollectionProbeTests covers that operand position: every discovered range and value set type, read from a captured collection through Contains, Any and Count, compared with the same projection in memory (oracles.md). Its lists are built from the rows as loaded, so the store's microsecond truncation does not count as a disagreement. It reuses the fuzz sweep's type discovery (FuzzTypes), links RangeProbes and SetProbes from the conventions project, and runs in about five seconds.

The Multirange Oracle

SmallModelMultirangeOracleTests covers what the other two do not reach. The single-range oracle only ever lifts one range into a set, so the multi-element algorithms — the greedy merge, the sorted merge behind Union, the two-pointer merge-join behind Except, the single-pass gap walk behind Complement — had no oracle at all and were covered by worked examples only.

Over a universe of eight grid points every subset is a representable multirange, so enumerating all 256 and sweeping all 65,536 ordered pairs is exhaustive over the whole multirange value space. The outermost grid points double as the unbounded sentinels, so unbounded elements and the infinite set appear without being special-cased.

Two things make it bite where the single-range oracle cannot:

  • It checks representation, not just membership. Results are compared element by element against the canonical run decomposition, so an unmerged neighbour, a stray empty or elements out of order fails — even though Contains(value) would answer identically. Count, Equals, GetHashCode, ToString, the indexer and the EF multirange literal all read the representation, and nothing asserted the invariant before this.
  • It feeds From deliberately non-canonical input. Building from maximal runs hands normalization something already normalized and asks it to do nothing. The From(shattered, reversed) path splits every run into one range per value and reverses them, which is the only thing here that exercises the sort and the greedy merge's adjacency arm. A seeded defect that disabled adjacency merging walked straight through every other check in this file.

Discrete domains only, and not incidentally. A run of consecutive grid points is the canonical decomposition exactly when consecutive points are contiguous — true of integers and days, false of reals. Over a half-step decimal grid, {0, 0.5} is both [0, 0.5] (which holds 0.25) and {0} ∪ {0.5} (which does not), and no grid resolution separates them. A continuous sweep built this way reported ~37,000 disagreements that were all the model being wrong. The merge-join algorithms are generic over the range type, so the discrete sweep does exercise all of them; what it cannot reach is continuous adjacency deciding a merge.

That half is swept against PostgreSQL instead, by ContinuousMultirangeParityTests in the integration suite — a point-set model is the wrong oracle for the reals, and the database is a working one. Between the two, the multirange surface is swept end to end: this file over the discrete domains with no database, that one over the continuous domain with one. The single-range oracle's two-element Set.Union/Set.Except cases and the worked examples in RangeSetTests remain as they were.

Discard Arms

docs/discard-triage.md records the one-time triage of every _ => arm outside Internals/, where the convention test cannot reach because plenty of them are correct. Read it before adding one: the short version is that a discard may be the last case of a closed set, a documented specification, or an EF "not mine" — never the answer to a shape nobody thought about.

The Value Set Oracle

SmallModelSetOracleTests (in the conventions project, which references all four shipping assemblies) does the same for the value set families — and there it is the only oracle. Intersect, Except and Add are deliberately client-side only, because a PostgreSQL array has no intersection, no difference and no sorted insert, so no amount of integration testing can cross-check them.

For each of the 30 set types it builds every subset of that family's probe universe — 2^n, so exhaustive over the whole value space rather than a sample — and checks:

  • per subset: Values, Count, IsEmpty, Contains, Add and Remove for every probe
  • per ordered pair: Union, Intersect, Except, Overlaps, the four subset/superset predicates, equality, and that equal sets hash equally
  • per construction path: reversed input, duplicated input, IEnumerable, repeated Add from empty, Parse(ToString()), and a JSON round trip — all of which must land on the identical canonical array, because canonical form is a property of the value and not of the order it arrived in

Its one axiom is that From(x) on a single element normalizes correctly, since element normalization is family-specific and internal (TextKey trims, the NodaTime calendar types convert to ISO) and cannot be derived externally the way a range's values can be derived from its bounds. That is pinned by ValueSetContractTests.EverySetType_FindsAnElementItWasBuiltFrom.

What it cannot check: the model reads the canonical order from CanonicalComparer, so it verifies that every path agrees with the declared order, not that the declared order is the specified one. Swapping StringComparer.Ordinal for StringComparer.InvariantCulture moves the model with the implementation. ValueSetContractTests.StringBackedFamilies_SortOrdinal is the independent check that closes it.

Probes and type discovery are shared with the contract tests through SetProbes — one table, so the two suites cannot disagree about which families exist or what to feed them.

Nulls are deliberately not in that table: it feeds sweeps that build valid sets. ValueSetNullContractTests covers them separately, asserting that every entry point — both From overloads, Add, Remove, Contains, an unquoted NULL in the array literal, a JSON null — refuses rather than drops. It runs over families whose element type is a reference type, which is StringSet alone today and whatever is added later without anyone having to remember.

The Sibling Parity Sweeps

A bug found at one API site tends to come back at a sibling site that makes the same promise. The changelog shows it repeatedly: IsAdjacentTo (6.2.1), then IsStrictlyLeftOf and Except (7.0.0), then RangeSet.Except(TRange) (8.0.0). The wrapper arities kept lagging their primitive set. And ==/!= went untested on 18 of 20 set types. A claim-driven oracle checks each site against its own document, so it can't see a promise two sites share without saying so. These three sweeps (conventions project, one [DynamicData] row per discovered type) compare the two siblings directly. They assert no ground truth: when the siblings disagree, one of them is wrong, and the failure line names both answers.

  • SetWrapperParityTests: each primitive set against its wrapper arity closed over an identity element (Int32Set against Int32Set<Same<int>>, StringSet against StringSet<SameText>, in IdentityWrappers.cs). An identity element parses, formats and compares exactly as its primitive does, so the family's own code is the only difference. That's the opposite of WrapperElements, whose elements get something wrong on purpose. The sweep runs the family's probes plus its extremes, every subset shape, array literals derived from the element texts (quoted, whitespace-padded, duplicated, out of order, empty, NULL, malformed) and JSON documents (reordered, mistyped, quoted, padded). It compares every operation, ParseValue and FormatValue, both overloads of Parse and TryParse, ToString with each format and provider, the TypeConverter, JSON token kinds, and each sibling reading the other's document. It wants the same element texts in the same order, or the same exception type.
  • RangeSetLiftParityTests: each range against the one-element RangeSet holding it, over the shape matrix DomainParityOracleTests uses (RangeSpecs.cs), extended with bounds at the domain's minimum and maximum. For every ordered pair it compares each predicate with the set-set and set-range overloads, and each operation with the named method and the operator (|, &, -). For every range it compares each accessor, Merge, Complement, the literal, and Contains(T) at every grid point. Results are compared as literals, so the bound values and inclusivity have to match, not just a few Contains answers.
  • LiteralEntryPointParityTests: one literal through every entry point: Parse and TryParse (string and span), a JSON string, the TypeConverter and IConfiguration binding. It covers every range union and each sealed variant (the union's answer narrowed to the variant's shape), every multirange (including its explicit IParsable implementation) and every value set. The entry points must produce equal values, or all refuse. A value set's JSON is its element array, so it skips the JSON step here; SetWrapperParityTests covers that array. The empty string is an absent value to the TypeConverter and the binder by design, so the converter and the binder never see it.

The sweeps run in a few seconds altogether. Each one is seeded (test/oracles/seeds/), and their disagreements are candidates in candidates.json: #141 (wrapper JSON reads accept tokens the primitive refuses) and #109 (IsUnboundedStart/IsUnboundedEnd on (,)), found again here between a range and its own multirange.

The Parse(string) trap. In generic code constrained on IRangeFactory, IValueSetFactory or ISpanParsable<TSelf> itself, TSet.Parse(text, provider) with a string argument calls the span overload. Overload resolution drops the base interface's IParsable<TSelf> candidate whenever the derived interface's span candidate applies, and the implicit string-to-span conversion makes it apply. So a sweep that means to compare the two overloads compares the span one with itself. Until #142, 33 calls in ten test classes did this, so no generic sweep ever called a type's string Parse or TryParse. Call the string overloads through StringOverloads (test/oracles/StringOverloads.cs, linked like Candidates.cs, constrained on IParsable<TSelf> alone). StringOverloadConventionTests binds every Parse/TryParse call in every test project with Roslyn and fails on one that reaches ISpanParsable's span overload with a string. The seed string-parse-localizes-decimal-separator breaks only DecimalRange.Parse(string), and the literal sweeps missed it until #142.

Shape-Combination Matrix

Every binary operation test must cover all five range shapes: Finite, UnboundedStart, UnboundedEnd, EmptyRange, Infinity. Tests instantiate the "other" operand in each shape and verify the result type and value.

Example from RangeContainsTests.cs:

// Tests Finite vs each shape of "other" — interior, left-of, right-of, overlapping

Boundary Inclusiveness Permutations

For Finite ranges, tests cover all four inclusiveness combinations:

  • [start, end] — both inclusive (default for discrete)
  • (start, end) — both exclusive
  • [start, end) — lower inclusive (default for continuous)
  • (start, end] — upper inclusive

Discrete vs Continuous Split

Tests are parameterized by range type:

  • Discrete (Int32Range, Int64Range, DateRange; satellite: LocalDateRange, YearMonthRange) — canonicalization, adjacency with step awareness
  • Continuous (DecimalRange, DateTimeRange, DateTimeOffsetRange, TimeRange; satellite: LocalDateTimeRange, InstantRange) — half-open defaults, equality via IEquatable

Assertion Style

Use MSTest assertions (Assert.IsTrue, Assert.IsFalse, Assert.AreEqual, Assert.AreSame). For structural shape checks, cast to the specific interface (e.g., IFiniteRange<int>) and verify properties.

RangeSet Type Aliases

Use local type aliases for readability in tests:

using IntSet = CodoMetis.ValueRanges.RangeSet<CodoMetis.ValueRanges.Int32Range, int>;
using DecimalRangeSet = CodoMetis.ValueRanges.RangeSet<CodoMetis.ValueRanges.DecimalRange, decimal>;

Alias names must not collide with the value set types (DecimalSet, DateSet, TimeSet, …): test namespaces live under CodoMetis.ValueRanges, and namespace-member lookup beats file-level aliases — prefer the *RangeSet naming.

Mutation Testing (manual, occasional)

Stryker.NET is installed as a local tool (.config/dotnet-tools.json) and configured in test/CodoMetis.ValueRanges.Tests/stryker-config.json. It mutates the core library and runs only the core unit project against each mutant, so it measures that project alone; the conventions and integration sweeps are not consulted. The level is Standard, string literals and exception messages are not mutated, coverage analysis is perTest, and the scope is Internals/, RangeSet.cs and Sets/. It is a manual tool, to be run occasionally and never per commit. Do not add it to CI or to any script that runs on every change. Write the reports outside the repository:

cd test/CodoMetis.ValueRanges.Tests
dotnet stryker --output /tmp/stryker-$(date +%F)

The first run, for 10.0.0 (#91), took 7½ minutes on a 16-core machine. The configured scope took 5 minutes 7 seconds and scored 74.6 %. The rest of the core, run once with the mutate patterns inverted, took 2 minutes 22 seconds and scored 82.1 %. Together that is 76.7 % (1,197 of 1,560 mutants detected). About 230 mutants did not compile and Stryker dropped them. Its safe mode takes out every mutant of a method that has one, so DiscreteEnumeration's walk and the range JSON reader were not measured at all.

The surviving mutants were triaged by hand. Each one was applied to a copy of the tree and the conventions suite, which builds in seconds, was run against it. That gave three outcomes:

  • Caught by a conventions sweep. A seed was added where the defect was a new kind for that sweep.
  • Caught by nothing. The missing test was added: the ISO offset grammar, structural literal checks, explicit-format display text, continuous bound ties in RangeSet, hash codes, null operands, and a value-set JSON number split across buffer segments. Two oracles also gained lines. The multirange oracle now checks IsAdjacentTo. The value-set oracle now checks == and !=, which no test had called on 18 of the 20 core set types.
  • Unreachable. These went to #112.

A future run should be compared against this one, not against 100 %. Most of what remains is equivalent: buffer capacities, cache-or-recompute choices, and tie-breaks that canonical form makes impossible.

What to Test

  • Every public method on range types and RangeSet
  • All shape combinations for binary operations (5×5 = 25 cases per operation)
  • Boundary inclusiveness permutations for Finite ranges
  • Normalization invariants: empty filtering, overlap merging, adjacency merging, Infinity collapse
  • Round-trips: parse → format should produce equivalent ranges (exact string match for same-shape inputs)
  • Edge cases: empty input to RangeSet.From(), single-element fast path, Infinite.Except() complement

What Not to Test

  • Framework internals (MSTest, System.Text.Json)
  • Simple property getters on range variants
  • Code paths that are provably unreachable (private constructors, sealed subtypes)