Skip to content
Merged
14 changes: 9 additions & 5 deletions .claude/probe-axes.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,13 +46,17 @@ and `GeneratedHarnessTests` in the conventions tests holds each file to its app'
- How a row ends: a check returns null or the failure text, and runs through `Rec.Run`, which fails the
test for a violation or a harness error and skips it, with its reasons, for any other outcome but a
pass; a check that throws ends as a harness error, never a violation, and no seed counts it caught. A check
says what it saw with `r.Note(NoteKeys.Name, detail)`. The key, a member of `NoteKeys` in `harness/Row.cs`,
alone decides the outcome. Keys are constants the compiler checks: a new one goes into `NoteKeys` with its
outcome on one line, and a string, a misspelled member or a key built from text does not compile (decision
103). A probe outside the namespace `Probes.Shapes` writes `using NoteKeys = Probes.Shapes.NoteKeys;`.
says how it compared with `r.Note(NoteKeys.Name, detail)`. The key, a member of `NoteKeys` in `harness/Row.cs`,
alone decides the note's outcome: every key is a pass key but `framework-refuses`, the three-sided
differential's expected refusal, and a check that did not apply, refused or could not tell says so through
its scope with a citation (below), never a note key. Keys are constants the compiler checks: a new one goes
into `NoteKeys` with its outcome on one line, and a string, a misspelled member or a key built from text does
not compile (decision 103). A probe outside the namespace `Probes.Shapes` writes `using NoteKeys = Probes.Shapes.NoteKeys;`.
Never match a failure's text to excuse it. A difference the framework makes on its own is shown by a
baseline in the same row (the document as ASP.NET writes it without `AddTypeKit()`,
`Hx.DocRefusedByAspNetAlone`), not by a substring. A row that would fail for an open finding calls
`Hx.DocRefusedByAspNetAlone`, a refusal compared by exception type and stage as `Sides.Alike` compares one),
not by a substring. A document both refuse alike is the framework's refusal for a check that compared nothing
else, and leaves a check that compared an answer on the wire first to be judged by that answer. A row that would fail for an open finding calls
`r.Skip("F7", why)`, the finding as a string literal: `register` and `publish` refuse a skip whose finding
is built from text or is not open in the ledger, naming the file and line. The gate refuses while any row
skips, and the replay fails a skip whose finding the ledger has fixed or dismissed.
Expand Down
4 changes: 2 additions & 2 deletions .claude/skills/probe-triage/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,8 +122,8 @@ dotnet tools/BuildVerdict/bin/Release/net10.0/build-verdict.dll test --project t

- the citation, where the corpus holds one: a `not-promised` entry of the claim whose `source` is the
fact's file and whose `sentence` is quoted from it (step 3's form), an `exclude` group whose `reason`
names `KB-NNN`, or a value in the axes file's `refused` object whose text names `KB-NNN`. Where the
check excuses the row by a note key, the key's line in `NoteKeys` names the fact in its comment;
names `KB-NNN`, or a value in the axes file's `refused` object whose text names `KB-NNN`, or the check's
own `Cite.Fact("KB-NNN")`;
- with a finding id:

```bash
Expand Down

Large diffs are not rendered by default.

5 changes: 3 additions & 2 deletions design/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,12 +105,13 @@ per area ([design](../plan.md)) link to the records they rest on.
| [0096](0096-ci-replays-every-probe-and-the-release-runs-the-gate.md) | CI replays every probe, from an empty state, on every push and pull request, and a failing probe fails the run; the release replays them and runs the gate before it packs, the gate advisory; held by `ReleaseWiringTests` | 2026-10-15 | accepted; the gate binds from 2.2.0 by [0097](0097-2-2-0-is-a-consolidation-release-and-the-gate-binds.md); the build workflow's replay runs by family and footprint by [0101](0101-ci-replays-by-family-and-footprint.md) |
| [0097](0097-2-2-0-is-a-consolidation-release-and-the-gate-binds.md) | 2.2.0 is a consolidation release: no public API, its additive items move to 2.3.0, and the probe gate binds from it | 2026-10-15 | accepted |
| [0098](0098-a-null-passes-no-comparison-in-a-query.md) | A nullable record-class value object compared with `<`, `<=`, `>` or `>=` in a query answers as SQL does, leaving out its null rows; documented, not refused | 2026-10-15 | accepted |
| [0099](0099-a-probe-row-says-how-it-ended.md) | A probe row says how it ended: notes are a closed catalogue of keys and outcomes, a row that earned nothing is a skipped test labelled with its settings, a skip for a finding fails the replay once the finding is fixed and the gate while it stands, and a refused document is judged against ASP.NET alone | 2026-10-15 | accepted; note keys are a typed catalogue the compiler checks by [0103](0103-a-harness-error-is-never-a-catch-and-note-keys-are-constants.md) |
| [0099](0099-a-probe-row-says-how-it-ended.md) | A probe row says how it ended: notes are a closed catalogue of keys and outcomes, a row that earned nothing is a skipped test labelled with its settings, a skip for a finding fails the replay once the finding is fixed and the gate while it stands, and a refused document is judged against ASP.NET alone | 2026-10-15 | accepted; note keys are a typed catalogue the compiler checks by [0103](0103-a-harness-error-is-never-a-catch-and-note-keys-are-constants.md); a refused document is compared by type and stage, and earns nothing in a check that compared nothing else, by [0108](0108-a-probe-check-earns-credit-against-a-baseline-and-ws2-closes.md) |
| [0100](0100-every-probe-is-shown-to-fail-against-a-seeded-defect.md) | Every registered probe names a seeded product defect, and `prober seeds` (local, a build per seed) requires a violation it causes; the prober tests hold that every seed exists and applies; pairwise coverage is a floor, three-way combinations belong to the variant catalogue | 2026-10-15 | accepted; a seed per obligation of a claim by [0104](0104-a-claim-states-its-obligations-and-each-has-a-seed.md); seeds run by footprint, a release `--all`, by [0107](0107-a-cited-fact-is-executable-and-seeds-run-by-footprint.md) |
| [0101](0101-ci-replays-by-family-and-footprint.md) | The build workflow replays the probes in legs by family, each running its probes in parallel, under a time limit: a push every probe with coverage and the merged state saved, a pull request only the probes it stales against its base branch's saved state and without a footprint; `prober merge` joins the legs and refuses a probe no leg saw | 2026-10-16 | accepted |
| [0102](0102-an-exclusion-carries-its-reason.md) | Each `exclude` entry of claims.json is a required one-sentence reason and its pairs; the prober refuses a claims file its schema does not match, naming the claim and the field, so an invented reason field is refused; `prober universe` prints each exclusion's reason beside the claim's narrowings; the prober writes JSON with only JSON's own escapes, so an unchanged publish leaves a tracked file as it is | 2026-10-16 | accepted |
| [0103](0103-a-harness-error-is-never-a-catch-and-note-keys-are-constants.md) | A harness error is never a catch: `Rec.Run` ends a row whose check throws as a harness error, not a violation, and `prober seeds` credits only a violation no harness error raised; note keys are members of a typed `NoteKeys` the compiler checks, and `register`, `publish` and a conventions test refuse a note or a skip whose key or finding is not written out | 2026-10-16 | accepted |
| [0104](0104-a-claim-states-its-obligations-and-each-has-a-seed.md) | A claim lists its `obligations`, each a sentence quoted verbatim from its source, and what it does not promise in `not-promised`, each quoted from the document or record that makes it intended; a claim without the field has one obligation, `claim`; a probe carries a seed per obligation it checks, and the check without a build refuses a seed naming an unlisted obligation and a listed obligation no seed names; `status` shows each claim's obligations, and `prober gaps` reports what the corpus leaves out, deciding nothing | 2026-10-16 | accepted |
| [0104](0104-a-claim-states-its-obligations-and-each-has-a-seed.md) | A claim lists its `obligations`, each a sentence quoted verbatim from its source, and what it does not promise in `not-promised`, each quoted from the document or record that makes it intended; a claim without the field has one obligation, `claim`; a probe carries a seed per obligation it checks, and the check without a build refuses a seed naming an unlisted obligation and a listed obligation no seed names; `status` shows each claim's obligations, and `prober gaps` reports what the corpus leaves out, deciding nothing | 2026-10-16 | accepted; `gaps` reads the last replay's evidence too by [0108](0108-a-probe-check-earns-credit-against-a-baseline-and-ws2-closes.md) |
| [0105](0105-every-tool-is-built-in-the-session-s-checkout.md) | SessionStart builds the guard in `$CLAUDE_PROJECT_DIR` and every other tool, through `tools/AgentTools.slnf`, in the session's own checkout; a hook, the prober, the smoke test and the verdict fall back to the SDK installer's `dotnet` when the one on PATH lacks the pinned SDK, and a session start that builds nothing tells the agent; the guard lets `dotnet msbuild -getProperty/-getItem` without a target through | 2026-10-17 | accepted |
| [0106](0106-a-parameter-of-a-value-object-over-a-described-type-is-inlined.md) | A value object over a type the JSON options describe is written inline as a route, query, header or form parameter and as an element of a query or header array, where value objects are components: the component is its JSON positions'; every other value object keeps its reference; the schema below a parameter is asked for with the parameter's source and type alone | 2026-10-17 | accepted |
| [0107](0107-a-cited-fact-is-executable-and-seeds-run-by-footprint.md) | A fact that prunes or excuses a row is executable: `docs-query check` fails `fact-not-executable-cited` where a refused entry, a not-promised entry, an obligation's declaration or the harness cites a fact without its test; a fact's measured answer is `KBNNN.fixture.json` beside its test, named by `fixture:`, read by a BCL-only reader the harness will use; `prober seeds` runs only the probes whose seeds identity or product footprint changed since every seed was caught, and a release runs `--all` | 2026-10-17 | accepted |
| [0108](0108-a-probe-check-earns-credit-against-a-baseline-and-ws2-closes.md) | A probe row runs a check per obligation, each against a baseline, and the ledger credits checks; an excuse is a closed citation, applicability is declared on the obligation, the http, json and text harness compares three sides by exception type and stage, decision 99's refused document among them, and the twins and the context are generated with the shapes; `NoteKeys` keeps its pass keys and `framework-refuses`; `prober fix`, a covering array that places no blocked pair, and `gaps` listing the pairs no passing check read both axes of; WS2 closes, and WS3 to WS6 are dropped | 2026-10-17 | accepted |
6 changes: 6 additions & 0 deletions design/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@ defect, a cost or a missing guard may land here; one that adds surface waits for

- Grow the probe corpus until `prober gate` passes on probes and reviewed waivers alone. Every claim
touched since v2.1.0 is covered pairwise, and no finding is open.
- The probing harness (WS2 of the agentic issue-finding concept) is done
([decision 108](adr/0108-a-probe-check-earns-credit-against-a-baseline-and-ws2-closes.md)). What it leaves the gate: the pairs of C12, C13, C16 and C21 that only a document
ASP.NET alone refuses had credited (a declaration of the document obligations' inapplicability under
`json.ReferenceHandler=Preserve` and `json.MaxDepth=3`, and rows for the rest), and C22's empty footprint.
WS3 (beyond pairs), WS4 (the mutation nightly), the second half of WS5 (the finder benchmark at equal budgets) and
WS6 (PostgreSQL in CI) were dropped on 2026-10-17 by the maintainer's decision, and are not scheduled.
- The generated-surface snapshot under "Generators" below.

## 2.3.0, additive
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ public sealed class ProbeNoteKeyTests
[Fact]
public void The_corpus_notes_are_found() =>
Calls(Sources()).Count(c => c.Method == "Note").ShouldBeGreaterThanOrEqualTo(100,
"the corpus made 136 notes when this rule was written, and 127 once the http family's outcome notes became checks' citations (WS2 step 4)");
"the corpus made 136 notes when this rule was written, 127 once the http family's outcome notes became checks' citations (WS2 step 4), " +
"and 119 at WS2's close, where every note is a pass key or framework-refuses");

[Fact]
public void Every_note_and_skip_in_the_corpus_names_its_key_as_written() =>
Expand Down
80 changes: 80 additions & 0 deletions test/CodoMetis.TypeKit.Prober.Tests/CoveringTests.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
using Prober;

namespace CodoMetis.TypeKit.Prober.Tests;

/// <summary>
/// The covering array never puts an excluded pair or one no obligation wants in a row, not even to fill an axis whose every
/// value would bring one (#54 and #59 saw `rows` place them): such an axis is left out of that row, and an axis with no wanted
/// pair at all is left out of every row.
/// </summary>
public sealed class CoveringTests
{
private static Inapplicability Declared(string cite, params (string Axis, string[] Values)[] axes) =>
new(axes.ToDictionary(a => a.Axis, a => a.Values.ToList(), StringComparer.Ordinal), cite);

private static Claim Over(string[] axes, IEnumerable<Pair>? excluded = null, params Inapplicability[] declared) => new()
{
Id = "C1", Source = "README.md#a", Surfaces = ["s"], Watched = [], Excluded = [.. excluded ?? []],
Obligations = declared.Length == 0 ? [] : [new("keys", "The claim.", new SortedDictionary<string, List<string>>(StringComparer.Ordinal)) { Inapplicable = declared }],
Axes = new(axes.ToDictionary(a => a, _ => new List<string> { "1", "2" }), StringComparer.Ordinal),
};

private static bool Holds(IReadOnlyDictionary<string, string> row, Pair pair) =>
new[] { pair.A, pair.B }.All(item => row.TryGetValue(item[..item.IndexOf('=', StringComparison.Ordinal)], out var v) && v == item[(item.IndexOf('=', StringComparison.Ordinal) + 1)..]);

private static IEnumerable<string> Shown(IEnumerable<Pair> pairs) => pairs.Select(p => p.ToString()).Order(StringComparer.Ordinal);

[Fact]
public void An_axis_whose_every_value_brings_an_excluded_pair_is_left_out_of_the_row()
{
// x=1 goes with no y (as shape.containing goes with no shape.position=root): a row holding x=1 holds no y.
Pair[] excluded = [new("x=1", "y=1"), new("x=1", "y=2")];
var claim = Over(["x", "y", "z"], excluded);
for (var seed = 1; seed <= 20; seed++)
{
var rows = Covering.Rows(claim, claim.Pairs(), 2000, seed);
rows.ShouldNotContain(r => excluded.Any(p => Holds(r.Row, p)), $"seed {seed}");
rows.Where(r => r.Row["x"] == "1").ShouldAllBe(r => !r.Row.ContainsKey("y"), $"seed {seed}");
Shown(rows.SelectMany(r => r.Pairs)).ShouldBe(Shown(claim.Pairs()), $"seed {seed}: every pair once, x=1 with z among them");
}
}

[Fact]
public void An_axis_whose_every_value_brings_a_pair_no_obligation_wants_is_left_out_of_the_row()
{
var claim = Over(["x", "y", "z"], null, Declared("record:0001", ("x", ["1"]), ("y", ["1", "2"])));
var dead = claim.DeclaredInapplicable().Keys.ToList();
dead.Count.ShouldBe(2);
for (var seed = 1; seed <= 20; seed++)
{
var rows = Covering.Rows(claim, claim.PairsWanted(), 2000, seed);
rows.ShouldNotContain(r => dead.Any(p => Holds(r.Row, p)), $"seed {seed}");
Shown(rows.SelectMany(r => r.Pairs)).ShouldBe(Shown(claim.PairsWanted()), $"seed {seed}");
}
}

[Fact]
public void An_axis_with_no_wanted_pair_is_in_no_row()
{
// every pair of z is excluded (as shape.refused-kind is where the refusal is not the claim): rows over x and y alone.
var claim = Over(["x", "y", "z"], [.. from x in new[] { "x=1", "x=2", "y=1", "y=2" } from z in new[] { "z=1", "z=2" } select new Pair(x, z)]);
for (var seed = 1; seed <= 5; seed++)
{
var rows = Covering.Rows(claim, claim.Pairs(), 2000, seed);
rows.ShouldAllBe(r => !r.Row.ContainsKey("z"), $"seed {seed}");
Shown(rows.SelectMany(r => r.Pairs)).ShouldBe(["x=1 y=1", "x=1 y=2", "x=2 y=1", "x=2 y=2"], $"seed {seed}");
}
}

[Fact]
public void A_seeded_row_leaves_out_an_axis_no_value_of_which_can_join_it()
{
// c=1 goes with no d: the row seeded from a=1 c=1 is a, b and c.
var claim = Over(["a", "b", "c", "d"], [new("c=1", "d=1"), new("c=1", "d=2")]);
Covering.Seeded(claim, new Pair("a=1", "c=1"), claim.DeclaredInapplicable())
.ShouldBe(new SortedDictionary<string, string>(StringComparer.Ordinal) { ["a"] = "1", ["b"] = "1", ["c"] = "1" });
Covering.Seeded(claim, new Pair("a=1", "b=2"), claim.DeclaredInapplicable())
.ShouldBe(new SortedDictionary<string, string>(StringComparer.Ordinal) { ["a"] = "1", ["b"] = "2", ["c"] = "1" },
"c=1 is c's first value, and d then has none");
}
}
Loading
Loading