Repository navigation
build: close WS2: its leftovers, the corpus replayed and seeded, record 0108 - #61
Merged
Merged
Conversation
WS2's conversions made every excuse a check's cited Inapplicable, Refused or Inconclusive, and the three-sided differential retired the last text-matching notes, so 41 outcome keys were declared and no harness or probe source called them: every inapplicable key (23), every expected-refusal key but framework-refuses (17) and the one inconclusive key. They are removed. The catalogue holds 27 keys: 26 pass keys and framework-refuses (63 at the start of WS2, 68 before this commit). - The NoteKeys summary says how a non-pass outcome is reached now; the probe-axes rules and the triage skill no longer speak of excusing a row by a note key. - C21's closed-set exclusion reason no longer says the probe notes closed-set-differs, a key that no longer exists. - ProbeNoteKeyTests' floor counts Note calls, not keys: it stays at 100, and its message records 119 calls at WS2's close (none was removed; every removed key was uncalled). Seeded: Sides.cs noting NoteKeys.StjMaxdepth instead of FrameworkRefuses fails the probes' build (prober try) with CS0117 'NoteKeys' does not contain a definition for 'StjMaxdepth'; restored, C10's rows pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
F3's fixed line (#45) was written into findings.jsonl by hand, since no command wrote one. `prober fix F<n> --commit <sha> [--reason TEXT]` appends the fixed line as finding-line.schema.json wants it: the open line's claim and probe, status fixed, the commit's full id and the reason when given. It refuses an id the ledger does not hold open (never seen, already fixed, withdrawn or dismissed) and a commit the repository does not have, and says so when the commit is not yet in HEAD's history, where the line counts once it is (Ledger.Findings' rule). FixTests (6 cases) fail against each seeded defect, one at a time: - the commit written as given, not resolved: the line test - any status accepted: the already-fixed refusal - an unknown id taken as open: the unknown-id case - the claim not copied, the probe not copied: the line test, each - any commit accepted: the missing-commit case Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ed pair #54 (C11) and #59 saw `prober rows` put excluded pairs in rows. Where every value of an axis brought a blocked pair (an excluded one, or one no obligation wants), the greedy pick took the least bad value anyway, and the seeded fallback filled an axis with its first value when none was free; the fallback also never looked at the exclusions. Now a candidate row that needs a blocked pair is dropped, and the seeded row searches (depth first, in axis and value order, bounded at 100,000 steps) for a fill that holds no excluded pair and no pair no obligation wants. A pair no such row can hold is given up, not placed: it stays in uncovered_after. With nothing blocked the rows are the same as before (the same values in the same random order). CoveringTests (4 cases) fail against each seeded defect, one at a time: - the greedy pick takes a blocked value: both no-blocked-pair cases - the seeded row checks the declared pairs only: the excluded case, the backtracking case and the give-up case - no backtracking: the backtracking case - a fill placed anyway when the search fails: three cases - a pair given up is placed in a first-value row: both no-blocked-pair cases Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#54 counted by hand the four C11 pairs no row could cover: they were wanted by the three parse-shape obligations, whose checks never read shape.wrapped, and the one obligation that reads it declared them inapplicable. `prober gaps` now ends with "pairs no passing check read both axes of (the last replay's evidence)": for each claim whose probes have evidence, every wanted pair for which no obligation that wants it has a passing check that read both of the pair's axes. More rows cannot cover such a pair; a check that reads both axes, an exclusion or a declaration can. It reads the state's evidence, beside the tracked corpus and the generated axes it read before, and still decides nothing. GapsTests (5 cases, one new) fail against each seeded defect: - a check that did not pass counted as reading its axes: the new case - an obligation's reads counted for a pair it does not want: the new case (x=1 z=1 is no longer listed) - a claim without evidence listed: the new case and both none cases - the section dropped: all five Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ToolchainTests.A_dotnet_without_the_pinned_sdk_falls_back_to_the_installers starts the built prober as a process of its own, which inherited the test runner's environment. Under PROBE_STATE_DIR (every replay script of WS2 set it) that prober recorded the run in the caller's state, not in the test repository's, and `status --json` found no claim C1: a NullReferenceException at ToolchainTests.cs:122, read as "flaky under load" in #53 and #57 and traced in #58. The test now removes PROBE_STATE_DIR, PROBE_CACHE_DIR and PROBE_HOLDER from the process's environment. Proven by the variable itself: before this change the test fails with PROBE_STATE_DIR set (1 of 12 ToolchainTests, the NullReferenceException) and passes without it; after it, all 12 pass with PROBE_STATE_DIR set, without it, and with PROBE_HOLDER and PROBE_CACHE_DIR set to strangers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…in a row The previous commit gave a pair up where no full row could hold it without a blocked pair. Measured with `rows --claim C --full` over the whole corpus (the claims at ac4c561), that made C09, C11 and C18 get no row at all and left 77 of C01's pairs uncovered (3288 pairs in all): their exclusions say an axis does not apply beside a value of another, so every full row holds one (shape.refused-kind wherever the claim excludes each of its pairs, shape.containing beside shape.position=root, shape.wrapped beside shape.kind=closed-set). The old tool filled such an axis with an excluded value, which the harness then ignores. Now an axis none of whose values can join the row without an excluded pair or one no obligation wants is left out of that row, by the greedy pick and by the seeded fill alike; no pair is given up. The harness already reads those axes as optional where they do not apply (a closed set reads no shape.wrapped, the root no shape.containing, C09, C11 and C18 never read shape.refused-kind); one that reads a left-out axis ends as a harness error, never a pass. The whole corpus, `rows --claim C --full` for its 44 claims, old tool (9e23403, the same Covering as ac4c561) against this one: 7746 rows of which 740 held an excluded pair, against 7842 rows of which none holds an excluded pair or one no obligation wants; 0 pairs left uncovered by either. Rows differ only in the ten claims whose old rows held one. CoveringTests (4 cases) fail against each seeded defect: - the greedy pick takes a blocked value: both left-out cases - the seeded fill takes the first value where none can join: the seeded case - the seeded fill checks the declared pairs only: the seeded case Leaving the axes no wanted pair takes part in out up front was tried and removed: the greedy pick already leaves them out, and no case told it apart (seeded, it failed nothing). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Decision 99's document rule was the last place a refusal's words decided a row: Hx.DocRefusedByAspNetAlone and FaultClaim's copy passed a refused document when ASP.NET alone refused it with the same message (#58, "Not changed"). Both now compare the two refusals as side (1) and side (3) are compared (Hx.DocAlike: Sides.Alike, the exception type and a JsonException's Path, no message), and Hx.Unplaced, which cut the line and byte out of a message, is gone. Where both refuse alike the document part has nothing to check (Hx.DocRefused): - a check that compared nothing before the document is the framework's refusal, framework-refuses, an expected refusal that earns nothing (C21's schema checks, C12's, C16's, C13's listing where the options do not reject an absent patch); - a check that compared an answer on the wire first (the fault claims, C03 and C05, C13's listing under RespectRequiredConstructorParameters) is judged by that answer, doc:refused-by-aspnet-alone, a pass, as decision 99 says ("judged by its other checks"). Measured: the http family replayed from empty states without footprint at 26e51e6 before and with this change, 13 passed both times. Row for row, 401 rows no longer pass (C12 156, C13 23, C16 93, C21 129); in each of them every check that passed before had noted a document both hosts refused and compared nothing else. 12 rows pass that did not (C07 1, C14 3, C15 8): under json.MaxDepth=3 both documents are refused with an InvalidOperationException whose messages differ, which ended the check on record:0099 although it had compared the wire. Checks: 12482 pass before, 10010 after. The pairs those 401 rows alone credited are no longer covered: C12 821, C13 48, C16 455, C21 677, nearly all under json.ReferenceHandler=Preserve or json.MaxDepth=3, where decision 99 says System.Text.Json refuses the document for plain ASP.NET too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record 0108 (0105, the number the brief named, was taken by "every tool is built in the session's checkout"), with its index row and status notes on 0099 (the document rule) and 0104 (gaps reads evidence). The roadmap's 2.2.0 section says WS2 is done, what it leaves the gate, and that WS3 to WS6 were dropped on 2026-10-17 by the maintainer's decision. design/plan.md lists no individual records and is unchanged. The record's evidence: 265309f against 5c5affe replayed from empty states (43 of 43 in 677 s, 45 of 45 in 818 s), row for row per probe (400 regained, 166 lost, each with its reason), the document rule's A/B, the covering array on the whole corpus, pairs per claim, C07's pass share by check, NoteKeys 63 -> 68 -> 27, the product defects found, seeds --all (170 of 170 caught, each by its own obligation's check) and the gate's output with what each refusal needs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WS2's close-out: its leftovers, the whole corpus replayed and seeded, and its record. The record is 0108, not 0105: 0105 was taken by "every tool is built in the session's checkout" on the same day, and records are append only.
The gate's verdict: refused, 17 refusals, 2,838 pairs uncovered across 16 claims, 0 open findings; 2,001 of those pairs come from leftover 3 (full output and what each refusal needs at the end).
Commits
test:NoteKeyskeeps the pass keys andframework-refuses: the 41 outcome keys no harness or probe source called are removed (68 → 27; 63 at the start of WS2). Seeded: a removed key called again fails the probes' build with CS0117.ProbeNoteKeyTests' floor countsNotecalls, not keys; none was removed (119 calls), so the floor stays 100 and its message says so.build:prober fix F<n> --commit <sha> [--reason]writes the fixed line asfinding-line.schema.jsonwants it (F3's was hand-written in #45). 6FixTests, 6 seeds caught.build:Covering.Rowsnever places an excluded pair or one no obligation wants, not even as a fallback fill; an axis none of whose values can join a row is left out of that row. The first commit gave such pairs up, which the whole-corpus measurement showed wrong (C09, C11, C18 got no rows; 3,288 pairs uncovered); the second replaced it. Whole corpus,rows --claim C --full× 44: 7,746 rows of which 740 held an excluded pair → 7,842 rows, none holding one, 0 pairs uncovered either way. 4CoveringTests, every seed caught.build:prober gapslists, from the last replay's evidence, the wanted pairs no obligation wanting them has a passing check that read both their axes of (#54's four C11 pairs). 1 newGapsTestscase, 4 seeds caught. On today's corpus: none.test:ToolchainTests.A_dotnet_without_the_pinned_sdk_falls_back_to_the_installerskeepsPROBE_STATE_DIR,PROBE_CACHE_DIRandPROBE_HOLDERout of the prober it starts: it failed withPROBE_STATE_DIRset (the "flaky under load" of #53 and #57), and passes with it set now.test:Hx.DocAlike=Sides.Alike);Hx.Unplacedis gone, no message comparison is kept. A check that compared an answer on the wire before the document is judged by it (pass); one that compared nothing else endsframework-refuses. See below: it loses no row that compared something, and uncovers 2,001 pairs.docs:design/plan.mdlists no individual records, so it is unchanged.Leftover 3, for review: what the document rule's conversion costs
The rule was "convert where that loses no row that compared something". Measured on the http family, row for row, at 26e51e6 before and with the change: 401 rows no longer pass (C12 156, C13 23, C16 93, C21 129), and in every one of them each check that had passed noted a document both hosts refused and compared nothing else (checked over the evidence, 0 exceptions). 12 rows pass that did not (C07 1, C14 3, C15 8:
MaxDepth=3documents refused with different messages, now alike by type, judged by the wire they compared). The fault claims keep their passes: their checks compare the wire first.The cost is coverage. Those 401 rows alone had credited 2,001 pairs (C12 821, C13 48, C16 455, C21 677), nearly all under
json.ReferenceHandler=Preserveorjson.MaxDepth=3, where decision 99 says System.Text.Json refuses the document for plain ASP.NET too. They are now uncovered and the gate refuses them. What they need: a declaration of the document obligations' inapplicability under those settings (citing record 0099 or a fact), and rows that hold the rest of those pairs without them. That is not done here. If you would rather keep the passes until those declarations exist, reverting 5c5affe alone restores them.Seeded: the value-object schema transformer throwing
InvalidOperationExceptionfails, in 8 sampled C21 rows, the 4 whose document ASP.NET alone writes, before and after. Of the 4 ASP.NET alone refuses, before the change 3 passed (same words) and #156 failed on a differing message; after it all 4 endframework-refuses. #156 is what comparing no message gives up.Measurements (all in the record)
seeds --probe Pfor all 45 probes in two lanes of separate worktrees from the final replay's state (about 3 h each): 170 seeds, 170 caught, each by a check of its own obligation; none missed, no seeded build failed. The table is in the record.docs-query check: 0 problems.Not done, recorded as open in 0108
The EF twins at the value object's nullability (#59); #59's untriaged candidate rows from regenerated arrays it did not commit (listed as open leads); the narrowing of an obligation's wanted pairs by its own axes (#54's option 2);
polymorphic-base-propertyas ashape.positionvalue (#51).The gate
src/andfootprintrefuses an empty one (#54):watchedpaths for the generators, as C18 has, so its footprint records them. Its 621 pairs follow.json.ReferenceHandler=Preserveandjson.MaxDepth=3, citing record 0099 or a fact, and rows (rows --probe) for the pairs that remain. C21's other 21 were open at ac4c561.rows --probe, which now leaves such an axis out instead.where-value-comparewith a wrapped type that has no>=(the probe substituteswhere-equals): an exclusion with that reason; and onegroup-byrow fornested-two-levels×DateTimeOffset.rows --claim, or exclusions with their reasons.None is a finding: 0 are open. Every refusal is answered by a probe's rows, a declaration, an exclusion, or a waiver naming the maintainer as reviewer (decision 97); this record answers none of them.
🤖 Generated with Claude Code