Repository navigation
test: the http family checks each obligation (WS2 step 4) - #53
Merged
Merged
Conversation
BaseProbe's wire half (HttpBaseline.Run): per side the echo, the request read with the row's input.json and the binding with a valid and a refused value are each a Row.Check of the implicit obligation claim (C07 promises nothing of the wire), against the twin, a closed set's own response read back, or the 400 the AspNetCore README's Binding section documents. An intended difference cites the sentence that makes it intended (Cite.Docs); one no document makes intended is compared (MVC under MaxValidationDepth=1, a byte array's base64) or, where the row's twin cannot be built or fails alike, ends inconclusive. C21's schema half (HttpBaseline.Schema) starts the host once and runs a check per obligation the row selects, every one, not stopping at the first failure; its excuses cite C21's not-promised entries. The text-matching sites stay, each marked with its baseline gap (HttpBaseline's depth and ValidationProblemDetails matches, the three OpenAPI 2.0 and enum schema matches, Positions.cs's depth match). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FaultClaim.Run tags each comparison with an Aspect (the start, a member left out, a member's status, fault, type and detail, the operation's statuses, their order, the problem schema and its problem types), and a claim's Spec.Of names the obligations each is checked for. The row runs one check per obligation it reaches, every check on one host with the answers fetched once, against the claim's README or CHANGELOG section, or the refusal for a host that must not start and a member left out. A document both refuse under MaxDepth=3 cites decision 99. C06's member-must-have-kind runs beside the rest instead of before it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ComposedClaims.WireAndDoc runs one check per obligation the claim names an aspect for (the answer's status, fault, type, detail and rule, and the operation's statuses, problem schema and problem types), on one host with the answers fetched once. C03's unmapped-set-stops-start and C05's unnamed-value-throws are checks against the refusal that run beside the rest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each of SchemaChecks' claims gathers what its host or hosts answered and wrote once, then runs one check per obligation: C08 per host's naming and leave-out, and the warning's names; C12 the source's own at a property, every position, and the decimal converter's parameter against ASP.NET's own; C13 the document's required against the wire, and what an absent patch reads; C16 the refusal, its type, its descriptions and the patch-held type apart, and the documents that build. C13's MVC body that MVC's validator refuses for the plain twin too ends inconclusive, uncited. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A struct value object over a class has no twin at its nullability either, not only an enum or Instant under a source-generated context; the reason now names the inputs that differ. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s the citations ProbeCiteTests read AgreeRun.Refused(c, ...) as a check's Refused without a citation; it is Refusal now. ProbeNoteKeyTests' floor of 130 Note calls guards against a walker that finds nothing: the http family's outcome notes became Check.Inapplicable/Refused citations, leaving 127, so the floor is 100. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ation The schema check decided from json.Converters alone that an application's own converter, for the value object or for the wrapped type, leaves nothing to compare (not-promised own-converter-for-the-value-object and own-converter-for-the-wrapped-type). Each of C21's five obligations now declares both (claims.json inapplicable, decision of PR #50), in place of the two exclusions of 168 pairs each: universe counts the same 336 pairs as wanted by no obligation, and Row.Check records such a check before its body runs. The code's lines stay for the schema half run under another claim. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Triage (.claude/skills/probe-triage) of the rows the conversion left failing: - 28 rows under mvc.MaxValidationDepth=1, "validated value object bound the refused value": triage 1 found no harness fault and no value object holding a refused value. Every answer was the value object's default (204 for a class, 200 with the default for a struct), and in one row per binding whose refused value is not its default (#122 body, #244 form, #238 header, #140 query, #313 as-parameters, #201 route) that is unambiguous. Triage 2: MVC's own behaviour, pinned as KB-037 (Facts KB037): under MaxValidationDepth = 1 an [ApiController] action whose simple parameter MVC cannot bind runs with the parameter at its default and answers 200, where the default depth gives a 400; a class from the query makes the validator throw at its property. The check now requires the value object to hold its default and sends the twin a text it cannot read in the same host: both at their default cite fact:KB-037, the twin rejecting it is a violation, anything else is inconclusive. - Row 83, a byte array: triage 2, KB-038 (Facts KB038): System.Text.Json writes byte[] as one base64 string. Triage 3: docs/json.md "The wrapped type's JSON" now says an array of value objects over byte is a JSON array of numbers, cited by the echo and read checks, and claim L01-json-bytes lists it as not promised (a-byte-array-is-base64), quoting that sentence. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WS2 step 4, the http family: every probe whose claim's first surface is http (BaseProbe, C03 to C08, C12 to C16, C21) runs a
Row.Checkper obligation its row reaches, named by the claim's obligation id and given its baseline, and every excuse of outcome inapplicable, expected-refusal or inconclusive is a citedc.Inapplicable(Cite…)/c.Refused(Cite…)or ac.Inconclusive(reason), except the text-matching sites this step keeps. Each http claim's obligations now earn credit on their own:prober statusshows 0 uncredited for all 12 http claims (before: every obligation of C03 to C16 uncredited), and every seed of every http claim is caught by its own obligation's check.What changed
HttpBaseline(BaseProbe's wire half, C21's schema half). Per side, the echo, the request read with the row'sinput.json, the binding of a valid value and of a valueCreaterefuses are each a check of their own. BaseProbe's checks are the implicitclaim: its rows are C07's, C07 promises nothing of the wire, and its seed names no obligation. Baselines: the twin; a closed set's own response read back (RoundTrip); the AspNetCore README's 400 for a refused binding (Docs(…#binding)). C21'sSchemastarts the host once and runs every obligation's check. Before,??=stopped at the first failure; now each obligation's check runs even when another fails.FaultClaim(C04, C06, C07, C14, C15). Each comparison is tagged with anAspect: the start, a member left out, a member's status, fault, type and detail, the operation's statuses, their order, the problem schema and the problem types it names. A claim'sSpec.Ofnames the obligations each aspect is checked for. One check per obligation runs against the claim's README or CHANGELOG section, or againstRefusalfor a host that must not start and for a member left out. The answers are fetched once, on one host.ComposedProbes(C03, C05). These follow the same shape. C03'sunmapped-set-stops-startand C05'sunnamed-value-throwsare refusal checks. They now run beside the rest instead of short-circuiting it.SchemaClaim(C08, C12, C13, C16). Each claim gathers what its host or hosts answered and wrote once, then judges it per obligation:Framework.requiredagainst the wire, and what an absent patch reads.NoteKeysis untouched. The http family no longer calls an outcome key except at the three text-matching sites below. The pass keys stay asc.Note.Commits:
b0195d3(the http baseline and C21),2dbb715(the fault claims),ab0057c(C03, C05),d15b2b4(the schema claims),90918a4(C21's inconclusive reason names the inputs),ddd146c(C16's judge renamed off the lint's.Refused(, and the Note floor),a481101(release/2.2.0 merged in, with #50's declared applicability),db7e3aa(C21 declares its own-converter rows inapplicable),8c92e32(release/2.2.0 merged in again, with #51 and #52),631e7c7andaf6e8d6(BaseProbe's 29 findings triaged: KB-037, KB-038, a docs sentence, L01's not-promised entry),658d173(release/2.2.0 merged in again, with #54, the build family).Before and after, per probe
Before: release/2.2.0 at
cfc9a2a, replayed from an empty state in a worktree of its own (replay --family http --no-footprint: 13 passed). After: this branch at658d173,replay --family httpwith footprints: 13 passed. Release/2.2.0 was merged in three times: ata481101(#50'sRow.csre-staled every probe),8c92e32(#51 and #52 changed the OpenAPI transformers and the generators the probes run) and658d173(#54). Before the triage, the replays atd15b2b4,ddd146c,db7e3aaand8c92e32gave the same outcomes, row for row: 12 passed, and BaseProbe failed on the 29 rows triaged below. The categories are each row's and each check's from the evidence. A row that runs noRow.Checkcounts as one check, its row. "Rows passing" is whatRec.Runpasses: no violation or harness error in the row or any of its checks, and at least one check that passed.The rows regained, 238 of them, are all rows that earned nothing before, and no row that passed before fails now.
requiredand the minimal side are checked beside it.reporting-left-out-throws, and C06'sdescription-as-detail,no-description-no-detailandmember-must-have-kind.record:0099.prober statusat658d173reports 0 uncredited obligations for every http claim (C03 5/5, C04 4/4, C05 5/5, C06 4/4, C07 6/6, C08 5/5, C12 4/4, C13 3/3, C14 4/4, C15 3/3, C16 5/5, C21 5/5), and 0 open findings. On release/2.2.0 every obligation of C03 to C16 was uncredited.prober status, the http claims, at 658d173
Seeds
prober seeds --probe Pfor each of the 13 probes, atdb7e3aa, after #50's merge (a first run atd15b2b4gave the same result). After #51 changed C12'sparameter-keeps-aspnetand C21'snullable-admits-null-as-the-wrapped-doespatches, C12's and C21's nine seeds ran again at8c92e32: all caught, by their own obligation's check. BaseProbe's seed ran again at658d173, after the triage: caught on 55 rows, 6 more than before, since the KB-037 check now catches the seededTryParsebypass undermvc.MaxValidationDepth=1, which the old excuse hid. 54 seeds, 54 caught, each by the check of its own obligation: the detail's example names it. BaseProbe's one seed names no obligation, so itsclaimcheck catches it.seeds output
Findings of this step, triaged
The conversion left 29 BaseProbe rows failing, all from excuses that had no citation and were compared instead. Each was triaged with
.claude/skills/probe-triage, in its order. None is a product defect, and no uncited excuse came back.mvc.MaxValidationDepth=1: "validated value object bound the refused value". These span every binding: body 7, form 7, header 4, query 4, as-parameters 3, route 3.0,null, an empty Guid,0001-01-01).Guid.Empty,0UL,DateTime.MinValue,DateTimeOffset.MinValue), an answer can't tell the two apart. So I read one row per binding whose refused value is not its default: body #122, form #244, header #238, query #140, as-parameters #313, route #201. Each holds its default.design/knowledge/KB-037-…, testtest/CodoMetis.TypeKit.Facts/KB037.cs, 18 cases). UnderMvcOptions.MaxValidationDepth = 1, an[ApiController]action whose simpleintparameter MVC cannot bind runs anyway, with the parameter at0, and answers 200. That holds for route, query, header, form and body. With the default depth (32), every such request is a 400. A[FromQuery]class makes the validator throw its depth refusal at the property instead, whether the property was bound or not.MaxValidationDepthsays only that the visitor throws past the depth and that the default is 32; it says nothing of binding errors.c.Inapplicable(Cite.Fact("KB-037")), 25 rows;KB038.cs, 4 cases). System.Text.Json writes abyte[]as one base64 string and reads it back only from one. AList<byte>, and an array of any other element, is a JSON array. Its source is the "Supported types in System.Text.Json" page: "byte[]is handled specially and serializes as a base64 string, not a JSON array."docs/json.md, "The wrapped type's JSON", gains a Good-to-know item: "An array of value objects overbyteis a JSON array of numbers. Each element is written as its byte, so you get[7,255], not the base64 string"B/8="that abyte[]becomes. Keep abyte[]where a client expects base64."Cite.Docs("docs/json.md#the-wrapped-types-json"), 2 checks.a-byte-array-is-base64(axesshape.wrapped=byte,shape.position=array-element), quoting the bold sentence, for the json family's conversion to cite.After the triage, 26 of the 29 rows pass by their other checks. The 3 string-wrapped ones end inapplicable on every check.
Uncited excuses
Each excuse below had no sentence, record or fact to cite. Three kinds of conversion:
mvc-maxvalidationdepth1-ignores-binding-errors(BaseProbe, refused binding): 28 violations, now compared with the twin and cited byfact:KB-037.byte-array-is-base64(BaseProbe, echo and read): 1 violation, now cited bydocs:docs/json.md#the-wrapped-types-json.bind:plain-unbindable(BaseProbe, valid binding): compared where the value object binds and the twin does not, which happened in no row.bind:plain-unbindablewhere both sides refuse the valid text, and the refused value where the wrapped type refuses it too (BaseProbe, 18 and 9 checks).not-bindable-from-text: a record class, or Instant in a minimal API. The wrapped type is not bound from text either, so neither side has an endpoint (BaseProbe, 22 checks in 19 rows).route-text-unsafe: a text with/,:,?or#cannot be carried by a route segment, and escaped it binds as other text. Comparing it anyway made a refused Uri bind ashttps:%2F%2F…on the first try, a harness artifact, so it stays a harness limit (BaseProbe, 24 checks in 11 rows).schema:nullability-not-comparable: no twin of the wrapped type at the value object's nullability. That is a struct value object over a class, or, under a source-generated context, an enum or Instant (C21, 10 checks in 6 rows).mvc:plain-nested-body-throws-too: MVC's validator throws for the plain twin's body as well (C13, 55 checks in 55 rows).stj-maxdepth(HttpBaseline twice, and Positions.cs 237 in the json family's code) andmvc-400-unwritable-under-source-gen.Text-matching sites, kept, each marked with its baseline gap
harness/HttpBaseline.cs:45maximum allowed depthunder MaxDepth=3 (notestj-maxdepth, uncited)harness/HttpBaseline.cs:59harness/HttpBaseline.cs:290ValidationProblemDetailsunder a source-generated context (notemvc-400-unwritable-under-source-gen, uncited)harness/HttpBaseline.cs:536.allOfpath and[{"x-nullable":true}](nowCite.NotPromised("openapi-2-nullable-component-loses-its-type"))harness/HttpBaseline.cs:544"schema":{"allOf":[{"x-nullable":true}]}(nowCite.NotPromised("openapi-2-nullable-body-keeps-only-its-null"))harness/HttpBaseline.cs:554$.parameters…enumpath (nowCite.NotPromised("enum-component-is-its-first-use"))harness/Positions.cs:237maximum allowed depthin the JSON family's parity (notestj-maxdepth, uncited)That is six sites in HttpBaseline. The design note counted eight at its line numbers. The two others I find there decide no outcome, so they are unmarked: the 404 that names a harness routing mistake (a failure, never an excuse), and the
[F1]prefix of a failure text.Not-promised entries added, and facts and docs
a-byte-array-is-base64, from the triage of row #83. It quotesdocs/json.md#the-wrapped-types-jsonand is the place the json family's conversion will cite.MaxValidationDepth = 1) and KB-038 (abyte[]is a base64 string). Each has its Facts test, and each has a row indesign/knowledge/README.md.docs/json.mdquoted above.The conversion itself added no not-promised entry:
Cite.Docs), not a not-promised entry of C07. BaseProbe runs on C07's rows as a calibration of the host, and C07 promises nothing of the wire. An entry of C07 saying "a struct value object refuses a JSON null" would put a JSON difference on the fault-responses claim. The citations are:docs/json.md#null-and-missingdocs/json.md#when-a-value-is-rejectedREADME.md#what-you-get(a closed set travels as its name)README.md#what-typekit-guarantees(StoredJsonConverterFactory)src/CodoMetis.TypeKit.Generators/README.md#json(preserved references)src/CodoMetis.TypeKit.AspNetCore/README.md#what-the-document-says(your own converter wins)#bindingfor the 400 as a baselineCite.Record("0099")).Declared applicability (#50)
Declared on the obligation (claims.json
inapplicable): C21's two own-converter cases, on each of its five obligations.{"json.Converters": ["own-for-value-object"]}citesnot-promised:own-converter-for-the-value-object.{"json.Converters": ["own-for-wrapped"]}citesnot-promised:own-converter-for-the-wrapped-type.The schema check decided both from that axis alone. They replace C21's two exclusions of 168 pairs each, and
universenow counts the same 336 pairs as wanted by no obligation (C21: 13802/14218, 336 inapplicable). The two lines inHttpBaseline.SchemaParitystay, for the schema half run under another claim (ShrinkProbe's baseline). For C21 they are no longer reached: no generated row has those values, andRow.Checkwould record the declaration first.Kept in code, because the shape instance or what the host did decides them:
claim, which no obligation entry declares.No other http claim has an excuse decided by axes alone. C04's
controller-scope-is-mvcand C16'sonly-converters-and-descriptions-comparedexcuse no check: the first is an exclusion, and the second's rows are checked bysame-converters-build.Verdicts
build-verdict test --project test/CodoMetis.TypeKit.Facts, TESTS OK: 49 passed, 9 skipped, ataf6e8d6. The 9 skipped are the PostgreSQL facts, which needTYPEKIT_PG. KB037 has 18 cases and KB038 4.sourcestarting with a quote, fixed inaf6e8d6.build-verdict test --project test/CodoMetis.TypeKit.Prober.Tests, TESTS OK: 245 passed, 0 skipped, atdb7e3aa,8c92e32andaf6e8d6(204 before the merge brought build: declared applicability: an obligation says where it does not apply, read by the universe, the covering generator, the gate and Row.Check #50's tests, at90918a4andddd146c).ddd146cfailedToolchainTests.A_dotnet_without_the_pinned_sdk_falls_back_to_the_installers. It passed alone, and in the full rerun (204 passed).build-verdict test --project test/CodoMetis.TypeKit.Conventions.Tests, TESTS OK: 1471 passed, 0 skipped, atddd146canddb7e3aa, 1475 at8c92e32(the merge brought four), and 1477 ataf6e8d6and658d173. The first run failed two tests, both fixed inddd146c:ProbeCiteTestsread a C16 helper,AgreeRun.Refused(c, …), as a check'sRefusedwithout aCite. The helper is nowRefusal.ProbeNoteKeyTests.The_corpus_notes_are_foundhas a floor of 130Notecalls, which keeps a walker that finds nothing from passing. The conversion leaves 127, since outcome notes became citations. The floor is 100 now. The build family's conversion lowers the count too, so that session may meet the same line.Left as it was, or not checked
application-json-declaredchecks the declared response's fault, rule and problem types. It does not check that the response keepsapplication/jsonas its media type, which no comparison held before either.claim, the implicit obligation, while C07 lists obligations. BaseProbe's seed names none.NoteKeysis unchanged. Its outcome keys the http family no longer calls stay declared for the other families until the end of WS2.🤖 Generated with Claude Code