Repository navigation
t27b ledger: name the 180 specs the reference passes and the ledger did not (Refs #10416) - #10470
Merged
Merged
Conversation
…id not (Refs #10416) PR #10445 merged with only suite_expectations.json: its last commit took the t27b ledger from master "to be re-blessed on the lab", and the re-bless never landed. t27b-native-ratchet has been red on every master commit since, with UNLISTED 180 and nothing else (no unexpected failure or pass, no stale entry, no moved blocker). Blessed from the t27b-native artifact of master 1ef60ae (arm64 Linux, full corpus, 2272 files) with `tri t27b ratchet --bless --accept-new`; no entry was refused and no existing entry changed. - 121 new `pass`, 58 new `blocked` (reason unimplemented), 1 new `codegen` (specs/t27mcp/tool_schema.t27, reason unimplemented: t27b's JIT reports a conditional branch out of range for the round-trip fixtures function). - max_not_pass 74 -> 130, exactly the 56 new non-pass entries. - source.commit fdeb1db -> 1ef60ae. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This was referenced Oct 18, 2026
This was referenced Oct 18, 2026
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #10416
Why
#10445 merged with only
docs/reports/suite_expectations.json. Its last commit (fd3952d3d) took the t27b ledger from master "to be re-blessed on the lab", and that re-bless never landed. Since thent27b-native-ratchethas been red on every master commit (b200c3a6a,a3ea55449,7552b89d0,1ef60ae34) and on every PR.The verdict is a single kind of finding:
UNLISTED 180. There is no UNEXPECTED FAILURE, UNEXPECTED PASS, STALE, OVER CAP, BAD REASON or MOVED. Nothing regressed; the ledger is just behind the corpus.What
docs/reports/t27b_expectations.jsononly, written by the ratchet's own bless mode with--accept-new(no hand edits, no entry refused). The run is thet27b-native-runartifact of master1ef60ae34(arm64 Linux, full corpus, 2272 files).pass, 58blocked, 1codegen.max_not_pass74 -> 130, exactly the 56 new non-pass entries.source.commitfdeb1db -> 1ef60ae.source.commitandmax_not_passlines).specs/t27mcp/tool_schema.t27iscodegenwith reasonunimplemented: the reference passes it (12/12 tests), and t27b's JIT reports "conditional branch out of range (function too large)" for the round-trip fixtures function. The way out of the ledger is to split that function in the spec, which is a separate change.How to check
The ratchet run on this PR's own merge ref should report no UNLISTED.
🤖 Generated with Claude Code