Repository navigation
spec(t27f): reel cells 15, 16, 24 cite the sealed NMSE run, drawn as earned drafts (Closes #10023) - #10031
Merged
Conversation
…earned drafts (Closes #10023) Cells 15, 16 and 24 (steps 16, 17, 25) of specs/t27f/reel_gf16_nmse.t27 named 'the sealed run PUB-02 asks for does not exist' as their gap. PR #10021 landed it: specs/benchmarks/gf16_bfloat16_nmse_run.t27 and bench/results/nmse-railway-x86_64-0x2545F4914F6CDD1D-2026-10-17.json. The three cells now cite those artefacts and are grounded. They are research DELIVER steps and #10021 has 0 reviews, so earned_status gives DRAFT. Counts move to 15 published, 1 verified, 9 drafts, 2 gaps. A new test pins that a test pass or a live page does not lift them and a review would. The reel quotes no NMSE value and claims nothing about silicon. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gHashTag
enabled auto-merge (squash)
October 17, 2026 17:09
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #10023
What changes
specs/t27f/reel_gf16_nmse.t27only. Cells 15, 16 and 24 (steps 16, 17, 25) gave "the sealed run PUB-02 asks for does not exist" as their gap. #10021 landed that run, so the cells now cite it:nmse_manifest+ manifestsamples_per_distribution,seal_hashschema_version, underschemas/nmse-protocol-v1.jsonresults.*.ratio, from the harnessspecs/benchmarks/gf16_bfloat16_nmse_run.t27bench/results/nmse-railway-x86_64-0x2545F4914F6CDD1D-2026-10-17.jsonWhy DRAFT and not higher. All three are research DELIVER steps.
earned_status(topic_engine.t27) needs a review on those, and #10021 has 0 reviews. A test pass counts only on PRACTICE steps, and the harness's tests check its procedure rather than the committed manifest, soTEST_PASSstays false. Both paths return 404 under https://t27.ai/t27/files/, soLIVEstays false, and it would not lift a DRAFT anyway.Counts (cells): published 15 (unchanged), verified 1 (unchanged), draft 6 -> 9, gap 5 -> 2. The two gaps left are cell 17 (the chip repo's manifest still says
measured_siliconfalse) and cell 25 (no Delta_dB in the paper draft yet).count_true(GROUNDED)= 25.Not claimed. The reel quotes no NMSE value, no ratio and no winner. It calls the manifest a software run on the Railway lab (
runner.host_archx86_64), not silicon. The only figures on the slides that come from the run are "5 distributions" (5 keys underresults) and "10M samples each" (samples_per_distribution10000000).New in the spec:
cites(src, path), a substring check.a_gap_is_named_not_fillednow asserts that each cell names its artefacts, plus two negative checks: cell 0 does not cite the manifest, and cell 16 does not cite the harness.the_run_is_a_draft_until_reviewed. It pins the three cells at DRAFT and checks withearned_statusthat a test pass plus a live page still give DRAFT, while a review gives VERIFIED (or PUBLISHED when the page is also live).Verification (Railway lab, never the Mac)
Tools built on the lab from master cc80a30: t27c x86_64 release, and t27b cross-built for aarch64 (its JIT runs only on arm64) and run under
qemu-aarch64 -L /usr/aarch64-linux-gnu. Zig 0.16.0. Spec sha2564f0f49a4533666577fa9fa1e44558cb5e030801f6ed37605c3ca6e65e74c5f4d.specs/t27f/reel_gf16_nmse.t27specs/t27f/topic_reel.t27specs/benchmarks/gf16_bfloat16_nmse.t27(cited by the reel, unchanged)Runtime asserts per reel test under t27c: 5, 54, 38, 14, 6, 1, 2.
Negative controls. Each was planted on the lab, run under both tools, then reverted; the sha256 matched afterwards.
STATUS[15]= VERIFIED with no reviewa_status_is_earned_not_chosen,the_run_is_a_draft_until_reviewedSOURCES[24]drops the harness patha_gap_is_named_not_filledREVIEWED[15]= true, STATUS left DRAFTa_status_is_earned_not_chosen,the_run_is_a_draft_until_reviewedGF16_MANT_BITS= 8both_layouts_are_sixteen_bits,gf16_trades_exponent_for_mantissaControls: 4 of 4 red under both tools, with identical failing tests.
Follow-ups (not in this PR)
specs/api/tri_net_api.t27:46-48still saysbench/results/does not exist. That spec is sealed, so changing it means re-sealing.🤖 Generated with Claude Code