Skip to content

spec(t27f): reel cells 15, 16, 24 cite the sealed NMSE run, drawn as earned drafts (Closes #10023) #10031

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gHashTag merged 1 commit into from
Oct 17, 2026
Merged

spec(t27f): reel cells 15, 16, 24 cite the sealed NMSE run, drawn as earned drafts (Closes #10023) #10031

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
147 changes: 104 additions & 43 deletions specs/t27f/reel_gf16_nmse.t27
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,37 +10,46 @@
; SOURCE: gHashTag/t27 at SOURCE_COMMIT, measured on MEASURED_ON. The protocol spec
; specs/benchmarks/gf16_bfloat16_nmse.t27 and its mirror docs/GF16_BFLOAT16_NMSE_PROTOCOL.md, the
; manifest schema schemas/nmse-protocol-v1.json, the roadmap docs/SCIENTIFIC_IMPROVEMENT_PLAN.md
; (PUB-01, PUB-02), and what the repo already holds about GF16. Every cell names its source in
; SOURCES; a cell with none is a gap and says in MISSING what would ground it.
; (PUB-01, PUB-02), the PUB-02 run harness specs/benchmarks/gf16_bfloat16_nmse_run.t27 and the
; manifest it wrote, RUN_MANIFEST (#10021), and what the repo already holds about GF16. Every cell
; names its source in SOURCES; a cell with none is a gap and says in MISSING what would ground it.

; BRIEF is the first three sentences of the protocol spec's header, verbatim. route_brief
; counts gf16 twice and benchmark once, no other list is hit, so the domain is DOMAIN_RESEARCH.

; LENSES: one row per lens. Steps 1-9 are the formats and why they compare at all; 10-18 the protocol
; and the one run it asks for; 19-27 what may be said and what the repo has said so far.

; MEASURED on 2026-10-17:
; MEASURED on 2026-10-18:
; - grounded: the source is in the repo at SOURCE_COMMIT (path and section in SOURCES).
; - test: t27c test-report and t27b test --check on the Railway lab passed all 7 tests of the
; protocol spec; a planted GF16_MANT_BITS = 8 turned 2 of them red under both, so they are real.
; - review: the PRs that wrote the protocol (#1399) and the spec rewrite (#5947) have 0 reviews.
; The harness's own tests check its procedure, not the committed manifest, so no step of the
; run (16, 17, 25) is marked as tested; on a DELIVER step a test would not count anyway.
; - review: the PRs that wrote the protocol (#1399), the spec rewrite (#5947) and the run (#10021)
; have 0 reviews.
; - live: GET https://t27.ai/t27/files/<path> returned 200 for the protocol spec and the schema,
; and 404 for the doc, FORMAT-SPEC-001.json, gf16_bench_results.json, the crosswalk and the paper
; draft; a made-up path returned 404, so 200 is not a catch-all.
; and 404 for the doc, FORMAT-SPEC-001.json, gf16_bench_results.json, the crosswalk, the paper
; draft, the harness and the manifest; a made-up path returned 404, so 200 is not a catch-all.
; The engine asks a research LEARN step for a source link, a PRACTICE step for a review or a passing
; test, and a DELIVER step for a review. So: 15 published (sourced, earned, and live), 1 verified
; (reporting rules: sourced, not live), 6 drafts (practice or deliver steps with neither a test nor a
; review) and 5 gaps.
; (reporting rules: sourced, not live), 9 drafts (practice or deliver steps with neither a test nor a
; review, three of them the run's own cells) and 2 gaps.

; THE GAPS are the point of this reel. PUB-02 asks for one sealed bench/results/nmse-*.json; the repo
; holds none, so the run, its schema check, the ratio per distribution and the paper's Delta_dB (which
; PUB-02 forbids before that manifest) are gaps, and so is the chip repo's manifest:
; gHashTag/tt-trinity-gamma tests/vectors/nmse_gf16_bf16.golden.json at GAMMA_COMMIT pins the seeds
; but says measured_silicon false and harness_committed false. The only GF16/BF16 numbers in the repo,
; conformance/gf16_bench_results.json (2026-04-05, no seed, no seal), are drawn as a draft that the
; protocol itself says does not count.
; THE RUN. PUB-02 asks for one sealed bench/results/nmse-*.json. Until #10021 the repo held none, and
; steps 16, 17 and 25 (the run, its schema check, the ratio per distribution) were gaps. They now cite
; RUN_HARNESS and RUN_MANIFEST, and they are DRAFTS, not verified: each is a DELIVER step, a research
; DELIVER step is verified by a review, and the run has none. The manifest is a software run on the
; Railway lab (runner.host_arch x86_64), not a silicon measurement.
; THE GAPS that remain: the chip repo's manifest, gHashTag/tt-trinity-gamma
; tests/vectors/nmse_gf16_bf16.golden.json at GAMMA_COMMIT, pins the seeds but says measured_silicon
; false and harness_committed false; and the paper draft has no Delta_dB yet (PUB-02 allowed one only
; after the manifest, and attaching the manifest to the draft is still open). The older GF16/BF16
; numbers, conformance/gf16_bench_results.json (2026-04-05, no seed, no seal), are drawn as a draft
; that the protocol itself says does not count.

; Not claimed: no NMSE value, no ratio, no winner. No GIF or MP4; stills only, for the chat.
; Not claimed: this reel quotes no NMSE value, no ratio and no winner; the manifest holds the numbers
; and is the only place to read them. Nothing here is silicon. No GIF or MP4; stills only, for the chat.
; phi^2 + 1/phi^2 = 3 | TRINITY

module reel_gf16_nmse;
Expand All @@ -63,6 +72,7 @@ use t27f::topic_engine::SALES_WORDS;
use t27f::topic_engine::keyword_hits;
use t27f::topic_engine::route_brief;
use t27f::topic_engine::step_stage;
use t27f::topic_engine::earned_status;
use t27f::topic_reel::count_status;
use t27f::topic_reel::count_true;
use t27f::topic_reel::statuses_earned;
Expand All @@ -73,11 +83,14 @@ use t27f::topic_reel::stage_slides_bare;
pub const TOPIC_ID : str = "gf16-nmse";
pub const SOURCE_REPO : str = "gHashTag/t27";
pub const SOURCE_SPEC : str = "specs/benchmarks/gf16_bfloat16_nmse.t27";
pub const SOURCE_COMMIT : str = "de2cff88243c179e5bbf21f8cb06e5da1ab833e8";
pub const SOURCE_COMMIT : str = "cc80a305dd5fef1cd21f930e1394ae47cbdacbfc";
pub const GAMMA_REPO : str = "gHashTag/tt-trinity-gamma";
pub const GAMMA_COMMIT : str = "de484c4ff249356624ec508127fcbab880ee25fc";
pub const LIVE_BASE : str = "https://t27.ai/t27/files/";
pub const MEASURED_ON : str = "2026-10-17";
pub const MEASURED_ON : str = "2026-10-18";
/// The PUB-02 run (#10021): the harness that wrote the manifest, and the manifest.
pub const RUN_HARNESS : str = "specs/benchmarks/gf16_bfloat16_nmse_run.t27";
pub const RUN_MANIFEST : str = "bench/results/nmse-railway-x86_64-0x2545F4914F6CDD1D-2026-10-17.json";

pub const BRIEF : str = "GF16 against bfloat16 NMSE protocol (TRI-NET benchmark 0). The standard comparison of GoldenFloat GF16 and bfloat16 numeric fidelity, as normalised mean squared error over named reference distributions. Protocol only: no silicon number is asserted.";
/// The domain route_brief gives that brief (see the_domain_is_routed_not_chosen).
Expand Down Expand Up @@ -132,40 +145,37 @@ pub const SOURCES : [27]str = [
"specs/benchmarks/gf16_bfloat16_nmse.t27 test identity_witnesses_hold_in_f64",
"specs/benchmarks/gf16_bfloat16_nmse.t27 test d_deep_mixture_weights_sum_to_one",
"specs/benchmarks/gf16_bfloat16_nmse.t27 test nmse_non_negative_and_zero_baseline_undefined",
"",
"",
"specs/benchmarks/gf16_bfloat16_nmse_run.t27 nmse_manifest; bench/results/nmse-railway-x86_64-0x2545F4914F6CDD1D-2026-10-17.json samples_per_distribution, seal_hash",
"bench/results/nmse-railway-x86_64-0x2545F4914F6CDD1D-2026-10-17.json schema_version, under schemas/nmse-protocol-v1.json",
"",
"specs/benchmarks/gf16_bfloat16_nmse.t27 header, the ratio",
"docs/GF16_BFLOAT16_NMSE_PROTOCOL.md section 8",
"specs/benchmarks/gf16_bfloat16_nmse.t27 NON_CLAIMS",
"specs/benchmarks/gf16_bfloat16_nmse.t27 test ratio_direction",
"conformance/gf16_bench_results.json",
"docs/GF16_BFLOAT16_NMSE_PROTOCOL.md section 6",
"",
"bench/results/nmse-railway-x86_64-0x2545F4914F6CDD1D-2026-10-17.json results.*.ratio, from specs/benchmarks/gf16_bfloat16_nmse_run.t27",
"",
"docs/WHITEPAPER/gf_paper_v3_imrad_draft.md (PUB-01)"
];

/// What would ground each gap; "" for a cell that has its source.
pub const MISSING : [27]str = [
"", "", "", "", "", "", "", "", "", "", "", "", "", "", "",
"bench/results/nmse-*.json, the sealed run PUB-02 asks for: the repo holds 0 such files",
"a manifest to check against schemas/nmse-protocol-v1.json: none exists",
"", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "",
"tt-trinity-gamma nmse_gf16_bf16.golden.json pins seeds only: measured_silicon false, harness_committed false",
"", "", "", "", "", "",
"an NMSE ratio per distribution: no run has produced one",
"a Delta_dB figure: PUB-02 forbids one before the manifest lands",
"", "", "", "", "", "", "",
"a Delta_dB figure in the paper draft: the manifest is not attached to it yet",
""
];

/// The measurements per cell (see the header for how each was taken).
pub const GROUNDED : [27]bool = [true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, false, false, false, true, true, true, true, true, true, false, false, true];
pub const GROUNDED : [27]bool = [true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, false, true, true, true, true, true, true, true, false, true];
pub const TEST_PASS : [27]bool = [false, false, false, true, true, true, false, false, false, false, false, false, true, true, true, false, false, false, false, false, false, true, false, false, false, false, false];
pub const REVIEWED : [27]bool = [false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false];
pub const LIVE : [27]bool = [true, true, true, true, true, true, false, false, false, true, true, true, true, true, true, false, false, false, true, false, true, true, false, false, false, false, false];

/// What a host passes the viewer, per cell: the status code and the step number as a label.
pub const STATUS : [27]u8 = [5, 5, 5, 5, 5, 5, 3, 3, 3, 5, 5, 5, 5, 5, 5, 0, 0, 0, 5, 4, 5, 5, 3, 3, 0, 0, 3];
pub const STATUS : [27]u8 = [5, 5, 5, 5, 5, 5, 3, 3, 3, 5, 5, 5, 5, 5, 5, 3, 3, 0, 5, 4, 5, 5, 3, 3, 3, 0, 3];
pub const LABELS : [27]str = ["1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24", "25", "26", "27"];

/// Slide text per demo scene, checked by topic_reel.t27 slides_fit (index scene*4+row).
Expand All @@ -174,7 +184,7 @@ pub const HEAD : [5]str = [
"Steps 1-9: formats",
"Witness first",
"Evidence",
"The missing run"
"The sealed run"
];
pub const ROWS : [20]str = [
"", "", "", "",
Expand All @@ -184,13 +194,13 @@ pub const ROWS : [20]str = [
"A failed witness aborts the run",
"No manifest is written after a failure",
"7 of 7 protocol tests pass on the lab",
"15 published, 1 verified, 6 drafts",
"5 gaps: no measured run exists yet",
"Measured 2026-10-17",
"15 published, 1 verified, 9 drafts",
"2 gaps: chip run, paper Delta_dB",
"Measured 2026-10-18",
"PUB-02: one sealed NMSE manifest",
"5 distributions, 10M samples each",
"Gaps stay gaps until a run lands",
""
"A software run, not silicon",
"Draft until someone reviews it"
];

/// The protocol spec's own test count, which the evidence slide quotes.
Expand All @@ -200,6 +210,29 @@ pub fn step_number(cell: i32) -> i32 {
return cell + 1;
}

/// Whether a SOURCES entry names a path: `path` occurs in `src` byte for byte.
pub fn cites(src: []const u8, n: usize, path: []const u8, m: usize) -> bool {
if (m == 0 || m > n) {
return false;
}
var i : usize = 0;
while (i + m <= n) {
var j : usize = 0;
var same : bool = true;
while (j < m && same) {
if (src[i + j] != path[j]) {
same = false;
}
j += 1;
}
if (same) {
return true;
}
i += 1;
}
return false;
}

test the_domain_is_routed_not_chosen {
assert DOMAIN == DOMAIN_RESEARCH;
// gf16, gf16, benchmark; "bfloat16" is not "gf16", and no other list is hit
Expand Down Expand Up @@ -229,29 +262,40 @@ test every_step_is_a_cell {

test a_gap_is_named_not_filled {
assert gaps_are_named(SOURCES, MISSING, GROUNDED);
assert count_true(GROUNDED) == 22;
assert count_true(GROUNDED) == 25;
// a gap is drawn as a gap, and only a gap is
var c : i32 = 0;
while (c < CELL_COUNT) {
const i : usize = @intCast(c);
assert (STATUS[i] == STATUS_GAP) == (GROUNDED[i] == false);
c = c + 1;
}
// every gap is a DELIVER step: what is missing is the run and what follows it, not the protocol
assert step_stage(15) == STAGE_DELIVER;
assert step_stage(16) == STAGE_DELIVER;
// the two gaps left are DELIVER steps: the chip repo's run and the paper's Delta_dB
assert STATUS[17] == STATUS_GAP;
assert STATUS[25] == STATUS_GAP;
assert step_stage(17) == STAGE_DELIVER;
assert step_stage(24) == STAGE_DELIVER;
assert step_stage(25) == STAGE_DELIVER;
// the run's own cells cite the run, by both paths
assert MISSING[15].len == 0;
assert MISSING[16].len == 0;
assert MISSING[24].len == 0;
assert cites(SOURCES[15], SOURCES[15].len, RUN_HARNESS, RUN_HARNESS.len);
assert cites(SOURCES[15], SOURCES[15].len, RUN_MANIFEST, RUN_MANIFEST.len);
assert cites(SOURCES[16], SOURCES[16].len, RUN_MANIFEST, RUN_MANIFEST.len);
assert cites(SOURCES[24], SOURCES[24].len, RUN_MANIFEST, RUN_MANIFEST.len);
assert cites(SOURCES[24], SOURCES[24].len, RUN_HARNESS, RUN_HARNESS.len);
// and cites is not a yes-machine: the protocol's own cells do not name the run
assert cites(SOURCES[0], SOURCES[0].len, RUN_MANIFEST, RUN_MANIFEST.len) == false;
assert cites(SOURCES[16], SOURCES[16].len, RUN_HARNESS, RUN_HARNESS.len) == false;
}

test a_status_is_earned_not_chosen {
assert statuses_earned(STATUS, DOMAIN, GROUNDED, TEST_PASS, REVIEWED, LIVE);
// the evidence slide's "15 published, 1 verified, 6 drafts" and "5 gaps"
// the evidence slide's "15 published, 1 verified, 9 drafts" and "2 gaps"
assert count_status(STATUS, STATUS_PUBLISHED) == 15;
assert count_status(STATUS, STATUS_VERIFIED) == 1;
assert count_status(STATUS, STATUS_DRAFT) == 6;
assert count_status(STATUS, STATUS_GAP) == 5;
assert count_status(STATUS, STATUS_DRAFT) == 9;
assert count_status(STATUS, STATUS_GAP) == 2;
assert count_true(REVIEWED) == 0;
// every passing test is one of the protocol's own tests, on a PRACTICE step
assert count_true(TEST_PASS) == PROTOCOL_TESTS;
Expand All @@ -267,6 +311,23 @@ test a_status_is_earned_not_chosen {
assert STATUS[22] == STATUS_DRAFT;
}

test the_run_is_a_draft_until_reviewed {
// steps 16, 17 and 25 now have the run as their source, and are DELIVER steps
assert STATUS[15] == STATUS_DRAFT;
assert STATUS[16] == STATUS_DRAFT;
assert STATUS[24] == STATUS_DRAFT;
assert step_stage(15) == STAGE_DELIVER;
assert step_stage(16) == STAGE_DELIVER;
assert step_stage(24) == STAGE_DELIVER;
assert REVIEWED[15] == false;
assert REVIEWED[16] == false;
assert REVIEWED[24] == false;
// a passing test and a live page do not lift a research DELIVER step: only a review does
assert earned_status(15, DOMAIN, true, true, false, true) == STATUS_DRAFT;
assert earned_status(15, DOMAIN, true, false, true, false) == STATUS_VERIFIED;
assert earned_status(15, DOMAIN, true, false, true, true) == STATUS_PUBLISHED;
}

test the_camera_is_the_demo_camera {
assert DEMO_COUNT == 5;
// scene 1 shows lens 0, the row HEAD[1] calls "Steps 1-9"
Expand Down
Loading