Skip to content

spec(nn): ternary STE, training step and data/checkpoint layout in t27 (Closes #10076) #10086

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gHashTag merged 1 commit into from
Oct 17, 2026
Merged

spec(nn): ternary STE, training step and data/checkpoint layout in t27 (Closes #10076) #10086

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 47 additions & 22 deletions specs/igla/own_model.t27
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,22 @@
; vocab-729 head and cross-entropy), its backward reaches the embedding, and every gradient is
; checked against a central difference; a mutant that drops one alpha fails 10 of the 46.
; Not in it: the per-block VSA path, f32, and the trainer's STE update (shadow weights).
; - test on 2026-10-18, same t27c, after the PR that added the training step (STE, backward, data):
; nn/ternary_ste 26/26, nn/hslm_backward 15/15, nn/hslm_data 39/39 pass, 0 vacuous. The STE is
; the TWN quantizer (delta 0.7 * mean abs, alpha = mean abs of the kept weights), the
; pass-through gradient, clip_grad_norm and the SGD step on the shadow weights. hslm_backward
; runs forward, backward, STE, clip, SGD and requantize on one 3-head slice of the model
; (ctx 27, head_dim 3): its trit, bias and input gradients match a central difference to 1e-6
; and 20 steps cut the loss below half. hslm_data holds the curriculum, next-token batches and
; the MSLH header and layout, whose byte count equals hslm_step_10000.bin (4971796). Not in
; them: the STE on the whole 3-block model, file I/O, heap, the tokenizer, shuffle, AdamW.
; - live: GET huggingface.co/api/models/playra/tern-tc-9m and .../tern-tc-9m-t27 returned 200;
; .../tern-tc-100m returned 401, the same as a made-up name, so no 100M is public.
; GET board-production-2dec.up.railway.app/state returned 200 and a made-up path 404; its
; snapshot is dated 2026-10-05, so the board is live and 12 days stale.
; - review: no PR behind any cell has a review, so no DELIVER cell is VERIFIED.
; The engine asks an engineering LEARN step for a source link, a PRACTICE step for a passing test
; and a DELIVER step for a review. So: 1 published, 12 verified, 7 drafts, 7 gaps.
; and a DELIVER step for a review. So: 1 published, 12 verified, 9 drafts, 5 gaps.

; PERCENT is a second, separate number per cell: an estimate of how much of that part exists, with
; its basis in SOURCES. It never feeds a status. The lens and overall figures are plain means, every
Expand All @@ -46,8 +55,9 @@
; THE ANSWER this map gives: HSLM was trained before, in Zig only (64 files); its best runs
; (PPL 2.96 at ctx 27) were wiped by EXP-002 and survive only in the ledger. In t27 the loss, three
; optimizers and the HSLM spec's forward and backward passes pass, and that spec is now the trained
; model but for its VSA path (see MEASURED); ternary STE with shadow weights, data loader,
; checkpoint I/O, heap and file I/O and `t27c --backend ml` do not exist yet.
; model but for its VSA path (see MEASURED); the ternary STE, the training step and the data and
; checkpoint layout now pass too, on a model slice and on bytes; the STE over the whole model,
; heap and file I/O and `t27c --backend ml` do not exist yet.

; Not claimed: no model quality beyond the numbers in SOURCES; no GIF or MP4, stills only.
; phi^2 + 1/phi^2 = 3 | TRINITY
Expand All @@ -63,6 +73,7 @@ use t27f::topic_engine::STATUS_DRAFT;
use t27f::topic_engine::STATUS_VERIFIED;
use t27f::topic_engine::STATUS_PUBLISHED;
use t27f::topic_engine::STAGE_PRACTICE;
use t27f::topic_engine::STAGE_DELIVER;
use t27f::topic_engine::DOMAIN_ENGINEERING;
use t27f::topic_engine::CONTENT_WORDS;
use t27f::topic_engine::ENGINEERING_WORDS;
Expand Down Expand Up @@ -144,9 +155,9 @@ pub const SOURCES : [27]str = [
"t27 specs/ml/loss/cross_entropy_loss.t27",
"t27 specs/ml/optimizer/{sgd,adam,lamb}.t27",
"t27 specs/nn/hslm.t27 and specs/hslm/forward_pass.t27",
"t27 specs/nn/hslm.t27 backward, finite-difference checked",
"",
"",
"t27 specs/nn/hslm.t27 backward and nn/hslm_backward.t27 training step on a head slice, finite-difference checked",
"t27 specs/nn/ternary_ste.t27, 26 tests; whole-model shadow weights not yet",
"t27 specs/nn/hslm_data.t27: batches, curriculum, MSLH layout = 4971796 bytes; no file I/O",
"igla-coder-gpu research/MODEL_ON_FPGA_PLAN.md",
"trinity-fpga b2bd5a30, 7682304/7682304 receipts",
"board-production-2dec.up.railway.app/state",
Expand All @@ -164,8 +175,8 @@ pub const MISSING : [27]str = [
"a public 100M checkpoint: playra/tern-tc-100m answers 401, like a made-up name",
"any run past 100M parameters: none exists",
"", "", "", "", "", "", "",
"a ternary straight-through estimator in any .t27 spec: none exists",
"a t27 reader for token streams and MSLH checkpoints: none exists",
"",
"",
"", "", "", "", "",
"heap allocation and file I/O lowered by t27c: neither exists",
"a t27c backend that lowers a model spec to a training binary: none exists",
Expand All @@ -174,17 +185,17 @@ pub const MISSING : [27]str = [
];

/// The measurements per cell (see the header for how each was taken).
pub const GROUNDED : [27]bool = [true, true, true, true, true, true, true, false, false, true, true, true, true, true, true, true, false, false, true, true, true, true, true, false, false, false, true];
pub const GROUNDED : [27]bool = [true, true, true, true, true, true, true, false, false, true, true, true, true, true, true, true, true, true, true, true, true, true, true, false, false, false, true];
pub const TEST_PASS : [27]bool = [false, false, false, false, false, false, false, false, false, false, false, false, true, true, true, false, false, false, false, false, false, true, false, false, false, false, false];
pub const REVIEWED : [27]bool = [false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false];
pub const LIVE : [27]bool = [false, false, false, false, false, false, true, false, false, false, false, false, false, false, false, false, false, false, false, false, true, false, false, false, false, false, false];

/// What a host passes the viewer, per cell: the earned status code and the step number as a label.
pub const STATUS : [27]u8 = [4, 4, 4, 3, 3, 3, 3, 0, 0, 4, 4, 4, 4, 4, 4, 3, 0, 0, 4, 4, 5, 4, 3, 0, 0, 0, 3];
pub const STATUS : [27]u8 = [4, 4, 4, 3, 3, 3, 3, 0, 0, 4, 4, 4, 4, 4, 4, 3, 3, 3, 4, 4, 5, 4, 3, 0, 0, 0, 3];
pub const LABELS : [27]str = ["1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24", "25", "26", "27"];

/// Percent of each part that exists, estimated from its source (see the header). Never a status.
pub const PCT : [27]u8 = [60, 70, 80, 80, 60, 10, 50, 0, 0, 100, 100, 20, 80, 65, 85, 70, 0, 0, 70, 15, 60, 40, 35, 0, 0, 0, 50];
pub const PCT : [27]u8 = [60, 70, 80, 80, 60, 10, 50, 0, 0, 100, 100, 20, 80, 65, 85, 75, 55, 40, 70, 15, 60, 40, 35, 0, 0, 0, 50];

/// The t27c test-report counts the PRACTICE cells quote.
pub const ML_SPECS_MEASURED : i32 = 17;
Expand All @@ -209,11 +220,11 @@ pub const ROWS : [20]str = [
"In t27: loss, optimizers, HSLM pass",
"6 of 17 ML specs compile on the lab",
"0 of 9 IGLA CODER specs compile",
"1 published, 12 verified, 7 drafts",
"7 gaps, about 44 percent overall",
"1 published, 12 verified, 9 drafts",
"5 gaps, about 48 percent overall",
"HSLM spec: add the VSA path",
"Ternary STE with shadow weights",
"Loader, checkpoint I/O, --backend ml",
"Shadow weights for the whole model",
"File I/O, heap, --backend ml",
""
];

Expand Down Expand Up @@ -266,7 +277,7 @@ test every_step_is_a_cell {

test a_gap_is_named_not_filled {
assert gaps_are_named(SOURCES, MISSING, GROUNDED);
assert count_true(GROUNDED) == 20;
assert count_true(GROUNDED) == 22;
var c : i32 = 0;
while (c < CELL_COUNT) {
const i : usize = @intCast(c);
Expand All @@ -280,11 +291,11 @@ test a_gap_is_named_not_filled {

test a_status_is_earned_not_chosen {
assert statuses_earned(STATUS, DOMAIN, GROUNDED, TEST_PASS, REVIEWED, LIVE);
// the evidence slide's "1 published, 12 verified, 7 drafts" and "7 gaps"
// the evidence slide's "1 published, 12 verified, 9 drafts" and "5 gaps"
assert count_status(STATUS, STATUS_PUBLISHED) == 1;
assert count_status(STATUS, STATUS_VERIFIED) == 12;
assert count_status(STATUS, STATUS_DRAFT) == 7;
assert count_status(STATUS, STATUS_GAP) == 7;
assert count_status(STATUS, STATUS_DRAFT) == 9;
assert count_status(STATUS, STATUS_GAP) == 5;
assert count_true(REVIEWED) == 0;
// every passing test is a PRACTICE step: loss, optimizers, forward pass, matvec
assert count_true(TEST_PASS) == 4;
Expand Down Expand Up @@ -319,10 +330,24 @@ test the_compile_counts_are_the_measured_ones {

test percent_per_lens_is_a_plain_mean {
assert mean_pct(0, 9) == 45;
assert mean_pct(9, 9) == 57;
assert mean_pct(9, 9) == 68;
assert mean_pct(18, 9) == 30;
// the evidence slide's "about 44 percent overall"
assert mean_pct(0, CELL_COUNT) == 44;
// the evidence slide's "about 48 percent overall"
assert mean_pct(0, CELL_COUNT) == 48;
}

test the_training_step_cells_are_drafts_until_reviewed {
// steps 16-18 are DELIVER steps: their specs pass (see MEASURED) but the engine asks a review
// for them, and none exists, so they are grounded drafts, not verified and not gaps
var c : i32 = 15;
while (c < 18) {
const i : usize = @intCast(c);
assert GROUNDED[i];
assert STATUS[i] == STATUS_DRAFT;
assert REVIEWED[i] == false;
assert step_stage(@intCast(c)) == STAGE_DELIVER;
c = c + 1;
}
}

test the_camera_is_the_demo_camera {
Expand Down
26 changes: 13 additions & 13 deletions specs/igla/own_model_board.t27
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,23 +48,23 @@ pub const LENS_CELLS : i32 = 9;

pub const JOB : str = "own_model";
pub const STAGE : str = "own_model";
pub const MSG : str = "1 published, 12 verified, 7 drafts, 7 gaps; 44%";
pub const MSG : str = "1 published, 12 verified, 9 drafts, 5 gaps; 48%";
/// What MSG says once its numbers are taken out.
pub const MSG_WORDS : str = " published, verified, drafts, gaps; %";
pub const OVERALL_PCT : u8 = 44;
pub const PROGRESS : f64 = 0.44;
pub const OVERALL_PCT : u8 = 48;
pub const PROGRESS : f64 = 0.48;

pub const KEY_LENS : str = "lens_pct";
pub const KEY_CELLS : str = "cells";
pub const KEY_STATUS : str = "status";
pub const KEY_PCT : str = "pct";

pub const LENS_PCT : [3]u8 = [45, 57, 30];
pub const CELL_STATUS : [27]u8 = [4, 4, 4, 3, 3, 3, 3, 0, 0, 4, 4, 4, 4, 4, 4, 3, 0, 0, 4, 4, 5, 4, 3, 0, 0, 0, 3];
pub const CELL_PCT : [27]u8 = [60, 70, 80, 80, 60, 10, 50, 0, 0, 100, 100, 20, 80, 65, 85, 70, 0, 0, 70, 15, 60, 40, 35, 0, 0, 0, 50];
pub const LENS_PCT : [3]u8 = [45, 68, 30];
pub const CELL_STATUS : [27]u8 = [4, 4, 4, 3, 3, 3, 3, 0, 0, 4, 4, 4, 4, 4, 4, 3, 3, 3, 4, 4, 5, 4, 3, 0, 0, 0, 3];
pub const CELL_PCT : [27]u8 = [60, 70, 80, 80, 60, 10, 50, 0, 0, 100, 100, 20, 80, 65, 85, 75, 55, 40, 70, 15, 60, 40, 35, 0, 0, 0, 50];

/// The size of the compact metrics object, in bytes, computed from the keys and values above.
pub const METRICS_BYTES : i32 = 622;
pub const METRICS_BYTES : i32 = 624;

pub fn is_digit(ch: u8) -> bool {
return ch >= '0' and ch <= '9';
Expand Down Expand Up @@ -262,8 +262,8 @@ test the_tally_sums_to_twenty_seven {
assert published + verified + drafts + gaps == CELL_COUNT;
assert published == 1;
assert verified == 12;
assert drafts == 7;
assert gaps == 7;
assert drafts == 9;
assert gaps == 5;
}

test the_message_is_the_tally {
Expand All @@ -273,12 +273,12 @@ test the_message_is_the_tally {
const n : i32 = @intCast(MSG.len);
assert n <= MSG_MAX;
// one number wrong, a word changed, or a number added: each refused
assert msg_matches("1 published, 11 verified, 7 drafts, 7 gaps; 44%", STATUS, overall) == false;
assert msg_matches("1 published, 12 verified, 7 drafts, 7 holes; 44%", STATUS, overall) == false;
assert msg_matches("1 published, 12 verified, 7 drafts, 7 gaps; 44% of 100", STATUS, overall) == false;
assert msg_matches("1 published, 11 verified, 9 drafts, 5 gaps; 48%", STATUS, overall) == false;
assert msg_matches("1 published, 12 verified, 9 drafts, 5 holes; 48%", STATUS, overall) == false;
assert msg_matches("1 published, 12 verified, 9 drafts, 5 gaps; 48% of 100", STATUS, overall) == false;
assert msg_matches(MSG, STATUS, 45) == false;
assert nth_number(MSG, 1) == 12;
assert nth_number(MSG, 4) == 44;
assert nth_number(MSG, 4) == 48;
assert nth_number(MSG, 5) == -1;
}

Expand Down
Loading
Loading