Skip to content

spec(igla): own_model -- take back the REVIEWED claim #10270 merged without a review (Closes #10272) #10273

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gHashTag merged 1 commit into from
Oct 18, 2026
Merged

spec(igla): own_model -- take back the REVIEWED claim #10270 merged without a review (Closes #10272) #10273

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 26 additions & 61 deletions specs/igla/own_model.t27
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,12 +94,12 @@
; .../tern-tc-100m returned 200 (tern-tc-100m answered 401 before 2026-10-18).
; GET board-production-2dec.up.railway.app/state returned 200 and a made-up path 404; its
; snapshot is dated 2026-10-05, so the board is live and 12 days stale.
; - review: #10164, #10179, #10186 and #10185 merged without one. #10185 was meant to carry the
; review of cells 7, 8, 16, 17, 18, 25 and 27, but auto-merge took it in on 2026-10-18 06:24
; with only a REQUEST_CHANGES on an older head (review 5561355779). So this change, PR
; REVIEW_PR (below), put the same evidence to review again, with auto-merge off: review
; REVIEW_ID is the reviewer bee's APPROVE of it. REVIEWED is true for those seven DELIVER cells
; and for no other cell.
; - review: no PR behind any cell has a review, so no DELIVER cell is VERIFIED. #10164 merged on
; 2026-10-18 02:43Z without one, and #10179, which put the evidence to review, was merged at
; 03:16Z by an auto-merge another session enabled, before the reviewer reached it; #10186
; (cell 25) merged at 03:42Z the same way. The PR that adds this line puts the evidence of
; cells 7, 8, 16, 17, 18, 25 and 27 to review again; REVIEWED still waits for a later PR that
; names the approving review.
; - board on 2026-10-18, cell 26: specs/fpga/ddr3_head_bandwidth.t27 (11/11 tests, 0 vacuous on the
; lab), lowered by t27b verilog around a LiteDRAM core, routed by openXC7 for xc7a200tfbg484-2 and
; flashed on the AX7203 with openFPGALoader -c digilent_hs2. Its UART receipt: 325000 beats
Expand All @@ -111,9 +111,8 @@
; worst path inside the LiteDRAM core, and 3 of 9 routes pass the BIOS memtest; the receipt is
; from those that do. This is a sequential read of a pattern, no weight on the board and no
; compute from DDR3, so the cell is a draft at 10 percent, a DELIVER step waiting for its review.
; Cell 26 has no review: its PR, #10248, merged without one, so it stays a draft.
; The engine asks an engineering LEARN step for a source link, a PRACTICE step for a passing test
; and a DELIVER step for a review. So: 3 published, 22 verified, 1 draft, 1 gap.
; and a DELIVER step for a review. So: 1 published, 17 verified, 8 drafts, 1 gap.

; PERCENT is a second, separate number per cell: an estimate of how much of that part exists, with
; its basis in SOURCES. It never feeds a status. The lens and overall figures are plain means, every
Expand Down Expand Up @@ -256,14 +255,11 @@ pub const MISSING : [27]str = [
/// The measurements per cell (see the header for how each was taken).
pub const GROUNDED : [27]bool = [true, true, true, true, true, true, true, true, false, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true];
pub const TEST_PASS : [27]bool = [false, false, false, true, true, true, false, false, false, false, false, false, true, true, true, false, false, false, false, false, false, true, true, true, false, false, false];
pub const REVIEWED : [27]bool = [false, false, false, false, false, false, true, true, false, false, false, false, false, false, false, true, true, true, false, false, false, false, false, false, true, false, true];
/// The approving review behind REVIEWED: PR and review id on gHashTag/t27.
pub const REVIEW_PR : i32 = 10270;
pub const REVIEW_ID : str = "REVIEW_ID_TBD";
pub const REVIEWED : [27]bool = [false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false];
pub const LIVE : [27]bool = [false, false, false, false, false, false, true, true, false, false, false, false, false, false, false, false, false, false, false, false, true, false, false, false, false, false, false];

/// What a host passes the viewer, per cell: the earned status code and the step number as a label.
pub const STATUS : [27]u8 = [4, 4, 4, 4, 4, 4, 5, 5, 0, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 4, 4, 4, 4, 3, 4];
pub const STATUS : [27]u8 = [4, 4, 4, 4, 4, 4, 3, 3, 0, 4, 4, 4, 4, 4, 4, 3, 3, 3, 4, 4, 5, 4, 4, 4, 3, 3, 3];
pub const LABELS : [27]str = ["1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24", "25", "26", "27"];

/// Percent of each part that exists, estimated from its source (see the header). Never a status.
Expand Down Expand Up @@ -292,7 +288,7 @@ pub const ROWS : [20]str = [
"In t27: loss, optimizers, HSLM pass",
"17 of 17 ML specs compile on the lab",
"9 of 9 IGLA CODER specs compile",
"3 published, 22 verified, 1 draft",
"1 published, 17 verified, 8 drafts",
"1 gap, about 61 percent overall",
"Batched f32 HSLM trainer",
"Shadow weights for the whole model",
Expand Down Expand Up @@ -363,12 +359,12 @@ test a_gap_is_named_not_filled {

test a_status_is_earned_not_chosen {
assert statuses_earned(STATUS, DOMAIN, GROUNDED, TEST_PASS, REVIEWED, LIVE);
// the evidence slide's "3 published, 22 verified, 1 draft" and "1 gap"
assert count_status(STATUS, STATUS_PUBLISHED) == 3;
assert count_status(STATUS, STATUS_VERIFIED) == 22;
assert count_status(STATUS, STATUS_DRAFT) == 1;
// the evidence slide's "1 published, 17 verified, 8 drafts" and "1 gap"
assert count_status(STATUS, STATUS_PUBLISHED) == 1;
assert count_status(STATUS, STATUS_VERIFIED) == 17;
assert count_status(STATUS, STATUS_DRAFT) == 8;
assert count_status(STATUS, STATUS_GAP) == 1;
assert count_true(REVIEWED) == 7;
assert count_true(REVIEWED) == 0;
// every passing test is a PRACTICE step: the t27 bench, MultiPL-E and coder specs (cells 4 to
// 6), loss, optimizers, forward pass, matvec, the ML specs that compile (cell 23) and the
// file I/O t27c lowers (cell 24)
Expand All @@ -381,51 +377,21 @@ test a_status_is_earned_not_chosen {
}
c = c + 1;
}
// the 9M model is live and its evidence reviewed, so it is published
// the 9M model is live, but a live page does not stand in for a review
assert LIVE[6];
assert REVIEWED[6];
assert STATUS[6] == STATUS_PUBLISHED;
assert STATUS[6] == STATUS_DRAFT;
// and the same holds for the public 100M model (step 8)
assert LIVE[7];
assert REVIEWED[7];
assert STATUS[7] == STATUS_PUBLISHED;
assert STATUS[7] == STATUS_DRAFT;
assert STATUS[23] == STATUS_VERIFIED;
// the slice training binary (step 25) is a reviewed DELIVER step that is not live: verified
// the slice training binary (step 25) is grounded, but a DELIVER step waits for a review
assert GROUNDED[24];
assert REVIEWED[24];
assert LIVE[24] == false;
assert step_stage(24) == STAGE_DELIVER;
assert STATUS[24] == STATUS_VERIFIED;
// the board is live and was sourced: the third published cell
assert STATUS[24] == STATUS_DRAFT;
// the board is live and was sourced, so it is the one published cell
assert STATUS[20] == STATUS_PUBLISHED;
}

test a_review_is_named_not_assumed {
// REVIEWED is true exactly at the grounded DELIVER cells PR REVIEW_PR put to review: every
// grounded DELIVER cell except step 26, whose evidence came in a PR with no review
assert REVIEW_PR == 10270;
assert REVIEW_ID.len > 0;
var c : i32 = 0;
while (c < CELL_COUNT) {
const i : usize = @intCast(c);
var want : bool = false;
if (step_stage(@intCast(c)) == STAGE_DELIVER and GROUNDED[i] and c != 25) {
want = true;
}
assert REVIEWED[i] == want;
c = c + 1;
}
// the gap, step 9, is a DELIVER step with nothing to review
assert step_stage(8) == STAGE_DELIVER;
assert GROUNDED[8] == false;
assert REVIEWED[8] == false;
// step 26 is grounded but not reviewed, so it stays a draft
assert step_stage(25) == STAGE_DELIVER;
assert GROUNDED[25];
assert REVIEWED[25] == false;
assert STATUS[25] == STATUS_DRAFT;
}

test the_compile_counts_are_the_measured_ones {
// the IGLA CODER specs cell: grounded, all nine compile and pass, a PRACTICE step with a
// passing test, so it is verified; its percent is the measured share of coder specs
Expand Down Expand Up @@ -459,16 +425,15 @@ test percent_per_lens_is_a_plain_mean {
assert mean_pct(0, CELL_COUNT) == 61;
}

test the_training_step_cells_are_verified_once_reviewed {
// steps 16-18 are DELIVER steps: their specs pass (see MEASURED) and the engine asks a review
// for them; review REVIEW_ID approved it, and they are not live, so they are verified
test the_training_step_cells_are_drafts_until_reviewed {
// steps 16-18 are DELIVER steps: their specs pass (see MEASURED) but the engine asks a review
// for them, and none exists, so they are grounded drafts, not verified and not gaps
var c : i32 = 15;
while (c < 18) {
const i : usize = @intCast(c);
assert GROUNDED[i];
assert STATUS[i] == STATUS_VERIFIED;
assert REVIEWED[i];
assert LIVE[i] == false;
assert STATUS[i] == STATUS_DRAFT;
assert REVIEWED[i] == false;
assert step_stage(@intCast(c)) == STAGE_DELIVER;
c = c + 1;
}
Expand Down
18 changes: 9 additions & 9 deletions specs/igla/own_model_board.t27
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ pub const LENS_CELLS : i32 = 9;

pub const JOB : str = "own_model";
pub const STAGE : str = "own_model";
pub const MSG : str = "3 published, 22 verified, 1 drafts, 1 gaps; 61%";
pub const MSG : str = "1 published, 17 verified, 8 drafts, 1 gaps; 61%";
/// What MSG says once its numbers are taken out.
pub const MSG_WORDS : str = " published, verified, drafts, gaps; %";
pub const OVERALL_PCT : u8 = 61;
Expand All @@ -60,7 +60,7 @@ pub const KEY_STATUS : str = "status";
pub const KEY_PCT : str = "pct";

pub const LENS_PCT : [3]u8 = [65, 70, 48];
pub const CELL_STATUS : [27]u8 = [4, 4, 4, 4, 4, 4, 5, 5, 0, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 4, 4, 4, 4, 3, 4];
pub const CELL_STATUS : [27]u8 = [4, 4, 4, 4, 4, 4, 3, 3, 0, 4, 4, 4, 4, 4, 4, 3, 3, 3, 4, 4, 5, 4, 4, 4, 3, 3, 3];
pub const CELL_PCT : [27]u8 = [60, 70, 80, 80, 60, 100, 50, 90, 0, 100, 100, 20, 80, 65, 95, 75, 55, 40, 70, 15, 60, 40, 100, 60, 30, 10, 50];

/// The size of the compact metrics object, in bytes, computed from the keys and values above.
Expand Down Expand Up @@ -260,9 +260,9 @@ test the_tally_sums_to_twenty_seven {
assert count_status(CELL_STATUS, STATUS_LOCKED) == 0;
assert count_status(CELL_STATUS, STATUS_OPEN) == 0;
assert published + verified + drafts + gaps == CELL_COUNT;
assert published == 3;
assert verified == 22;
assert drafts == 1;
assert published == 1;
assert verified == 17;
assert drafts == 8;
assert gaps == 1;
}

Expand All @@ -273,11 +273,11 @@ test the_message_is_the_tally {
const n : i32 = @intCast(MSG.len);
assert n <= MSG_MAX;
// one number wrong, a word changed, or a number added: each refused
assert msg_matches("3 published, 22 verified, 0 drafts, 1 gaps; 61%", STATUS, overall) == false;
assert msg_matches("3 published, 22 verified, 1 drafts, 1 holes; 61%", STATUS, overall) == false;
assert msg_matches("3 published, 22 verified, 1 drafts, 1 gaps; 61% of 100", STATUS, overall) == false;
assert msg_matches("1 published, 17 verified, 7 drafts, 1 gaps; 61%", STATUS, overall) == false;
assert msg_matches("1 published, 17 verified, 8 drafts, 1 holes; 61%", STATUS, overall) == false;
assert msg_matches("1 published, 17 verified, 8 drafts, 1 gaps; 61% of 100", STATUS, overall) == false;
assert msg_matches(MSG, STATUS, 60) == false;
assert nth_number(MSG, 2) == 1;
assert nth_number(MSG, 2) == 8;
assert nth_number(MSG, 4) == 61;
assert nth_number(MSG, 5) == -1;
}
Expand Down
Loading