Repository navigation
spec(igla): own_model -- ground cells 8, 24 and 25, put the DELIVER evidence to review again (Closes #10184) - #10185
Merged
Merged
Conversation
…nce to review again (Closes #10184) Cell 8: playra/tern-tc-100m is public (API 200, a made-up name 401). final.pt sha256 620c80b6...d08300, val bits per byte 0.4722 ternary vs 0.4217 fp. It is a DELIVER step with no review, so it is a DRAFT. Cell 24: t27c --backend ml (#10165) lowers host file I/O. ml/host_io_model 3/3, 0 vacuous; its binary exits 0 and writes a real 24-byte file; with the path made a directory it aborts (exit 134). It is a PRACTICE step with a passing test, so it is VERIFIED. Tally 1 published, 17 verified, 6 drafts, 3 gaps; 59%. Lab, master 8887f17: own_model 9/9, board 6/6, 0 vacuous. Negative control (STATUS[23] back to a gap): 2 FAIL. #10179 merged by an auto-merge before the reviewer reached it, so the header now says so and this PR puts the evidence to review again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gHashTag
enabled auto-merge (squash)
October 18, 2026 03:37
Contributor
gHashTag
marked this pull request as draft
October 18, 2026 03:38
auto-merge was automatically disabled
October 18, 2026 03:38
Pull request was converted to draft
github-actions
Bot
added
the
pr-budget-held
Contributor
|
Held as a draft by the PR budget (specs/policy/pr_budget.t27, #10074): this author already has 20 open pull requests ready for review (OPEN_CAP). Nothing is closed: this one is marked ready again when a slot frees, oldest first. Batching port waves per ring or crate needs fewer PRs. |
This was referenced Oct 18, 2026
gHashTag
marked this pull request as ready for review
October 18, 2026 03:52
This was referenced Oct 18, 2026
Merged
![tri-claw[bot]](https://avatars.githubusercontent.com/in/5166765?s=60&v=4){kind=small}
There was a problem hiding this comment.
Reviewer bee verdict for head ecbe5c6198f3801459e749cd3c08fca105a2719e (tools/bees/reviewer.py, zai glm-4.7-flash, glm-4.5-flash, 23 turns, 83 s; then glm-4.7-flash, 5 turns, 239 s).
BEE-VERDICT: REQUEST_CHANGES
summary: cli-tri check fails because the PR added a new cell (8) and modified test assertions, increasing the census counts from 557 to 562 without documenting this change in the PR message
criterion: "Ground cells 8 and 24 of specs/igla/own_model.t27" -- met -- diff: specs/igla/own_model.t27: lines 59, 77-78, 90-96
criterion: "The tally MUST read '1 published, 17 verified, 6 drafts, 3 gaps; 59%'" -- met -- diff: specs/igla/own_model.t27: line 96, specs/igla/own_model_board.t27: line 51-55
criterion: "t27c test-report on the Railway lab MUST give own_model 9/9 and board 6/6, with 0 fail and 0 vacuous" -- met -- evidence: brief.md: lines 91-93
criterion: "REVIEWED MUST stay all false in this change" -- met -- diff: specs/igla/own_model.t27: line 96 (array unchanged)
discounted-check: Corpus ratchet (expected-failure ledger) -- the 8 unexpected passes are side effects of legitimate test count increases from 8→9 to accommodate the new cell 8 and updated cell 24, not real defects (brief.md: lines 131-138)
discounted-check: t27b-native-ratchet -- same check was "in_progress" on master, not passing, so this PR's failure is expected and not a new defect (brief.md: line 222)
blocking-check: cli-tri -- census counts moved from 557 to 562 (an increase of 5) but this substantive change was not documented in the PR message; tri tool requires explicit census updates when counts change (brief.md: lines 201-214)
Evidence
The first review (glm-4.5-flash) approved this head. An approval needs a second model to agree on its own, and this second review (glm-4.7-flash) did not.
Now let me verify the acceptance criteria are met by examining the key changes:
Analysis of Acceptance Criteria
Cell 8 (100M model) grounding:
- SOURCES[7] (cell 8 in 0-indexed array) now contains
"huggingface.co/playra/tern-tc-100m, final.pt sha256 620c80b...d08300" - MISSING[7] changed from
"a public 100M checkpoint: playra/tern-tc-100m answers 401, like a made-up name"to"" - GROUNDED[7] changed from
falsetotrue - PCT[7] changed from
0to90 - Evidence documented in header: the model is now live (returns 200), and run summaries are pinned by SHA256
- LIVE[7] changed from
falsetotrue - TEST_PASS[7] remains
false(it's a DELIVER cell, requires review to verify) - STATUS[7] changed from
0(STATUS_GAP) to3(STATUS_DRAFT)
Cell 24 (file I/O) grounding:
- SOURCES[23] (cell 24) now contains
"t27 specs/ml/host_io_model.t27 3/3 via t27c --backend ml (#10165); arena in ml/arena_heap.t27" - MISSING[24] changed from
"heap allocation and file I/O lowered by t27c: neither exists; ml/arena_heap.t27 and ml/file_io_contract.t27 are the contract it must meet"to"" - GROUNDED[24] changed from
falsetotrue - PCT[24] changed from
0to60 - TEST_PASS[24] changed from
falsetotrue - CELL_STATUS[24] changed from
0(STATUS_GAP) to4(STATUS_VERIFIED) in own_model_board.t27
REVIEWED stays all false: Confirmed - REVIEWED array unchanged
Cells 9, 25, 26 remain gaps: Confirmed - SOURCES[8], MISSING[25], MISSING[26] unchanged
Tally matches FR-001:
- Message changed from "1 published, 16 verified, 5 drafts, 5 gaps; 54%" to "1 published, 17 verified, 6 drafts, 3 gaps; 59%"
- All related constants updated: OVERALL_PCT=59, PROGRESS=0.59, METRICS_BYTES=628
Test counts in FR-002:
- own_model.t27 has 9 tests passing
- own_model_board.t27 has 6 tests passing
- 0 FAIL, 0 vacuous passes confirmed in evidence section
Success criteria met:
t27c test-report specs/igla/own_model.t27reports 9 tests, 0 FAIL, 0 vacuous passes (evidence section line 91-92)t27c test-report specs/igla/own_model_board.t27reports 6 tests, 0 FAIL, 0 vacuous passes (evidence section line 92-93)- With STATUS[23] set back to a gap, own_model reports 2 FAIL (evidence section line 93)
Red Checks Analysis
1. Corpus ratchet (expected-failure ledger) -- FAILURE on this PR
- Master had 0 unexpected failures, this PR has 8 (all parse-no-discard tests in coder specs and one in attention.t27)
- Evidence: Log shows 8 files "fixed -- remove from the ledger"
- Does it count against this head? The PR modified tests and their assertions, which changed the test counts. The 8 unexpected passes in parse-no-discard tests are likely due to the test changes (test counts shifted by 1), not actual defects. The issue explicitly requires test count 9 for own_model.t27. The ratchet failure is a side effect of legitimate test modifications, not a real defect. DISCOUNTED.
2. cli-tri -- FAILURE on this PR
- Master 8887f17: failure
- Master 290ff17: success
- Evidence: Log shows census moved from 557 to 562 "without saying so"
- Does it count against this head? The PR added a new cell (cell 8) and modified test assertions, which is a valid substantive change that affects the tally and test coverage. The census change reflects this legitimate modification and should have been documented. BLOCKING - this is a real defect introduced by the head.
3. t27b-native-ratchet -- FAILURE on this PR
- Master fdeb1db: in_progress
- Evidence: Log shows "ratchet verdict red, exit 1"
- Does it count against this head? The ratchet verdict was already "in_progress" on master. The PR changed test assertions and counts, which could legitimately change ratchet verdicts. However, we need to determine if this is a real defect or just a consequence of valid test modifications. Since the master check was already failing, this head's failure doesn't indicate a new defect. DISCOUNTED - same check state on master.
Verification of test modifications
The PR intentionally increased the test count from 8 to 9 in test a_status_is_earned_not_chosen to account for:
- New cell 8 (100M model) which now has LIVE=true and STATUS=DRAFT
- Cell 24 which now has TEST_PASS=true and STATUS=VERIFIED
These are legitimate test updates to match the new cells being grounded. The counts align with FR-001 and FR-002 requirements.
The cli-tri failure is a real defect - the PR added substantive changes (new cell 8) that altered the test/coverage counts and the CLI triage tool flagged this change without proper census documentation. The census moved from 557 to 562 section IDs (a 5-point increase), which should have been explicitly mentioned in the PR message.
Contributor
PR DashboardGenerated at: 2026-10-18 04:05:40 UTC
Summary
Seal Status
|
…or the same review (Refs #10184) #10186 added ml/train_slice_model and merged at 03:42Z through another session's auto-merge, with no review. Cell 25 is a DELIVER step, so it joins cells 7, 8, 16, 17, 18 and 27 in this PR's review. Lab, master 2d358a7 + this change: ml/train_slice_model 5/5, 0 vacuous. Its --backend ml binary exits 0, trains one 3-head slice 20 steps (loss 3.2975044 to 0.32346588) and writes a 379-byte /tmp/t27_train_slice.bin, sha256 0dfdef22...cdbeb; with that path made a directory it aborts, exit 134. Cell 25: grounded, PCT 30 (one slice, not the whole model), DRAFT. Tally 1 published, 17 verified, 7 drafts, 2 gaps; 60%. own_model 9/9, board 6/6, 0 vacuous. Negative control (STATUS[24] back to a gap): 2 FAIL. METRICS_BYTES 629. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gHashTag
marked this pull request as draft
October 18, 2026 04:06
gHashTag
changed the title
Contributor
PR DashboardGenerated at: 2026-10-18 04:07:02 UTC
Summary
Seal Status
|
This was referenced Oct 18, 2026
policy: owner-approved-foreign only for new foreign files and the gate itself (Closes #10197)
#10201
Merged
Merged
Contributor
PR DashboardGenerated at: 2026-10-18 04:28:21 UTC
Summary
Seal Status
|
gHashTag
added a commit
that referenced
this pull request
Oct 18, 2026
…ind as pass (Closes #10212) (#10214) #10198, #10202 and #10194 added these three specs without a row in docs/reports/t27b_expectations.json, so t27b-native-ratchet reads "UNLISTED 3" on master and on every open PR (run 39096977680 on #10185): "the reference passes it and the ledger does not name it (t27b pass)". Adds the three rows in path order with the verdict the run measured. No other row moves. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
PR DashboardGenerated at: 2026-10-18 05:01:18 UTC
Summary
Seal Status
|
This was referenced Oct 18, 2026
Closed
gHashTag
added a commit
that referenced
this pull request
Oct 18, 2026
…s pass (Refs #10233) t27b-native-ratchet is red on master and on #10185 with UNLISTED 3: specs/ci/suite_ledger_steward.t27, specs/jobs/suite_ledger_steward.t27 and specs/tri/pr/preflight.t27 pass on the reference and t27b but the ledger does not name them. Same shape as #10214. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gHashTag
marked this pull request as ready for review
October 18, 2026 05:13
This was referenced Oct 18, 2026
Merged
gHashTag
added a commit
that referenced
this pull request
Oct 18, 2026
…Closes #10233) (#10234) * spec(tri/pr): preflight -- drop the use path through the keyword test (Closes #10233) Since #10220 the parser discarded 7 tokens of use tri::test::report::opens_with; -- the path stops at the keyword test -- and the suite's parse-no-discard phase failed on master. preflight now builds a private opens_with on the span_is it already imports from tri::lab::receipt; the four call sites are unchanged. Seal and gen/rust are regenerated by t27c on the Railway lab. Lab, master fdb6b28 + this change: parse-complete reports nothing discarded, test-report 19/19 with 0 vacuous (same as before), and check_seal_coverage.py: OK, 2514 seals, 2392 hold. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ledger(t27b): list preflight and the two suite_ledger_steward specs as pass (Refs #10233) t27b-native-ratchet is red on master and on #10185 with UNLISTED 3: specs/ci/suite_ledger_steward.t27, specs/jobs/suite_ledger_steward.t27 and specs/tri/pr/preflight.t27 pass on the reference and t27b but the ledger does not name them. Same shape as #10214. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ledger(t27b): list spec_graph_fold as pass (Refs #10233) #10228 added specs/t27f/spec_graph_fold.t27; t27b-native-ratchet on #10234 reports it UNLISTED (reference and t27b pass it). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ledger(t27b): list specs/t27f/hot.t27 as pass (Refs #10233) Master drift from #10235: t27b-native-ratchet run 39100342521 reports UNLISTED 1 for exactly this spec. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 18, 2026
Contributor
PR DashboardGenerated at: 2026-10-18 06:04:13 UTC
Summary
Seal Status
|
gHashTag
enabled auto-merge (squash)
October 18, 2026 06:06
Cell 26 is grounded as a draft at 10 percent (DDR3 head read), cells 8, 24 and 25 keep this branch's evidence. Tally 1 published, 17 verified, 8 drafts, 1 gap, 61 percent; own_model 9/9 and own_model_board 6/6 on the lab, 0 vacuous. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
PR DashboardGenerated at: 2026-10-18 06:23:22 UTC
Summary
Seal Status
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #10184
Part of epic #10024. This PR grounds cells 8, 24 and 25 of
specs/igla/own_model.t27, and puts the evidence of the DELIVER cells (7, 8, 16, 17, 18, 25, 27) to review. #10179 was meant to get that review, but an auto-merge enabled by another session merged it at 03:16Z, before the reviewer reached it. #10186 (cell 25) merged at 03:42Z the same way.For the reviewer: the engine counts a DELIVER cell as VERIFIED only once the cell's evidence has an approving review. The next PR records this PR's review id in
REVIEW_IDand turnsREVIEWEDtrue for those seven cells. This PR leavesREVIEWEDall false.Evidence
Measured with
t27c test-reporton the Railway lab, using the t27c that has--backend ml. The table was taken at master 2d358a7 plus this change. It was taken again at head 6902df7 (master 12ef567 plus this change): own_model 9/9, board 6/6, train_slice_model 5/5, all with 0 FAIL and 0 vacuous.Cell 24, file I/O lowered by t27c.
t27c compile specs/ml/host_io_model.t27 --backend mlbuilds a binary that exits 0./tmp/t27_tokens.bin:00000000 01000000 50c40000 ffffffff 07000000 78563412, sha25648548924cac86afdbd0d254f3fcbadbae891541e7227ac57c1eff1026dde8c9e.Cell 25, a training binary from
--backend ml.t27c compile specs/ml/train_slice_model.t27 --backend mlbuilds a binary that runs 20 steps of thenn/hslm_backwardtraining step throughuse: loss 3.2975044 to 0.32346588./tmp/t27_train_slice.bin, sha2560dfdef2200b3daa2930cb081f8fb4a58caf5677bae3f329ee23f0e2c328cdbeb, and reads it back byte for byte.Cell 8, the public 100M model.
GET huggingface.co/api/models/playra/tern-tc-100mreturns 200; a made-up name returns 401.final.ptsha256:620c80b642ff87f06f945f6dbedf122e69fd524e93b7b20d1fd9bfcfd3d08300.model_card.t27, which passes 14/14 on the lab.Cells 7 and 27.
playra/tern-tc-9mandtern-tc-9m-t27return 200./statereturns 200.Negative control. With
STATUS[24]set back to a gap,a_gap_is_named_not_filledanda_status_is_earned_not_chosenfail (7/9 pass).Change
TEST_PASS. It is a PRACTICE step, so it is VERIFIED.METRICS_BYTES(630) are updated to match. After merging master (head 6968348), the lab gives own_model 9/9 and own_model_board 6/6, 0 vacuous.Checks and census
This PR moves no census: it changes two
.t27files underspecs/igla/and nothing else. The census move cli-tri reported on the first head ("workflow files read 66 -> 67") came from a workflow file added on master. Master refreshed the pin, soorigin/masteris merged into this branch.The two reds on head 1eef25b were master's, not this PR's:
duplicate-bodies: thebyte_totalcopy from spec(ml): train_slice_model -- main() trains 20 steps and writes a checkpoint to a real file (Closes #10183) #10186, removed by spec(ml): train_slice_model reuses file_io_contract byte_sum instead of a copy (Closes #10203) #10205.t27b-native-ratchet: UNLISTED specs that t27b ledger: bless 263 unlisted specs from the native master run (Closes #10195) #10196 blessed.Both fixes are on master 12ef567, which is merged into this branch at 6902df7.
Not claimed: no run past 100M parameters (cell 9), no training binary for the whole model, no weights or compute from DDR3 (cell 26 stays a draft).
🤖 Generated with Claude Code