Repository navigation
spec(nn): ternary STE, training step and data/checkpoint layout in t27 (Closes #10076) - #10086
Merged
Merged
Conversation
Closes #10076) Cells 16-18 of the own-model map (epic #10024): - nn/ternary_ste.t27: TWN quantizer, pass-through STE, clip_grad_norm, SGD on shadow weights, requantize (26 tests). - nn/hslm_backward.t27: forward, backward, STE, clip, SGD on a 3-head ctx-27 slice; gradients checked against central differences (15 tests). - nn/hslm_data.t27: curriculum, next-token batches, MSLH header and layout; byte count equals hslm_step_10000.bin, 4971796 (39 tests). - igla/own_model.t27 and own_model_board.t27: cells 16-18 now grounded drafts (DELIVER steps, no review yet): 1 published, 12 verified, 9 drafts, 5 gaps; 48%. Not covered: STE over the whole model, file I/O, heap, tokenizer, shuffle, AdamW. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This was referenced Oct 17, 2026
spec(policy): l2_generation -- gen/json/ is t27c gen-json's backend directory (Closes #10079)
#10082
Merged
gHashTag
added a commit
that referenced
this pull request
Oct 17, 2026
Adds the per-block VSA stream of trinity-training's HSLM (@3e9648c1, trinity_block.zig, model.zig forwardTrain) to nn/hslm: bind and rotate, the max_sim >= PHI_INV gate, +0.1 * project(context) when it fires, and the block output handed to the next block. The offset never reads a float parameter, so the existing finite-difference gradient checks still hold with the stream on. The ternary STE update is left to its own lane (#10076, PR #10086). nn/hslm on the Railway lab: 61/61 tests pass, 12 invariants proved, 0 vacuous (master: 46 tests). Mutation control: gate > for >= fails 2 tests, VSA gain 0 fails 1. HSLM and nn_HSLM are re-sealed on the lab. own_model: the forward-pass cell goes 85 to 95 percent (lens 2 now 58, overall still 44); own_model_board mirrors it. Cells 15-17 (STE) are untouched. Both pass 8/8 and 6/6, 0 vacuous. Deletes conformance/nn_hslm_vectors.json: it described the 6-layer, FF 972, ctx 81 model and nothing read it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
This was referenced Oct 17, 2026
Merged
gHashTag
enabled auto-merge (squash)
October 17, 2026 19:29
gHashTag
added a commit
that referenced
this pull request
Oct 17, 2026
…#10099) Conflicts in specs/igla/own_model.t27 and own_model_board.t27 with #10086 (ternary STE, training step, data layout): kept #10086's cells 15-17 (85/75/55/40 -> STE cells 75/55/40) and its MEASURED bullet, plus this PR's forward-pass cell 14 at 95 and its VSA bullet. Lens 2 re-derived: mean of 9..17 = 630 / 9 = 70 (was 68 on master); overall stays 48 and METRICS_BYTES stays 624 (no digit count changed). The "add the VSA path" gap row becomes "Batched f32 HSLM trainer". duplicate-bodies: copy_floats in nn/hslm had copy_trits' body; it is now save_block_out, which copies act_out into block_out_scratch. The four groups master already failed on (fill, ternary_matmul, matmul_probe, matmul_rect_probe: nn/hslm's helpers copied into nn/attention and nn/hslm_backward) are recorded in the ledger with dupe_scan --bless; the ledger gains exactly those four lines. Railway lab after the merge: nn/hslm 61/61 tests, 12 invariants, own_model 9/9, own_model_board 6/6, all 0 FAIL, 0 vacuous. HSLM and nn_HSLM re-sealed on the lab, sha256 matched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gHashTag
added a commit
that referenced
this pull request
Oct 17, 2026
…0100) Adds the per-block VSA stream of trinity-training's HSLM (@3e9648c1, trinity_block.zig, model.zig forwardTrain) to nn/hslm: bind and rotate, the max_sim >= PHI_INV gate, +0.1 * project(context) when it fires, and the block output handed to the next block. The offset never reads a float parameter, so the existing finite-difference gradient checks still hold with the stream on. The ternary STE update is left to its own lane (#10076, PR #10086). nn/hslm on the Railway lab: 61/61 tests pass, 12 invariants proved, 0 vacuous (master: 46 tests). Mutation control: gate > for >= fails 2 tests, VSA gain 0 fails 1. HSLM and nn_HSLM are re-sealed on the lab. own_model: the forward-pass cell goes 85 to 95 percent (lens 2 now 58, overall still 44); own_model_board mirrors it. Cells 15-17 (STE) are untouched. Both pass 8/8 and 6/6, 0 vacuous. Deletes conformance/nn_hslm_vectors.json: it described the 6-layer, FF 972, ctx 81 model and nothing read it. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #10076
Moves cells 16 (backward pass), 17 (ternary STE) and 18 (data loader and checkpoint I/O) of the own-model map (epic #10024).
What is new (all
.t27, nothing else)specs/nn/ternary_ste.t27: TWN quantizer (delta = 0.7 * mean abs, alpha = mean abs of kept weights), vanilla/progressive modes, pass-through STE,clip_grad_norm, SGD on the shadow weights, requantize. Source: trinity-trainingste.zig/autograd.zig/trainer.zig@ 3e9648c1.specs/nn/hslm_backward.t27: one training step (forward, backward, STE, clip, SGD, requantize) on a 3-head slice of the model, ctx 27 = 3^3, head_dim 3. Trit, bias and input gradients match a central finite difference to 1e-6; 20 steps cut the loss below half.specs/nn/hslm_data.t27: curriculum (5 schedules), next-token batches (shift by one, wrap, pad), MSLH header read/write, f32 bit codec, and the checkpoint layout: its byte count is 4971796, the size ofhslm_step_10000.bin; the real header bytes decode to step 10000, loss 5.552.specs/igla/own_model.t27,own_model_board.t27: cells 16-18 are now grounded drafts (they are DELIVER steps and no review exists, so the engine does not verify them): 1 published, 12 verified, 9 drafts, 5 gaps; 48 percent.Measured on the Railway lab (same t27c as the map)
hslm_backwardwas mutation-checked (4 mutants, all caught).Still a gap
STE over the whole 3-block model (a shadow copy of about 2M weights), file I/O and heap (cells 24, 25),
--backend ml, the VSA path, AdamW, tokenizer, shuffle.nn/hslm.t27andhslm/forward_pass.t27are untouched.🤖 Generated with Claude Code