Skip to content

spec(hslm): nn/hslm is the model trinity-training trained (Closes #10062) - #10068

Merged
gHashTag merged 1 commit into
masterfrom
feat/hslm-match-trained
Oct 17, 2026
Merged

spec(hslm): nn/hslm is the model trinity-training trained (Closes #10062)#10068
gHashTag merged 1 commit into
masterfrom
feat/hslm-match-trained

Conversation

@gHashTag

@gHashTag gHashTag commented Oct 17, 2026

Copy link
Copy Markdown
Owner

Closes #10062

What changed

specs/nn/hslm.t27 now describes the HSLM that trinity-training actually trained (src/hslm @3e9648c1). The header cites constants.zig, embedding.zig, sacred_attention.zig, trinity_block.zig, model.zig, ste.zig, trainer.zig, and FOUND_EXPERIMENTS_SUMMARY.md @ee512359 for the context sweep (ctx 18: PPL 5.58, ctx 27: 2.96, ctx 54: 6.05).

before now (= trained)
blocks 6 3
context 81 27 (the PPL 2.96 run)
FFN 243-972-243, GELU 243-729-243, ReLU, biases b_up/b_down, TWN alphas
input none token embedding + phi position encoding
attention RMSNorm(gamma), alpha-scaled Q/K/V/O, phi-RoPE on pairs (i, i+40), causal softmax with scale 1/81^(phi^-3)
output none plain RMSNorm, alpha_out * (n @ W_out) / 243^(phi^-3) + b_out, vocab 729, cross-entropy on the last position

The spec still leaves out three things, and the header says so:

  • the per-block VSA path;
  • f32 (the spec uses f64);
  • the trainer's STE update: no shadow weights, and the STE backward drops alpha.

Gradient checks

The backward pass now reaches the embedding (g_embed[token]). Each gradient is checked against a central difference, using a test hook that makes one ternary weight real-valued:

  • embedding;
  • Q, K, V and O in the last block, and Q in the first;
  • W_up and W_down;
  • b_up, b_down and b_out;
  • both RMSNorm gammas;
  • W_out.

W_out is also checked exactly against a trit flip, and the last block's W_down against a trit flip within 0.2%. The tests also cover ReLU, RoPE (identity at position 0, norm-preserving, invertible), the head, the cross-entropy gradient and the attention cache.

Lab results (Railway lab, t27c @efe3c94)

spec tests FAIL vacuous
specs/nn/hslm.t27 46/46 0 0 (10 invariants proved)
specs/igla/own_model.t27 8/8 0 0
specs/igla/own_model_board.t27 6/6 0 0

Mutation control: a copy with alpha_q dropped from the attention forward fails 10 of the 46 tests: the embedding, cross-entropy, Q, first-block W_up, bias and gamma finite-difference checks.

own_model

  • PCT: forward pass 60 -> 85, backward pass 40 -> 70.
  • HSLM lens 51 -> 57, overall 42 -> 44.
  • STATUS and the tally (1/12/7/7) are unchanged. Cell 16 is a DELIVER step that waits for review, so it stays DRAFT.
  • ROWS: the next steps are now "HSLM spec: add the VSA path" and "Ternary STE with shadow weights".
  • MEASURED and THE ANSWER now say the spec is the trained model except for the VSA path.
  • own_model_board.t27 mirrors all of this: MSG, OVERALL_PCT, PROGRESS, LENS_PCT, CELL_PCT and the one-off negative cases. METRICS_BYTES is unchanged at 622.

Seals

HSLM.json and nn_HSLM.json were re-saved on the lab with seal specs/nn/hslm.t27 --save:

  • spec_hash sha256:df24bc0c..., 46/46, 0 vacuous;
  • the files were copied back by sha256 (HSLM 5be91b2b..., nn_HSLM e214680d...).

Not in this PR

conformance/nn_hslm_vectors.json (generated 2026-04) still describes the old 6-layer / FF 972 / ctx 81 shape. Nothing in the tree reads it; it was already stale before #7778.

Only .t27 files and compiler-written seals change; no hand-written foreign code.

🤖 Generated with Claude Code

@gHashTag @claude
spec(hslm): nn/hslm is the model trinity-training trained (Closes #10062
1a6e210 
)

specs/nn/hslm.t27 now describes src/hslm in trinity-training @3e9648c1:
3 blocks, context 27 (the PPL 2.96 run), a ReLU FFN 243-729-243 with
biases and TWN alphas, phi position encoding and phi-RoPE, a token
embedding and the vocab-729 head with cross-entropy on the last position.
The header cites the trained source files and names what is not modelled
(the per-block VSA path, f32, the trainer's STE update).

The backward pass reaches the embedding; every gradient (embedding, Q/K/V/O,
W_up/W_down, biases, both RMSNorm gammas, W_out, b_out) is checked against
a central difference or an exact trit flip. Lab (t27c @efe3c94):
46/46 pass, 0 vacuous, 10 invariants proved. A mutant that drops alpha_q
fails 10 of the 46.

own_model: forward 60 -> 85, backward 40 -> 70; HSLM lens 51 -> 57,
overall 42 -> 44; rows and header updated. own_model_board mirrors it.
Lab: own_model 8/8, own_model_board 6/6, 0 vacuous.

Seals HSLM and nn_HSLM re-saved on the lab with t27c seal --save.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gHashTag
gHashTag enabled auto-merge (squash) October 17, 2026 18:06
@gHashTag

gHashTag commented Oct 17, 2026

Copy link
Copy Markdown
Owner Author

cli-tri is red on master too, the last 3 master runs of the same workflow failed with the same error, so this PR inherits it: cli/tri/src/gates.rs include_str!s .github/workflows/brain-seal-refresh.yml, which #9117 deleted. It is tracked in #9634. This PR touches only .t27 specs and seals and does not change cli/tri. The fix is a Rust edit, which needs the owner-approved-foreign label, so it is not made here.

@gHashTag
gHashTag merged commit ee3da59 into master Oct 17, 2026
25 of 28 checks passed
@github-actions

github-actions Bot commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-17 18:17:29 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 50
PRs with All Checks Green 0
READY 0
FAILING 50
PENDING 0
NO CHECKS YET 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=e769ef021713 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

This was referenced Oct 17, 2026
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

spec(nn): make specs/nn/hslm.t27 describe the trained HSLM (3 blocks, ctx 27, ReLU FFN with bias and alpha, embedding, vocab-729 head)

1 participant

@gHashTag