Skip to content

spec(nn): make specs/nn/hslm.t27 describe the trained HSLM (3 blocks, ctx 27, ReLU FFN with bias and alpha, embedding, vocab-729 head) #10062

Closed as not planned

Description

@gHashTag

What

specs/nn/hslm.t27 compiles and passes since #10055, but it does not describe the HSLM that gHashTag/trinity-training trained (src/hslm, commit 3e9648c1). Its own header lists the gaps:

  • NUM_LAYERS 6; the trained model has DEFAULT_BLOCKS 3.
  • CONTEXT_LEN 81; the best run (PPL 2.96, trinity docs/experiments/FOUND_EXPERIMENTS_SUMMARY.md) trained at context 27.
  • FFN 243 -> 972 -> 243 with GELU, no bias, behind a second RMSNorm; the trained block is 243 -> 729 -> 243 with ReLU, a TWN alpha per matrix, a bias after each projection, and no norm.
  • No token embedding (table + phi position encoding) and no vocab-729 output head (RMSNorm, ternary W_out, alpha, 1/243^(phi^-3), bias).
  • The header says attention matches. It does not: the trained attention scales Q/K/V/O by TWN alphas and rotates Q and K with phi-RoPE.

Done when

  • specs/nn/hslm.t27 describes the trained forward pass, cites the trained source files, and names what still differs.
  • The backward pass is still checked by finite differences, including ReLU, the biases, the embedding and the head.
  • specs/igla/own_model.t27 and its mirror specs/igla/own_model_board.t27 count it, and both pass.
  • hslm, own_model and own_model_board report 0 FAIL and 0 vacuous on the Railway lab; HSLM and nn_HSLM are re-sealed.

This is the own-model board row "Match the spec to the trained HSLM".

Activity

  1. added
    shape-refusedClosed by the issue-shape gate (specs/policy/issue_shape.t27); a passing edit reopens it
    on Oct 17, 2026
  2. github-actions commented on Oct 17, 2026

    @github-actions
    Contributor

    Closed by the issue-shape gate (specs/policy/issue_shape.t27, #7498): an issue must have the template's sections. Edit the body; an edit that passes reopens it.

    • missing: ## Boundary -- list every file this task may touch, one per line (- specs/x.t27); without it the Queen reserves nothing and no bee can take the issue
    • missing: ## User Scenarios -- at least one Given/When/Then scenario
    • missing: ## Requirements -- numbered obligations, FR-001: ... MUST ...
  3. added a commit that references this issue on Oct 17, 2026
    ee3da59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    shape-refusedClosed by the issue-shape gate (specs/policy/issue_shape.t27); a passing edit reopens it

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions