Repository navigation
Conversation
… and LAMB, writes MSLH checkpoints (Refs #10372) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the flat vector, xorshift guards zero (Refs #10372) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion and full sequence (Refs #10372) main() reads /tmp/hslm/eval.cfg (step, windows, stride, check), loads ckpt_<step>.state and valid.tok through train_hslm's own readers, and scores non-overlapping 28-token windows two ways: the last position only (the trainer's valid_loss) and every position (27 terms per window). It prints and writes eval step <s> windows <n> positions <p> last_loss .. last_ppl .. full_loss .. full_ppl .. to /tmp/hslm/eval_<step>.txt. train_hslm.t27 is reached through use and is not touched. 10 tests, sealed. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
gHashTag
added
the
no-auto-merge
Contributor
This was referenced Oct 18, 2026
Merged
Merged
Merged
This was referenced Oct 18, 2026
Merged
Merged
Merged
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #10372 (epic #10024, own_model cell 25). Stacked on #10387 (feat/train-hslm, head 45ea9a9).
What it measures
specs/ml/eval_hslm.t27is a t27 program that scores a trained HSLM checkpoint on held-out tokens (/tmp/hslm/valid.tok) two ways, on the same windows of 28 tokens (27 inputs, targets are tokens 1..27):last_loss/last_ppl: cross-entropy of the last position only. This is the numbertrain_hslmprints asvalid_lossand the way run R5's PPL 2.96 was reported.full_loss/full_ppl: cross-entropy at every position (27 terms per window, mean over all of them).Position 26's term is the same number in both, so
full_lossis what the model does on the 26 positions the trainer never scores.main()reads/tmp/hslm/eval.cfg(key=value lines:step,windows,stride,check; defaults 0 / 243 / 0 / 0), loadsckpt_<step>.stateandvalid.tok, and prints and writes to/tmp/hslm/eval_<step>.txt:stride=0spaces the windows evenly exactly astr_valid_lossdoes; a set stride below 28 is raised to 28 so windows never overlap. A window that does not fit is refused and not counted.check=1also runs the trainer's owntr_valid_lossand prints both numbers and whether they are the same bits. The checkpoint must be the.statefile: the MSLH.binholds only the ternary shadows and biases.It changes no training path
use ml::train_hslmreaches the forward pass,cross_entropy,tr_load_state,tr_read_tok,tr_valid_lossand the formatters.specs/ml/train_hslm.t27is not touched: zero lines changed, nothing madepub. The diff is two new files,specs/ml/eval_hslm.t27and.trinity/seals/ml_eval_hslm.json. What the new module declares itself: the shape consts, the modulevarstate the spliced code binds to (use_resolve splices novar), and two kernels (ternary_matmul,trit_int; train_hslm's speed-1 forms, tested there to give the reference logits to the bit) because the name is declared by both nn/hslm and train_hslm and is not resolved.Evidence (Railway lab, t27c 0.5.2
/root/dupdecl/target/release/t27c, zig 0.16.0)Tests:
t27c test-report --specs-dir /root/eval-hslm/specs specs/ml/eval_hslm.t27: 10 tests, 10 pass, 0 vacuous. Seal:t27c seal --save(tests 10/10 pass), saved.trinity/seals/ml_eval_hslm.json.Negative controls (break one line, run test-report, restore; spec hash back to 09c6729a... each time):
p + 1 == CONTEXT_LEN->p + 2 == ...tokens[start + p + 1]->tokens[start + p]ev_mean_fulldivides byev_npos + 1start + EV_WINDOW > len->start + CONTEXT_LEN > lenBinary:
t27c compile specs/ml/eval_hslm.t27 --backend ml(0 UNRESOLVED), thenzig build-exe -O ReleaseFast, run undernice -n 19while the training run stayed live.Cross-check against
/tmp/hslm/run.log(243 evenly spaced windows,check=1):Both match run.log to the printed digits, and to the bit against
tr_valid_loss.2000 windows (54000 positions, 126 s for step 2000 on a loaded 48-core box):
Reading: the last-position loss falls 5.75 -> 4.50 between steps 2000 and 4000, while the full-sequence loss rises 6.20 -> 6.54 (PPL 493 -> 690). The trainer only scores position 26, so the other 26 positions are not being trained on.
Only
eval_*.txtand a temporaryeval.cfg(removed) were written in/tmp/hslm; nockpt_*orrun.logwas touched.Draft, label
no-auto-merge.🤖 Generated with Claude Code