Skip to content

spec(ml): adamw.t27 heap-free rewrite, 16/16 tests pass, 0 vacuous (Closes #10091) - #10093

Closed
gHashTag wants to merge 1 commit into
masterfrom
spec/adamw-heap-free-10091
Closed

spec(ml): adamw.t27 heap-free rewrite, 16/16 tests pass, 0 vacuous (Closes #10091)#10093
gHashTag wants to merge 1 commit into
masterfrom
spec/adamw-heap-free-10091

Conversation

@gHashTag

@gHashTag gHashTag commented Oct 17, 2026

Copy link
Copy Markdown
Owner

Closes #10091
Refs #10024 (cell 23)

What

Heap-free rewrite of specs/ml/optimizer/adamw.t27. The old file called allocate_array (not lowered by t27c), had comment-only function bodies, and carried two copies of the optimizer (pre-B1 and Phase B1). The new file is one module in the adam.t27 / lamb.t27 style: the caller owns params, grads, m, v, v_max; adamw_step is the real update:

theta = theta - lr * (m_hat / (sqrt(v_hat) + eps) + wd * theta)

with bias correction, optional AMSGrad (v_max tracks raw v, bias-corrected afterwards, as in PyTorch), and refusal of NaN/inf gradients. beta1, beta2 and epsilon defaults and powi / is_finite are taken from adam.t27 through use, not copied.

Measured (Railway lab, t27c test-report, zig 0.16.0)

tests pass FAIL BLOCKED vacuous invariants
16 16 0 0 0 of 16 3 proved (comptime)

Each test executes 1 runtime assert (a chained and). Before: BLOCKED, allocate_array.

Mutation check (applied to a copy on the lab, then deleted)

Mutant Tests failing
update sign flipped (theta + lr * ...) 8 of 16
decay sign flipped (- wd * theta) 4 of 16
bias correction dropped (m / 1.0) 6 of 16
AMSGrad ignores v_max (v_used = v) 1 of 16 (amsgrad_keeps_the_largest_second_moment)

All four mutants are killed.

Tests that can fail

  • Hand-computed 2-element step (lr 0.1, wd 0.1): params (1, -2) -> (0.89, -1.88); moments m, v.
  • Hand-computed second step with changing gradient: update 0.965187.
  • wd = 0 equals adam.t27 over five steps with changing gradients (and the parameter must have moved).
  • Decoupled decay: zero gradient gives 1.98 (AdamW) against 1.9 (Adam with L2); with gradient 0.89 against 0.9.
  • Decay-only geometric law: 2 * 0.99^10 = 1.808764.
  • Bias correction values and m_hat = g at step 1.
  • AMSGrad: v = 0.25 but v_max = 0.5; second step 8.591752 against 8.422650 without it; v_max untouched when off.
  • NaN and inf refused, params and m unchanged.
  • x^2 from 5 descends (with and without decay).
  • Ablation arm constants (phi^-1, phi^-3, 0.9/phi, 4/7, 7/30) and L5 identity.

Dropped from the old spec, and why

  • gf16 storage types, PHI_BETA1 / PHI_BETA2 and the _V1 / _v2 twin copies: duplicates; f32 matches the passing siblings.
  • PhiVariant enum and tuple-returning get_effective_betas (its body returned a single value, so the 4+4 variant tests could never pass): replaced by variant_beta1(which) / variant_weight_decay(which) with real tests.
  • PHI_DAMPED_BETA2 and PHI_CANONICAL_LR: unused by any arm.
  • OptimizerStepResult record (step_norm never computed).
  • All bench blocks: they timed undefined helpers (random_input, steps_per_second, any_adamw_config), so none could run.
  • b1_control_ablation_produces_distinct_bpb: called an undefined run_adamw; the BPB ablation is a training experiment, not a unit test.
  • b1_phi_anchors_labeled_open_conjecture: assert true placeholder. The [Open conjecture] label stays in comments.
  • Correction: the old test comment gave 0.9/phi as 0.55557; it is 0.556230 (checked by a test now).

Not in this PR

.trinity/seals/AdamW.json and optimizer_AdamW.json still hold the old spec_hash and the allocate_array blocked note; the reseal workflows refresh them (sibling spec PRs do not touch seals).

🤖 Generated with Claude Code

@gHashTag @claude
spec(ml): adamw.t27 heap-free rewrite, 16/16 tests pass, 0 vacuous (C…
eca4f33 
…loses #10091)

The old spec called allocate_array (not lowered by t27c), left every function
body as a comment and carried two copies of the optimizer. Rewritten in the
adam.t27 / lamb.t27 style: caller-owned buffers, real AdamW update with bias
correction and optional AMSGrad, tests that fail under a sign flip.

Refs #10024

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gHashTag

gHashTag commented Oct 17, 2026

Copy link
Copy Markdown
Owner Author

Closing as a duplicate: #10088 rewrites the same file (specs/ml/optimizer/adamw.t27) without allocate_array, carving m, v and v_max from specs/ml/arena_heap.t27, with resealed seals and the map count update. Two rewrites of one spec would conflict on whichever lands second. This PR's extra tests (wd=0 equals Adam, decoupled decay differs from L2, bias correction at step 1) can be folded into the #10088 version in a follow-up. Part of #10024.

@gHashTag gHashTag closed this Oct 17, 2026
@github-actions

github-actions Bot commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-17 19:02:17 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 47
PRs with All Checks Green 3
READY 0
FAILING 47
PENDING 0
NO CHECKS YET 0

These columns do not partition: 0 + 47 + 0 + 0 = 47, and there are 50 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=e769ef021713 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

spec(ml): adamw.t27 heap-free rewrite so it compiles and its tests can fail (Refs #10024)

1 participant

@gHashTag