Skip to content

HSLM backward elementwise PTX: RMSNorm, ReLU, softmax-CE backward and the f32-to-int8 quantise step, with CPU references #10932

Open

Description

@gHashTag

The HSLM backward GEMMs (dX, dW) already have int8 tensor-core PTX written by t27 (specs/igla/coder/hslm_bwd_ptx.t27, merged in #10855, issue #10847). The rest of the backward pass of specs/ml/train_hslm.t27 is elementwise and still CPU-only. This issue adds the PTX emitters and CPU references for those elementwise steps in a new spec, on the same skeleton and text-pin style as hslm_bwd_ptx.t27. The GPU run is a separate step on a 4090 pod; the bees need only t27c.

Refs #10847. Out of bounds (do not edit): specs/ml/train_hslm.t27, specs/igla/coder/hslm_bwd_ptx.t27, any Rust source under bootstrap/src.

Boundary

  • specs/igla/coder/hslm_bwd_elem_ptx.t27
  • bootstrap/gen/rust/igla/coder/hslm_bwd_elem_ptx.rs
  • .trinity/seals/coder_igla-coder-hslm-bwd-elem-ptx.json

User Scenarios

  • Given the new spec, when t27c test-report runs it, then every test passes: each kernel's PTX text is pinned (length and key lines), and each CPU reference matches hand-computed cases in the layout train_hslm.t27 uses.
  • Given a quantised activation row, when the f32-to-int8 per-row quantise reference runs, then the int8 values and the scale reproduce the f64 input within the stated bound.

Requirements

  • FR-001: The spec MUST give a PTX emitter plus an f64 CPU reference for each of: RMSNorm backward, ReLU backward, softmax-cross-entropy backward, and the f32 to int8 per-row and per-column quantise step that feeds hslm_dx_s8 and hslm_dw_s8.
  • FR-002: Each reference MUST follow the index layout and the formulas of the matching function in specs/ml/train_hslm.t27 (read it; do not edit it).
  • FR-003: The spec MUST state the error bound of the quantise step and test it against an f64 reference.
  • FR-004: The spec MUST hold at least one test or invariant per kernel (law L4), be ASCII-only (L3), and have no vacuous test.
  • FR-005: Shared helpers MUST be reused from specs/igla/coder/hslm_bwd_ptx.t27 by use, not copied (the duplicate-body ratchet fails on a copy).

Success Criteria

  • t27c test-report specs/igla/coder/hslm_bwd_elem_ptx.t27 exits 0.
  • t27c seal specs/igla/coder/hslm_bwd_elem_ptx.t27 exits 0 and the seal file lists all tests as passed.
  • The duplicate-body ratchet (tools/dupe_scan) reports no new duplicate body.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions