Skip to content

feat(igla): hslm_bwd_ptx kernel 3 -- split-K dW, 4.2-4.9x faster than kernel 2 at K=1782 (Closes #10895) - #10914

Draft
gHashTag wants to merge 5 commits into
masterfrom
claude/hslm-dw-splitk
Draft

gHashTag wants to merge 5 commits into
masterfrom
claude/hslm-dw-splitk

Conversation

@gHashTag

@gHashTag gHashTag commented Oct 19, 2026 •

Copy link
Copy Markdown
Owner

Closes #10895

Follows #10855 (merged, fc611e7): kernel 3 builds on kernels 1 and 2. Rebased onto master after the squash; the diff is only the split-K change (spec, its regenerated Rust and seal, and driver.rs, net 0 lines).

CI note: t27b-native-ratchet, Corpus ratchet (expected-failure ledger), duplicate-bodies, the orphan ceiling and the types Range check were already red when #10855 merged (non-required). This PR does not bless the ledgers: that needs a full t27b lab corpus run and would also bless three unrelated specs (sar_adc, ngram_hslm, reap) that sit UNLISTED today, so it belongs in a ledger-steward change.

What

specs/igla/coder/hslm_bwd_ptx.t27 gains kernel 3, hslm_dw_splitk_s8: kernel 2 (dW = X^T x dY) split along k.

  • grid z = slices; slice z sums k in [zper32, (z+1)per32), per = ceil(ceil(k/32)/nz) read from %nctaid.z in the kernel.
  • Each slice ends with red.global.add.s32 of its i32 partials into C. The host zeroes C before each launch (zero_first()), so the result equals kernel 2's bit for bit.
  • grid_z() aims at SPLIT_TARGET_BLOCKS = 64 blocks, never more slices than k-steps, and drops slices the rounding would leave empty (no idle block).
  • A warp covers one 16-byte chunk and 32 k rows (kernel 2: 8 lanes per row), so the transposed byte stores spread over 8 banks. Measured effect alone: small (0.034 -> 0.0325 ms at the old target).
  • Kernels 0, 1, 2, their integer CPU references and their text pins are unchanged.

Measured (RTX 4090, RunPod, t27c ptx-run --iters 500 --stride 7, mismatches=0 everywhere; a stride-1 run at 256x736x1782 also reports mismatches=0)

m x n x k kernel 2 ms kernel 3 ms CUTLASS s8 ms kernel 3 / CUTLASS
256 x 256 x 1782 0.0747 0.0153 0.0066 2.3x
256 x 736 x 1782 0.0752 0.0180 0.0068 2.6x
736 x 256 x 1782 0.0746 0.0180 0.0068 2.6x
256 x 256 x 27 0.0037 0.0072 - -

Kernel 3 is 4.2-4.9x faster than kernel 2 at K=1782 (was 11x behind CUTLASS, now 2.3-2.6x).

The goal was not reached

The issue asked for within about 1.5x of CUTLASS s8. It is at 2.3-2.6x. Honest notes:

  • The ptx-run time is wall time per call and for kernel 3 includes the zero-fill launch. At K=27 (one step, nothing to compute) kernel 3 costs 0.0072 ms against 0.0037 ms for kernel 2: that is two launches against one, about 3.7 us each from the CPU. The CUTLASS figures are GPU-event times without that floor, so the table is not apples to apples; about 7 us of the 15-18 us is launch cost.
  • What remains on the GPU is the red traffic (every slice adds the full 128x128 tile into C) and the 128x128 tile walk with u8 transposed shared stores. The sweep of SPLIT_TARGET_BLOCKS (8..256) is U-shaped with its floor at 48..64: fewer slices starve the SMs, more slices pay in reds.
  • Next steps if the owner wants the last 1.5x: fuse the zero fill (e.g. a graph or a cooperative first-slice store), or a second-pass reduction instead of reds. Both are separate changes.

Foreign-line accounting (shrink-only budget)

bootstrap/src/host/driver.rs: net 0 lines. The cuMemsetD32_v2 declaration joins an existing extern line; the launch closure zeroes C when zero_first(kernel) and passes grid_z(...) as the launch's z dimension, in place. bootstrap/gen/rust/... and the seal are regenerated on the Railway lab (test-report 26/26, 0 vacuous; gen-rust; seal --save).

Test plan

  • t27c test-report specs/igla/coder/hslm_bwd_ptx.t27 on the lab: 26/26, 0 vacuous
  • gen-rust and seal regenerated on the lab, hashes verified after transfer
  • t27c ptx-run --kernel 3 on RTX 4090: mismatches=0 for 256x256, 256x736, 736x256 at K=1782 and 256x256 at K=27

🤖 Generated with Claude Code

@gHashTag gHashTag added the no-auto-merge Opt out of the issue gate's default squash auto-merge (issue-gate.yml) label Oct 19, 2026
@gHashTag
gHashTag force-pushed the claude/hslm-dw-splitk branch from 330075d to 96eb8c6 Compare October 19, 2026 07:04
@gHashTag
gHashTag changed the base branch from claude/hslm-ptx-backward-0d1f39 to master October 19, 2026 07:04
@gHashTag
gHashTag force-pushed the claude/hslm-dw-splitk branch from 96eb8c6 to caed7b0 Compare October 19, 2026 07:11
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-19 07:15:23 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 46
PRs with All Checks Green 4
READY 2
FAILING 46
PENDING 0
NO CHECKS YET 0

These columns do not partition: 2 + 46 + 0 + 0 = 48, and there are 50 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=88b8cb73ceda != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

This was referenced Oct 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-auto-merge Opt out of the issue gate's default squash auto-merge (issue-gate.yml)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HSLM dW kernel: split-K variant (kernel 3) to close the 11x gap to CUTLASS s8 at K=1782

1 participant