Skip to content

Phase 5: NAIVE schedule, K/V overlap, performance baselines - #2

Merged
deaneeth merged 2 commits into
mainfrom
feat/phase-5-overlap-naive
Oct 25, 2026
Merged

deaneeth merged 2 commits into
mainfrom
feat/phase-5-overlap-naive

Conversation

@deaneeth

Copy link
Copy Markdown
Member

Summary

Phase 5 of the TinyAttention engine (Phase 0-4 are on main via PR #1).

  • NAIVE schedule (naive_ctrl): the standard 3-pass safe softmax, run block by block. Z and P tiles round-trip through HBM (HBM grows to 2048 words). The four passes reuse the tile core through new operations: OP_NZ, OP_NM, OP_NP, OP_NPV.
  • K/V overlap (cfg_overlap, FLASH only): double-buffered K/V with a prefetch sequencer that loads tile kj+1 while tile kj computes.
  • Performance baseline: sim/perf_baseline.json pins busy cycles for FLASH (with and without overlap) and NAIVE over N, causal, cfg_qrows and latency. The sequential FLASH values match the closed-form model exactly; overlap is checked never to be slower.

Verification

  • bash sim/lint.sh: -Wall clean.
  • bash sim/run_unit.sh: 15 suites pass.
  • SEEDS=200 bash sim/run_e2e.sh: 3 tests pass. Bit-exact O for FLASH and NAIVE, with and without overlap, against the golden model, for every N, causal setting, block size and latency. HBM bytes, tile counts and K/V loads match traffic.py. The perf grid matches the baseline.
  • Mutation check: a wrong alpha in the NAIVE O update fails the end-to-end test.

Notes

  • Overlap is FLASH only. NAIVE runs sequentially.
  • NAIVE's 2.7x slowdown versus FLASH at N = 32 is recorded but not yet broken down by pass.
  • See docs/ARCHITECTURE.md section 10 for the design and measured numbers.

…lines

naive_ctrl runs the four NAIVE passes block by block (Z and P tiles round-trip through HBM,
HBM grows to 2048 words). flash_ctrl gains double-buffered K/V prefetch (cfg_overlap). The tile
core gains OP_NZ/OP_NM/OP_NP/OP_NPV. E2E: bit-exact O for FLASH and NAIVE, with and without
overlap, traffic and tile counts exact, FLASH sequential cycles exact against the model,
overlap and NAIVE cycles pinned in sim/perf_baseline.json.
@deaneeth
deaneeth merged commit f594f4a into main Oct 25, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant