Skip to content

[bench] final CUDA benchmark numbers from NVIDIA A10 - #8

Merged
peterajhgraham merged 1 commit into
mainfrom
claude/add-cuda-benchmarks-OUmAL
May 18, 2026
Merged

peterajhgraham merged 1 commit into
mainfrom
claude/add-cuda-benchmarks-OUmAL

Conversation

@peterajhgraham

Copy link
Copy Markdown
Owner

Summary

  • benchmarks/serving/results.md: adds a new Phase 5 CUDA section recording ~5 ms/batch inference time on NVIDIA A10 24GB (Lambda Cloud), notes the asyncio scheduling bug tracked separately, and confirms the p99 < 30 ms SLO is achievable at this inference time.
  • README.md: updates the decoding accuracy table to Cortex-S R² = 0.60 and Wiener filter R² = 0.48 (real trial-aligned A10 numbers); adds a "Speedup (A10)" column to the Triton kernels table showing up to 27× on sparse cross-attention; updates the serving latency note to ~5 ms/batch measured on A10.
  • benchmarks/training/results.md: records the final trial-aligned R² of 0.60 (Cortex-S) vs 0.48 (Wiener filter) from the NVIDIA A10 run, replacing the "pending CUDA" placeholder.

Test plan

  • Verify README hero metrics table renders correctly (R² values, new speedup column)
  • Verify benchmarks/serving/results.md Phase 5 section is accurate and clearly notes the asyncio scheduling caveat
  • Verify benchmarks/training/results.md trial-aligned results table reflects A10 numbers

https://claude.ai/code/session_01GVUkAT2SLAqooWAavzTSp7


Generated by Claude Code

- benchmarks/serving/results.md: add Phase 5 CUDA section with ~5ms/batch
  inference time on A10 24GB (Lambda Cloud), note asyncio scheduling bug
  tracked separately, confirm p99 <30ms SLO is achievable
- README.md: update Cortex-S R² to 0.60 and Wiener R² to 0.48 (real
  trial-aligned A10 numbers); add 27x speedup column for sparse
  cross-attention Triton kernel; update serving latency note to ~5ms/batch
  measured on A10
- benchmarks/training/results.md: record final trial-aligned R² of 0.60
  (Cortex-S) vs 0.48 (Wiener filter) from NVIDIA A10 run

https://claude.ai/code/session_01GVUkAT2SLAqooWAavzTSp7
@peterajhgraham
peterajhgraham marked this pull request as ready for review May 18, 2026 23:32
@peterajhgraham
peterajhgraham merged commit d5f9101 into main May 18, 2026
4 checks passed
@peterajhgraham
peterajhgraham deleted the claude/add-cuda-benchmarks-OUmAL branch May 18, 2026 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants