Skip to content

HW CI: back to one capture per build; keep measure.py --runs - #899

Merged
bwhitman merged 1 commit into
mainfrom
claude/hwci-single-run
Jul 15, 2026
Merged

HW CI: back to one capture per build; keep measure.py --runs#899
bwhitman merged 1 commit into
mainfrom
claude/hwci-single-run

Conversation

@bwhitman

Copy link
Copy Markdown
Collaborator

Follow-up to #897. The 3× averaging trial (re-bench of #896) answered the noise question: run-to-run spread within one flashed binary is only ±1–2 µs (PR: 3704/3702/3703 · main: 3674/3672/3672), while the ~±30 µs full-chord delta on that no-op PR showed up identically in every run. The noise is per-binary (code/flash-cache layout jitter between builds), so averaging board resets can't reduce it — it just costs ~80 s per bench job.

  • amy-hwci.yml: drop --runs 3 from both measure invocations, restore the single-run comment text, and note the finding in the header comment. The per-run artifact globs stay (harmless, and correct if --runs is ever re-enabled).
  • measure.py / hwci_report.py: unchanged — full --runs support (per-run stats, cross-run means, spread line in the report) stays for ad-hoc bench investigation.
  • README: records the finding so nobody re-runs this experiment from scratch.

🤖 Generated with Claude Code

The 3x-averaging trial (#897, re-bench of #896) showed run-to-run spread
of only ±1-2 µs within one flashed binary, while the ~±30 µs deltas on
no-op PRs persisted identically across all three runs — they're
per-binary code-layout jitter, not measurement noise, so averaging
resets buys nothing. Drop --runs 3 from the workflow (saves ~80 s per
bench job); measure.py/hwci_report.py keep full --runs support for
ad-hoc use.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

🎛️ AMY HW CI (AMYboard bench)

Flashed this PR's AMY (LoadTestChord: 6-voice Juno patch=1, one held note every 2 s) onto the physical AMYboard and measured the smoothed render load as the chord grows — back-to-back with the same sketch built at the PR's merge base, so Δ is this PR's own cost. Each build is flashed once and captured 3 times (board reset between runs); the numbers are per-run means.

PASS — the bench ran the test to completion.

notes held main @ 555c0a6 this PR Δ
1 1603 1606 +3
2 1799 1802 +3
3 2481 2485 +4
4 2720 2720 +0
5 3464 3468 +4
6 3673 3676 +3

Full chord settled render μs: 3676 (was 3672, Δ +0.1%) (peak 3686, 117 samples, mean of 3 runs)

per-run full-chord μs — this PR: 3678 / 3675 / 3676 · main @ 555c0a6: 3673 / 3672 / 3672

⬇️ Artifacts: serial log · load trace · report

Self-hosted bench (amyboardci). FAIL means only that the test could not run — the load values are informational, with no threshold and no audio compare. See tools/arduino_loadsweep/.

@bwhitman
bwhitman merged commit 821f455 into main Jul 15, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant