From 8cd2850922a180abc07a6a2d81062b968b822594 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 17 Sep 2026 22:26:55 +0000 Subject: [PATCH] bench: seed the instruction-count baselines from the first main run Copies the merged baselines artifact of the first bench.yml push-to-main run after #21 (run 35281280300 at df482d1) into bench/baselines.json: five keys (m4-softfp, m4f, m33, m55, m55-ooura), two scenarios each (rfft_f32_512, rfft_f32_2048), measured on the vendored Ooura C with arm-none-eabi-gcc 13.2.1 (15:13.2.rel1-2) and QEMU 8.2.2 on ubuntu-24.04. bench/README.md records one row per (key, scenario) with the run URL, the main SHA and the toolchain versions, as the seeding rule requires. From this commit every push and pull request compares against these numbers at +/-3%; the bench.yml run on this PR is the first compare-mode run and must report 0% delta (identical tree). Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_019ZPTzNxo5Fe4EtpXXKf7Sy --- bench/README.md | 11 ++++++++++- bench/baselines.json | 25 ++++++++++++++++++++----- 2 files changed, 30 insertions(+), 6 deletions(-) diff --git a/bench/README.md b/bench/README.md index 158e081..57142be 100644 --- a/bench/README.md +++ b/bench/README.md @@ -118,7 +118,16 @@ shape is fixed so the record stays greppable: | key | scenario | before | after | delta | reason | main run (URL) | main SHA | arm-none-eabi-gcc | qemu-system-arm | |---|---|---|---|---|---|---|---|---|---| -| all | all | — | — | — | **unseeded**: the skeleton is empty by design; numbers are measured on CI, never typed in | — | — | — | — | +| m33 | rfft_f32_2048 | — | 116,385,409 | — | **seeded** from the first push-to-main run after #21 merged (vendored Ooura C, `TAP_DSP_BENCH_ENGINE=reference_c`; m55 = CMSIS-DSP Helium, m55-ooura = the Ooura fallback) | [run 35281280300](https://github.com/tap/DspTap/actions/runs/35281280300) | df482d1 | 13.2.1 (15:13.2.rel1-2) | 8.2.2 (1:8.2.2+ds-0ubuntu1.18) | +| m33 | rfft_f32_512 | — | 102,248,169 | — | **seeded** from the first push-to-main run after #21 merged (vendored Ooura C, `TAP_DSP_BENCH_ENGINE=reference_c`; m55 = CMSIS-DSP Helium, m55-ooura = the Ooura fallback) | [run 35281280300](https://github.com/tap/DspTap/actions/runs/35281280300) | df482d1 | 13.2.1 (15:13.2.rel1-2) | 8.2.2 (1:8.2.2+ds-0ubuntu1.18) | +| m4-softfp | rfft_f32_2048 | — | 2,296,984,479 | — | **seeded** from the first push-to-main run after #21 merged (vendored Ooura C, `TAP_DSP_BENCH_ENGINE=reference_c`; m55 = CMSIS-DSP Helium, m55-ooura = the Ooura fallback) | [run 35281280300](https://github.com/tap/DspTap/actions/runs/35281280300) | df482d1 | 13.2.1 (15:13.2.rel1-2) | 8.2.2 (1:8.2.2+ds-0ubuntu1.18) | +| m4-softfp | rfft_f32_512 | — | 1,868,441,244 | — | **seeded** from the first push-to-main run after #21 merged (vendored Ooura C, `TAP_DSP_BENCH_ENGINE=reference_c`; m55 = CMSIS-DSP Helium, m55-ooura = the Ooura fallback) | [run 35281280300](https://github.com/tap/DspTap/actions/runs/35281280300) | df482d1 | 13.2.1 (15:13.2.rel1-2) | 8.2.2 (1:8.2.2+ds-0ubuntu1.18) | +| m4f | rfft_f32_2048 | — | 111,859,257 | — | **seeded** from the first push-to-main run after #21 merged (vendored Ooura C, `TAP_DSP_BENCH_ENGINE=reference_c`; m55 = CMSIS-DSP Helium, m55-ooura = the Ooura fallback) | [run 35281280300](https://github.com/tap/DspTap/actions/runs/35281280300) | df482d1 | 13.2.1 (15:13.2.rel1-2) | 8.2.2 (1:8.2.2+ds-0ubuntu1.18) | +| m4f | rfft_f32_512 | — | 98,090,666 | — | **seeded** from the first push-to-main run after #21 merged (vendored Ooura C, `TAP_DSP_BENCH_ENGINE=reference_c`; m55 = CMSIS-DSP Helium, m55-ooura = the Ooura fallback) | [run 35281280300](https://github.com/tap/DspTap/actions/runs/35281280300) | df482d1 | 13.2.1 (15:13.2.rel1-2) | 8.2.2 (1:8.2.2+ds-0ubuntu1.18) | +| m55 | rfft_f32_2048 | — | 54,858,120 | — | **seeded** from the first push-to-main run after #21 merged (vendored Ooura C, `TAP_DSP_BENCH_ENGINE=reference_c`; m55 = CMSIS-DSP Helium, m55-ooura = the Ooura fallback) | [run 35281280300](https://github.com/tap/DspTap/actions/runs/35281280300) | df482d1 | 13.2.1 (15:13.2.rel1-2) | 8.2.2 (1:8.2.2+ds-0ubuntu1.18) | +| m55 | rfft_f32_512 | — | 52,382,331 | — | **seeded** from the first push-to-main run after #21 merged (vendored Ooura C, `TAP_DSP_BENCH_ENGINE=reference_c`; m55 = CMSIS-DSP Helium, m55-ooura = the Ooura fallback) | [run 35281280300](https://github.com/tap/DspTap/actions/runs/35281280300) | df482d1 | 13.2.1 (15:13.2.rel1-2) | 8.2.2 (1:8.2.2+ds-0ubuntu1.18) | +| m55-ooura | rfft_f32_2048 | — | 107,806,480 | — | **seeded** from the first push-to-main run after #21 merged (vendored Ooura C, `TAP_DSP_BENCH_ENGINE=reference_c`; m55 = CMSIS-DSP Helium, m55-ooura = the Ooura fallback) | [run 35281280300](https://github.com/tap/DspTap/actions/runs/35281280300) | df482d1 | 13.2.1 (15:13.2.rel1-2) | 8.2.2 (1:8.2.2+ds-0ubuntu1.18) | +| m55-ooura | rfft_f32_512 | — | 94,561,954 | — | **seeded** from the first push-to-main run after #21 merged (vendored Ooura C, `TAP_DSP_BENCH_ENGINE=reference_c`; m55 = CMSIS-DSP Helium, m55-ooura = the Ooura fallback) | [run 35281280300](https://github.com/tap/DspTap/actions/runs/35281280300) | df482d1 | 13.2.1 (15:13.2.rel1-2) | 8.2.2 (1:8.2.2+ds-0ubuntu1.18) | `before` is `—` for a seed. `main SHA` is the commit on `main` whose push run measured the numbers: a pull-request head SHA stops resolving after diff --git a/bench/baselines.json b/bench/baselines.json index df01ae1..2c0bf95 100644 --- a/bench/baselines.json +++ b/bench/baselines.json @@ -1,7 +1,22 @@ { - "m33": {}, - "m4-softfp": {}, - "m4f": {}, - "m55": {}, - "m55-ooura": {} + "m33": { + "rfft_f32_2048": 116385409, + "rfft_f32_512": 102248169 + }, + "m4-softfp": { + "rfft_f32_2048": 2296984479, + "rfft_f32_512": 1868441244 + }, + "m4f": { + "rfft_f32_2048": 111859257, + "rfft_f32_512": 98090666 + }, + "m55": { + "rfft_f32_2048": 54858120, + "rfft_f32_512": 52382331 + }, + "m55-ooura": { + "rfft_f32_2048": 107806480, + "rfft_f32_512": 94561954 + } }