Harden the ratchet, tests and CI ahead of the monorepo migration (step P) - #48
Merged
Merged
Conversation
- Port both Pico 2 examples to the current snake_case API and add the dsptap include path they need since the DspTap substrate move; both firmwares build again against Pico SDK 2.1.1. - Make the book's trace tool compile against both the current tree and the pre-fix commit 045de5d (whose API predates the renames), and pass the dsptap include path from book_figures.py. - Point .git-blame-ignore-revs at the real reformat commit (b84020e) and add the namespace-rename reflow (e2f5a48); the old entry named a commit that exists in no history. - Quote RatioTap's current cross-validation floors (-98/-90 dB). Step P.1 of docs/MONOREPO_PLAN.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VR1VC4SDGxHZQQsQvPBaA
- icount.py: --exact (integer equality), --json-out/--compare-json for same-job A/B measurement, a failure when a recorded workload has no binary, a fatal error on a missing baselines file, and the workload's checksum printed and recorded. Hexagon workloads now run from one fixed path with a fixed argv[0] and an empty environment: qemu-hexagon copies those onto the guest stack and static musl's startup walks them, so the binary path and the job's environment were leaking into the count. qemu-hexagon is resolved to an absolute path first, since the cleared environment has no PATH. Hexagon baselines are re-recorded separately. - Tests carry an "async." ctest prefix and an async label (the bare-metal entry too), so names stay unique once they share a tree with RatioTap's. - New OutputHash suite: FNV-1a hashes of the datapath's output for every format, preset and two rate offsets, printed for same-job comparison, never pinned (float differs across FMA/libm; Q15/Q31 measured identical on x86-64 and Cortex-M33). About 60 s on M33, so the M33 leg's timeout rises to 40 min. - CI: push runs only on main plus pull_request and dispatch (a PR push no longer runs twice); cancellation spares main and the migration PR; contents: read permissions; ubuntu-24.04 pinned for QEMU and ratchet jobs, with the image OS in the plugin-qemu cache key and image/toolchain versions logged; --no-tests=error on every ctest; QEMU legs keep and upload full test logs. - New Pico 2 firmware build job, so the examples cannot silently rot again. - ci-arm64: the TSan ring stress filtered on SpscRing, which has matched nothing since the snake_case rename; it now selects spsc_ring and fails on an empty selection. style.yml gets the same triggers, permissions and a SHA-pinned checkout. Step P.2 of the monorepo migration plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VR1VC4SDGxHZQQsQvPBaA
The previous commit runs Hexagon workloads from a fixed path with a fixed argv[0] and an empty environment. The first isolated CI run (PR #48) measured every scenario 3,656 to 3,724 instructions lower (-0.0008% to -0.0020%): the path and environment strings static musl's startup used to walk. Inside tolerance, but re-recorded so exact comparisons start from the new harness; ledger entry in docs/PERFORMANCE.md, README table regenerated. M33/M55 are unaffected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VR1VC4SDGxHZQQsQvPBaA
- requirements.in / requirements.lock: the notebook environment pinned with hashes (numpy 2.4.6, scipy 1.17.1, matplotlib 3.11.2, samplerate 0.2.4 and soxr 1.1.0 as the committed comparison outputs record, plus jupyter/nbconvert); identical in RatioTap. README points at it. - The ctypes bridges now rebuild the C ABI library incrementally on every run (quietly; the log prints only on failure) instead of only when it is missing, so a library left over from an older checkout can no longer be measured silently. - notebooks/figure_digest.py wraps plt.show() to print a digest of each figure's plotted data, quantized to 9 significant digits of each array's peak, so a changed curve shows up as changed text; stable across re-executions. The rbj notebook's wall-clock timing cell is tagged "nondeterministic". - All four notebooks re-executed in the pinned environment. The comparison and rbj numbers reproduce exactly. Two notebooks' committed outputs predated the compensated prototype design (2026-07-04) and had drifted: the demo's SNR is 125.9 dB (was 126.4) and the block-size study's wideband floors are 61.5/58.1/54.0 dB (were 60.9/58.2/53.1), its final-ppm snapshots moved with them. The prose that quotes them (notebook summaries, README, book, COMPARISON.md) now states the re-executed values. Step P.3 of the monorepo migration plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VR1VC4SDGxHZQQsQvPBaA
tap
marked this pull request as ready for review
September 27, 2026 20:40
tap
pushed a commit
that referenced
this pull request
Sep 27, 2026
The previous commit runs Hexagon workloads from a fixed path with a fixed argv[0] and an empty environment. The first isolated CI run (PR #48) measured every scenario 3,656 to 3,724 instructions lower (-0.0008% to -0.0020%): the path and environment strings static musl's startup used to walk. Inside tolerance, but re-recorded so exact comparisons start from the new harness; ledger entry in docs/PERFORMANCE.md, README table regenerated. M33/M55 are unaffected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VR1VC4SDGxHZQQsQvPBaA
tap
pushed a commit
that referenced
this pull request
Sep 28, 2026
Moves submodules/sampleratetap 2b4dff1 -> 5e2057f, SampleRateTap's main after its step-P PR (#48). That range changes no header under include/ and keeps the same DspTap pin (0eb09fa), so the tree the cross-validation compiles against is unchanged: 82/82 tests pass and the four cross-validation lines are byte-identical to before. The monorepo migration's step 0 snapshots against this pin, so the cross-validation lines it records come from the same async tree the import compiles (step P.4 of the plan). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VR1VC4SDGxHZQQsQvPBaA
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
This PR is SampleRateTap's half of step P, the pre-work for the monorepo migration. The plan is
docs/MONOREPO_PLAN.mdonclaude/sample-rate-expansion-strategies-ezqzu6, and the companion PR is tap/RatioTap#18. The PR repairs breakage that nothing was catching. It also makes the ratchet, the tests, CI and the notebooks produce the evidence the migration's gates will compare.P.1: repairs (98cbe9b)
scripts/book_figures_trace.cppcompiles against both the current tree and the pre-fix commit045de5d.book_figures.pypasses the dsptap include path..git-blame-ignore-revsis fixed. Its only entry named a commit that exists in no history. It now listsb84020eande2f5a48.P.2: harness and CI (865e32d, 64509e1)
scripts/icount.py:--exactrequires identical counts.--json-outand--compare-jsonsupport same-job A/B runs.argv[0]and an empty environment. qemu-hexagon copies these onto the guest stack, and static musl's startup walks them, so the old counts included them.docs/PERFORMANCE.md, and the README table was regenerated.async.ctest prefix and anasynclabel, including the bare-metal entry.OutputHashsuite. It prints FNV-1a hashes of the datapath output. They are compared within one job, never pinned.main; branches run throughpull_requestand dispatch. This removes the double runs on PR branches.mainand the migration PR, but still applies to other PRs, including this one.permissions: contents: read.ubuntu-24.04, with the image OS added to the plugin-qemu cache key and image and toolchain versions logged.--no-tests=erroron every ctest call.ci-arm64: the TSan ring stress filtered onSpscRing, which matched nothing, and now filters onspsc_ring.P.3: notebooks (3e39365)
requirements.in/requirements.lock. The notebook environment is pinned with hashes, and the same file is in RatioTap. The README points to it.notebooks/figure_digest.pyprints a data digest for each figure. A changed curve then shows up as changed text. The rbj notebook's wall-clock timing cell is taggednondeterministic.Why
The migration gates key tests by unique names and compare exact icount, output hashes, full on-target logs and notebook outputs. All of that has to exist before the snapshot. Several checks were also passing while testing nothing: the arm64 TSan filter, the Pico builds, the trace tool and the blame-ignore file. The snapshot must not record those as green.
Verification
-Werror: 77/77 tests pass, all namedasync.*and labelledasync.-Werror: clean.icount.pyon M33 under real QEMU with the plugin:pip install --require-hashes -r requirements.lockon Python 3.11.15. The digest lines are identical across two re-executions.ci-arm64, because it is dispatch- and schedule-only.Notes for the reviewer
async.prefix.-Rand-Esubstring filters are unaffected.🤖 Generated with Claude Code
https://claude.ai/code/session_015VR1VC4SDGxHZQQsQvPBaA