srdif: tune for Hexagon (clang) and gate Hexagon in the ratchet; no output bit moves

.github/workflows/bench.yml

@@ -11,6 +11,8 @@ name: Bench
 #   m55        Cortex-M55, Helium            (mps3-an547)  CMSIS-DSP (deployed profile)
 #   m55-ooura  Cortex-M55, TAP_DSP_FFT_CMSIS=OFF (mps3-an547)  srdif float32 fallback
 #              (the key's name is historical; baselines are keyed on it)
+#   hexagon    Hexagon v68 + HVX-128, clang 19 (qemu-hexagon user mode)  srdif float32
+#              and double (a hosted target, so rfft_f64_512 is built and gated too)
 #
 # Every key measures what ships through tap::dsp::basic_real_fft: the srdif
 # engine (fft/srdif.h), or CMSIS-DSP behind the same class on the `m55` key,
@@ -53,7 +55,7 @@ on:
 permissions:
   contents: read
 
-# The expensive workflow: five QEMU jobs. One run per ref at a time.
+# The expensive workflow: six QEMU jobs. One run per ref at a time.
 concurrency:
   group: bench-${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
@@ -115,6 +117,20 @@ jobs:
             text_ceiling_f32: 29056
             text_ceiling_q15: 27968
             text_ceiling_q31: 27392
+          # Hexagon: MuTap's third target (its Hexagon CI leg and ratchet use
+          # this toolchain, these flags and this QEMU), added when the srdif
+          # engine was found to cost more instructions there than the engine
+          # it replaced, which no key here measured (bench/README.md).
+          # clang, not GCC; hosted (static musl) under qemu-hexagon, which
+          # the job builds with plugin support. The size probes are measured
+          # with the toolchain's llvm-size.
+          - key: hexagon
+            arch: hexagon
+            toolchain: cmake/hexagon-linux-musl.cmake
+            flags: ""
+            text_ceiling_f32: 245824
+            text_ceiling_q15: 251968
+            text_ceiling_q31: 251456
     env:
       # Commit the tag pointed at when pinned (tags are movable; commit SHAs
       # are not); the header's digest is verified on download. This SHA is
@@ -125,6 +141,14 @@ jobs:
       # that is never shipped (tools/qemu_insn_plugin/insn_count.c).
       QEMU_PLUGIN_HEADER_URL: https://raw.githubusercontent.com/qemu/qemu/11aa0b1ff115b86160c4d37e7c37e6a6b13b77ea/include/qemu/qemu-plugin.h
       QEMU_PLUGIN_HEADER_SHA256: "c53a2af163e80e3f4bc6c60dbdfc84003db329d757e37cd8a16a77e1d82606ff"
+      # The hexagon key only: the QEMU release its qemu-hexagon is built
+      # from (digest-verified, the plugin header's release) and the
+      # CodeLinaro toolchain (MuTap's pins).
+      QEMU_SRC_URL: https://download.qemu.org/qemu-8.2.2.tar.xz
+      QEMU_SRC_SHA256: 847346c1b82c1a54b2c38f6edbd85549edeb17430b7d4d3da12620e2962bc4f3
+      HEXAGON_TOOLCHAIN: clang+llvm-19.1.5-cross-hexagon-unknown-linux-musl
+      HEXAGON_TOOLCHAIN_SHA256: 55b41922318f6331590ab7baa7f5dbdd99c109327a9c44a52c5e9878fab148c1
+      HEXAGON_TOOLCHAIN_VERSION: clang version 19.1.5
       PLUGIN: /tmp/libinsncount.so
       BUILD_DIR: build-${{ matrix.key }}
       SIZE_DIR: build-${{ matrix.key }}-minsizerel
@@ -133,18 +157,87 @@ jobs:
       - uses: actions/checkout@v4
 
       - name: Install Arm toolchain and QEMU
+        if: matrix.arch != 'hexagon'
         run: >
           sudo apt-get update -q &&
           sudo apt-get install -y -q gcc-arm-none-eabi qemu-system-arm
           libglib2.0-dev pkg-config
 
-      # These versions go into bench/README.md next to the numbers when a key
-      # is seeded or re-recorded; the counts are only comparable within a
-      # toolchain/QEMU pair.
+      # Neither Debian's nor CodeLinaro's qemu-hexagon enables TCG plugins,
+      # so the hexagon key builds its own from the pinned release (the same
+      # pin and digest as MuTap's hexagon job, cached on the digest).
+      - name: Install Hexagon build dependencies
+        if: matrix.arch == 'hexagon'
+        run: >
+          sudo apt-get update -q &&
+          sudo apt-get install -y -q libglib2.0-dev pkg-config ninja-build flex bison zstd
+
+      - name: Cache plugin-enabled qemu-hexagon
+        if: matrix.arch == 'hexagon'
+        id: qemu-hex
+        uses: actions/cache@v4
+        with:
+          path: ~/qemu-hexagon-plugins
+          key: qemu-hexagon-plugins-${{ env.QEMU_SRC_SHA256 }}-1
+
+      - name: Build plugin-enabled qemu-hexagon
+        if: matrix.arch == 'hexagon' && steps.qemu-hex.outputs.cache-hit != 'true'
+        run: |
+          curl -sfLo /tmp/qemu-src.tar.xz "$QEMU_SRC_URL"
+          actual=$(sha256sum /tmp/qemu-src.tar.xz | cut -d' ' -f1)
+          if [ "$actual" != "$QEMU_SRC_SHA256" ]; then
+            echo "::error::qemu source checksum mismatch"; exit 1
+          fi
+          tar -xJf /tmp/qemu-src.tar.xz -C /tmp
+          cd /tmp/qemu-*/
+          ./configure --target-list=hexagon-linux-user --enable-plugins \
+            --disable-docs --disable-tools --disable-system
+          ninja -C build qemu-hexagon
+          mkdir -p ~/qemu-hexagon-plugins
+          cp build/qemu-hexagon ~/qemu-hexagon-plugins/
+
+      # The compiler determines every hexagon count, so it is pinned like the
+      # QEMU source: digest-verified on download, and its version asserted.
+      - name: Download the Hexagon toolchain
+        if: matrix.arch == 'hexagon'
+        run: |
+          set -eo pipefail
+          curl -fsSLo /tmp/hexagon-toolchain.tar.zst \
+            "https://artifacts.codelinaro.org/artifactory/codelinaro-toolchain-for-hexagon/19.1.5/${HEXAGON_TOOLCHAIN}.tar.zst"
+          echo "$HEXAGON_TOOLCHAIN_SHA256  /tmp/hexagon-toolchain.tar.zst" | sha256sum -c -
+          mkdir -p "$RUNNER_TEMP/hexagon-toolchain"
+          tar --zstd -x -C "$RUNNER_TEMP/hexagon-toolchain" -f /tmp/hexagon-toolchain.tar.zst
+          rm /tmp/hexagon-toolchain.tar.zst
+          cxx=$(find "$RUNNER_TEMP/hexagon-toolchain" -maxdepth 5 \( -type f -o -type l \) \
+                  -name 'hexagon-unknown-linux-musl-clang++' | sort | head -1)
+          test -n "$cxx" || { echo "::error::no hexagon clang++ in the toolchain archive"; exit 1; }
+          "$cxx" --version | head -1 | grep -qF "$HEXAGON_TOOLCHAIN_VERSION" \
+            || { echo "::error::the toolchain is not $HEXAGON_TOOLCHAIN_VERSION"; exit 1; }
+          echo "HEXAGON_TOOLCHAIN_ROOT=$(dirname "$(dirname "$cxx")")" >> "$GITHUB_ENV"
+          echo "$HOME/qemu-hexagon-plugins" >> "$GITHUB_PATH"
+
+      # linux-user qemu prints "unknown option 'plugin'" when built without
+      # plugin support; a missing binary is caught first.
+      - name: Check qemu-hexagon has plugin support
+        if: matrix.arch == 'hexagon'
+        run: |
+          set -o pipefail
+          q="$HOME/qemu-hexagon-plugins/qemu-hexagon"
+          test -x "$q" || { echo "::error::no qemu-hexagon at $q"; exit 1; }
+          if "$q" -plugin help 2>&1 | grep -q "unknown option"; then
+            echo "::error::built qemu-hexagon lacks plugin support"; exit 1
+          fi
+
       - name: Toolchain versions
         run: |
-          arm-none-eabi-gcc --version | head -1
-          qemu-system-arm --version | head -1
+          set -o pipefail
+          if [ "${{ matrix.arch }}" = hexagon ]; then
+            "$HEXAGON_TOOLCHAIN_ROOT/bin/hexagon-unknown-linux-musl-clang++" --version | head -1
+            qemu-hexagon --version | head -1
+          else
+            arm-none-eabi-gcc --version | head -1
+            qemu-system-arm --version | head -1
+          fi
           cmake --version | head -1
 
       - name: Build counting plugin
@@ -187,6 +280,8 @@ jobs:
           CEILING_Q31: ${{ matrix.text_ceiling_q31 }}
         run: |
           set -eo pipefail
+          SIZE_TOOL=arm-none-eabi-size
+          if [ "${{ matrix.arch }}" = hexagon ]; then SIZE_TOOL="$HEXAGON_TOOLCHAIN_ROOT/bin/llvm-size"; fi
           cmake -S . -B "$SIZE_DIR" \
             -DCMAKE_BUILD_TYPE=MinSizeRel \
             -DCMAKE_TOOLCHAIN_FILE=${{ matrix.toolchain }} \
@@ -198,7 +293,7 @@ jobs:
             profile=${entry%%:*}
             ceiling=${entry#*:}
             probe="$SIZE_DIR/bench/tap_dsp_size_probe_rfft_${profile}_512"
-            arm-none-eabi-size -A "$probe" | tee "size-$profile.txt"
+            $SIZE_TOOL -A "$probe" | tee "size-$profile.txt"
             text=$(awk '$1 == ".text" { print $2 }' "size-$profile.txt")
             case "$text" in
               ''|*[!0-9]*) echo "::error::$KEY: $profile probe .text unavailable (no numeric .text row in size-$profile.txt: '$text')"; exit 1 ;;
@@ -289,7 +384,7 @@ jobs:
   # Folds the per-key seeding artifacts (each has one key filled in) into the
   # single file the seeding commit copies to bench/baselines.json. Fails when
   # any icount job failed, so a partial seed from a half-red run can never be
-  # merged. NEVER a required check: the five `icount <key>` jobs are the
+  # merged. NEVER a required check: the six `icount <key>` jobs are the
   # required checks once seeded (bench/README.md).
   seed-summary:
     name: Merge seeding artifacts (never a required check)

CLAUDE.md

@@ -40,8 +40,9 @@ asset's contract summary.
   each floor a measured number against the double profile. The floating profiles run the srdif
   engine (`fft/srdif.h`): the N real samples as N/2 complex values, a split-radix
   decimation-in-frequency kernel of length N/2 (fused two-level passes, compile-time blocks of 32
-  and 64, register leaves of 16 and fewer), the bit-reversal permutation and the real post-pass
-  derived for the fixed-point engine (#39), written clean-room from the literature the header cites
+  and 64, one level at a time in memory below that, forced inlining under every clang, tuned on
+  Hexagon), the bit-reversal permutation and the real post-pass derived for the fixed-point engine
+  (#39), written clean-room from the literature the header cites
   (tap/DspTap#42; it replaced, at the same contract, the C++20 port of Ooura's `rdft`,
   `fft/split_radix.h`, that the floating profiles ran from Stage 2b, so DspTap ships no code derived
   from Ooura's package — the maintainer's judgement, `NOTICE.md`). Since Stage 4 the engine is a
@@ -102,7 +103,9 @@ parity suite runs for real. Each leg builds `tap_dsp_tests` one-shot (`tests/bar
 with the `MAIN_FILTER` in `tests/CMakeLists.txt` naming what the emulation budget excludes, plus
 the srdif fingerprints, with every FFT sweep capped at `TAP_DSP_TEST_MAX_FFT_N=4096`. The
 instruction-count ratchet (`bench.yml`, `bench/README.md`) runs the same four cores under five
-baseline keys at ±3 %, with `.text` ceilings per profile; a red ratchet is a failing check. The
+baseline keys at ±3 %, plus a `hexagon` key (clang 19, v68 + HVX, under `qemu-hexagon`; it
+counts packets, not instructions), with `.text` ceilings per profile; a red ratchet is a failing
+check. The
 hosted build and battery need no Arm toolchain; where `gcc-arm-none-eabi` and `qemu-system-arm`
 are installed, `cmake -S . -B build-m33 -DCMAKE_TOOLCHAIN_FILE=cmake/arm-cortex-m33-mps2.cmake`
 reproduces a leg.

README.md

@@ -25,7 +25,7 @@ its size range and shareability as contract numbers:
 | Engine | Profiles | Selected by | Size range | Shareable | What it is |
 |---|---|---|---|---|---|
 | srdif (`fft/srdif.h`) | `double`; `float` unless a backend is on | default | 4 … 2^30 | yes | a split-radix decimation-in-frequency kernel of length N/2 (Duhamel–Hollmann 1984; Sorensen–Heideman–Burrus 1986), the bit-reversal permutation and the real post-pass of Cooley–Lewis–Welch 1970 / Sorensen et al. 1987 (the fixed-point engine's structure in floating point), written from the literature; exactly the split-radix operation count, 2N log₂N − 2N − 2 real operations per forward transform; tables from integer arithmetic (no libm), built in the constructor, so without fp-contraction the output bits are one row for every compiler and target CI runs (pinned as output fingerprints in `tests/test_fft_srdif_fingerprint.cpp`). Since tap/DspTap#42, written clean-room; it replaced, at the same contract, the C++20 port of Ooura's `rdft` (`fft/split_radix.h`) that the floating profiles ran from Stage 2b (#31) until #42 (see Provenance) |
-| CMSIS-DSP Helium (`fft/backends/cmsis.h`) | `float` | `TAP_DSP_FFT_CMSIS` (default ON exactly when the compiler targets floating-point Helium, `__ARM_FEATURE_MVE & 2`: the Cortex-M55 / M85 class; OFF for every other target, other Cortex-M cores included) | **32 … 4096** (CMSIS-DSP's own init table; the constructor checks the init status since Stage 4, in a debug build — outside this range the library never initializes, and a release build is undefined behaviour with no fault promised: measured under QEMU, N = 16 and 8192 give a wrong spectrum silently, N = 4 a wrong spectrum and a corrupted heap) | no | Arm's radix-4/8 MVE real FFT, re-presented in the same contract to float epsilon; on the M55 the srdif engine (the `m55-ooura` icount key) executes 1.60× (N = 512) / 1.78× (N = 2048) the instructions of the CMSIS build (the `m55` key) over the whole ratchet scenario, transform plus the class's copy loop, 2/N scaling and checksum (`bench/README.md`) |
+| CMSIS-DSP Helium (`fft/backends/cmsis.h`) | `float` | `TAP_DSP_FFT_CMSIS` (default ON exactly when the compiler targets floating-point Helium, `__ARM_FEATURE_MVE & 2`: the Cortex-M55 / M85 class; OFF for every other target, other Cortex-M cores included) | **32 … 4096** (CMSIS-DSP's own init table; the constructor checks the init status since Stage 4, in a debug build — outside this range the library never initializes, and a release build is undefined behaviour with no fault promised: measured under QEMU, N = 16 and 8192 give a wrong spectrum silently, N = 4 a wrong spectrum and a corrupted heap) | no | Arm's radix-4/8 MVE real FFT, re-presented in the same contract to float epsilon; on the M55 the srdif engine (the `m55-ooura` icount key) executes 1.60× (N = 512) / 1.77× (N = 2048) the instructions of the CMSIS build (the `m55` key) over the whole ratchet scenario, transform plus the class's copy loop, 2/N scaling and checksum (`bench/README.md`) |
 | Apple vDSP (`fft/backends/accelerate.h`) | `float` | `TAP_DSP_FFT_ACCELERATE` (default ON on Apple) | 4 … 2^20 | no | `vDSP_fft_zrip`, same contract to float epsilon; ~3× faster per transform on Apple Silicon is MuTap's transform-only measurement against the engine the library carried before (tap/MuTap#31), not re-measured in this repo against srdif (the same-binary parity *test* exists since Stage 4; the microbenchmark needs a Mac) |
 | int32 radix-4 (`fft/fixed_point.h`) | Q15, Q31 | the sample type (the second template argument is the scaling policy here) | 4 … 65536 | Q31 yes, Q15 no | one kernel over Q1.30 twiddles under two scaling policies, returning an exponent |