Skip to content

perf(sha256): compress blocks on ARMv8 SHA2 / x86 SHA-NI (#2441) - #2575

Merged
DeusData merged 1 commit into
mainfrom
perf/sha256-hw
Oct 9, 2026
Merged

DeusData merged 1 commit into
mainfrom
perf/sha256-hw

Conversation

@DeusData

@DeusData DeusData commented Oct 9, 2026

Copy link
Copy Markdown
Owner

What

Every process start computes the SHA-256 of its own executable (the daemon build identity). With the portable transform that took about 1.2 s of CPU for the ~300 MB binary, before hook-augment could do anything else. On the reporters' machines this pushed the hook past its 2000 ms deadline, so it exited 0 with empty stdout (#2441).

This PR makes that hash about 12 times faster without changing a single digest. Both changes are in src/foundation/sha256.c:

  • Block-direct update. cbm_sha256_update compresses whole 64-byte blocks straight from the input instead of copying every byte through the context buffer.

  • Hardware rounds. Block compression uses the CPU's SHA-256 instructions when it has them:

    • ARMv8 SHA2: probed with sysctl on macOS, HWCAP on Linux, IsProcessorFeaturePresent on Windows;
    • x86 SHA-NI: probed with CPUID.

    The backend is probed once, and the portable transform stays the fallback.

No new configuration, files or public API. Two test-only seams (backend name, force portable) are compiled only with CBM_ENABLE_TEST_SEAMS.

Measurements

Setup: Apple arm64 Mac, macOS 26.6. Release builds of main 72a2c0b and of this branch. hook-augment with a SessionStart payload in fresh isolated HOME, cache and runtime dirs. 9 interleaved runs, medians.

main this PR
hook-augment wall 1.448 s 0.376 s
hook-augment user CPU 1.162 s 0.097 s
SHA-256 of the 303 MB binary, CPU per pass 1.18 s 0.095 s
CBM_HOOK_DEADLINE_MS=1000 (stands in for a slower machine) exit 0, empty stdout after 1.01 s full 188-byte answer in 0.37 s

The block-direct update alone, with portable rounds, takes the hash from 1.18 s to 0.74 s, so CPUs without SHA instructions gain too.

Correctness

  • The build id logged by each binary equals the file's SHA-256, and the hook output is byte-identical between main and this PR.
  • New test cli_sha256_hardware_matches_portable_issue2441:
    • compares the hardware and portable backends at every length 0..299 and at 4096, 65535, 65536, 65537 and 1 MiB + 13 bytes, under eight different update chunkings;
    • checks the NIST million-'a' vector;
    • on Apple arm64, asserts that the hardware backend is in use.
  • Mutation checks: breaking one step of the ARM message schedule fails the test, and so does making the Apple probe report no SHA2.
  • x86 SHA-NI was checked under qemu-x86_64 -cpu max: 2,436 comparisons equal and the million-'a' vector correct. On a CPU model without SHA-NI the probe falls back to portable with the same digests.
  • On a Windows arm64 VM the probe selects the hardware path. Hashing the 303 MB product binary in memory took 0.094 s, against 0.82 s on the forced portable path, and both digests equal sha256sum's.
  • Full test suites on this tree:
    • macOS arm64: 8,952 passed, 0 failed;
    • Linux arm64 (GCC container): 8,790 passed, 0 failed;
    • Windows arm64 VM: 8,782 passed, 0 failed.

Refs #2441. Both reports there are macOS arm64, where the hardware path always applies.

Every process start fingerprints its own executable for the daemon build
identity, and the hook path pays for that before it can answer. With the
portable transform that was ~1.2 s of CPU for the ~300 MB binary, so on a
slower machine hook-augment hit its 2000 ms deadline and printed nothing
(#2441).

Two changes in src/foundation/sha256.c:
- cbm_sha256_update compresses whole blocks straight from the input instead
  of copying every byte through the context buffer.
- Block compression runs on the CPU's SHA-256 instructions when it has them:
  ARMv8 SHA2 (sysctl on macOS, HWCAP on Linux, IsProcessorFeaturePresent on
  Windows) or x86 SHA-NI (CPUID). The backend is probed once; the portable
  transform stays the fallback.

Measured on an Apple arm64 Mac (macOS 26.6), release builds of main 72a2c0b
and of this change, hook-augment SessionStart in fresh isolated dirs, 9
interleaved runs, medians:
- hook-augment: wall 1.448 s -> 0.376 s, user CPU 1.162 s -> 0.097 s
- SHA-256 of the 303 MB binary: 1.18 s -> 0.095 s CPU per pass (the
  block-direct update alone, portable rounds: 0.74 s)
- with CBM_HOOK_DEADLINE_MS=1000 (a slower machine): before, exit 0 with
  empty stdout after 1.01 s; after, the full 188-byte answer in 0.37 s

Digests are unchanged: the build id the new binary logs equals the file's
SHA-256, and the hook output is byte-identical. A new test compares the
hardware and portable backends at every length 0..299 and at block-boundary
sizes up to 1 MiB under eight update chunkings, plus the NIST million-'a'
vector; it fails if the ARM schedule is broken or if Apple arm64 falls back
to portable. The x86 path was checked under qemu-x86_64 -cpu max (2,436
comparisons equal, million-'a' correct) and falls back to portable on a CPU
without SHA-NI. On a Windows arm64 VM the probe picks the hardware path:
0.094 s against 0.82 s portable for the 303 MB binary, digest equal to
sha256sum's.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant