Skip to content

perf(bench): move the release baseline to 2.8.0 - #228

Merged
orieg merged 1 commit into
mainfrom
perf/baseline-2.8.0
Oct 9, 2026
Merged

perf(bench): move the release baseline to 2.8.0#228
orieg merged 1 commit into
mainfrom
perf/baseline-2.8.0

Conversation

@orieg

@orieg orieg commented Oct 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Summary

Release-cycle step 7, in its own dedicated PR as required by README: a baseline move never rides along with a feature change.

1. What changed and where

  • .github/workflows/ci.yml: Bump pie install --skip-enable-extension orieg/judy:2.6.0 to 2.8.0.
  • baselines/latest.json: Replaced with benchmark-linux-php8.4 artifact from CI run on tagged commit df9ca33 (run 37872132328).

2. What was verified by running

  • Verified orieg/judy:2.8.0 is published and indexed on Packagist.
  • Verified artifact metadata matches baseline specifications (PHP 8.4.26, Linux x86_64, size 500,000, 7 iterations, suite all).
  • Interleaved release comparison (2.6.0 -> 2.8.0 on Ubuntu runner, 5 groups x 5 rounds x 2 arms | 250s): 
    Run-wide median delta: +0.44% (PHP-array control +0.01%) -> clean
Summary: 0 faster, 0 regressions, 100 unchanged, 0 unstable, 0 new, 0 removed
  • PHP-array control (+0.01%) rules out runner contention; 0 regressions observed.

3. Invariants preserved

  • Release-over-release benchmark comparison remains apples-to-apples.
  • Baseline parameters (size, iterations, platform, suite) unchanged.

4. Downstream impacts

  • Future PRs will benchmark against v2.8.0.

allow-golden-update: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
allow-regression: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33

@orieg
perf(bench): move the release baseline to 2.8.0
8674288 
Release-cycle step 7, in its own dedicated commit as required by README:
a baseline move never rides along with a feature change.

`baselines/latest.json` is replaced with the `benchmark-linux-php8.4`
artifact from the CI run on the tagged commit `df9ca33` (run 37872132328)
-- PHP 8.4.26, Linux x86_64, size 500000, iterations 7, suite `all`,
reporting judy_version 2.8.0, matching the parameters of the baseline it
replaces. The `pie install orieg/judy:` pin in `.github/workflows/ci.yml`
moves 2.6.0 -> 2.8.0 now that 2.8.0 resolves on Packagist / PIE.

The interleaved comparison this baseline supersedes, 2.6.0 -> 2.8.0 on the
same runner (5 groups x 5 rounds x 2 arms | 250s):

  Interleaved comparison: 2.6.0 -> 2.8.0 | 5 groups x 5 rounds x 2 arms | 250s
  Run-wide median delta: +0.44% (PHP-array control +0.01%) -> clean
  Summary: 0 faster, 0 regressions, 100 unchanged, 0 unstable, 0 new, 0 removed

The PHP-array control at +0.01% rules out runner contention, and 0
regressions confirms behavior tightening, strict validation, and memory-safety
fixes across 2.8.0 carry no measurable performance overhead.
@github-actions

github-actions Bot commented Oct 9, 2026 •
edited
Loading

Copy link
Copy Markdown

discipline: passed

Base main (origin/main) (merge base df9ca33f01) · 27 gate(s) passed · 2167 item(s) examined · 0 warning(s) · 165 override(s)

Findings lifted by an override:

  • golden-output on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33
  • bench-regression on baselines/latest.json: baselines/latest.json 2.8.0 baseline update from CI run on tagged commit df9ca33

discipline explain <gate> prints what a gate checks and how a finding is lifted. This comment is updated on every run; the check's status is the verdict.

@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown

Test Results

PHP Platform Arch TS Tests Pass Fail Skip Duration
✅ 8.1 Linux x64 - 258 258 0 0 7.5s
✅ 8.2 Linux x64 - 258 258 0 0 7.9s
✅ 8.3 Linux x64 - 258 258 0 0 10.0s
✅ 8.4 Linux x64 - 258 258 0 0 10.0s
✅ 8.5 Linux x64 - 258 258 0 0 9.3s
✅ 🧪 8.6 Linux x64 - 258 258 0 0 9.7s
✅ Total 1548 1548 0 0

Benchmark Results

500,000 elements, 7 iterations (median), Judy 2.8.0

Run Status

Platform PHP 8.1 PHP 8.2 PHP 8.3 PHP 8.4 PHP 8.5 PHP 8.6
Linux ✅ ✅ ✅ ✅ ✅ ✅

Memory (Linux, PHP 8.6)

Peak RSS of a child process building one structure, n = 500,000, minus an empty-process floor. Judy's memory advantage is type- and scale-dependent — see BENCHMARK.md for the curve across sizes on a dedicated host.

Type Judy PHP array Judy vs PHP array
Bitset < 440.0 KB 8.1 MB > 19x smaller
Int to Int 4.3 MB 8.1 MB 1.9x smaller
Int to Mixed 40.4 MB 38.1 MB 1.06x larger
Int to Packed 16.3 MB 38.1 MB 2.3x smaller
String to Int 10.3 MB 38.1 MB 3.7x smaller
String to Mixed 46.4 MB 68.1 MB 1.5x smaller
String to Int (Hash) 31.5 MB 38.2 MB 1.2x smaller
String to Mixed (Hash) 67.5 MB 68.0 MB about the same
String to Int (Adaptive) 31.6 MB 38.2 MB 1.2x smaller
String to Mixed (Adaptive) 67.6 MB 68.2 MB about the same

< marks a structure smaller than this instrument can resolve at n = 500,000: run-to-run movement in the empty-process floor puts the resolution at 440.0 KB. The bound is real, the exact figure is not available at this size — BENCHMARK.md measures the same types up to n = 8M, where they are well clear of the floor.

API Speedups (Linux, PHP 8.6)

Speedup = PHP time / Judy time. Bold = Judy faster.

Operation Judy PHP Speedup
putAll() 10.9 ms 15.4 ms 1.4x
fromArray() 10.8 ms 15.4 ms 1.4x
toArray() 12.3 ms 20.9 ms 1.7x
getAll() 0.60 ms 1.1 ms 1.8x
keys() 5.5 ms 11.7 ms 2.1x
values() 5.6 ms 11.6 ms 2.1x
sumValues() 4.0 ms 8.8 ms 2.2x
populationCount() 0.00 ms 9.1 ms O(1) vs O(n)
deleteRange() 20.1 ms 27.7 ms 1.4x
equals() 5.2 ms 22.0 ms 4.2x
union() Bitset 15.2 ms 2.8 ms 0.2x
intersect() Bitset 6.5 ms 3.1 ms 0.5x
diff() Bitset 6.5 ms 1.8 ms 0.3x
xor() Bitset 13.4 ms 6.7 ms 0.5x
mergeWith() Int→Int 25.2 ms 32.0 ms 1.3x
mergeWith() Str→Int 138.2 ms 147.3 ms 1.1x
forEach() Int→Int 13.1 ms 10.6 ms 0.8x
filter() Int→Int 20.9 ms 20.4 ms ~1.0x
map() Int→Int 23.1 ms 23.7 ms ~1.0x
forEach() Str→Int 41.7 ms 34.7 ms 0.8x
filter() Str→Int 71.3 ms 69.2 ms ~1.0x
map() Str→Int 93.9 ms 94.7 ms ~1.0x

O(1) vs O(n) marks a row whose ratio is decided by the complexity classes rather than measured: populationCount() reads libJudy's cached population, the PHP arm counts every element. It scales with n and says nothing about this change. judy-bench.php declines the same comparison for size($lo, $hi) for the same reason.

Core Types (Linux, PHP 8.6)

Speedup = PHP time / Judy time. Bold = Judy faster.

Type Op Judy PHP array Speedup
Bitset Write 9.1 ms 3.8 ms 0.4x
Bitset Read 5.6 ms 2.0 ms 0.4x
Bitset Iter 8.5 ms 1.6 ms 0.2x
Bitset Free 0.00 ms 0.24 ms 74.4x
Int to Int Write 12.2 ms 3.9 ms 0.3x
Int to Int Read 6.3 ms 2.0 ms 0.3x
Int to Int Iter 8.6 ms 1.6 ms 0.2x
Int to Int Free 0.10 ms 0.24 ms 2.4x
Int to Mixed Write 18.1 ms 7.2 ms 0.4x
Int to Mixed Read 7.5 ms 3.3 ms 0.4x
Int to Mixed Iter 9.4 ms 2.3 ms 0.2x
Int to Mixed Free 8.6 ms 4.2 ms 0.5x
String to Int Write 53.8 ms 9.6 ms 0.2x
String to Int Read 23.9 ms 4.7 ms 0.2x
String to Int Iter 33.1 ms 2.2 ms 0.1x
String to Int Free 12.2 ms 0.88 ms 0.1x
String to Int (Adaptive) Write 100.6 ms 9.2 ms 0.1x
String to Int (Adaptive) Read 14.8 ms 4.1 ms 0.3x
String to Int (Adaptive) Iter 41.0 ms 2.2 ms 0.1x
String to Int (Adaptive) Free 27.6 ms 0.82 ms 0.0x
All types detail

Core Types (Linux, PHP 8.6) — All

Speedup = PHP time / Judy time. Bold = Judy faster.

Type Op Judy PHP array Speedup
Bitset Write 9.1 ms 3.8 ms 0.4x
Bitset Read 5.6 ms 2.0 ms 0.4x
Bitset Iter 8.5 ms 1.6 ms 0.2x
Bitset Free 0.00 ms 0.24 ms 74.4x
Int to Int Write 12.2 ms 3.9 ms 0.3x
Int to Int Read 6.3 ms 2.0 ms 0.3x
Int to Int Iter 8.6 ms 1.6 ms 0.2x
Int to Int Free 0.10 ms 0.24 ms 2.4x
Int to Mixed Write 18.1 ms 7.2 ms 0.4x
Int to Mixed Read 7.5 ms 3.3 ms 0.4x
Int to Mixed Iter 9.4 ms 2.3 ms 0.2x
Int to Mixed Free 8.6 ms 4.2 ms 0.5x
Int to Packed Write 23.6 ms 6.9 ms 0.3x
Int to Packed Read 15.0 ms 3.4 ms 0.2x
Int to Packed Iter 17.4 ms 2.4 ms 0.1x
Int to Packed Free 4.5 ms 4.3 ms 0.9x
String to Int Write 53.8 ms 9.6 ms 0.2x
String to Int Read 23.9 ms 4.7 ms 0.2x
String to Int Iter 33.1 ms 2.2 ms 0.1x
String to Int Free 12.2 ms 0.88 ms 0.1x
String to Mixed Write 74.0 ms 13.2 ms 0.2x
String to Mixed Read 27.3 ms 6.9 ms 0.3x
String to Mixed Iter 44.2 ms 4.3 ms 0.1x
String to Mixed Free 46.5 ms 6.8 ms 0.1x
String to Int (Hash) Write 106.6 ms 9.4 ms 0.1x
String to Int (Hash) Read 15.4 ms 4.8 ms 0.3x
String to Int (Hash) Iter 42.3 ms 2.2 ms 0.1x
String to Int (Hash) Free 30.2 ms 0.87 ms 0.0x
String to Mixed (Hash) Write 164.1 ms 13.3 ms 0.1x
String to Mixed (Hash) Read 18.3 ms 6.8 ms 0.4x
String to Mixed (Hash) Iter 65.9 ms 4.1 ms 0.1x
String to Mixed (Hash) Free 91.6 ms 6.5 ms 0.1x
String to Int (Adaptive) Write 100.6 ms 9.2 ms 0.1x
String to Int (Adaptive) Read 14.8 ms 4.1 ms 0.3x
String to Int (Adaptive) Iter 41.0 ms 2.2 ms 0.1x
String to Int (Adaptive) Free 27.6 ms 0.82 ms 0.0x
String to Mixed (Adaptive) Write 168.9 ms 12.6 ms 0.1x
String to Mixed (Adaptive) Read 17.4 ms 6.7 ms 0.4x
String to Mixed (Adaptive) Iter 70.1 ms 4.3 ms 0.1x
String to Mixed (Adaptive) Free 91.4 ms 6.6 ms 0.1x
Cross-version detail (Linux)

Core Types — Linux

Each cell: Judy ms (speedup vs PHP array).

Type Op PHP 8.1 PHP 8.2 PHP 8.3 PHP 8.4 PHP 8.5 PHP 8.6
Bitset Write 15.3 ms (0.4x) 13.1 ms (0.4x) 17.9 ms (0.4x) 17.2 ms (0.4x) 14.3 ms (0.5x) 9.1 ms (0.4x)
Bitset Read 9.3 ms (0.4x) 9.2 ms (0.3x) 11.1 ms (0.3x) 11.2 ms (0.4x) 10.2 ms (0.4x) 5.6 ms (0.4x)
Int to Int Write 20.9 ms (0.3x) 18.2 ms (0.3x) 24.8 ms (0.3x) 34.9 ms (0.2x) 21.2 ms (0.3x) 12.2 ms (0.3x)
Int to Int Read 8.6 ms (0.4x) 9.0 ms (0.3x) 11.2 ms (0.3x) 11.2 ms (0.4x) 10.7 ms (0.3x) 6.3 ms (0.3x)
Int to Mixed Write 28.7 ms (0.3x) 29.0 ms (0.3x) 36.4 ms (0.3x) 36.7 ms (0.3x) 31.9 ms (0.3x) 18.1 ms (0.4x)
Int to Mixed Read 11.1 ms (0.4x) 9.7 ms (0.4x) 14.3 ms (0.4x) 14.2 ms (0.4x) 12.9 ms (0.5x) 7.5 ms (0.4x)
Int to Packed Write 41.0 ms (0.2x) 35.9 ms (0.2x) 49.2 ms (0.2x) 49.4 ms (0.2x) 44.7 ms (0.2x) 23.6 ms (0.3x)
Int to Packed Read 22.8 ms (0.2x) 24.5 ms (0.2x) 30.6 ms (0.2x) 29.8 ms (0.2x) 29.3 ms (0.2x) 15.0 ms (0.2x)
String to Int Write 72.4 ms (0.2x) 80.3 ms (0.2x) 91.4 ms (0.2x) 92.1 ms (0.2x) 89.7 ms (0.2x) 53.8 ms (0.2x)
String to Int Read 31.3 ms (0.2x) 31.7 ms (0.2x) 37.9 ms (0.2x) 37.2 ms (0.2x) 37.7 ms (0.2x) 23.9 ms (0.2x)
String to Mixed Write 92.4 ms (0.2x) 103.6 ms (0.1x) 114.8 ms (0.2x) 113.3 ms (0.2x) 108.7 ms (0.2x) 74.0 ms (0.2x)
String to Mixed Read 33.6 ms (0.2x) 33.5 ms (0.2x) 41.5 ms (0.3x) 40.7 ms (0.2x) 39.3 ms (0.2x) 27.3 ms (0.3x)
String to Int (Hash) Write 133.2 ms (0.1x) 137.1 ms (0.1x) 161.5 ms (0.1x) 167.8 ms (0.1x) 159.8 ms (0.1x) 106.6 ms (0.1x)
String to Int (Hash) Read 20.7 ms (0.3x) 19.2 ms (0.3x) 24.4 ms (0.3x) 25.0 ms (0.3x) 22.7 ms (0.3x) 15.4 ms (0.3x)
String to Mixed (Hash) Write 196.6 ms (0.1x) 209.0 ms (0.1x) 234.2 ms (0.1x) 241.2 ms (0.1x) 228.4 ms (0.1x) 164.1 ms (0.1x)
String to Mixed (Hash) Read 22.6 ms (0.3x) 18.9 ms (0.3x) 27.5 ms (0.4x) 28.5 ms (0.4x) 25.4 ms (0.4x) 18.3 ms (0.4x)
String to Int (Adaptive) Write 129.0 ms (0.1x) 141.2 ms (0.1x) 162.4 ms (0.1x) 170.6 ms (0.1x) 160.5 ms (0.1x) 100.6 ms (0.1x)
String to Int (Adaptive) Read 20.4 ms (0.3x) 19.1 ms (0.3x) 23.6 ms (0.3x) 24.5 ms (0.3x) 22.2 ms (0.4x) 14.8 ms (0.3x)
String to Mixed (Adaptive) Write 194.5 ms (0.1x) 226.2 ms (0.1x) 220.1 ms (0.1x) 248.9 ms (0.1x) 251.6 ms (0.1x) 168.9 ms (0.1x)
String to Mixed (Adaptive) Read 22.8 ms (0.3x) 19.0 ms (0.3x) 26.4 ms (0.4x) 27.6 ms (0.4x) 23.6 ms (0.4x) 17.4 ms (0.4x)

API — Linux

Operation PHP 8.1 PHP 8.2 PHP 8.3 PHP 8.4 PHP 8.5 PHP 8.6
putAll() 17.9 ms (1.3x) 15.8 ms (1.4x) 20.7 ms (1.4x) 20.7 ms (1.9x) 17.8 ms (1.6x) 10.9 ms (1.4x)
fromArray() 17.2 ms (1.3x) 15.7 ms (1.4x) 20.6 ms (1.4x) 20.6 ms (1.9x) 18.1 ms (1.5x) 10.8 ms (1.4x)
toArray() 17.2 ms (1.6x) 19.4 ms (1.4x) 22.4 ms (1.6x) 23.5 ms (1.6x) 24.5 ms (1.5x) 12.3 ms (1.7x)
getAll() 0.84 ms (1.9x) 0.91 ms (1.7x) 1.0 ms (2.0x) 1.1 ms (1.8x) 1.2 ms (1.8x) 0.60 ms (1.8x)
keys() 9.8 ms (2.2x) 9.2 ms (2.0x) 11.3 ms (2.1x) 11.3 ms (2.2x) 10.0 ms (2.1x) 5.5 ms (2.1x)
values() 11.4 ms (1.8x) 9.6 ms (1.9x) 11.7 ms (2.0x) 12.1 ms (2.1x) 11.1 ms (1.9x) 5.6 ms (2.1x)
sumValues() 7.2 ms (2.5x) 6.4 ms (2.5x) 7.7 ms (2.6x) 8.5 ms (2.6x) 6.4 ms (2.5x) 4.0 ms (2.2x)
populationCount() 0.00 ms (204092x) 0.00 ms (163548x) 0.00 ms (207224x) 0.00 ms (232333x) 0.00 ms (166827x) 0.00 ms (∞)
deleteRange() 31.1 ms (1.5x) 28.8 ms (1.5x) 39.0 ms (1.5x) 38.6 ms (1.5x) 34.3 ms (1.4x) 20.1 ms (1.4x)
equals() 9.0 ms (4.2x) 9.4 ms (4.2x) 11.2 ms (4.2x) 12.0 ms (3.9x) 9.8 ms (4.1x) 5.2 ms (4.2x)
union() Bitset 25.1 ms (0.2x) 24.2 ms (0.2x) 30.3 ms (0.2x) 30.6 ms (0.2x) 25.6 ms (0.3x) 15.2 ms (0.2x)
intersect() Bitset 10.7 ms (0.4x) 11.8 ms (0.4x) 13.1 ms (0.4x) 13.2 ms (0.4x) 12.2 ms (0.5x) 6.5 ms (0.5x)
diff() Bitset 10.4 ms (0.3x) 11.7 ms (0.3x) 12.9 ms (0.2x) 12.9 ms (0.3x) 12.2 ms (0.3x) 6.5 ms (0.3x)
xor() Bitset 20.9 ms (0.6x) 24.3 ms (0.5x) 25.8 ms (0.4x) 26.0 ms (0.4x) 24.3 ms (0.6x) 13.4 ms (0.5x)
mergeWith() Int→Int 42.2 ms (1.3x) 39.2 ms (1.3x) 53.3 ms (1.3x) 61.7 ms (1.4x) 44.0 ms (1.3x) 25.2 ms (1.3x)
mergeWith() Str→Int 180.7 ms (1.1x) 186.0 ms (1.1x) 229.6 ms (1.1x) 245.2 ms (1.1x) 231.8 ms (1.1x) 138.2 ms (1.1x)
forEach() Int→Int 21.2 ms (0.9x) 22.9 ms (0.8x) 27.1 ms (0.8x) 28.0 ms (0.8x) 24.5 ms (0.7x) 13.1 ms (0.8x)
filter() Int→Int 33.4 ms (~1.0x) 35.2 ms (0.8x) 44.6 ms (0.9x) 45.9 ms (~1.0x) 39.1 ms (0.8x) 20.9 ms (~1.0x)
map() Int→Int 37.4 ms (~1.0x) 36.8 ms (0.9x) 50.5 ms (0.9x) 52.4 ms (0.9x) 42.1 ms (~1.0x) 23.1 ms (~1.0x)
forEach() Str→Int 50.0 ms (0.9x) 53.4 ms (0.9x) 63.6 ms (0.9x) 67.8 ms (0.9x) 60.9 ms (0.9x) 41.7 ms (0.8x)
filter() Str→Int 98.0 ms (~1.0x) 102.7 ms (0.9x) 116.3 ms (~1.0x) 125.6 ms (0.9x) 118.6 ms (0.9x) 71.3 ms (~1.0x)
map() Str→Int 135.0 ms (~1.0x) 135.8 ms (~1.0x) 160.3 ms (~1.0x) 164.2 ms (~1.0x) 157.9 ms (~1.0x) 93.9 ms (~1.0x)

Release Comparison (2.8.0 → 2.8.0)

Interleaved A/B comparison against v2.8.0 (pie install) on the same CI runner: 5 benchmark groups × 5 rounds × 2 arms, 300,000 elements, median of 3 per child. Each row is the median of the paired current/baseline ratios with a 95% percentile-bootstrap CI; it is flagged only when the whole drift-adjusted CI clears ±10%. Advisory — benchmarks never gate a merge.

Run-wide median delta +0.09% (PHP-array control -0.04%); runner common-mode -0.04% divided out of every row.

Summary: 0 regressions, 99 unchanged, 1 not evaluated (unstable)

Benchmark Baseline Current Delta 95% CI Status
adv.filter.int_to_int 27.4 ms 27.8 ms +0.7% [-3.1%, +5.1%] ~same
adv.filter.string_to_int 72.9 ms 73.5 ms +0.3% [-3.5%, +2.0%] ~same
adv.forEach.int_to_int 16.7 ms 16.5 ms -0.9% [-2.3%, +2.8%] ~same
adv.forEach.string_to_int 40.0 ms 39.8 ms -1.4% [-3.5%, -0.0%] ~same
adv.map.int_to_int 30.3 ms 30.5 ms +0.8% [-2.2%, +2.3%] ~same
adv.map.string_to_int 96.1 ms 95.6 ms -0.3% [-1.9%, +0.4%] ~same
adv.optiter.foreach.string_to_int_adaptive.default 62.4 ms 59.7 ms -5.5% [-12.0%, +3.9%] ~same
adv.optiter.foreach.string_to_int_adaptive.optimized 42.5 ms 41.7 ms -4.1% [-10.8%, +2.4%] ~same
adv.optiter.foreach.string_to_int_hash.default 55.3 ms 50.0 ms -1.3% [-12.4%, +8.4%] ~same
adv.optiter.foreach.string_to_int_hash.optimized 32.6 ms 32.7 ms +0.7% [-2.5%, +2.2%] ~same
adv.optiter.increment.string_to_int_hash.default 33.9 ms 35.0 ms +3.0% [-18.5%, +5.3%] ~same
adv.optiter.increment.string_to_int_hash.optimized 41.5 ms 39.4 ms +4.6% [-6.9%, +6.1%] ~same
adv.optiter.overwrite.string_to_int_adaptive.default 54.4 ms 53.0 ms -3.4% [-7.7%, +3.6%] ~same
adv.optiter.overwrite.string_to_int_adaptive.optimized 49.6 ms 48.9 ms +0.9% [-12.7%, +7.6%] ~same
adv.optiter.overwrite.string_to_int_hash.optimized 38.5 ms 40.6 ms +7.8% [-15.9%, +15.6%] ~same
adv.optiter.ratio.foreach.string_to_int_adaptive 68.7 ms 69.4 ms -1.4% [-5.6%, +7.7%] ~same
adv.optiter.ratio.foreach.string_to_int_hash 59.1 ms 64.3 ms +2.8% [-5.6%, +12.7%] ~same
adv.optiter.ratio.increment.string_to_int_hash 109.8 ms 113.0 ms +2.9% [-11.1%, +14.3%] ~same
adv.optiter.ratio.overwrite.string_to_int_adaptive 91.2 ms 92.2 ms +0.5% [-5.3%, +11.4%] ~same
adv.optiter.ratio.overwrite.string_to_int_hash 105.6 ms 111.2 ms +5.9% [-0.2%, +15.2%] ~same
adv.optiter.ratio.toArray.string_to_int_adaptive 65.0 ms 65.6 ms +1.1% [-6.1%, +3.0%] ~same
adv.optiter.ratio.toArray.string_to_int_hash 55.6 ms 56.2 ms -1.9% [-6.3%, +13.4%] ~same
adv.optiter.ratio.values.string_to_int_adaptive 59.5 ms 57.4 ms -4.0% [-7.7%, +30.2%] ~same
adv.optiter.ratio.values.string_to_int_hash 50.3 ms 54.2 ms -0.8% [-5.5%, +13.2%] ~same
adv.optiter.toArray.string_to_int_adaptive.default 68.4 ms 67.6 ms -1.5% [-3.1%, +1.7%] ~same
adv.optiter.toArray.string_to_int_adaptive.optimized 44.7 ms 44.1 ms -0.2% [-4.5%, +2.0%] ~same
adv.optiter.toArray.string_to_int_hash.default 59.0 ms 58.5 ms +1.3% [-11.1%, +7.3%] ~same
adv.optiter.toArray.string_to_int_hash.optimized 32.7 ms 32.7 ms -0.3% [-1.6%, +0.9%] ~same
adv.optiter.values.string_to_int_adaptive.default 55.8 ms 56.4 ms +1.6% [-24.2%, +9.3%] ~same
adv.optiter.values.string_to_int_adaptive.optimized 33.4 ms 32.9 ms -1.3% [-4.7%, +1.9%] ~same
adv.optiter.values.string_to_int_hash.default 47.2 ms 43.3 ms +1.4% [-15.4%, +7.2%] ~same
adv.optiter.values.string_to_int_hash.optimized 23.8 ms 23.6 ms +0.5% [-4.2%, +2.6%] ~same
api.deleteRange.int_to_int 23.4 ms 23.2 ms -0.6% [-4.1%, +1.9%] ~same
api.equals.int_to_int 7.0 ms 6.8 ms -1.1% [-5.3%, +2.6%] ~same
api.fromArray.int_to_int 13.6 ms 13.9 ms +2.2% [+0.4%, +2.3%] ~same
api.getAll.int_to_int 0.53 ms 0.52 ms -0.3% [-8.3%, +5.9%] ~same
api.increment.int_to_int 14.1 ms 14.2 ms +1.4% [-0.7%, +3.3%] ~same
api.keys.int_to_int 6.3 ms 6.3 ms +0.3% [-0.5%, +3.1%] ~same
api.mergeWith.int_to_int 31.7 ms 31.7 ms +0.2% [-2.1%, +2.6%] ~same
api.mergeWith.string_to_int 142.8 ms 142.6 ms -0.7% [-1.5%, +0.7%] ~same
api.populationCount.int_to_int 0.00 ms 0.00 ms 0.0% [+0.0%, +0.0%] ~same
api.putAll.int_to_int 13.8 ms 13.9 ms +1.1% [+0.5%, +1.9%] ~same
api.range.keys.int_to_int 0.57 ms 0.57 ms -0.6% [-4.8%, +6.9%] ~same
api.range.size.string_to_int 2.3 ms 2.3 ms 0.0% [-0.2%, +0.5%] ~same
api.range.size1000.int_to_int 0.71 ms 0.72 ms +0.1% [-4.0%, +0.5%] ~same
api.setop.diff.bitset 7.6 ms 7.6 ms +0.1% [-0.5%, +1.3%] ~same
api.setop.diff.int_to_int 10.9 ms 10.6 ms -0.9% [-4.3%, +1.8%] ~same
api.setop.diff.string_to_int 63.2 ms 62.8 ms -0.1% [-3.2%, +4.0%] ~same
api.setop.intersect.bitset 7.7 ms 7.8 ms +0.8% [-2.1%, +2.3%] ~same
api.setop.intersect.int_to_int 10.3 ms 10.3 ms +0.5% [-3.6%, +2.9%] ~same
api.setop.intersect.string_to_int 70.7 ms 70.4 ms +0.7% [-2.1%, +1.9%] ~same
api.setop.union.bitset 18.3 ms 18.3 ms +0.1% [-2.9%, +4.2%] ~same
api.setop.union.int_to_int 26.4 ms 26.1 ms -1.4% [-2.3%, +3.8%] ~same
api.setop.union.string_to_int 163.9 ms 163.3 ms -1.1% [-1.5%, -0.2%] ~same
api.setop.xor.bitset 15.4 ms 15.4 ms +0.1% [-0.6%, +3.1%] ~same
api.setop.xor.int_to_int 20.9 ms 20.7 ms -2.0% [-5.8%, +2.6%] ~same
api.sumValues.int_to_int 4.4 ms 4.6 ms +6.5% [-15.5%, +11.8%] ~same
api.toArray.int_to_int 14.6 ms 14.8 ms +0.4% [-1.2%, +1.8%] ~same
api.values.int_to_int 6.5 ms 6.5 ms +0.6% [-4.9%, +4.3%] ~same
core.bitset.free 0.00 ms 0.00 ms -4.1% [-8.8%, +9.3%] ~same
core.bitset.iter 10.9 ms 10.9 ms +0.2% [-1.6%, +5.1%] ~same
core.bitset.read 6.7 ms 6.7 ms +0.1% [-1.1%, +1.2%] ~same
core.bitset.write 10.3 ms 10.3 ms -0.2% [-1.4%, -0.2%] ~same
core.int_to_int.free 0.30 ms 0.28 ms -3.9% [-13.4%, +15.2%] ~same
core.int_to_int.iter 11.6 ms 11.6 ms -0.1% [-2.3%, +3.5%] ~same
core.int_to_int.read 6.7 ms 6.8 ms +0.1% [-6.0%, +5.5%] ~same
core.int_to_int.write 14.8 ms 14.9 ms +0.6% [-0.6%, +2.9%] ~same
core.int_to_mixed.free 7.9 ms 7.9 ms +0.3% [-2.1%, +0.6%] ~same
core.int_to_mixed.iter 12.6 ms 12.5 ms -0.6% [-2.3%, +0.3%] ~same
core.int_to_mixed.read 8.4 ms 8.4 ms +0.2% [-0.4%, +1.2%] ~same
core.int_to_mixed.write 21.5 ms 21.1 ms -1.2% [-2.4%, +0.5%] ~same
core.int_to_packed.free 5.2 ms 5.2 ms -0.2% [-4.5%, +1.0%] ~same
core.int_to_packed.iter 23.6 ms 23.4 ms -0.9% [-2.3%, +0.5%] ~same
core.int_to_packed.read 17.6 ms 17.8 ms +1.1% [-1.0%, +1.9%] ~same
core.int_to_packed.write 29.0 ms 28.6 ms -0.4% [-2.2%, +0.9%] ~same
core.string_to_int.free 10.0 ms 10.1 ms +1.1% [-0.1%, +2.7%] ~same
core.string_to_int.iter 33.4 ms 33.3 ms -0.2% [-0.6%, +0.3%] ~same
core.string_to_int.read 21.2 ms 21.4 ms +0.3% [-0.6%, +1.6%] ~same
core.string_to_int.write 54.3 ms 53.3 ms -1.3% [-2.0%, -0.4%] ~same
core.string_to_int_adaptive.free 21.0 ms 20.6 ms -2.3% [-5.5%, -0.0%] ~same
core.string_to_int_adaptive.iter 39.7 ms 39.7 ms -0.6% [-1.0%, +0.8%] ~same
core.string_to_int_adaptive.read 14.4 ms 14.4 ms +0.5% [-0.2%, +0.6%] ~same
core.string_to_int_adaptive.write 91.5 ms 92.5 ms +1.0% [-2.4%, +5.4%] ~same
core.string_to_int_hash.free 20.8 ms 21.5 ms +1.7% [-1.1%, +9.3%] ~same
core.string_to_int_hash.iter 40.5 ms 40.4 ms -0.4% [-1.3%, +1.2%] ~same
core.string_to_int_hash.read 14.6 ms 14.8 ms +0.5% [-0.1%, +2.5%] ~same
core.string_to_int_hash.write 91.9 ms 93.5 ms -0.4% [-2.1%, +2.8%] ~same
core.string_to_mixed.free 33.5 ms 34.2 ms +0.3% [-0.6%, +2.8%] ~same
core.string_to_mixed.iter 36.6 ms 36.2 ms -1.7% [-3.5%, +0.3%] ~same
core.string_to_mixed.read 23.6 ms 23.6 ms +0.1% [-1.5%, +2.2%] ~same
core.string_to_mixed.write 64.2 ms 64.3 ms +0.2% [-3.7%, +1.0%] ~same
core.string_to_mixed_adaptive.free 64.1 ms 62.9 ms -1.9% [-4.1%, +4.9%] ~same
core.string_to_mixed_adaptive.iter 48.0 ms 47.8 ms -1.2% [-2.2%, +3.5%] ~same
core.string_to_mixed_adaptive.read 16.1 ms 16.1 ms -0.3% [-1.6%, +1.7%] ~same
core.string_to_mixed_adaptive.write 131.5 ms 131.1 ms +0.6% [-2.7%, +4.7%] ~same
core.string_to_mixed_hash.free 61.7 ms 62.4 ms +1.1% [-0.6%, +4.8%] ~same
core.string_to_mixed_hash.iter 49.7 ms 47.3 ms -1.9% [-8.4%, +2.5%] ~same
core.string_to_mixed_hash.read 16.6 ms 16.5 ms -0.5% [-1.9%, +1.4%] ~same
core.string_to_mixed_hash.write 128.1 ms 127.2 ms 0.0% [-4.1%, +2.7%] ~same
1 benchmarks not evaluated (within-arm scatter exceeded the effect being claimed)
Benchmark Baseline Current Delta 95% CI Status
adv.optiter.overwrite.string_to_int_hash.default 37.3 ms 33.9 ms +2.8% [-27.0%, +12.4%] — unstable

Memory Comparison

Memory: 0 regressions, 10 unchanged

Type Baseline Current Delta Status
Bitset 160 B 160 B 0.0% ~same
Int to Int 160 B 160 B 0.0% ~same
Int to Mixed 4.6 MB 4.6 MB 0.0% ~same
Int to Packed 8.0 MB 8.0 MB 0.0% ~same
String to Int 64.2 KB 64.2 KB 0.0% ~same
String to Int (Adaptive) 64.2 KB 64.2 KB 0.0% ~same
String to Int (Hash) 64.2 KB 64.2 KB 0.0% ~same
String to Mixed 4.6 MB 4.6 MB 0.0% ~same
String to Mixed (Adaptive) 4.6 MB 4.6 MB 0.0% ~same
String to Mixed (Hash) 4.6 MB 4.6 MB 0.0% ~same
Full benchmark output

Linux — PHP 8.6

══════════════════════════════════════════════════════════════════════════════════════════
  php-judy Benchmark Suite — 500,000 elements, 7 iterations (median)
  PHP 8.6.0-dev | Judy ext 2.8.0 | Linux x86_64
  Suite: all | Groups: core.int,core.str,api.batch,api.setops,adv.iter,adv.sso
  memory_limit: -1
  Memory: peak RSS of a child process, empty-process floor subtracted, median of 3
══════════════════════════════════════════════════════════════════════════════════════════

── Core: Integer-Keyed Types ────────────────────────────────────────────────

  Type                         Write(ms)      Read(ms)      Iter(ms)      Free(ms)        Memory
  ────────────────────────  ────────────  ────────────  ────────────  ────────────  ────────────
  BITSET                            9.06          5.57          8.51          0.00    < 440.0 KB
    PHP array                       3.80          2.01          1.63          0.24        8.1 MB
  INT_TO_INT                       12.20          6.26          8.61          0.10        4.3 MB
    PHP array                       3.87          2.01          1.63          0.24        8.1 MB
  INT_TO_MIXED                     18.13          7.50          9.40          8.59       40.4 MB
    PHP array                       7.18          3.28          2.31          4.23       38.1 MB
  INT_TO_PACKED                    23.64         15.04         17.39          4.50       16.3 MB
    PHP array                       6.86          3.35          2.36          4.26       38.1 MB

── Core: String-Keyed Types ────────────────────────────────────────────────

  Type                         Write(ms)      Read(ms)      Iter(ms)      Free(ms)        Memory
  ────────────────────────  ────────────  ────────────  ────────────  ────────────  ────────────
  STRING_TO_INT                    53.83         23.92         33.09         12.22       10.3 MB
    PHP array                       9.60          4.67          2.21          0.88       38.1 MB
  STRING_TO_MIXED                  74.04         27.27         44.20         46.49       46.4 MB
    PHP array                      13.20          6.92          4.29          6.85       68.1 MB
  STR_TO_INT_HASH                 106.60         15.41         42.32         30.22       31.5 MB
    PHP array                       9.40          4.77          2.20          0.87       38.2 MB
  STR_TO_MIX_HASH                 164.10         18.33         65.91         91.56       67.5 MB
    PHP array                      13.26          6.82          4.11          6.49       68.0 MB
  STR_INT_ADAPTIVE                100.62         14.76         40.96         27.65       31.6 MB
    PHP array                       9.19          4.14          2.19          0.82       38.2 MB
  STR_MIX_ADAPTIVE                168.94         17.42         70.11         91.43       67.6 MB
    PHP array                      12.59          6.73          4.26          6.63       68.2 MB

── API: Batch Operations (INT_TO_INT) ────────────────────────────────────────

  Operation                           Judy(ms)       PHP(ms)     Speedup
  ──────────────────────────────  ────────────  ────────────  ──────────
  putAll()                               10.95         15.39        1.4x
  fromArray()                            10.83         15.39        1.4x
  toArray()                              12.35         20.94        1.7x
  getAll(50100 keys)                      0.60          1.07        1.8x
  increment(500000 ops/1000 keys)         12.22          5.34        0.4x

── API: Native Methods (keys, values, sum, popCount, deleteRange, equals) ────

  Operation                           Judy(ms)       PHP(ms)     Speedup
  ──────────────────────────────  ────────────  ────────────  ──────────
  keys() INT_TO_INT                       5.52         11.69        2.1x
  values() INT_TO_INT                     5.61         11.55        2.1x
  keys($lo,$hi) INT_TO_INT                0.43         11.43       26.3x
    vs slice()->keys()                    1.34             -        3.1x
  size($lo,$hi) INT x1000                 0.43             -        O(1)
  size($lo,$hi) STRING_TO_INT             2.44          3.10        1.3x
  sumValues() INT_TO_INT                  4.00          8.77        2.2x
  populationCount() INT_TO_INT            0.00          9.08   226953.4x
  deleteRange() INT_TO_INT mid50%         20.08         27.71        1.4x
  equals() INT_TO_INT                     5.20         21.96        4.2x

── API: Set Operations (BITSET, 50% overlap) ────────────────────────────────

  Operation                           Judy(ms)       PHP(ms)     Speedup
  ──────────────────────────────  ────────────  ────────────  ──────────
  union() BITSET                         15.24          2.76        0.2x
  intersect() BITSET                      6.53          3.08        0.5x
  diff() BITSET                           6.54          1.78        0.3x
  xor() BITSET                           13.40          6.75        0.5x
  union() INT_TO_INT                     19.94         40.84        2.0x
  intersect() INT_TO_INT                  8.07         19.34        2.4x
  diff() INT_TO_INT                       8.17         20.57        2.5x
  xor() INT_TO_INT                       15.72         39.79        2.5x
  union() STRING_TO_INT                 173.45        189.09        1.1x
  intersect() STRING_TO_INT              76.00         89.35        1.2x
  diff() STRING_TO_INT                   75.09         84.42        1.1x
  mergeWith() INT_TO_INT                 25.17         32.01        1.3x
  mergeWith() STRING_TO_INT             138.18        147.26        1.1x

── Advanced: C-Level forEach/filter/map ──────────────────────────────────────

  Operation                        C-level(ms)       PHP(ms)     Speedup
  ──────────────────────────────  ────────────  ────────────  ──────────
  forEach() INT_TO_INT                   13.12         10.60        0.8x
  forEach() STRING_TO_INT                41.66         34.72        0.8x
  filter() INT_TO_INT (50% pass)         20.92         20.37        1.0x
  filter() STRING_TO_INT (50% pass)         71.25         69.16        1.0x
  map() INT_TO_INT (*2)                  23.07         23.74        1.0x
  map() STRING_TO_INT (*2)               93.90         94.70        1.0x

── Advanced: optimizeIteration trade (opt-in key-index payload mirror) ──────

  Operation                        default(ms)    optim.(ms)   optim./def.
  ──────────────────────────────  ────────────  ────────────  ────────────
  foreach() HASH kl16                    44.36         24.67         55.6%
  values() HASH kl16                     45.21         20.25         44.8%
  toArray() HASH kl16                    57.21         25.75         45.0%
  overwrite() HASH kl16                  42.30         40.26         95.2%
  increment() HASH kl16                  43.52         40.81         93.8%
    HASH kl16: honoured?          yes
  foreach() ADAPTIVE kl40                50.47         32.47         64.3%
  values() ADAPTIVE kl40                 46.25         27.38         59.2%
  toArray() ADAPTIVE kl40                57.86         35.09         60.6%
  overwrite() ADAPTIVE kl40              50.46         46.22         91.6%
    ADAPTIVE kl40: honoured?      yes
  skipped — increment() ADAPTIVE kl40: Judy::increment() is only supported for INT_TO_INT, STRING_TO_INT and STRING_TO_INT_HASH types

  Ratios are optimized as a percentage of default, measured inside this
  build; below 100 is the traversal win, above 100 the write price. Lower
  is better on every row, so a lost win reads as a rise.

── Advanced: Adaptive SSO — short keys (<8 bytes) via JudyL ─────────────────

  Operation                   STR_TO_INT    STR_INT_HASH   STR_INT_ADPTV    Best ratio
  ──────────────────────  ──────────────  ──────────────  ──────────────  ────────────
  insert (short keys)          44.26 ms       74.80 ms       70.10 ms       1.7x
  read (random access)         20.87 ms       16.75 ms       16.57 ms       1.3x
  foreach (iterate all)        29.12 ms       56.41 ms       62.69 ms       1.9x

── Advanced: Adaptive SSO — long keys (>=8 bytes) fallback to JudyHS ────────

  Operation                   STR_TO_INT    STR_INT_HASH   STR_INT_ADPTV    Best ratio
  ──────────────────────  ──────────────  ──────────────  ──────────────  ────────────
  insert (long keys)           75.54 ms      141.13 ms      141.78 ms       1.9x
  read (random access)         34.24 ms       29.46 ms       28.54 ms       1.2x
  foreach (iterate all)        77.02 ms       76.96 ms       72.01 ms       1.1x

══════════════════════════════════════════════════════════════════════════════════════════
  Benchmark complete — 2026-10-09 04:28:27
  All timings: median of 7 iterations via hrtime(true), 1 warmup run
  Memory: peak RSS of a child process building one structure of 500,000 elements,
          minus an empty-process floor. Measured with getrusage(), not
          memory_get_usage(), for the reason above. Judy's memory advantage is
          scale- and type-dependent — see BENCHMARK.md for the curve.
══════════════════════════════════════════════════════════════════════════════════════════

  JSON results written to: bench.json

@orieg
orieg merged commit 388c75b into main Oct 9, 2026
40 of 44 checks passed
@orieg
orieg deleted the perf/baseline-2.8.0 branch October 9, 2026 04:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@orieg