Skip to content

test: fix allocator-sensitive memory assertion that broke the valgrind nightly (#222) - #223

Merged
orieg merged 1 commit into
mainfrom
fix/nightly-valgrind-test
Oct 8, 2026
Merged

orieg merged 1 commit into
mainfrom
fix/nightly-valgrind-test

Conversation

@orieg

@orieg orieg commented Oct 8, 2026

Copy link
Copy Markdown
Owner

What

Fixes the failure behind issue #222: the nightly valgrind memcheck job failed its first post-#217 run on tests/string_to_entry_delete_range_001.phpt.

Root cause

The test's secondary assertion measured a memory_get_usage() delta around deleteRange() and required delta > 0:

$before = memory_get_usage();
$deleted = $j->deleteRange('k0000', 'k0999');
$delta = $before - memory_get_usage();   // "memory freed > 0: yes"

Under valgrind memcheck the delta was 0 (no(0)). Freed zvals land on the Zend MM freelist, so memory_get_usage() does not reliably drop; and the Judy trie plus the zvals it holds are invisible to memory_get_usage() either way (AGENTS.md, "Debugging and profiling Judy code"). This is the same trap the repo documents — the test happened to be stable on the non-valgrind PR runs, then broke under the nightly's memcheck allocator.

The primary leak assertion is what actually matters and it passed: __destruct fired: 1000/1000 (the pre-#217 bug leaked the entry struct + refcount and never ran the destructor). Valgrind itself reported 0 definite leaks.

Fix

Swap the secondary measurement to Judy::memoryUsage() — the extension's own string-keyed accounting (approx_payload_bytes), which deleteRange() decrements by every confirmed removal and which is independent of the allocator:

  • deterministic: 38000 -> 0 on this test (1000 entries)
  • unaffected by valgrind (pure internal counter, not a memory read)
  • aligns with the repo's documented guidance to use memoryUsage() over memory_get_usage() for Judy memory

The test identity (--TEST-- title and file name) is unchanged, so the test-floor ratchet baseline in reports/junit.xml still matches exactly (verified: 258/258 IDs identical, 0 dropped, 0 added).

Validation

  • Modified test passes locally (make test TESTS=tests/string_to_entry_delete_range_001.phpt)
  • Full local suite: 258/258 pass, JUnit IDs identical to the committed reports/junit.xml baseline
  • Local v0.18.0 discipline check --base origin/main: PASS (27 gates, 0 errors, 0 warnings)

…d nightly

The secondary assertion in tests/string_to_entry_delete_range_001.phpt
measured a memory_get_usage() delta around deleteRange() and required it
to be > 0. Under valgrind memcheck the delta came out 0: freed zvals land
on the Zend MM freelist, and memory_get_usage() does not reliably drop,
while the Judy trie (and its held zvals) is invisible to it entirely.
The nightly valgrind job failed its first post-#217 run on this (issue
#222), even though the primary leak assertion -- __destruct fires 1000/1000
-- passed and valgrind reported 0 definite leaks.

Replace the reading with Judy::memoryUsage(), the extension's own
string-keyed accounting (approx_payload_bytes), which deleteRange()
decrements by every confirmed removal and which is independent of the
allocator: 38000 -> 0 deterministically on this test. Test identity
(--TEST-- title, file name) is unchanged, so the test-floor ratchet
baseline in reports/junit.xml still matches.
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown

discipline: passed

Base main (origin/main) (merge base 7d896385c1) · 27 gate(s) passed · 2163 item(s) examined · 0 warning(s) · 0 override(s)

discipline explain <gate> prints what a gate checks and how a finding is lifted. This comment is updated on every run; the check's status is the verdict.

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown

Test Results

PHP Platform Arch TS Tests Pass Fail Skip Duration
✅ 8.1 Linux x64 - 258 258 0 0 7.2s
✅ 8.2 Linux x64 - 258 258 0 0 9.2s
✅ 8.3 Linux x64 - 258 258 0 0 9.8s
✅ 8.4 Linux x64 - 258 258 0 0 8.2s
✅ 8.5 Linux x64 - 258 258 0 0 10.6s
✅ 🧪 8.6 Linux x64 - 258 258 0 0 11.8s
✅ 8.1 Linux (Expanse) x64 - 258 258 0 0 10.0s
✅ 8.5 Linux (Expanse) x64 - 258 258 0 0 11.0s
✅ 8.1 Windows x64 nts 258 258 0 0 15.5s
✅ 8.5 Windows x64 nts 258 258 0 0 15.9s
✅ 8.1 Windows (Expanse) x64 nts 258 258 0 0 14.1s
✅ 8.5 Windows (Expanse) x64 nts 258 258 0 0 15.5s
✅ Total 3096 3096 0 0

Benchmark Results

500,000 elements, 7 iterations (median), Judy 2.7.1

Run Status

Platform PHP 8.1 PHP 8.2 PHP 8.3 PHP 8.4 PHP 8.5 PHP 8.6
Linux ✅ ✅ ✅ ✅ ✅ ✅
Windows ✅ — — — ✅ —

Memory (Linux, PHP 8.6)

Peak RSS of a child process building one structure, n = 500,000, minus an empty-process floor. Judy's memory advantage is type- and scale-dependent — see BENCHMARK.md for the curve across sizes on a dedicated host.

Type Judy PHP array Judy vs PHP array
Bitset 248.0 KB 8.1 MB 33.6x smaller
Int to Int 4.2 MB 8.0 MB 1.9x smaller
Int to Mixed 40.3 MB 38.1 MB 1.06x larger
Int to Packed 16.4 MB 38.2 MB 2.3x smaller
String to Int 10.3 MB 38.0 MB 3.7x smaller
String to Mixed 46.3 MB 68.1 MB 1.5x smaller
String to Int (Hash) 31.5 MB 38.2 MB 1.2x smaller
String to Mixed (Hash) 67.6 MB 68.0 MB about the same
String to Int (Adaptive) 31.5 MB 38.1 MB 1.2x smaller
String to Mixed (Adaptive) 67.5 MB 68.2 MB about the same

API Speedups (Linux, PHP 8.6)

Speedup = PHP time / Judy time. Bold = Judy faster.

Operation Judy PHP Speedup
putAll() 17.8 ms 23.9 ms 1.3x
fromArray() 17.5 ms 23.9 ms 1.4x
toArray() 17.5 ms 27.7 ms 1.6x
getAll() 0.88 ms 1.7 ms 1.9x
keys() 9.9 ms 20.9 ms 2.1x
values() 10.9 ms 20.7 ms 1.9x
sumValues() 7.2 ms 17.2 ms 2.4x
populationCount() 0.00 ms 19.4 ms O(1) vs O(n)
deleteRange() 31.7 ms 45.2 ms 1.4x
equals() 9.4 ms 38.1 ms 4.1x
union() Bitset 26.0 ms 3.5 ms 0.1x
intersect() Bitset 11.5 ms 4.2 ms 0.4x
diff() Bitset 10.9 ms 2.6 ms 0.2x
xor() Bitset 21.8 ms 8.8 ms 0.4x
mergeWith() Int→Int 43.0 ms 54.5 ms 1.3x
mergeWith() Str→Int 188.6 ms 201.7 ms 1.1x
forEach() Int→Int 21.8 ms 19.4 ms 0.9x
filter() Int→Int 33.4 ms 31.9 ms ~1.0x
map() Int→Int 37.2 ms 40.0 ms 1.1x
forEach() Str→Int 50.2 ms 48.2 ms ~1.0x
filter() Str→Int 95.3 ms 89.9 ms 0.9x
map() Str→Int 121.4 ms 123.2 ms ~1.0x

O(1) vs O(n) marks a row whose ratio is decided by the complexity classes rather than measured: populationCount() reads libJudy's cached population, the PHP arm counts every element. It scales with n and says nothing about this change. judy-bench.php declines the same comparison for size($lo, $hi) for the same reason.

Core Types (Linux, PHP 8.6)

Speedup = PHP time / Judy time. Bold = Judy faster.

Type Op Judy PHP array Speedup
Bitset Write 15.1 ms 5.8 ms 0.4x
Bitset Read 9.8 ms 3.8 ms 0.4x
Bitset Iter 16.4 ms 2.6 ms 0.2x
Bitset Free 0.01 ms 0.29 ms 42.3x
Int to Int Write 20.6 ms 5.8 ms 0.3x
Int to Int Read 9.6 ms 3.7 ms 0.4x
Int to Int Iter 17.1 ms 2.6 ms 0.2x
Int to Int Free 0.62 ms 0.28 ms 0.5x
Int to Mixed Write 30.6 ms 10.0 ms 0.3x
Int to Mixed Read 11.2 ms 5.4 ms 0.5x
Int to Mixed Iter 17.5 ms 3.4 ms 0.2x
Int to Mixed Free 14.8 ms 4.8 ms 0.3x
String to Int Write 72.7 ms 13.3 ms 0.2x
String to Int Read 32.4 ms 6.6 ms 0.2x
String to Int Iter 45.4 ms 3.8 ms 0.1x
String to Int Free 15.7 ms 1.0 ms 0.1x
String to Int (Adaptive) Write 140.0 ms 13.8 ms 0.1x
String to Int (Adaptive) Read 21.1 ms 6.1 ms 0.3x
String to Int (Adaptive) Iter 56.2 ms 3.8 ms 0.1x
String to Int (Adaptive) Free 39.6 ms 0.99 ms 0.0x
All types detail

Core Types (Linux, PHP 8.6) — All

Speedup = PHP time / Judy time. Bold = Judy faster.

Type Op Judy PHP array Speedup
Bitset Write 15.1 ms 5.8 ms 0.4x
Bitset Read 9.8 ms 3.8 ms 0.4x
Bitset Iter 16.4 ms 2.6 ms 0.2x
Bitset Free 0.01 ms 0.29 ms 42.3x
Int to Int Write 20.6 ms 5.8 ms 0.3x
Int to Int Read 9.6 ms 3.7 ms 0.4x
Int to Int Iter 17.1 ms 2.6 ms 0.2x
Int to Int Free 0.62 ms 0.28 ms 0.5x
Int to Mixed Write 30.6 ms 10.0 ms 0.3x
Int to Mixed Read 11.2 ms 5.4 ms 0.5x
Int to Mixed Iter 17.5 ms 3.4 ms 0.2x
Int to Mixed Free 14.8 ms 4.8 ms 0.3x
Int to Packed Write 38.2 ms 8.6 ms 0.2x
Int to Packed Read 22.7 ms 5.3 ms 0.2x
Int to Packed Iter 30.4 ms 3.5 ms 0.1x
Int to Packed Free 8.4 ms 5.0 ms 0.6x
String to Int Write 72.7 ms 13.3 ms 0.2x
String to Int Read 32.4 ms 6.6 ms 0.2x
String to Int Iter 45.4 ms 3.8 ms 0.1x
String to Int Free 15.7 ms 1.0 ms 0.1x
String to Mixed Write 93.8 ms 17.2 ms 0.2x
String to Mixed Read 34.3 ms 9.4 ms 0.3x
String to Mixed Iter 60.1 ms 6.1 ms 0.1x
String to Mixed Free 58.3 ms 7.1 ms 0.1x
String to Int (Hash) Write 137.5 ms 13.6 ms 0.1x
String to Int (Hash) Read 21.4 ms 5.8 ms 0.3x
String to Int (Hash) Iter 56.7 ms 3.8 ms 0.1x
String to Int (Hash) Free 39.4 ms 0.99 ms 0.0x
String to Mixed (Hash) Write 209.1 ms 17.1 ms 0.1x
String to Mixed (Hash) Read 23.6 ms 8.5 ms 0.4x
String to Mixed (Hash) Iter 85.5 ms 5.7 ms 0.1x
String to Mixed (Hash) Free 115.2 ms 7.5 ms 0.1x
String to Int (Adaptive) Write 140.0 ms 13.8 ms 0.1x
String to Int (Adaptive) Read 21.1 ms 6.1 ms 0.3x
String to Int (Adaptive) Iter 56.2 ms 3.8 ms 0.1x
String to Int (Adaptive) Free 39.6 ms 0.99 ms 0.0x
String to Mixed (Adaptive) Write 218.1 ms 17.1 ms 0.1x
String to Mixed (Adaptive) Read 23.4 ms 8.7 ms 0.4x
String to Mixed (Adaptive) Iter 101.4 ms 5.9 ms 0.1x
String to Mixed (Adaptive) Free 121.0 ms 7.4 ms 0.1x
Cross-version detail (Linux)

Core Types — Linux

Each cell: Judy ms (speedup vs PHP array).

Type Op PHP 8.1 PHP 8.2 PHP 8.3 PHP 8.4 PHP 8.5 PHP 8.6
Bitset Write 9.2 ms (0.5x) 15.3 ms (0.4x) 18.0 ms (0.4x) 9.6 ms (0.4x) 17.4 ms (0.4x) 15.1 ms (0.4x)
Bitset Read 6.3 ms (0.4x) 9.1 ms (0.4x) 11.3 ms (0.4x) 5.3 ms (0.4x) 10.4 ms (0.4x) 9.8 ms (0.4x)
Int to Int Write 13.8 ms (0.4x) 18.5 ms (0.3x) 25.3 ms (0.3x) 13.1 ms (0.3x) 23.8 ms (0.3x) 20.6 ms (0.3x)
Int to Int Read 11.6 ms (0.2x) 8.5 ms (0.4x) 11.8 ms (0.3x) 6.3 ms (0.3x) 11.1 ms (0.4x) 9.6 ms (0.4x)
Int to Mixed Write 19.9 ms (0.4x) 27.7 ms (0.3x) 34.0 ms (0.3x) 17.5 ms (0.4x) 35.6 ms (0.3x) 30.6 ms (0.3x)
Int to Mixed Read 7.9 ms (0.5x) 11.1 ms (0.4x) 14.1 ms (0.4x) 7.1 ms (0.4x) 14.0 ms (0.4x) 11.2 ms (0.5x)
Int to Packed Write 24.9 ms (0.3x) 35.5 ms (0.3x) 45.5 ms (0.2x) 24.1 ms (0.3x) 46.9 ms (0.2x) 38.2 ms (0.2x)
Int to Packed Read 15.5 ms (0.2x) 21.9 ms (0.2x) 30.4 ms (0.2x) 15.7 ms (0.2x) 29.3 ms (0.2x) 22.7 ms (0.2x)
String to Int Write 56.8 ms (0.2x) 71.5 ms (0.2x) 91.5 ms (0.2x) 56.0 ms (0.2x) 91.4 ms (0.2x) 72.7 ms (0.2x)
String to Int Read 25.4 ms (0.2x) 31.7 ms (0.2x) 36.6 ms (0.2x) 25.0 ms (0.2x) 36.9 ms (0.2x) 32.4 ms (0.2x)
String to Mixed Write 76.7 ms (0.2x) 94.8 ms (0.2x) 113.4 ms (0.2x) 72.9 ms (0.2x) 113.6 ms (0.2x) 93.8 ms (0.2x)
String to Mixed Read 28.8 ms (0.2x) 34.5 ms (0.3x) 40.4 ms (0.2x) 27.2 ms (0.2x) 41.0 ms (0.2x) 34.3 ms (0.3x)
String to Int (Hash) Write 115.2 ms (0.1x) 135.7 ms (0.1x) 161.0 ms (0.1x) 95.1 ms (0.1x) 157.4 ms (0.1x) 137.5 ms (0.1x)
String to Int (Hash) Read 17.7 ms (0.3x) 20.6 ms (0.3x) 24.8 ms (0.3x) 15.8 ms (0.3x) 25.0 ms (0.3x) 21.4 ms (0.3x)
String to Mixed (Hash) Write 163.9 ms (0.1x) 204.4 ms (0.1x) 226.6 ms (0.1x) 154.6 ms (0.1x) 230.9 ms (0.1x) 209.1 ms (0.1x)
String to Mixed (Hash) Read 19.0 ms (0.3x) 23.8 ms (0.4x) 27.8 ms (0.4x) 17.3 ms (0.4x) 28.3 ms (0.4x) 23.6 ms (0.4x)
String to Int (Adaptive) Write 112.0 ms (0.1x) 136.6 ms (0.1x) 160.9 ms (0.1x) 94.4 ms (0.1x) 169.3 ms (0.1x) 140.0 ms (0.1x)
String to Int (Adaptive) Read 16.0 ms (0.3x) 20.0 ms (0.3x) 24.4 ms (0.3x) 15.4 ms (0.3x) 24.7 ms (0.3x) 21.1 ms (0.3x)
String to Mixed (Adaptive) Write 178.2 ms (0.1x) 211.5 ms (0.1x) 227.3 ms (0.1x) 159.4 ms (0.1x) 256.2 ms (0.1x) 218.1 ms (0.1x)
String to Mixed (Adaptive) Read 18.5 ms (0.3x) 23.0 ms (0.4x) 27.2 ms (0.4x) 16.5 ms (0.4x) 27.9 ms (0.4x) 23.4 ms (0.4x)

API — Linux

Operation PHP 8.1 PHP 8.2 PHP 8.3 PHP 8.4 PHP 8.5 PHP 8.6
putAll() 11.6 ms (1.4x) 16.2 ms (1.4x) 19.8 ms (1.5x) 11.1 ms (1.5x) 19.3 ms (1.4x) 17.8 ms (1.3x)
fromArray() 11.5 ms (1.4x) 16.5 ms (1.3x) 20.0 ms (1.5x) 11.2 ms (1.5x) 19.3 ms (1.4x) 17.5 ms (1.4x)
toArray() 14.0 ms (1.6x) 21.1 ms (1.5x) 22.5 ms (1.6x) 12.3 ms (1.7x) 26.5 ms (1.5x) 17.5 ms (1.6x)
getAll() 0.58 ms (2.0x) 0.82 ms (2.0x) 1.1 ms (2.2x) 0.57 ms (2.0x) 1.1 ms (1.8x) 0.88 ms (1.9x)
keys() 6.9 ms (2.2x) 9.6 ms (2.2x) 10.7 ms (2.3x) 5.4 ms (2.1x) 11.9 ms (2.2x) 9.9 ms (2.1x)
values() 8.9 ms (1.7x) 10.8 ms (2.0x) 11.2 ms (2.1x) 5.5 ms (2.0x) 12.9 ms (2.0x) 10.9 ms (1.9x)
sumValues() 3.8 ms (2.5x) 6.7 ms (2.6x) 6.9 ms (2.8x) 3.2 ms (2.7x) 7.7 ms (2.6x) 7.2 ms (2.4x)
populationCount() 0.00 ms (99945x) 0.00 ms (199257x) 0.00 ms (204571x) 0.00 ms (91173x) 0.00 ms (219349x) 0.00 ms (193563x)
deleteRange() 21.7 ms (1.4x) 30.1 ms (1.4x) 38.2 ms (1.5x) 20.8 ms (1.4x) 39.4 ms (1.5x) 31.7 ms (1.4x)
equals() 7.1 ms (3.6x) 8.7 ms (4.3x) 10.5 ms (4.5x) 5.4 ms (4.4x) 11.8 ms (4.2x) 9.4 ms (4.1x)
union() Bitset 15.1 ms (0.3x) 23.8 ms (0.2x) 31.7 ms (0.2x) 15.3 ms (0.2x) 31.4 ms (0.2x) 26.0 ms (0.1x)
intersect() Bitset 6.8 ms (0.4x) 10.2 ms (0.4x) 13.7 ms (0.4x) 6.5 ms (0.5x) 13.2 ms (0.4x) 11.5 ms (0.4x)
diff() Bitset 6.6 ms (0.3x) 9.7 ms (0.3x) 13.7 ms (0.2x) 6.6 ms (0.3x) 13.0 ms (0.2x) 10.9 ms (0.2x)
xor() Bitset 13.3 ms (0.7x) 19.1 ms (0.5x) 29.6 ms (0.4x) 13.1 ms (0.5x) 26.5 ms (0.4x) 21.8 ms (0.4x)
mergeWith() Int→Int 27.0 ms (1.3x) 39.0 ms (1.3x) 53.9 ms (1.3x) 26.8 ms (1.3x) 48.6 ms (1.3x) 43.0 ms (1.3x)
mergeWith() Str→Int 146.9 ms (1.1x) 181.9 ms (1.1x) 232.5 ms (1.1x) 139.7 ms (1.1x) 237.1 ms (1.1x) 188.6 ms (1.1x)
forEach() Int→Int 14.2 ms (0.9x) 21.4 ms (0.9x) 28.3 ms (0.8x) 12.7 ms (0.8x) 28.1 ms (0.9x) 21.8 ms (0.9x)
filter() Int→Int 22.2 ms (0.9x) 32.5 ms (0.9x) 42.1 ms (0.9x) 19.9 ms (0.9x) 45.1 ms (0.9x) 33.4 ms (~1.0x)
map() Int→Int 25.0 ms (~1.0x) 36.5 ms (~1.0x) 49.5 ms (0.9x) 21.8 ms (~1.0x) 47.2 ms (~1.0x) 37.2 ms (1.1x)
forEach() Str→Int 38.7 ms (0.9x) 50.6 ms (0.9x) 66.1 ms (0.9x) 38.0 ms (0.9x) 65.6 ms (0.9x) 50.2 ms (~1.0x)
filter() Str→Int 83.6 ms (0.9x) 94.8 ms (~1.0x) 119.5 ms (0.9x) 72.4 ms (0.9x) 123.9 ms (0.9x) 95.3 ms (0.9x)
map() Str→Int 104.5 ms (~1.0x) 123.9 ms (~1.0x) 162.0 ms (~1.0x) 96.3 ms (~1.0x) 166.1 ms (~1.0x) 121.4 ms (~1.0x)
Cross-version detail (Windows)

Core Types — Windows

Each cell: Judy ms (speedup vs PHP array).

Type Op PHP 8.1 PHP 8.5
Bitset Write 22.9 ms (0.6x) 22.3 ms (0.5x)
Bitset Read 17.0 ms (0.5x) 18.1 ms (0.5x)
Int to Int Write 34.4 ms (0.4x) 35.7 ms (0.3x)
Int to Int Read 18.3 ms (0.5x) 18.6 ms (0.5x)
Int to Mixed Write 47.3 ms (0.5x) 51.2 ms (0.3x)
Int to Mixed Read 21.8 ms (0.5x) 20.3 ms (0.5x)
Int to Packed Write 67.8 ms (0.3x) 68.9 ms (0.2x)
Int to Packed Read 49.3 ms (0.2x) 48.1 ms (0.2x)
String to Int Write 148.0 ms (0.2x) 143.0 ms (0.2x)
String to Int Read 67.8 ms (0.2x) 61.0 ms (0.2x)
String to Mixed Write 191.3 ms (0.2x) 186.4 ms (0.2x)
String to Mixed Read 68.9 ms (0.3x) 63.5 ms (0.2x)
String to Int (Hash) Write 263.4 ms (0.1x) 249.4 ms (0.1x)
String to Int (Hash) Read 47.7 ms (0.3x) 44.5 ms (0.3x)
String to Mixed (Hash) Write 377.1 ms (0.1x) 361.8 ms (0.1x)
String to Mixed (Hash) Read 50.7 ms (0.3x) 48.7 ms (0.3x)
String to Int (Adaptive) Write 259.6 ms (0.1x) 244.8 ms (0.1x)
String to Int (Adaptive) Read 49.1 ms (0.2x) 42.9 ms (0.3x)
String to Mixed (Adaptive) Write 399.8 ms (0.1x) 371.4 ms (0.1x)
String to Mixed (Adaptive) Read 50.7 ms (0.3x) 48.6 ms (0.3x)

API — Windows

Operation PHP 8.1 PHP 8.5
putAll() 31.1 ms (1.7x) 32.0 ms (1.5x)
fromArray() 31.4 ms (1.7x) 31.5 ms (1.5x)
toArray() 35.1 ms (1.4x) 28.9 ms (1.7x)
getAll() 1.4 ms (2.2x) 1.6 ms (1.9x)
keys() 16.6 ms (2.2x) 12.7 ms (2.5x)
values() 17.1 ms (2.1x) 13.3 ms (2.3x)
sumValues() 7.7 ms (3.1x) 7.0 ms (3.6x)
populationCount() 0.00 ms (256486x) 0.00 ms (130109x)
deleteRange() 52.2 ms (1.5x) 57.7 ms (1.4x)
equals() 12.0 ms (5.3x) 11.2 ms (5.5x)
union() Bitset 36.0 ms (0.4x) 37.1 ms (0.3x)
intersect() Bitset 16.3 ms (0.5x) 17.4 ms (0.5x)
diff() Bitset 15.5 ms (0.4x) 16.9 ms (0.3x)
xor() Bitset 31.1 ms (0.8x) 34.4 ms (0.5x)
mergeWith() Int→Int 64.0 ms (1.3x) 65.7 ms (1.3x)
mergeWith() Str→Int 321.0 ms (1.1x) 341.0 ms (1.1x)
forEach() Int→Int 29.2 ms (~1.0x) 35.8 ms (0.8x)
filter() Int→Int 43.8 ms (1.1x) 47.4 ms (~1.0x)
map() Int→Int 49.7 ms (1.2x) 54.3 ms (1.1x)
forEach() Str→Int 79.1 ms (~1.0x) 85.9 ms (~1.0x)
filter() Str→Int 153.1 ms (~1.0x) 160.7 ms (~1.0x)
map() Str→Int 214.6 ms (~1.0x) 225.8 ms (~1.0x)
Windows results (PHP 8.5)

Memory (Windows, PHP 8.5)

Peak RSS of a child process building one structure, n = 500,000, minus an empty-process floor. Judy's memory advantage is type- and scale-dependent — see BENCHMARK.md for the curve across sizes on a dedicated host.

Type Judy PHP array Judy vs PHP array
Bitset 156.0 KB 9.3 MB 60.8x smaller
Int to Int 4.4 MB 9.3 MB 2.1x smaller
Int to Mixed 42.6 MB 38.2 MB 1.11x larger
Int to Packed 17.2 MB 38.2 MB 2.2x smaller
String to Int 10.6 MB 38.0 MB 3.6x smaller
String to Mixed 48.7 MB 69.0 MB 1.4x smaller
String to Int (Hash) 32.1 MB 38.0 MB 1.2x smaller
String to Mixed (Hash) 70.2 MB 69.0 MB about the same
String to Int (Adaptive) 32.1 MB 38.0 MB 1.2x smaller
String to Mixed (Adaptive) 70.2 MB 69.0 MB about the same

API Speedups (Windows, PHP 8.5)

Speedup = PHP time / Judy time. Bold = Judy faster.

Operation Judy PHP Speedup
putAll() 32.0 ms 46.5 ms 1.5x
fromArray() 31.5 ms 46.5 ms 1.5x
toArray() 28.9 ms 48.0 ms 1.7x
getAll() 1.6 ms 3.1 ms 1.9x
keys() 12.7 ms 31.9 ms 2.5x
values() 13.3 ms 31.0 ms 2.3x
sumValues() 7.0 ms 25.4 ms 3.6x
populationCount() 0.00 ms 26.0 ms O(1) vs O(n)
deleteRange() 57.7 ms 81.9 ms 1.4x
equals() 11.2 ms 61.2 ms 5.5x
union() Bitset 37.1 ms 9.3 ms 0.3x
intersect() Bitset 17.4 ms 8.2 ms 0.5x
diff() Bitset 16.9 ms 4.4 ms 0.3x
xor() Bitset 34.4 ms 17.9 ms 0.5x
mergeWith() Int→Int 65.7 ms 85.8 ms 1.3x
mergeWith() Str→Int 341.0 ms 369.0 ms 1.1x
forEach() Int→Int 35.8 ms 29.0 ms 0.8x
filter() Int→Int 47.4 ms 48.7 ms ~1.0x
map() Int→Int 54.3 ms 61.6 ms 1.1x
forEach() Str→Int 85.9 ms 81.9 ms ~1.0x
filter() Str→Int 160.7 ms 158.4 ms ~1.0x
map() Str→Int 225.8 ms 223.6 ms ~1.0x

O(1) vs O(n) marks a row whose ratio is decided by the complexity classes rather than measured: populationCount() reads libJudy's cached population, the PHP arm counts every element. It scales with n and says nothing about this change. judy-bench.php declines the same comparison for size($lo, $hi) for the same reason.

Core Types (Windows, PHP 8.5)

Speedup = PHP time / Judy time. Bold = Judy faster.

Type Op Judy PHP array Speedup
Bitset Write 22.3 ms 12.2 ms 0.5x
Bitset Read 18.1 ms 8.3 ms 0.5x
Bitset Iter 24.7 ms 7.0 ms 0.3x
Bitset Free 0.01 ms 0.69 ms 98.6x
Int to Int Write 35.7 ms 12.0 ms 0.3x
Int to Int Read 18.6 ms 8.7 ms 0.5x
Int to Int Iter 23.5 ms 6.9 ms 0.3x
Int to Int Free 0.59 ms 0.69 ms 1.2x
Int to Mixed Write 51.2 ms 16.0 ms 0.3x
Int to Mixed Read 20.3 ms 9.7 ms 0.5x
Int to Mixed Iter 24.4 ms 7.5 ms 0.3x
Int to Mixed Free 18.4 ms 4.7 ms 0.3x
String to Int Write 143.0 ms 25.4 ms 0.2x
String to Int Read 61.0 ms 12.0 ms 0.2x
String to Int Iter 75.7 ms 8.1 ms 0.1x
String to Int Free 30.7 ms 1.9 ms 0.1x
String to Int (Adaptive) Write 244.8 ms 25.1 ms 0.1x
String to Int (Adaptive) Read 42.9 ms 11.7 ms 0.3x
String to Int (Adaptive) Iter 108.7 ms 8.1 ms 0.1x
String to Int (Adaptive) Free 60.3 ms 1.9 ms 0.0x
All types detail

Core Types (Windows, PHP 8.5) — All

Speedup = PHP time / Judy time. Bold = Judy faster.

Type Op Judy PHP array Speedup
Bitset Write 22.3 ms 12.2 ms 0.5x
Bitset Read 18.1 ms 8.3 ms 0.5x
Bitset Iter 24.7 ms 7.0 ms 0.3x
Bitset Free 0.01 ms 0.69 ms 98.6x
Int to Int Write 35.7 ms 12.0 ms 0.3x
Int to Int Read 18.6 ms 8.7 ms 0.5x
Int to Int Iter 23.5 ms 6.9 ms 0.3x
Int to Int Free 0.59 ms 0.69 ms 1.2x
Int to Mixed Write 51.2 ms 16.0 ms 0.3x
Int to Mixed Read 20.3 ms 9.7 ms 0.5x
Int to Mixed Iter 24.4 ms 7.5 ms 0.3x
Int to Mixed Free 18.4 ms 4.7 ms 0.3x
Int to Packed Write 68.9 ms 15.9 ms 0.2x
Int to Packed Read 48.1 ms 9.5 ms 0.2x
Int to Packed Iter 53.0 ms 7.4 ms 0.1x
Int to Packed Free 13.4 ms 5.0 ms 0.4x
String to Int Write 143.0 ms 25.4 ms 0.2x
String to Int Read 61.0 ms 12.0 ms 0.2x
String to Int Iter 75.7 ms 8.1 ms 0.1x
String to Int Free 30.7 ms 1.9 ms 0.1x
String to Mixed Write 186.4 ms 28.9 ms 0.2x
String to Mixed Read 63.5 ms 14.8 ms 0.2x
String to Mixed Iter 89.8 ms 9.4 ms 0.1x
String to Mixed Free 98.6 ms 8.7 ms 0.1x
String to Int (Hash) Write 249.4 ms 25.0 ms 0.1x
String to Int (Hash) Read 44.5 ms 12.3 ms 0.3x
String to Int (Hash) Iter 109.2 ms 7.6 ms 0.1x
String to Int (Hash) Free 62.4 ms 1.9 ms 0.0x
String to Mixed (Hash) Write 361.8 ms 28.8 ms 0.1x
String to Mixed (Hash) Read 48.7 ms 14.4 ms 0.3x
String to Mixed (Hash) Iter 143.4 ms 9.2 ms 0.1x
String to Mixed (Hash) Free 186.7 ms 8.2 ms 0.0x
String to Int (Adaptive) Write 244.8 ms 25.1 ms 0.1x
String to Int (Adaptive) Read 42.9 ms 11.7 ms 0.3x
String to Int (Adaptive) Iter 108.7 ms 8.1 ms 0.1x
String to Int (Adaptive) Free 60.3 ms 1.9 ms 0.0x
String to Mixed (Adaptive) Write 371.4 ms 29.1 ms 0.1x
String to Mixed (Adaptive) Read 48.6 ms 15.2 ms 0.3x
String to Mixed (Adaptive) Iter 151.6 ms 9.0 ms 0.1x
String to Mixed (Adaptive) Free 199.8 ms 8.2 ms 0.0x

Release Comparison (2.6.0 → 2.7.1)

Interleaved A/B comparison against v2.6.0 (pie install) on the same CI runner: 5 benchmark groups × 5 rounds × 2 arms, 300,000 elements, median of 3 per child. Each row is the median of the paired current/baseline ratios with a 95% percentile-bootstrap CI; it is flagged only when the whole drift-adjusted CI clears ±10%. Advisory — benchmarks never gate a merge.

Run-wide median delta +0.59% (PHP-array control +0.20%); runner common-mode +0.20% divided out of every row.

Summary: 0 regressions, 83 unchanged, 17 not evaluated (unstable)

Benchmark Baseline Current Delta 95% CI Status
adv.filter.int_to_int 13.0 ms 13.1 ms -0.7% [-4.6%, +4.1%] ~same
adv.forEach.int_to_int 8.1 ms 7.8 ms -4.2% [-9.6%, -0.5%] ~same
adv.forEach.string_to_int 22.1 ms 22.1 ms -0.3% [-9.9%, +14.2%] ~same
adv.map.int_to_int 13.9 ms 13.4 ms -3.7% [-7.8%, -1.1%] ~same
adv.map.string_to_int 55.8 ms 56.8 ms +0.9% [-4.6%, +15.9%] ~same
adv.optiter.foreach.string_to_int_adaptive.default 41.3 ms 43.9 ms +3.7% [-16.5%, +12.5%] ~same
adv.optiter.foreach.string_to_int_hash.optimized 23.3 ms 24.9 ms +7.1% [-2.0%, +14.6%] ~same
adv.optiter.overwrite.string_to_int_adaptive.optimized 35.2 ms 35.5 ms -0.5% [-5.5%, +6.7%] ~same
adv.optiter.ratio.overwrite.string_to_int_adaptive 95.7 ms 94.1 ms -1.9% [-27.5%, +13.1%] ~same
adv.optiter.ratio.overwrite.string_to_int_hash 114.3 ms 100.4 ms -12.2% [-21.8%, +9.4%] ~same
adv.optiter.ratio.toArray.string_to_int_adaptive 68.2 ms 67.2 ms -2.3% [-3.2%, +9.8%] ~same
adv.optiter.ratio.toArray.string_to_int_hash 59.9 ms 57.1 ms -1.7% [-21.0%, +2.3%] ~same
adv.optiter.ratio.values.string_to_int_adaptive 67.5 ms 65.1 ms -2.8% [-16.8%, +9.6%] ~same
adv.optiter.ratio.values.string_to_int_hash 60.5 ms 59.1 ms -4.0% [-18.5%, +14.9%] ~same
adv.optiter.toArray.string_to_int_adaptive.default 50.9 ms 51.6 ms -0.1% [-4.7%, +2.8%] ~same
adv.optiter.toArray.string_to_int_adaptive.optimized 34.9 ms 34.7 ms -0.2% [-2.2%, +9.3%] ~same
adv.optiter.toArray.string_to_int_hash.optimized 24.9 ms 24.9 ms +0.5% [+0.1%, +4.0%] ~same
adv.optiter.values.string_to_int_adaptive.default 36.9 ms 38.4 ms -4.7% [-10.2%, +26.9%] ~same
adv.optiter.values.string_to_int_adaptive.optimized 25.0 ms 24.6 ms -1.8% [-7.1%, +11.2%] ~same
adv.optiter.values.string_to_int_hash.optimized 18.7 ms 19.3 ms +3.2% [-3.4%, +14.1%] ~same
api.deleteRange.int_to_int 12.5 ms 12.7 ms +1.7% [-3.6%, +5.3%] ~same
api.equals.int_to_int 3.4 ms 3.6 ms +3.8% [-0.5%, +15.4%] ~same
api.fromArray.int_to_int 7.9 ms 7.7 ms -0.4% [-2.3%, +0.5%] ~same
api.getAll.int_to_int 0.29 ms 0.28 ms -1.7% [-3.2%, +0.5%] ~same
api.increment.int_to_int 7.3 ms 7.4 ms +1.2% [-0.4%, +2.0%] ~same
api.keys.int_to_int 3.2 ms 3.2 ms +0.5% [-1.1%, +5.1%] ~same
api.mergeWith.int_to_int 15.9 ms 16.1 ms +1.6% [-3.9%, +4.0%] ~same
api.mergeWith.string_to_int 83.9 ms 84.9 ms +2.1% [-3.6%, +6.3%] ~same
api.populationCount.int_to_int 0.00 ms 0.00 ms -0.2% [-100.0%, -0.2%] ~same
api.putAll.int_to_int 7.8 ms 7.7 ms -2.0% [-5.7%, +3.7%] ~same
api.range.keys.int_to_int 0.27 ms 0.27 ms -0.3% [-2.2%, +2.5%] ~same
api.range.size.string_to_int 1.5 ms 1.5 ms +0.5% [-1.0%, +4.0%] ~same
api.range.size1000.int_to_int 0.43 ms 0.40 ms -5.0% [-6.0%, -3.6%] ~same
api.setop.diff.bitset 3.8 ms 3.9 ms +0.6% [-3.4%, +4.0%] ~same
api.setop.diff.int_to_int 5.0 ms 4.8 ms -3.6% [-8.6%, -0.7%] ~same
api.setop.diff.string_to_int 43.9 ms 42.8 ms +0.7% [-12.6%, +11.1%] ~same
api.setop.intersect.bitset 3.8 ms 4.0 ms +4.5% [+0.1%, +8.7%] ~same
api.setop.intersect.int_to_int 5.0 ms 4.8 ms -4.4% [-6.9%, -0.2%] ~same
api.setop.intersect.string_to_int 49.4 ms 46.8 ms -2.1% [-5.5%, +1.2%] ~same
api.setop.union.bitset 9.1 ms 9.5 ms +2.4% [-3.8%, +7.2%] ~same
api.setop.union.int_to_int 13.3 ms 12.9 ms -4.5% [-6.3%, -1.8%] ~same
api.setop.union.string_to_int 107.2 ms 105.7 ms +0.3% [-9.3%, +6.0%] ~same
api.setop.xor.bitset 7.7 ms 7.9 ms +3.2% [-2.1%, +4.1%] ~same
api.setop.xor.int_to_int 10.0 ms 9.8 ms -1.9% [-9.2%, +0.2%] ~same
api.sumValues.int_to_int 2.1 ms 2.1 ms -1.4% [-6.5%, +3.6%] ~same
api.toArray.int_to_int 8.4 ms 8.4 ms +0.6% [-4.8%, +4.0%] ~same
api.values.int_to_int 3.3 ms 3.5 ms +6.0% [-1.6%, +10.0%] ~same
core.bitset.free 0.00 ms 0.00 ms +23.6% [+4.5%, +47.3%] ~same
core.bitset.iter 5.2 ms 5.1 ms -1.4% [-6.0%, +2.6%] ~same
core.bitset.read 3.3 ms 3.3 ms -0.3% [-6.2%, +2.8%] ~same
core.bitset.write 5.6 ms 5.5 ms -0.7% [-6.0%, +3.3%] ~same
core.int_to_int.free 0.21 ms 0.20 ms -6.0% [-13.7%, +14.8%] ~same
core.int_to_int.write 8.0 ms 7.9 ms +1.8% [-4.6%, +4.4%] ~same
core.int_to_mixed.free 4.5 ms 4.5 ms 0.0% [-7.7%, +17.5%] ~same
core.int_to_mixed.iter 5.7 ms 5.6 ms -4.2% [-7.5%, -0.9%] ~same
core.int_to_mixed.read 4.3 ms 4.4 ms +3.5% [-8.5%, +12.3%] ~same
core.int_to_packed.free 2.6 ms 2.6 ms +0.4% [-3.2%, +1.0%] ~same
core.int_to_packed.iter 10.8 ms 10.9 ms +1.3% [-0.4%, +3.4%] ~same
core.int_to_packed.read 9.2 ms 9.7 ms +5.3% [+1.7%, +31.2%] ~same
core.string_to_int.free 7.0 ms 7.2 ms +1.6% [+0.0%, +6.9%] ~same
core.string_to_int.iter 19.4 ms 20.5 ms +5.6% [-2.5%, +6.2%] ~same
core.string_to_int.read 13.8 ms 14.3 ms +2.5% [+1.1%, +12.6%] ~same
core.string_to_int.write 30.4 ms 31.9 ms +3.7% [-1.1%, +8.7%] ~same
core.string_to_int_adaptive.free 16.3 ms 15.9 ms -3.0% [-6.4%, -2.1%] ~same
core.string_to_int_adaptive.iter 25.4 ms 25.1 ms -1.6% [-4.4%, +3.1%] ~same
core.string_to_int_adaptive.read 9.2 ms 9.5 ms +5.0% [-2.5%, +15.4%] ~same
core.string_to_int_adaptive.write 57.9 ms 56.0 ms -0.5% [-3.7%, +2.7%] ~same
core.string_to_int_hash.free 16.3 ms 16.7 ms -5.7% [-9.1%, +6.9%] ~same
core.string_to_int_hash.iter 25.7 ms 25.2 ms -2.1% [-5.5%, +4.8%] ~same
core.string_to_int_hash.read 9.9 ms 9.7 ms -2.0% [-5.1%, +7.3%] ~same
core.string_to_int_hash.write 61.3 ms 59.7 ms -2.8% [-5.9%, -1.1%] ~same
core.string_to_mixed.free 25.6 ms 26.2 ms +1.6% [-8.6%, +12.5%] ~same
core.string_to_mixed.iter 24.9 ms 25.5 ms +2.4% [-5.0%, +10.2%] ~same
core.string_to_mixed.read 16.0 ms 15.9 ms -0.9% [-4.4%, +2.0%] ~same
core.string_to_mixed.write 42.5 ms 43.3 ms +0.8% [-7.8%, +8.1%] ~same
core.string_to_mixed_adaptive.free 51.8 ms 53.0 ms +5.0% [-7.8%, +6.8%] ~same
core.string_to_mixed_adaptive.iter 36.0 ms 35.3 ms -2.1% [-3.1%, +2.2%] ~same
core.string_to_mixed_adaptive.read 11.0 ms 11.1 ms +0.6% [-3.8%, +3.3%] ~same
core.string_to_mixed_adaptive.write 97.4 ms 94.1 ms -2.8% [-9.5%, +3.6%] ~same
core.string_to_mixed_hash.free 47.4 ms 49.5 ms +2.4% [-8.6%, +5.1%] ~same
core.string_to_mixed_hash.iter 38.7 ms 38.9 ms -2.3% [-5.3%, +2.1%] ~same
core.string_to_mixed_hash.read 11.3 ms 11.4 ms +1.0% [-3.0%, +6.0%] ~same
core.string_to_mixed_hash.write 87.9 ms 88.9 ms +1.3% [-1.3%, +7.0%] ~same
17 benchmarks not evaluated (within-arm scatter exceeded the effect being claimed)
Benchmark Baseline Current Delta 95% CI Status
adv.filter.string_to_int 41.1 ms 42.2 ms +3.5% [-22.8%, +12.6%] — unstable
adv.optiter.foreach.string_to_int_adaptive.optimized 29.0 ms 31.1 ms 0.0% [-13.6%, +50.3%] — unstable
adv.optiter.foreach.string_to_int_hash.default 33.2 ms 35.7 ms +7.4% [-0.6%, +24.3%] — unstable
adv.optiter.increment.string_to_int_hash.default 22.0 ms 26.2 ms +8.1% [-22.1%, +25.9%] — unstable
adv.optiter.increment.string_to_int_hash.optimized 25.7 ms 27.3 ms +4.5% [-7.4%, +29.8%] — unstable
adv.optiter.overwrite.string_to_int_adaptive.default 37.3 ms 38.3 ms +1.2% [-16.6%, +34.9%] — unstable
adv.optiter.overwrite.string_to_int_hash.default 24.2 ms 27.8 ms +14.9% [-1.6%, +39.0%] — unstable
adv.optiter.overwrite.string_to_int_hash.optimized 26.3 ms 27.3 ms +7.8% [-3.9%, +22.3%] — unstable
adv.optiter.ratio.foreach.string_to_int_adaptive 70.4 ms 70.7 ms +3.3% [-9.0%, +33.3%] — unstable
adv.optiter.ratio.foreach.string_to_int_hash 71.8 ms 68.2 ms -5.1% [-8.0%, +7.6%] — unstable
adv.optiter.ratio.increment.string_to_int_hash 114.7 ms 104.2 ms -3.6% [-11.7%, +42.6%] — unstable
adv.optiter.toArray.string_to_int_hash.default 40.4 ms 43.7 ms +2.0% [-2.3%, +31.4%] — unstable
adv.optiter.values.string_to_int_hash.default 32.3 ms 32.7 ms +3.1% [-7.2%, +39.6%] — unstable
core.int_to_int.iter 5.5 ms 5.3 ms -8.1% [-44.1%, -0.1%] — unstable
core.int_to_int.read 3.7 ms 3.9 ms +10.4% [-2.7%, +73.8%] — unstable
core.int_to_mixed.write 11.2 ms 10.4 ms -7.1% [-36.7%, -3.9%] — unstable
core.int_to_packed.write 14.9 ms 14.5 ms -2.1% [-50.9%, +3.5%] — unstable

Memory Comparison

Memory: 0 regressions, 10 unchanged

Type Baseline Current Delta Status
Bitset 160 B 160 B 0.0% ~same
Int to Int 160 B 160 B 0.0% ~same
Int to Mixed 4.6 MB 4.6 MB 0.0% ~same
Int to Packed 8.0 MB 8.0 MB 0.0% ~same
String to Int 64.2 KB 64.2 KB 0.0% ~same
String to Int (Adaptive) 64.2 KB 64.2 KB 0.0% ~same
String to Int (Hash) 64.2 KB 64.2 KB 0.0% ~same
String to Mixed 4.6 MB 4.6 MB 0.0% ~same
String to Mixed (Adaptive) 4.6 MB 4.6 MB 0.0% ~same
String to Mixed (Hash) 4.6 MB 4.6 MB 0.0% ~same
Full benchmark output

Linux — PHP 8.6

══════════════════════════════════════════════════════════════════════════════════════════
  php-judy Benchmark Suite — 500,000 elements, 7 iterations (median)
  PHP 8.6.0-dev | Judy ext 2.7.1 | Linux x86_64
  Suite: all | Groups: core.int,core.str,api.batch,api.setops,adv.iter,adv.sso
  memory_limit: -1
  Memory: peak RSS of a child process, empty-process floor subtracted, median of 3
══════════════════════════════════════════════════════════════════════════════════════════

── Core: Integer-Keyed Types ────────────────────────────────────────────────

  Type                         Write(ms)      Read(ms)      Iter(ms)      Free(ms)        Memory
  ────────────────────────  ────────────  ────────────  ────────────  ────────────  ────────────
  BITSET                           15.06          9.82         16.43          0.01      248.0 KB
    PHP array                       5.75          3.76          2.61          0.29        8.1 MB
  INT_TO_INT                       20.61          9.58         17.09          0.62        4.2 MB
    PHP array                       5.81          3.69          2.61          0.28        8.0 MB
  INT_TO_MIXED                     30.64         11.18         17.48         14.79       40.3 MB
    PHP array                      10.00          5.39          3.41          4.85       38.1 MB
  INT_TO_PACKED                    38.20         22.74         30.41          8.37       16.4 MB
    PHP array                       8.57          5.26          3.46          5.02       38.2 MB

── Core: String-Keyed Types ────────────────────────────────────────────────

  Type                         Write(ms)      Read(ms)      Iter(ms)      Free(ms)        Memory
  ────────────────────────  ────────────  ────────────  ────────────  ────────────  ────────────
  STRING_TO_INT                    72.69         32.36         45.38         15.71       10.3 MB
    PHP array                      13.26          6.62          3.79          1.03       38.0 MB
  STRING_TO_MIXED                  93.83         34.26         60.11         58.30       46.3 MB
    PHP array                      17.20          9.36          6.06          7.14       68.1 MB
  STR_TO_INT_HASH                 137.52         21.40         56.72         39.44       31.5 MB
    PHP array                      13.61          5.80          3.78          0.99       38.2 MB
  STR_TO_MIX_HASH                 209.06         23.64         85.48        115.15       67.6 MB
    PHP array                      17.05          8.53          5.71          7.55       68.0 MB
  STR_INT_ADAPTIVE                139.97         21.07         56.15         39.57       31.5 MB
    PHP array                      13.83          6.08          3.77          0.99       38.1 MB
  STR_MIX_ADAPTIVE                218.08         23.40        101.44        120.97       67.5 MB
    PHP array                      17.11          8.69          5.87          7.41       68.2 MB

── API: Batch Operations (INT_TO_INT) ────────────────────────────────────────

  Operation                           Judy(ms)       PHP(ms)     Speedup
  ──────────────────────────────  ────────────  ────────────  ──────────
  putAll()                               17.80         23.93        1.3x
  fromArray()                            17.45         23.93        1.4x
  toArray()                              17.55         27.72        1.6x
  getAll(50100 keys)                      0.88          1.70        1.9x
  increment(500000 ops/1000 keys)         18.37          9.16        0.5x

── API: Native Methods (keys, values, sum, popCount, deleteRange, equals) ────

  Operation                           Judy(ms)       PHP(ms)     Speedup
  ──────────────────────────────  ────────────  ────────────  ──────────
  keys() INT_TO_INT                       9.85         20.90        2.1x
  values() INT_TO_INT                    10.93         20.74        1.9x
  keys($lo,$hi) INT_TO_INT                0.86         21.57       25.0x
    vs slice()->keys()                    2.49             -        2.9x
  size($lo,$hi) INT x1000                 0.77             -        O(1)
  size($lo,$hi) STRING_TO_INT             3.22          4.12        1.3x
  sumValues() INT_TO_INT                  7.22         17.18        2.4x
  populationCount() INT_TO_INT            0.00         19.36   241953.8x
  deleteRange() INT_TO_INT mid50%         31.67         45.22        1.4x
  equals() INT_TO_INT                     9.36         38.05        4.1x

── API: Set Operations (BITSET, 50% overlap) ────────────────────────────────

  Operation                           Judy(ms)       PHP(ms)     Speedup
  ──────────────────────────────  ────────────  ────────────  ──────────
  union() BITSET                         25.98          3.45        0.1x
  intersect() BITSET                     11.46          4.24        0.4x
  diff() BITSET                          10.92          2.57        0.2x
  xor() BITSET                           21.81          8.83        0.4x
  union() INT_TO_INT                     36.17         70.20        1.9x
  intersect() INT_TO_INT                 14.77         33.82        2.3x
  diff() INT_TO_INT                      14.61         34.70        2.4x
  xor() INT_TO_INT                       28.90         70.15        2.4x
  union() STRING_TO_INT                 227.30        252.50        1.1x
  intersect() STRING_TO_INT              99.22        115.65        1.2x
  diff() STRING_TO_INT                   95.29        112.12        1.2x
  mergeWith() INT_TO_INT                 43.02         54.53        1.3x
  mergeWith() STRING_TO_INT             188.62        201.68        1.1x

── Advanced: C-Level forEach/filter/map ──────────────────────────────────────

  Operation                        C-level(ms)       PHP(ms)     Speedup
  ──────────────────────────────  ────────────  ────────────  ──────────
  forEach() INT_TO_INT                   21.78         19.40        0.9x
  forEach() STRING_TO_INT                50.20         48.20        1.0x
  filter() INT_TO_INT (50% pass)         33.41         31.92        1.0x
  filter() STRING_TO_INT (50% pass)         95.29         89.89        0.9x
  map() INT_TO_INT (*2)                  37.21         40.03        1.1x
  map() STRING_TO_INT (*2)              121.44        123.19        1.0x

── Advanced: optimizeIteration trade (opt-in key-index payload mirror) ──────

  Operation                        default(ms)    optim.(ms)   optim./def.
  ──────────────────────────────  ────────────  ────────────  ────────────
  foreach() HASH kl16                    48.20         28.05         58.2%
  values() HASH kl16                     42.53         21.55         50.7%
  toArray() HASH kl16                    56.22         28.87         51.4%
  overwrite() HASH kl16                  47.23         43.90         92.9%
  increment() HASH kl16                  35.17         46.05        130.9%
    HASH kl16: honoured?          yes
  foreach() ADAPTIVE kl40                60.86         41.15         67.6%
  values() ADAPTIVE kl40                 61.99         33.45         54.0%
  toArray() ADAPTIVE kl40                65.14         44.42         68.2%
  overwrite() ADAPTIVE kl40              55.52         56.15        101.1%
    ADAPTIVE kl40: honoured?      yes
  skipped — increment() ADAPTIVE kl40: Judy::increment() is only supported for INT_TO_INT, STRING_TO_INT and STRING_TO_INT_HASH types

  Ratios are optimized as a percentage of default, measured inside this
  build; below 100 is the traversal win, above 100 the write price. Lower
  is better on every row, so a lost win reads as a rise.

── Advanced: Adaptive SSO — short keys (<8 bytes) via JudyL ─────────────────

  Operation                   STR_TO_INT    STR_INT_HASH   STR_INT_ADPTV    Best ratio
  ──────────────────────  ──────────────  ──────────────  ──────────────  ────────────
  insert (short keys)          62.18 ms       99.15 ms       96.17 ms       1.6x
  read (random access)         24.69 ms       21.44 ms       18.98 ms       1.3x
  foreach (iterate all)        42.91 ms       72.69 ms       70.56 ms       1.7x

── Advanced: Adaptive SSO — long keys (>=8 bytes) fallback to JudyHS ────────

  Operation                   STR_TO_INT    STR_INT_HASH   STR_INT_ADPTV    Best ratio
  ──────────────────────  ──────────────  ──────────────  ──────────────  ────────────
  insert (long keys)          103.63 ms      184.09 ms      183.80 ms       1.8x
  read (random access)         39.72 ms       32.48 ms       34.35 ms       1.2x
  foreach (iterate all)        78.94 ms       84.78 ms       86.48 ms       1.1x

══════════════════════════════════════════════════════════════════════════════════════════
  Benchmark complete — 2026-10-08 16:52:21
  All timings: median of 7 iterations via hrtime(true), 1 warmup run
  Memory: peak RSS of a child process building one structure of 500,000 elements,
          minus an empty-process floor. Measured with getrusage(), not
          memory_get_usage(), for the reason above. Judy's memory advantage is
          scale- and type-dependent — see BENCHMARK.md for the curve.
══════════════════════════════════════════════════════════════════════════════════════════

  JSON results written to: bench.json

Windows — PHP 8.5

══════════════════════════════════════════════════════════════════════════════════════════
  php-judy Benchmark Suite — 500,000 elements, 7 iterations (median)
  PHP 8.5.11 | Judy ext 2.7.1 | WINNT AMD64
  Suite: all | Groups: core.int,core.str,api.batch,api.setops,adv.iter,adv.sso
  memory_limit: 4G
  Memory: peak RSS of a child process, empty-process floor subtracted, median of 3
══════════════════════════════════════════════════════════════════════════════════════════

── Core: Integer-Keyed Types ────────────────────────────────────────────────

  Type                         Write(ms)      Read(ms)      Iter(ms)      Free(ms)        Memory
  ────────────────────────  ────────────  ────────────  ────────────  ────────────  ────────────
  BITSET                           22.31         18.07         24.65          0.01      156.0 KB
    PHP array                      12.19          8.33          7.00          0.69        9.3 MB
  INT_TO_INT                       35.70         18.59         23.50          0.59        4.4 MB
    PHP array                      12.04          8.70          6.90          0.69        9.3 MB
  INT_TO_MIXED                     51.16         20.33         24.38         18.39       42.6 MB
    PHP array                      15.97          9.69          7.47          4.72       38.2 MB
  INT_TO_PACKED                    68.86         48.13         53.02         13.40       17.2 MB
    PHP array                      15.93          9.53          7.38          5.05       38.2 MB

── Core: String-Keyed Types ────────────────────────────────────────────────

  Type                         Write(ms)      Read(ms)      Iter(ms)      Free(ms)        Memory
  ────────────────────────  ────────────  ────────────  ────────────  ────────────  ────────────
  STRING_TO_INT                   142.99         61.04         75.71         30.69       10.6 MB
    PHP array                      25.42         12.01          8.10          1.92       38.0 MB
  STRING_TO_MIXED                 186.43         63.52         89.75         98.61       48.7 MB
    PHP array                      28.93         14.78          9.42          8.68       69.0 MB
  STR_TO_INT_HASH                 249.35         44.48        109.24         62.39       32.1 MB
    PHP array                      25.04         12.29          7.64          1.92       38.0 MB
  STR_TO_MIX_HASH                 361.76         48.73        143.41        186.65       70.2 MB
    PHP array                      28.81         14.35          9.23          8.16       69.0 MB
  STR_INT_ADAPTIVE                244.79         42.85        108.68         60.29       32.1 MB
    PHP array                      25.13         11.73          8.08          1.93       38.0 MB
  STR_MIX_ADAPTIVE                371.35         48.56        151.59        199.83       70.2 MB
    PHP array                      29.10         15.23          9.04          8.24       69.0 MB

── API: Batch Operations (INT_TO_INT) ────────────────────────────────────────

  Operation                           Judy(ms)       PHP(ms)     Speedup
  ──────────────────────────────  ────────────  ────────────  ──────────
  putAll()                               32.03         46.54        1.5x
  fromArray()                            31.54         46.54        1.5x
  toArray()                              28.94         48.02        1.7x
  getAll(50100 keys)                      1.61          3.06        1.9x
  increment(500000 ops/1000 keys)         33.75         24.29        0.7x

── API: Native Methods (keys, values, sum, popCount, deleteRange, equals) ────

  Operation                           Judy(ms)       PHP(ms)     Speedup
  ──────────────────────────────  ────────────  ────────────  ──────────
  keys() INT_TO_INT                      12.70         31.94        2.5x
  values() INT_TO_INT                    13.32         31.03        2.3x
  keys($lo,$hi) INT_TO_INT                0.97         32.07       33.1x
    vs slice()->keys()                    3.39             -        3.5x
  size($lo,$hi) INT x1000                 0.74             -        O(1)
  size($lo,$hi) STRING_TO_INT             5.45          7.63        1.4x
  sumValues() INT_TO_INT                  7.03         25.38        3.6x
  populationCount() INT_TO_INT            0.00         26.02   130109.5x
  deleteRange() INT_TO_INT mid50%         57.72         81.88        1.4x
  equals() INT_TO_INT                    11.23         61.24        5.5x

── API: Set Operations (BITSET, 50% overlap) ────────────────────────────────

  Operation                           Judy(ms)       PHP(ms)     Speedup
  ──────────────────────────────  ────────────  ────────────  ──────────
  union() BITSET                         37.12          9.31        0.3x
  intersect() BITSET                     17.38          8.16        0.5x
  diff() BITSET                          16.86          4.43        0.3x
  xor() BITSET                           34.45         17.88        0.5x
  union() INT_TO_INT                     54.89        108.05        2.0x
  intersect() INT_TO_INT                 20.77         51.50        2.5x
  diff() INT_TO_INT                      19.82         56.76        2.9x
  xor() INT_TO_INT                       40.90        111.79        2.7x
  union() STRING_TO_INT                 383.44        449.03        1.2x
  intersect() STRING_TO_INT             174.97        214.24        1.2x
  diff() STRING_TO_INT                  160.93        197.91        1.2x
  mergeWith() INT_TO_INT                 65.73         85.80        1.3x
  mergeWith() STRING_TO_INT             341.01        369.01        1.1x

── Advanced: C-Level forEach/filter/map ──────────────────────────────────────

  Operation                        C-level(ms)       PHP(ms)     Speedup
  ──────────────────────────────  ────────────  ────────────  ──────────
  forEach() INT_TO_INT                   35.79         29.04        0.8x
  forEach() STRING_TO_INT                85.90         81.88        1.0x
  filter() INT_TO_INT (50% pass)         47.45         48.68        1.0x
  filter() STRING_TO_INT (50% pass)        160.75        158.40        1.0x
  map() INT_TO_INT (*2)                  54.34         61.64        1.1x
  map() STRING_TO_INT (*2)              225.85        223.58        1.0x

── Advanced: optimizeIteration trade (opt-in key-index payload mirror) ──────

  Operation                        default(ms)    optim.(ms)   optim./def.
  ──────────────────────────────  ────────────  ────────────  ────────────
  foreach() HASH kl16                    94.72         58.23         61.5%
  values() HASH kl16                     82.58         46.91         56.8%
  toArray() HASH kl16                    95.03         56.10         59.0%
  overwrite() HASH kl16                  52.59         61.50        117.0%
  increment() HASH kl16                  58.68         62.11        105.8%
    HASH kl16: honoured?          yes
  foreach() ADAPTIVE kl40               118.65         53.46         45.1%
  values() ADAPTIVE kl40                109.41         42.09         38.5%
  toArray() ADAPTIVE kl40               133.91         63.10         47.1%
  overwrite() ADAPTIVE kl40              91.90         99.06        107.8%
    ADAPTIVE kl40: honoured?      yes
  skipped — increment() ADAPTIVE kl40: Judy::increment() is only supported for INT_TO_INT, STRING_TO_INT and STRING_TO_INT_HASH types

  Ratios are optimized as a percentage of default, measured inside this
  build; below 100 is the traversal win, above 100 the write price. Lower
  is better on every row, so a lost win reads as a rise.

── Advanced: Adaptive SSO — short keys (<8 bytes) via JudyL ─────────────────

  Operation                   STR_TO_INT    STR_INT_HASH   STR_INT_ADPTV    Best ratio
  ──────────────────────  ──────────────  ──────────────  ──────────────  ────────────
  insert (short keys)         105.66 ms      205.21 ms      183.43 ms       1.9x
  read (random access)         37.27 ms       36.79 ms       32.77 ms       1.1x
  foreach (iterate all)        66.13 ms      158.10 ms      156.96 ms       2.4x

── Advanced: Adaptive SSO — long keys (>=8 bytes) fallback to JudyHS ────────

  Operation                   STR_TO_INT    STR_INT_HASH   STR_INT_ADPTV    Best ratio
  ──────────────────────  ──────────────  ──────────────  ──────────────  ────────────
  insert (long keys)          179.63 ms      313.94 ms      306.04 ms       1.7x
  read (random access)         65.99 ms       57.99 ms       58.02 ms       1.1x
  foreach (iterate all)       119.83 ms      175.20 ms      177.31 ms       1.5x

══════════════════════════════════════════════════════════════════════════════════════════
  Benchmark complete — 2026-10-08 16:54:01
  All timings: median of 7 iterations via hrtime(true), 1 warmup run
  Memory: peak RSS of a child process building one structure of 500,000 elements,
          minus an empty-process floor. Measured with getrusage(), not
          memory_get_usage(), for the reason above. Judy's memory advantage is
          scale- and type-dependent — see BENCHMARK.md for the curve.
══════════════════════════════════════════════════════════════════════════════════════════

  JSON results written to: bench.json

@orieg
orieg merged commit e6f3fe0 into main Oct 8, 2026
24 checks passed
@orieg
orieg deleted the fix/nightly-valgrind-test branch October 8, 2026 18:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant