ci: fix test-floor ratchet never engaging (base path mismatch) by orieg · Pull Request #224 · orieg/php-judy

orieg · 2026-10-08T17:07:39Z

What

Fixes the test-floor identity ratchet wired in #221, which was a silent no-op: it never enforced anything. Two bugs:

1. Path mismatch (inspection)

test_report: reports/junit-head.xml

git mode reads the base report from the base ref at the same path as test_report, and the head report from disk at that path. But reports/junit-head.xml is a name the merge step generates per-run and is never committed — the committed baseline lives at reports/junit.xml. So the base-ref lookup always missed and the step passed trivially:

test-floor: test report 'reports/junit-head.xml' not present in base ref;
identity ratchet will be enforced on subsequent changes

That message's promise could never come true: junit-head.xml will never exist in any base ref. Every PR since #221 passed the ratchet unconditionally, so a count-preserving test drop (the exact attack the ratchet exists to block) would sail through.

2. Serialization mismatch (found by the first live run of bug-1's fix)

Pointing test_report at reports/junit.xml made the ratchet fire — but on four false "drops": increment_string_to_int_hash_001, iterator_next_invalid_behavior_001, prune_expired_reentrancy_001, regression_delete_range_destructor_reentrancy_001. The artifact and baseline names were byte-identical, yet the merged head disagreed. Cause: the merge step built the head with xml.dom.minidom.toprettyxml(), which normalizes attribute quoting/escaping — name='...offsetSet's...' became name="...offsetSet's...". The committed baseline is raw run-tests.php output (' entities), so every test name containing an apostrophe disagreed with the head, and test-floor compares name strings as written (raw), not XML-decoded:

error [test-floor] Test Dropped From Suite

Fix

Point test_report at the committed reports/junit.xml baseline. Git mode then finds both halves: base from the base ref (reports/junit.xml is committed on main), head from disk at the same path.
Merge the head by concatenating the raw <testsuite> blocks from each artifact verbatim (no parse/re-serialize), preserving the exact escaping the baseline uses.

Validation (local v0.18.0 CLI — the same engine CI runs)

Fixed wiring, head == baseline: verified 258 passed base test identity/identities against head report — this is now the live CI result on this PR (test-floor on 258 0, PASS).
minidom-re-serialized head: 4 false Test Dropped From Suite errors — reproduced locally, then eliminated by raw concat.
A head with one testcase genuinely dropped: caught as Test Dropped From Suite.
Full local discipline check --base origin/main: PASS (27 gates, 0 errors).

The CI run for this PR itself exercises the fix end-to-end: base ref (main) carries the 258-test baseline, the merge step writes the raw-concat head over it, and the ratchet compares them for real — green.

github-actions · 2026-10-08T17:07:59Z

discipline: passed

Base main (origin/main) (merge base e6f3fe0a1f) · 27 gate(s) passed · 2163 item(s) examined · 0 warning(s) · 0 override(s)

_{discipline explain <gate> prints what a gate checks and how a finding is lifted. This comment is updated on every run; the check's status is the verdict.}

github-actions · 2026-10-08T17:40:34Z

Test Results

	PHP	Platform	Arch	TS	Tests	Pass	Fail	Skip	Duration
✅	8.4	Linux	x64	-	258	258	0	0	9.5s
✅	Total				258	258	0	0

Benchmark Results

500,000 elements, 7 iterations (median), Judy 2.7.1

Run Status

Platform	PHP 8.4
Linux	✅

Memory (Linux, PHP 8.4)

Peak RSS of a child process building one structure, n = 500,000, minus an empty-process floor. Judy's memory advantage is type- and scale-dependent — see BENCHMARK.md for the curve across sizes on a dedicated host.

Type	Judy	PHP array	Judy vs PHP array
Bitset	< 632.0 KB	10.1 MB	> 16x smaller
Int to Int	4.3 MB	10.0 MB	2.3x smaller
Int to Mixed	42.2 MB	40.0 MB	1.06x larger
Int to Packed	18.3 MB	40.0 MB	2.2x smaller
String to Int	10.3 MB	40.0 MB	3.9x smaller
String to Mixed	48.3 MB	70.1 MB	1.5x smaller
String to Int (Hash)	31.4 MB	40.0 MB	1.3x smaller
String to Mixed (Hash)	69.4 MB	70.0 MB	about the same
String to Int (Adaptive)	31.4 MB	40.2 MB	1.3x smaller
String to Mixed (Adaptive)	69.5 MB	70.0 MB	about the same

< marks a structure smaller than this instrument can resolve at n = 500,000: run-to-run movement in the empty-process floor puts the resolution at 632.0 KB. The bound is real, the exact figure is not available at this size — BENCHMARK.md measures the same types up to n = 8M, where they are well clear of the floor.

API Speedups (Linux, PHP 8.4)

Speedup = PHP time / Judy time. Bold = Judy faster.

Operation	Judy	PHP	Speedup
putAll()	18.2 ms	27.8 ms	1.5x
fromArray()	18.1 ms	27.8 ms	1.5x
toArray()	24.7 ms	37.5 ms	1.5x
getAll()	1.1 ms	2.1 ms	2.0x
keys()	10.2 ms	20.2 ms	2.0x
values()	10.9 ms	19.7 ms	1.8x
sumValues()	6.4 ms	14.9 ms	2.3x
populationCount()	0.00 ms	15.5 ms	O(1) vs O(n)
deleteRange()	34.7 ms	49.0 ms	1.4x
equals()	9.9 ms	39.1 ms	3.9x
union() Bitset	25.6 ms	6.3 ms	0.2x
intersect() Bitset	11.9 ms	5.6 ms	0.5x
diff() Bitset	11.5 ms	3.3 ms	0.3x
xor() Bitset	23.4 ms	14.9 ms	0.6x
mergeWith() Int→Int	44.4 ms	56.6 ms	1.3x
mergeWith() Str→Int	231.8 ms	246.3 ms	1.1x
forEach() Int→Int	23.2 ms	16.5 ms	0.7x
filter() Int→Int	39.7 ms	31.2 ms	0.8x
map() Int→Int	44.4 ms	38.1 ms	0.9x
forEach() Str→Int	60.9 ms	54.9 ms	0.9x
filter() Str→Int	120.3 ms	111.7 ms	0.9x
map() Str→Int	160.5 ms	154.5 ms	~1.0x

O(1) vs O(n) marks a row whose ratio is decided by the complexity classes rather than measured: populationCount() reads libJudy's cached population, the PHP arm counts every element. It scales with n and says nothing about this change. judy-bench.php declines the same comparison for size($lo, $hi) for the same reason.

Core Types (Linux, PHP 8.4)

Speedup = PHP time / Judy time. Bold = Judy faster.

Type	Op	Judy	PHP array	Speedup
Bitset	Write	14.7 ms	7.2 ms	0.5x
Bitset	Read	9.9 ms	3.9 ms	0.4x
Bitset	Iter	13.8 ms	3.3 ms	0.2x
Bitset	Free	0.00 ms	0.48 ms	120x
Int to Int	Write	21.6 ms	7.6 ms	0.4x
Int to Int	Read	10.4 ms	3.8 ms	0.4x
Int to Int	Iter	14.3 ms	3.3 ms	0.2x
Int to Int	Free	0.43 ms	0.46 ms	1.1x
Int to Mixed	Write	32.2 ms	11.9 ms	0.4x
Int to Mixed	Read	12.5 ms	5.9 ms	0.5x
Int to Mixed	Iter	15.8 ms	3.9 ms	0.2x
Int to Mixed	Free	13.9 ms	5.8 ms	0.4x
String to Int	Write	89.7 ms	21.5 ms	0.2x
String to Int	Read	38.0 ms	8.2 ms	0.2x
String to Int	Iter	53.0 ms	4.7 ms	0.1x
String to Int	Free	19.6 ms	2.7 ms	0.1x
String to Int (Adaptive)	Write	162.6 ms	20.4 ms	0.1x
String to Int (Adaptive)	Read	22.0 ms	8.0 ms	0.4x
String to Int (Adaptive)	Iter	67.1 ms	4.6 ms	0.1x
String to Int (Adaptive)	Free	38.4 ms	2.6 ms	0.1x

All types detail

Core Types (Linux, PHP 8.4) — All

Speedup = PHP time / Judy time. Bold = Judy faster.

Type	Op	Judy	PHP array	Speedup
Bitset	Write	14.7 ms	7.2 ms	0.5x
Bitset	Read	9.9 ms	3.9 ms	0.4x
Bitset	Iter	13.8 ms	3.3 ms	0.2x
Bitset	Free	0.00 ms	0.48 ms	120x
Int to Int	Write	21.6 ms	7.6 ms	0.4x
Int to Int	Read	10.4 ms	3.8 ms	0.4x
Int to Int	Iter	14.3 ms	3.3 ms	0.2x
Int to Int	Free	0.43 ms	0.46 ms	1.1x
Int to Mixed	Write	32.2 ms	11.9 ms	0.4x
Int to Mixed	Read	12.5 ms	5.9 ms	0.5x
Int to Mixed	Iter	15.8 ms	3.9 ms	0.2x
Int to Mixed	Free	13.9 ms	5.8 ms	0.4x
Int to Packed	Write	43.6 ms	12.5 ms	0.3x
Int to Packed	Read	29.0 ms	5.9 ms	0.2x
Int to Packed	Iter	33.7 ms	3.8 ms	0.1x
Int to Packed	Free	8.2 ms	5.7 ms	0.7x
String to Int	Write	89.7 ms	21.5 ms	0.2x
String to Int	Read	38.0 ms	8.2 ms	0.2x
String to Int	Iter	53.0 ms	4.7 ms	0.1x
String to Int	Free	19.6 ms	2.7 ms	0.1x
String to Mixed	Write	108.7 ms	22.3 ms	0.2x
String to Mixed	Read	39.6 ms	9.2 ms	0.2x
String to Mixed	Iter	59.9 ms	5.6 ms	0.1x
String to Mixed	Free	62.9 ms	8.4 ms	0.1x
String to Int (Hash)	Write	162.6 ms	20.4 ms	0.1x
String to Int (Hash)	Read	22.9 ms	8.2 ms	0.4x
String to Int (Hash)	Iter	67.4 ms	4.7 ms	0.1x
String to Int (Hash)	Free	38.8 ms	2.6 ms	0.1x
String to Mixed (Hash)	Write	232.6 ms	22.5 ms	0.1x
String to Mixed (Hash)	Read	24.7 ms	9.3 ms	0.4x
String to Mixed (Hash)	Iter	93.7 ms	5.7 ms	0.1x
String to Mixed (Hash)	Free	121.2 ms	8.5 ms	0.1x
String to Int (Adaptive)	Write	162.6 ms	20.4 ms	0.1x
String to Int (Adaptive)	Read	22.0 ms	8.0 ms	0.4x
String to Int (Adaptive)	Iter	67.1 ms	4.6 ms	0.1x
String to Int (Adaptive)	Free	38.4 ms	2.6 ms	0.1x
String to Mixed (Adaptive)	Write	258.7 ms	23.7 ms	0.1x
String to Mixed (Adaptive)	Read	23.5 ms	9.1 ms	0.4x
String to Mixed (Adaptive)	Iter	104.3 ms	5.6 ms	0.1x
String to Mixed (Adaptive)	Free	129.2 ms	8.2 ms	0.1x

Release Comparison (2.6.0 → 2.7.1)

Interleaved A/B comparison against v2.6.0 (pie install) on the same CI runner: 5 benchmark groups × 5 rounds × 2 arms, 300,000 elements, median of 3 per child. Each row is the median of the paired current/baseline ratios with a 95% percentile-bootstrap CI; it is flagged only when the whole drift-adjusted CI clears ±10%. Advisory — benchmarks never gate a merge.

Run-wide median delta -0.26% (PHP-array control -0.53%); runner common-mode -0.53% divided out of every row.

Summary: 0 regressions, 100 unchanged

Benchmark	Baseline	Current	Delta	95% CI	Status
adv.filter.int_to_int	23.7 ms	24.3 ms	+3.3%	[+3.0%, +13.6%]	~same
adv.filter.string_to_int	70.8 ms	71.4 ms	+1.7%	[+0.8%, +3.2%]	~same
adv.forEach.int_to_int	14.3 ms	14.0 ms	-2.4%	[-3.0%, +1.1%]	~same
adv.forEach.string_to_int	35.6 ms	35.4 ms	-0.2%	[-0.3%, +0.1%]	~same
adv.map.int_to_int	26.0 ms	26.4 ms	+2.2%	[+1.6%, +2.3%]	~same
adv.map.string_to_int	92.0 ms	93.6 ms	+2.2%	[+2.2%, +4.0%]	~same
adv.optiter.foreach.string_to_int_adaptive.default	80.3 ms	80.7 ms	+5.0%	[-0.5%, +7.5%]	~same
adv.optiter.foreach.string_to_int_adaptive.optimized	35.2 ms	35.8 ms	+1.5%	[-0.1%, +3.6%]	~same
adv.optiter.foreach.string_to_int_hash.default	67.7 ms	66.2 ms	-0.8%	[-3.3%, +1.7%]	~same
adv.optiter.foreach.string_to_int_hash.optimized	31.7 ms	31.3 ms	-1.2%	[-2.9%, +1.6%]	~same
adv.optiter.increment.string_to_int_hash.default	50.0 ms	51.2 ms	+0.5%	[-3.9%, +4.5%]	~same
adv.optiter.increment.string_to_int_hash.optimized	56.1 ms	57.9 ms	+2.5%	[-6.0%, +5.8%]	~same
adv.optiter.overwrite.string_to_int_adaptive.default	64.2 ms	65.1 ms	+4.9%	[-7.6%, +14.0%]	~same
adv.optiter.overwrite.string_to_int_adaptive.optimized	68.3 ms	68.6 ms	+1.2%	[-14.7%, +13.5%]	~same
adv.optiter.overwrite.string_to_int_hash.default	52.1 ms	52.1 ms	-0.6%	[-6.1%, +4.0%]	~same
adv.optiter.overwrite.string_to_int_hash.optimized	56.9 ms	55.9 ms	-2.9%	[-4.9%, +2.6%]	~same
adv.optiter.ratio.foreach.string_to_int_adaptive	44.2 ms	44.0 ms	-1.9%	[-6.1%, +4.4%]	~same
adv.optiter.ratio.foreach.string_to_int_hash	47.1 ms	47.5 ms	-1.2%	[-1.6%, +3.9%]	~same
adv.optiter.ratio.increment.string_to_int_hash	111.4 ms	113.1 ms	+1.8%	[-1.7%, +5.8%]	~same
adv.optiter.ratio.overwrite.string_to_int_adaptive	108.1 ms	105.3 ms	-5.9%	[-10.8%, +9.4%]	~same
adv.optiter.ratio.overwrite.string_to_int_hash	111.4 ms	105.7 ms	-1.8%	[-6.6%, +9.9%]	~same
adv.optiter.ratio.toArray.string_to_int_adaptive	50.5 ms	49.3 ms	+1.6%	[-5.9%, +16.4%]	~same
adv.optiter.ratio.toArray.string_to_int_hash	48.1 ms	47.4 ms	-1.8%	[-2.9%, +18.9%]	~same
adv.optiter.ratio.values.string_to_int_adaptive	37.5 ms	37.8 ms	-0.9%	[-6.6%, +3.9%]	~same
adv.optiter.ratio.values.string_to_int_hash	39.3 ms	40.0 ms	+2.8%	[-5.2%, +4.8%]	~same
adv.optiter.toArray.string_to_int_adaptive.default	95.6 ms	96.1 ms	+0.5%	[-0.8%, +9.7%]	~same
adv.optiter.toArray.string_to_int_adaptive.optimized	47.5 ms	47.4 ms	+2.6%	[-2.8%, +16.2%]	~same
adv.optiter.toArray.string_to_int_hash.default	74.9 ms	76.2 ms	+2.1%	[-0.7%, +5.7%]	~same
adv.optiter.toArray.string_to_int_hash.optimized	35.5 ms	35.5 ms	+0.6%	[-3.0%, +19.5%]	~same
adv.optiter.values.string_to_int_adaptive.default	75.6 ms	76.6 ms	+1.9%	[-3.2%, +9.8%]	~same
adv.optiter.values.string_to_int_adaptive.optimized	28.6 ms	29.0 ms	+0.1%	[-0.9%, +3.9%]	~same
adv.optiter.values.string_to_int_hash.default	61.0 ms	60.7 ms	-1.1%	[-2.8%, +5.7%]	~same
adv.optiter.values.string_to_int_hash.optimized	24.3 ms	24.3 ms	+1.1%	[-0.5%, +2.1%]	~same
api.deleteRange.int_to_int	20.7 ms	20.8 ms	+0.4%	[-0.8%, +1.6%]	~same
api.equals.int_to_int	5.6 ms	5.9 ms	+4.2%	[+1.8%, +6.2%]	~same
api.fromArray.int_to_int	11.8 ms	11.7 ms	-0.8%	[-3.7%, +6.7%]	~same
api.getAll.int_to_int	0.53 ms	0.55 ms	+1.2%	[-2.5%, +8.1%]	~same
api.increment.int_to_int	12.6 ms	12.2 ms	-1.8%	[-6.4%, +3.4%]	~same
api.keys.int_to_int	6.2 ms	6.0 ms	-3.4%	[-5.3%, -0.7%]	~same
api.mergeWith.int_to_int	26.5 ms	25.9 ms	-2.4%	[-3.4%, -1.4%]	~same
api.mergeWith.string_to_int	136.9 ms	135.4 ms	-0.5%	[-1.8%, +2.2%]	~same
api.populationCount.int_to_int	0.00 ms	0.00 ms	+0.5%	[+0.5%, +0.5%]	~same
api.putAll.int_to_int	11.8 ms	11.8 ms	-0.3%	[-1.7%, +3.1%]	~same
api.range.keys.int_to_int	0.51 ms	0.50 ms	-1.6%	[-5.5%, -0.4%]	~same
api.range.size.string_to_int	2.3 ms	2.3 ms	+0.1%	[-0.1%, +0.3%]	~same
api.range.size1000.int_to_int	0.46 ms	0.42 ms	-8.0%	[-9.3%, -7.1%]	~same
api.setop.diff.bitset	6.8 ms	6.8 ms	+1.2%	[+0.7%, +1.6%]	~same
api.setop.diff.int_to_int	8.7 ms	8.7 ms	-0.2%	[-0.9%, +8.4%]	~same
api.setop.diff.string_to_int	64.5 ms	65.1 ms	+1.1%	[-1.3%, +1.8%]	~same
api.setop.intersect.bitset	6.8 ms	7.0 ms	+4.6%	[+3.5%, +5.0%]	~same
api.setop.intersect.int_to_int	8.8 ms	8.9 ms	+0.6%	[-1.0%, +7.8%]	~same
api.setop.intersect.string_to_int	73.6 ms	74.0 ms	+0.8%	[-0.8%, +3.7%]	~same
api.setop.union.bitset	14.9 ms	15.2 ms	+2.5%	[-5.6%, +3.6%]	~same
api.setop.union.int_to_int	22.9 ms	22.5 ms	+0.2%	[-1.9%, +1.9%]	~same
api.setop.union.string_to_int	159.6 ms	163.3 ms	+2.6%	[+1.3%, +4.5%]	~same
api.setop.xor.bitset	13.6 ms	13.9 ms	+2.4%	[+1.4%, +2.8%]	~same
api.setop.xor.int_to_int	18.2 ms	17.7 ms	-1.3%	[-3.1%, +3.1%]	~same
api.sumValues.int_to_int	4.0 ms	3.7 ms	-5.3%	[-8.2%, -2.6%]	~same
api.toArray.int_to_int	17.2 ms	17.1 ms	-0.4%	[-3.2%, +3.2%]	~same
api.values.int_to_int	6.8 ms	6.5 ms	-3.4%	[-6.2%, +6.8%]	~same
core.bitset.free	0.00 ms	0.00 ms	+3.2%	[-40.5%, +19.6%]	~same
core.bitset.iter	7.7 ms	8.3 ms	+8.3%	[+8.2%, +8.8%]	~same
core.bitset.read	6.0 ms	6.0 ms	+1.2%	[-0.2%, +1.9%]	~same
core.bitset.write	8.8 ms	8.5 ms	-0.5%	[-4.0%, +1.5%]	~same
core.int_to_int.free	0.35 ms	0.37 ms	+5.6%	[-4.6%, +36.2%]	~same
core.int_to_int.iter	8.9 ms	8.6 ms	-2.5%	[-3.4%, -1.6%]	~same
core.int_to_int.read	6.1 ms	6.3 ms	+4.8%	[-3.3%, +9.1%]	~same
core.int_to_int.write	12.6 ms	12.6 ms	+0.1%	[-1.1%, +1.6%]	~same
core.int_to_mixed.free	8.4 ms	8.1 ms	-3.0%	[-19.8%, -2.4%]	~same
core.int_to_mixed.iter	9.5 ms	9.4 ms	-1.0%	[-2.2%, -0.1%]	~same
core.int_to_mixed.read	7.6 ms	7.6 ms	+0.8%	[-1.5%, +6.4%]	~same
core.int_to_mixed.write	19.4 ms	19.1 ms	-1.3%	[-2.7%, -0.7%]	~same
core.int_to_packed.free	5.0 ms	4.8 ms	-4.0%	[-4.9%, -3.0%]	~same
core.int_to_packed.iter	19.7 ms	20.1 ms	+2.7%	[+0.7%, +3.2%]	~same
core.int_to_packed.read	17.7 ms	17.4 ms	-0.9%	[-1.1%, +0.5%]	~same
core.int_to_packed.write	27.1 ms	25.9 ms	-4.0%	[-4.5%, -3.6%]	~same
core.string_to_int.free	10.8 ms	11.0 ms	+2.6%	[+2.4%, +3.3%]	~same
core.string_to_int.iter	31.3 ms	30.8 ms	-1.0%	[-1.8%, -0.8%]	~same
core.string_to_int.read	21.6 ms	21.4 ms	-0.1%	[-1.8%, +1.4%]	~same
core.string_to_int.write	51.9 ms	51.2 ms	-0.8%	[-3.4%, +0.3%]	~same
core.string_to_int_adaptive.free	21.8 ms	21.7 ms	-0.1%	[-1.3%, +0.8%]	~same
core.string_to_int_adaptive.iter	40.1 ms	39.5 ms	-0.7%	[-2.2%, -0.5%]	~same
core.string_to_int_adaptive.read	13.3 ms	13.3 ms	+0.6%	[-1.5%, +2.6%]	~same
core.string_to_int_adaptive.write	92.2 ms	93.0 ms	+1.9%	[+1.0%, +2.9%]	~same
core.string_to_int_hash.free	21.7 ms	21.5 ms	+0.3%	[-0.9%, +2.6%]	~same
core.string_to_int_hash.iter	39.1 ms	39.6 ms	+1.2%	[-1.0%, +1.7%]	~same
core.string_to_int_hash.read	13.8 ms	13.7 ms	-0.6%	[-3.2%, +1.1%]	~same
core.string_to_int_hash.write	92.8 ms	93.4 ms	+1.3%	[+0.9%, +2.0%]	~same
core.string_to_mixed.free	35.4 ms	35.8 ms	+1.5%	[-0.2%, +2.4%]	~same
core.string_to_mixed.iter	32.9 ms	32.2 ms	-1.5%	[-3.9%, -1.2%]	~same
core.string_to_mixed.read	22.5 ms	22.6 ms	+1.1%	[-0.5%, +1.3%]	~same
core.string_to_mixed.write	62.5 ms	62.5 ms	+0.9%	[-0.2%, +1.2%]	~same
core.string_to_mixed_adaptive.free	68.1 ms	67.5 ms	-0.2%	[-2.9%, +2.0%]	~same
core.string_to_mixed_adaptive.iter	53.8 ms	50.1 ms	-6.8%	[-9.3%, -5.0%]	~same
core.string_to_mixed_adaptive.read	14.1 ms	14.2 ms	+1.3%	[+0.3%, +2.2%]	~same
core.string_to_mixed_adaptive.write	132.8 ms	131.0 ms	-0.5%	[-2.0%, +1.2%]	~same
core.string_to_mixed_hash.free	58.8 ms	58.7 ms	+0.4%	[-0.6%, +1.1%]	~same
core.string_to_mixed_hash.iter	47.5 ms	46.9 ms	-0.9%	[-2.1%, +1.5%]	~same
core.string_to_mixed_hash.read	14.6 ms	14.8 ms	+2.0%	[+0.2%, +16.1%]	~same
core.string_to_mixed_hash.write	122.0 ms	122.6 ms	+1.2%	[-0.6%, +1.4%]	~same

Memory Comparison

Memory: 0 regressions, 10 unchanged

Type	Baseline	Current	Delta	Status
Bitset	160 B	160 B	0.0%	~same
Int to Int	160 B	160 B	0.0%	~same
Int to Mixed	4.6 MB	4.6 MB	0.0%	~same
Int to Packed	8.0 MB	8.0 MB	0.0%	~same
String to Int	64.2 KB	64.2 KB	0.0%	~same
String to Int (Adaptive)	64.2 KB	64.2 KB	0.0%	~same
String to Int (Hash)	64.2 KB	64.2 KB	0.0%	~same
String to Mixed	4.6 MB	4.6 MB	0.0%	~same
String to Mixed (Adaptive)	4.6 MB	4.6 MB	0.0%	~same
String to Mixed (Hash)	4.6 MB	4.6 MB	0.0%	~same

Full benchmark output

Linux — PHP 8.4

══════════════════════════════════════════════════════════════════════════════════════════
  php-judy Benchmark Suite — 500,000 elements, 7 iterations (median)
  PHP 8.4.26 | Judy ext 2.7.1 | Linux x86_64
  Suite: all | Groups: core.int,core.str,api.batch,api.setops,adv.iter,adv.sso
  memory_limit: -1
  Memory: peak RSS of a child process, empty-process floor subtracted, median of 3
══════════════════════════════════════════════════════════════════════════════════════════

── Core: Integer-Keyed Types ────────────────────────────────────────────────

  Type                         Write(ms)      Read(ms)      Iter(ms)      Free(ms)        Memory
  ────────────────────────  ────────────  ────────────  ────────────  ────────────  ────────────
  BITSET                           14.70          9.90         13.83          0.00    < 632.0 KB
    PHP array                       7.19          3.86          3.35          0.48       10.1 MB
  INT_TO_INT                       21.56         10.45         14.29          0.43        4.3 MB
    PHP array                       7.58          3.83          3.34          0.46       10.0 MB
  INT_TO_MIXED                     32.18         12.53         15.81         13.92       42.2 MB
    PHP array                      11.93          5.92          3.93          5.83       40.0 MB
  INT_TO_PACKED                    43.58         28.96         33.70          8.16       18.3 MB
    PHP array                      12.46          5.93          3.82          5.70       40.0 MB

── Core: String-Keyed Types ────────────────────────────────────────────────

  Type                         Write(ms)      Read(ms)      Iter(ms)      Free(ms)        Memory
  ────────────────────────  ────────────  ────────────  ────────────  ────────────  ────────────
  STRING_TO_INT                    89.73         37.98         52.99         19.63       10.3 MB
    PHP array                      21.47          8.22          4.68          2.73       40.0 MB
  STRING_TO_MIXED                 108.73         39.63         59.94         62.92       48.3 MB
    PHP array                      22.33          9.22          5.62          8.43       70.1 MB
  STR_TO_INT_HASH                 162.59         22.90         67.43         38.82       31.4 MB
    PHP array                      20.42          8.21          4.72          2.60       40.0 MB
  STR_TO_MIX_HASH                 232.64         24.68         93.73        121.16       69.4 MB
    PHP array                      22.46          9.26          5.67          8.53       70.0 MB
  STR_INT_ADAPTIVE                162.56         22.00         67.05         38.38       31.4 MB
    PHP array                      20.38          7.97          4.59          2.59       40.2 MB
  STR_MIX_ADAPTIVE                258.68         23.51        104.28        129.21       69.5 MB
    PHP array                      23.73          9.13          5.60          8.18       70.0 MB

── API: Batch Operations (INT_TO_INT) ────────────────────────────────────────

  Operation                           Judy(ms)       PHP(ms)     Speedup
  ──────────────────────────────  ────────────  ────────────  ──────────
  putAll()                               18.20         27.77        1.5x
  fromArray()                            18.08         27.77        1.5x
  toArray()                              24.75         37.45        1.5x
  getAll(50100 keys)                      1.07          2.08        2.0x
  increment(500000 ops/1000 keys)         20.55          8.51        0.4x

── API: Native Methods (keys, values, sum, popCount, deleteRange, equals) ────

  Operation                           Judy(ms)       PHP(ms)     Speedup
  ──────────────────────────────  ────────────  ────────────  ──────────
  keys() INT_TO_INT                      10.18         20.20        2.0x
  values() INT_TO_INT                    10.87         19.67        1.8x
  keys($lo,$hi) INT_TO_INT                0.84         18.33       21.9x
    vs slice()->keys()                    2.49             -        3.0x
  size($lo,$hi) INT x1000                 0.49             -        O(1)
  size($lo,$hi) STRING_TO_INT             3.93          4.88        1.2x
  sumValues() INT_TO_INT                  6.36         14.91        2.3x
  populationCount() INT_TO_INT            0.00         15.48   253771.5x
  deleteRange() INT_TO_INT mid50%         34.70         48.97        1.4x
  equals() INT_TO_INT                     9.94         39.11        3.9x

── API: Set Operations (BITSET, 50% overlap) ────────────────────────────────

  Operation                           Judy(ms)       PHP(ms)     Speedup
  ──────────────────────────────  ────────────  ────────────  ──────────
  union() BITSET                         25.61          6.25        0.2x
  intersect() BITSET                     11.94          5.65        0.5x
  diff() BITSET                          11.53          3.33        0.3x
  xor() BITSET                           23.37         14.92        0.6x
  union() INT_TO_INT                     36.93         71.72        1.9x
  intersect() INT_TO_INT                 14.97         35.02        2.3x
  diff() INT_TO_INT                      14.50         35.45        2.4x
  xor() INT_TO_INT                       29.36         71.28        2.4x
  union() STRING_TO_INT                 278.13        309.71        1.1x
  intersect() STRING_TO_INT             119.07        135.59        1.1x
  diff() STRING_TO_INT                  112.55        130.82        1.2x
  mergeWith() INT_TO_INT                 44.36         56.59        1.3x
  mergeWith() STRING_TO_INT             231.81        246.26        1.1x

── Advanced: C-Level forEach/filter/map ──────────────────────────────────────

  Operation                        C-level(ms)       PHP(ms)     Speedup
  ──────────────────────────────  ────────────  ────────────  ──────────
  forEach() INT_TO_INT                   23.19         16.54        0.7x
  forEach() STRING_TO_INT                60.88         54.89        0.9x
  filter() INT_TO_INT (50% pass)         39.69         31.18        0.8x
  filter() STRING_TO_INT (50% pass)        120.29        111.70        0.9x
  map() INT_TO_INT (*2)                  44.37         38.08        0.9x
  map() STRING_TO_INT (*2)              160.45        154.53        1.0x

── Advanced: optimizeIteration trade (opt-in key-index payload mirror) ──────

  Operation                        default(ms)    optim.(ms)   optim./def.
  ──────────────────────────────  ────────────  ────────────  ────────────
  foreach() HASH kl16                    63.53         30.58         48.1%
  values() HASH kl16                     59.07         24.05         40.7%
  toArray() HASH kl16                    74.59         35.23         47.2%
  overwrite() HASH kl16                  50.50         54.21        107.3%
  increment() HASH kl16                  48.78         51.01        104.6%
    HASH kl16: honoured?          yes
  foreach() ADAPTIVE kl40                75.59         34.73         46.0%
  values() ADAPTIVE kl40                 68.09         28.29         41.6%
  toArray() ADAPTIVE kl40                89.89         46.14         51.3%
  overwrite() ADAPTIVE kl40              60.61         64.90        107.1%
    ADAPTIVE kl40: honoured?      yes
  skipped — increment() ADAPTIVE kl40: Judy::increment() is only supported for INT_TO_INT, STRING_TO_INT and STRING_TO_INT_HASH types

  Ratios are optimized as a percentage of default, measured inside this
  build; below 100 is the traversal win, above 100 the write price. Lower
  is better on every row, so a lost win reads as a rise.

── Advanced: Adaptive SSO — short keys (<8 bytes) via JudyL ─────────────────

  Operation                   STR_TO_INT    STR_INT_HASH   STR_INT_ADPTV    Best ratio
  ──────────────────────  ──────────────  ──────────────  ──────────────  ────────────
  insert (short keys)          70.30 ms      123.40 ms      109.56 ms       1.8x
  read (random access)         23.79 ms       21.68 ms       21.97 ms       1.1x
  foreach (iterate all)        43.03 ms       87.59 ms       90.73 ms       2.0x

── Advanced: Adaptive SSO — long keys (>=8 bytes) fallback to JudyHS ────────

  Operation                   STR_TO_INT    STR_INT_HASH   STR_INT_ADPTV    Best ratio
  ──────────────────────  ──────────────  ──────────────  ──────────────  ────────────
  insert (long keys)          120.00 ms      212.77 ms      212.44 ms       1.8x
  read (random access)         41.93 ms       32.72 ms       32.81 ms       1.3x
  foreach (iterate all)        73.17 ms       94.03 ms       92.68 ms       1.3x

══════════════════════════════════════════════════════════════════════════════════════════
  Benchmark complete — 2026-10-08 18:48:14
  All timings: median of 7 iterations via hrtime(true), 1 warmup run
  Memory: peak RSS of a child process building one structure of 500,000 elements,
          minus an empty-process floor. Measured with getrusage(), not
          memory_get_usage(), for the reason above. Judy's memory advantage is
          scale- and type-dependent — see BENCHMARK.md for the curve.
══════════════════════════════════════════════════════════════════════════════════════════

  JSON results written to: bench.json

The ratchet wired in #221 never actually enforced anything. Two bugs: 1. Path mismatch. test_report pointed at reports/junit-head.xml, a filename the merge step generates but that is never committed. In git mode discipline reads the BASE report from the base ref at the same path as test_report, so the lookup always missed and the step passed trivially with "not present in base ref; identity ratchet will be enforced on subsequent changes" -- a promise that could never come true. 2. Serialization mismatch (found on the first live run of the fix). Having pointed test_report at reports/junit.xml, the ratchet then flagged four tests as "dropped" for no reason. Cause: the merge step built the head with xml.dom.minidom.toprettyxml(), which normalizes attribute quoting and HTML-entity escaping (name="...offsetSet's..." instead of name='...offsetSet's...'). The committed baseline is raw run-tests.php output, so every test name containing an apostrophe disagreed with the head and test-floor read it as a drop. Discipline compares the name strings as written, not XML-decoded. Fix both: point test_report at the committed reports/junit.xml baseline (git mode then finds base from the base ref and head from disk at the same path), and merge the head by concatenating the raw <testsuite> blocks from each artifact verbatim, preserving the escaping the baseline uses. Verified with the local v0.18.0 CLI (the same engine CI runs): - fixed wiring, head == baseline: "verified 258 passed base test identity/identities against head report", PASS - minidom-re-serialized head: 4 false "Test Dropped From Suite" errors (the exact names CI flagged) -- reproduced, then fixed by raw concat - one testcase genuinely dropped: caught as Test Dropped From Suite - full local discipline check on the diff: PASS (27 gates)

orieg force-pushed the fix/test-floor-ratchet-path branch from 53b7500 to e9d37ef Compare October 8, 2026 17:32

orieg force-pushed the fix/test-floor-ratchet-path branch from e9d37ef to 43f4659 Compare October 8, 2026 18:36

orieg merged commit 5b9c8da into main Oct 8, 2026
19 checks passed

orieg deleted the fix/test-floor-ratchet-path branch October 8, 2026 18:50

orieg mentioned this pull request Oct 8, 2026

🚨 [Nightly CI Failure] Valgrind Memory Leak Detected on main #222

Closed

Navigation Menu

Repository navigation

ci: fix test-floor ratchet never engaging (base path mismatch) - #224

ci: fix test-floor ratchet never engaging (base path mismatch)#224
orieg merged 1 commit into
mainfrom
fix/test-floor-ratchet-path

orieg commented Oct 8, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 8, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 8, 2026 •

edited

Loading

Core Types (Linux, PHP 8.4) — All

Linux — PHP 8.4

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

orieg commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

1. Path mismatch (inspection)

2. Serialization mismatch (found by the first live run of bug-1's fix)

Fix

Validation (local v0.18.0 CLI — the same engine CI runs)

Uh oh!

github-actions Bot commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

discipline: passed

Uh oh!

github-actions Bot commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Benchmark Results

Run Status

Memory (Linux, PHP 8.4)

API Speedups (Linux, PHP 8.4)

Core Types (Linux, PHP 8.4)

Core Types (Linux, PHP 8.4) — All

Release Comparison (2.6.0 → 2.7.1)

Memory Comparison

Linux — PHP 8.4

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

orieg commented Oct 8, 2026 •

edited

Loading

github-actions Bot commented Oct 8, 2026 •

edited

Loading

github-actions Bot commented Oct 8, 2026 •

edited

Loading