Check freed debug block fill patterns 32 bytes at a time - #97
Conversation
…by checking chunks of 32 bytes at a time.
|
Thanks for the suggestion. I've further improved the routine by coaxing the compiler into reducing the number of branches. The code isn't pretty though. This routine seems like a good candidate for a SIMD assembly implementation. I'll put it on my to-do list. |
I had a go at it out of curiosity, and it turned out well enough to be worth handing over rather than sitting on. No PR - you said you would rather ask for one, and this is on your list, so this is just the idea plus the numbers. Take it, change it, or ignore it. The idea
Two choices that keep it low risk:
SSE2 is enough, so it fits what the file already does for the move routines: guaranteed under 64-bit, and gated on NumbersYour alloc/free workload in debug mode with stack traces off, so the routine runs once per iteration on the reuse path. Median of 25 alternating pairs, each sample in a fresh process, Delphi 13.1, one machine (Ryzen 9 7950X):
The part I did not expect: Win32 gains the most, which is exactly where #97 came out weakest (+3.30% on Intel Win32 at 4 KiB). So this lands on the side that the scalar widening could not reach, rather than overlapping with it. CorrectnessIn the real allocator: 12 medium block sizes chosen to cover exact multiples of 64, 16/32/48 byte vector remainders and 1/2/3/7/15 byte scalar tails, with every one of 34,949 byte positions corrupted in turn - all detected, on Win32 and Win64, and the unmodified build gives identical results. Separately, the routine in isolation against a plain byte loop for sizes 1 to 600, clean and with every byte position corrupted, deliberately unaligned since a user area is only pointer aligned. Two caveatsOne CPU and one compiler version, so this is not the cross architecture picture janrysavy produced - the direction was consistent everywhere I looked, but the magnitudes will move. Also a methodology warning, in case anyone benchmarks this routine: measuring both variants inside one executable is useless here. Below the threshold both run identical code, and that null control still swung by up to 30% from code layout interaction alone. Separate executables and a fresh process per sample brought the noise floor down to about 3%, which is why the table above is worth anything at all. Happy to send the patch, or a PR if you would prefer one after all - otherwise the snippet above is the whole trick and the rest is bookkeeping. |
|
Correction to the table above: the CPU is an Intel Core i9-12900K, not a Ryzen 9 7950X. I took the AMD designation from the tables in #99 instead of stating the machine I actually measured on. The numbers themselves are unchanged - only the hardware attribution was wrong. It does change how one line should be read, and in a useful direction. I presented the Win32 gains as landing where #97 came out weakest, and that weak case (+3.30% at 4 KiB) was Intel Win32. Since my measurements are Intel too, that is a closer comparison than I implied, not a cross-vendor one: different generation (Coffee Lake versus Alder Lake), same vendor. One methodology detail that belongs with the corrected hardware: the 12900K is a hybrid design, and the benchmark pins itself to one logical processor, which on this part is a P-core. E-cores would give different figures. The pinning is identical for both builds and the pairs alternate, so it does not bias the comparison, but the absolute times are P-core times. Apologies for the noise - better to have it right in the thread than to leave a wrong CPU attached to the numbers. |
|
Thanks for sharing this. I reconstructed @TetzkatLipHoka's SIMD approach from the published description and assembly loop, then measured it on an AMD Ryzen 9 7950X and an Intel Core i7-8750H, with both Win32 and Win64. This is not a byte-for-byte rerun of the original Core i9-12900K candidate because the complete patch was not posted. Both builds are based on upstream commit The straightforward reconstructed Win64 integration added a SIMD helper call inside the Pascal checker. Although the helper was not called for the control sizes, its presence made the compiler save five nonvolatile registers on every check and moved the following allocator routines. That produced a repeatable Intel regression of 3.58% at 64 B, with a bootstrap 95% interval reaching -3.92%. I tried a different Win64 integration to remove that cost. The scalar checker keeps the upstream instruction flow, tests the 128 B crossover once, and tail-jumps to an out-of-line SSE2 checker only for larger blocks. Corruption logging is also an out-of-line cold path.
The refined Win64 integration removes the control regression. The AMD 64 B result is statistically neutral, with a bootstrap 95% interval of [-0.89%, +0.44%], while the Intel 64 B result is +1.08% with [0.57%, 1.12%]. The material gains begin at 128 B on Win64 and continue to grow through the KiB sizes. The Win32 values in the table are the earlier measurements because the Win32 implementation did not change. The Core i7-8750H results also reproduce the shape of the corrected Core i9-12900K results, although the exact magnitudes differ by generation. For correctness, I used 12 real medium-block sizes totaling exactly 34,949 byte positions, covering the 0/16/32/48-byte vector remainders and 1/2/3/7/15-byte scalar tails. Every position was corrupted individually and every corruption was detected by the final binaries on AMD and Intel, Win32 and Win64. The existing debug-pattern test passes, the upstream The refined patch is available as janrysavy/FastMM5@9e0ce1a on the This suggests that the SSE2 loop is worthwhile for debug-mode freed-block validation. The layout-preserving Win64 implementation has a larger assembly surface than the straightforward helper call, but it avoids charging small blocks for a SIMD path they do not use. |
Replaces the bulk of CheckFreedDebugBlockFillPatternIntact with an SSE2 loop that compares 16 bytes at a time and accumulates the results with pand instead of branching on them, so the single loop exit and the data independent run time of the scalar version are preserved. Only whole 16 byte units are vectorised and only from 128 of them upward; the freed object marker and the trailing bytes stay on the existing scalar path, which is left untouched. Smaller blocks therefore execute exactly the code they did before. Offered under PR pleriche#97, where the SIMD implementation was put on the to-do list. No pull request opened by request; this branch keeps the work ready. Tests/ carries the correctness and timing harness: exhaustive.dpr corrupts all 34,949 byte positions of 12 freed block sizes covering every vector remainder and scalar tail, and Measure.ps1 pairs two builds as separate processes.
|
Thanks for taking the time to reconstruct and measure this - the reproduction found something our own numbers were hiding in plain sight. I reran the control sizes on the Core i9-12900K the original candidate was measured on, this time with a paired percentile bootstrap over the pairwise gains rather than a p5/p95 spread. Baseline
The Win64 64 B interval does not contain zero, so the regression reproduces here as well - smaller than your 3.58% on the i7-8750H, but the same sign and the same cause. Win32 is unaffected, which matches your reading. That corrects our own table: it reported -1.0% at exactly that size and we read it as noise. With 25 pairs and a percentile spread that was not a distinguishable claim in either direction; the bootstrap makes it one. The general point is worth more than the number - a threshold guarantees that small blocks execute the same instructions, but not that the surrounding code keeps the same addresses, and the second half is what a control size is there to catch and ours did not. On the test side, the exhaustive checker is now Still no pull request from me, per Pierre's request in #101. If the routine is wanted, the layout-preserving integration is plainly the one to take, and your branch already has it. |
|
@TetzkatLipHoka, have you benchmarked this in a real-world scenario using an actual application in debug mode? I’m curious what kind of speedup we could expect in practice. |
|
Not before you asked - now there is one, and it is in the branch so it can be rerun anywhere.
The distribution matters more than the total, so the driver reports it ( Four fifths of the freed blocks are too small to reach the vector path, but they carry under a tenth of the bytes, and the check costs bytes rather than blocks. End to end, 300 rounds per run, 25 alternating pairs, fresh process per sample, bootstrap CI:
So roughly 4% of total run time on Win64 and 8% on Win32, for a program that spends a substantial share of its time allocating in debug mode. Well short of the routine level figures, which I think is the honest headline: the check is one part of an allocation, and most allocations are too small to care. Two details from the same runs bear on your finding. The per allocation stack trace costs about what the check costs at medium sizes, so at the default depth it dilutes the routine level gain by roughly half - 4 KiB goes from +64% to +32%. It does not remove it. And the 64 B control moves from -1.12% [-1.68, -0.85] at depth 0 to +3.59% [+3.36, +4.74] at depth 20. Both intervals exclude zero, and at 64 B both builds execute the same instructions - so the layout effect is not only real, its sign depends on the configuration. That strikes me as a stronger argument for your layout-preserving integration than a single figure would be: it is not a fixed 3.58% one could note and live with. The tools are on the branch next to the test: |
|
Thank you for this info. I will try it on our applications with heavy XML parsers. |
3 participants
Summary
This changes
CheckFreedDebugBlockFillPatternIntactto validate 32 bytes per main-loop iteration instead of one 8-byte word. At 4 KiB, throughput improved by 40.18% on AMD Win64 and 14.45% on AMD Win32; Intel improved by 38.31% on Win64 but only 3.30% on Win32. The change adds fourUInt64comparisons before the existing 8-byte remainder loop, while preserving the freed-object prefix check, 4/2/1-byte remainder handling, corruption result, and logging path. The diff adds 16 lines and changes one comment.This is complementary to #92: that PR can disable fill-pattern diagnostics for selected configurations, while this change speeds up the unchanged diagnostic path when freed-block validation remains enabled.
Tracking issue: #99
Benchmark
The benchmark repeatedly allocates and frees one block in FastMM5 debug mode with stack-trace collection disabled. Results are medians of 25 alternating baseline/candidate process pairs after three warmups. Throughput gain is
(baseline time / candidate time - 1) * 100; the 95% interval is a deterministic 10,000-resample bootstrap interval for the paired median.AMD Ryzen 9 7950X:
Intel Core i7-8750H:
The 4 KiB and 64 KiB gains reproduced on AMD Win32 and Win64 and on Intel Win64, but Intel Win32 improved by only 3.30% to 4.29%, so the result depends strongly on architecture and CPU. Additional 25-pair tests cover 15, 39, 40, and 41 bytes. The 39-byte case exercises the new failed 32-byte loop guard before the existing 8-byte path; the 40/41-byte cases enter the new loop.
The worst AMD result was -2.12% at 39 bytes on Win32, with all Win64 controls positive. The worst Intel result was -1.58% at 15 bytes on Win32 (CI -2.93% to -0.33%), again with all Win64 controls positive. Avoiding those boundary losses would require an additional fallback path, which does not seem justified for debug mode.
Tests ran on an AMD Ryzen 9 7950X and an Intel Core i7-8750H under Windows 11 build 26200, using Delphi Win32/Win64 37.0.59082.6021 with release-style
-O+builds. The Intel runs use fewer operations to keep process duration similar; both builds still perform identical work within each pair, and the pairing and analysis are unchanged.Correctness
FastMMDebugPatternTest.dprpasses for the DCC 37 baseline and candidate on Win32 and Win64 on both CPUs. Additional compatibility runs pass with DCC 35 and 36 Win32/Win64 and with DCC 37PurePascal. The test verifies every allocated fill byte for sizes 1 through 1025, then corrupts and restores every byte for freed-block sizes 1 through 65 and 257. It accepts only the expectedEInvalidPointerand requires a clean scan afterward, covering the object marker and every 32/8/4/2/1-byte validation remainder.Full benchmark source, raw AMD and Intel samples, summaries, and correctness tests: https://github.com/janrysavy/FastMM5/tree/ed3229e678bbe4b2455b8435168442d51f430996/perf-review
Scope
The change keeps the current comparison strategy and only reduces loop overhead. It adds no SIMD requirement and does not change the public API or diagnostic contract.