Repository navigation
Fix 2-D/3-D compressed size missing the rate-array alignment padding - #14
Merged
hyfshishen merged 1 commit intoOct 13, 2026
Merged
Conversation
The 2-D and 3-D compress entry points reported glob_sync + blockNum, but the kernels place block payloads after the rate array padded to a multiple of 4 bytes (rate_ofs = (blockNum + 3) / 4 * 4). Callers that store exactly cmpSize bytes lost the last 0-3 bytes, the top bit plane of the last block with a payload, so values there could miss the error bound by a power-of-two multiple of 2*eb. Report the padded length, matching rate_ofs and the 1-D entry points.
Welcome to Codecov 🎉Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests. Thanks for integrating Codecov - We've got you covered ☂️ |
Collaborator
|
Hi Skyler, I just merged the PR. Thanks for the great effort! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Title: Fix 2-D/3-D compressed size missing the rate-array alignment padding
Problem
The 2-D and 3-D compress entry points report
but the kernels place the block payloads after the per-block rate array padded to a
multiple of 4 bytes:
So the reported size is short by
(-blockNum) mod 4bytes (0-3). Any caller thatstores or transfers exactly
cmpSizebytes (the example CLI does both: it copiescmpSizebytes to the host and writes them to the.cuszpfile) drops the end ofthe stream. Decompression derives offsets from the rate bytes, not from
cmpSize, soit reads the missing bytes as zero. Those final bytes are the most significant bit
plane of the last block with a payload, so up to 16-24 values in that block lose
their top quantization bit and miss the error bound by a power-of-two multiple of
2*eb.
The 1-D entry points are correct: they add the padded count, which matches the 1-D
kernels' offset.
Reproduction (cuSZp v3.0.0 and current master
f581dcf)-m fixed2-D, REL 1e-4:blockNum = 225*450 = 101250, 2 byteslost; 16 values exceed the bound, the largest by about 1.3e5 x eb (the CLI's own check,
which allows 1.1 x eb, reports them).
blockNumis 390625 and 1953125, 3 bytes lost.violations in fixed, plain and outlier modes (2-D and 3-D, f32 and f64) has this cause.
With the change below, all 362 cells satisfy the bound (worst 1.009 x eb; the only
remaining overshoots are float64 rounding at <= 1.0000001 x eb). Compressed sizes grow by
0-3 bytes.
Fix
Report the padded rate-array length in all 12 2-D/3-D compress entry points (fixed,
plain, outlier x 2-D, 3-D x f32, f64):
This matches
rate_ofsin the corresponding kernels and the 1-D entry points.