Skip to content

clamp origin texel before narrowing to int in load_image_block#648

Open
sahvx655-wq wants to merge 1 commit into
ARM-software:mainfrom
sahvx655-wq:hdr-origin-texel-clamp
Open

clamp origin texel before narrowing to int in load_image_block#648
sahvx655-wq wants to merge 1 commit into
ARM-software:mainfrom
sahvx655-wq:hdr-origin-texel-clamp

Conversation

@sahvx655-wq

Copy link
Copy Markdown
Contributor

When compressing with the HDR-RGB/LDR-alpha profile, load_image_block builds the origin texel for each block and feeds blk.texel(0) straight into float_to_int() so it can be re-encoded through the LNS path. The RGB lanes have already passed through float_to_lns() and sit in [0, 65535], but the alpha lane takes the LDR unorm route and is just the input value scaled by 65535. A crafted HDR source (an .exr or .hdr carrying a large alpha) therefore presents a float well above INT_MAX to the scalar float_to_int(), which is undefined behaviour. UBSan flags it at astcenc_image.cpp:254 from an ordinary astcenc_compress_image() call: "3.9321e+09 is outside the range of representable values of type int".

The narrowed alpha lane is discarded by the later lns_mask select, so the value is never used for that channel, but the conversion still runs across all four lanes and trips the UB regardless. Clamping the vector into the valid LNS range [0, 65535] before the narrowing mirrors the clampzo guard this file already applies ahead of every other float_to_int/float_to_int_rtn, keeps the result bit-identical for in-range data, and folds NaN onto the low bound. Putting the clamp at the conversion site keeps the encoder sound whatever the image loaders hand it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant