[astorise]

Summary

Nvfp4LinearPacked::forward and dequantize_nvfp4 (candle-transformers/src/quantized_nvfp4.rs)
both decode a block's F8E4M3 weight_scale byte to f32 inside the
per-column loop, instead of once per block_size-wide block:

for (col, val) in row_buf.iter_mut().enumerate() {
    let byte = packed_row[col / 2];
    let nibble = if col % 2 == 0 { byte & 0xf } else { byte >> 4 };
    let block_scale = scale_row[col / self.block_size].to_f32(); // redone every column
    *val = unpack_e2m1(nibble) * block_scale * self.weight_scale_2;
}

With block_size = 16, that's 16x more F8E4M3::to_f32 conversions than
the format requires — the same cost as unpacking a nibble, redone on every
column of the block instead of once per block.

Fixes #3860.

Change

Both loops now walk block-by-block: decode block_scale once per block,
then iterate only the columns inside that block reusing it. Behavior is
unchanged — this is a decode-cost fix, not a numeric one.

Testing

• cargo test -p candle-transformers --lib quantized_nvfp4 — all 8 tests
  pass, including nvfp4_linear_packed_matches_dense_dequantization
  (bit-exact packed-vs-dense parity) and
  dequantize_nvfp4_matches_manual_computation.
• cargo clippy -p candle-transformers --lib — no new warnings.
• cargo fmt -p candle-transformers -- --check — clean.

This only touches the CPU decode path (Nvfp4LinearPacked and the dense
dequantize_nvfp4 fallback); the nvfp4-cuda fused kernel path
(cuda::Nvfp4LinearCuda) is untouched, so no GPU testing is needed for
this change.

[astorise]

« Closing — this carries kernel crates that don't exist upstream. Moving it to a separate crate. »