Skip to content

Fix sparse duplicates and quantized CPU split boundaries - #39

Merged
dev-rinchin merged 1 commit into
masterfrom
fix/treelite-sparse-quantization
Nov 2, 2026
Merged

dev-rinchin merged 1 commit into
masterfrom
fix/treelite-sparse-quantization

Conversation

@dev-rinchin

@dev-rinchin dev-rinchin commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Compiled CPU inference can return different predictions for equivalent sparse matrices, or after enabling quantize=1. For example, a CSR coordinate stored as 0.4 + 0.4 follows the wrong branch at a 0.5 split, and quantization changes one prediction in a 128-tree regression fixture by up to 0.08035.

This fixes both cases while retaining float32 inference and threshold quantization. Follow-up to #37.

  • Sum duplicate sparse coordinates in the input dtype before casting to float32 or dropping NaNs. Copy noncanonical CSR buffers before summation, preserve explicit zeros, and leave caller-owned matrices unchanged.
  • Preserve split boundaries affected by TL2cgen 1.0's eight-digit float-array formatting. On a separate compilation copy, rewrite an affected x <= t comparison as the equivalent float32 comparison x < nextafter(t, +inf). Apply this at compile time so existing Treelite 4 checkpoints also benefit; keep the original model and compiler parameter dictionary unchanged.
  • Add 40 regression cases covering CSR/CSC/COO, float32/float64 values, int32/int64 indices, cancellation, NaN duplicates, threshold equality and neighboring values, both NaN directions, scalar/vector outputs, checkpoint loading, and serial/parallel compilation. Update the README with the supported behavior.

Validation:

  • Full Linux/NVIDIA A100 suite on the resulting tree: 419 passed, 1 Windows-only test skipped.
  • Built sdist and wheel from the final branch; installed the wheel and ran the full suite outside the source checkout without CuPy: 116 passed, 206 skipped. GPU tests and the actual Windows loader test are skipped in this environment.
  • CPU integration and loader suites on Python 3.8 / NumPy 1.24.4 / Treelite 4.1.2, Python 3.12 / Treelite 4.1.2, and Python 3.14 / Treelite 4.8.1: 110 passed, 1 skipped in each environment, with TL2cgen 1.0.0.
  • Original 128-tree fixture: quantized maximum absolute error falls from 0.08035 to 3.8e-6, matching ordinary compilation. An additional probe of 64 affected float32 thresholds across signs and exponents passes for equality, neighboring values, NaNs, and infinities.
  • Ruff syntax/undefined-name checks, uv pip check, and git diff --check pass. Local Windows execution is not available; the existing CI matrix covers GCC and MSVC.
  • GitHub Actions: all 8 Linux/Windows jobs passed, including GCC/MSVC and the minimum Treelite version. Successful CI run.

@dev-rinchin
dev-rinchin merged commit 5bef19e into master Nov 2, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant