Migrate CPU inference to Treelite 4 and TL2cgen

.github/workflows/python-compatibility.yml

@@ -10,22 +10,53 @@ permissions:
 
 jobs:
   cpu:
-    runs-on: ubuntu-latest
+    runs-on: ${{ matrix.os }}
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.13", "3.14"]
+        include:
+          - os: ubuntu-24.04
+            python-version: "3.8"
+            treelite: "treelite==4.1.2"
+          - os: ubuntu-latest
+            python-version: "3.12"
+            treelite: "treelite>=4.1.2,<5"
+          - os: ubuntu-latest
+            python-version: "3.12"
+            treelite: "treelite==4.1.2"
+          - os: ubuntu-latest
+            python-version: "3.13"
+            treelite: "treelite>=4.1.2,<5"
+          - os: ubuntu-latest
+            python-version: "3.14"
+            treelite: "treelite>=4.1.2,<5"
+          - os: windows-latest
+            python-version: "3.12"
+            treelite: "treelite>=4.1.2,<5"
+          - os: windows-latest
+            python-version: "3.12"
+            treelite: "treelite==4.1.2"
+          - os: windows-latest
+            python-version: "3.14"
+            treelite: "treelite>=4.1.2,<5"
+    defaults:
+      run:
+        shell: bash
+    env:
+      OMP_NUM_THREADS: "2"
     steps:
       - uses: actions/checkout@v7
       - uses: actions/setup-python@v7
         with:
           python-version: ${{ matrix.python-version }}
+      - uses: ilammy/msvc-dev-cmd@v1
+        if: runner.os == 'Windows'
       - name: Build distributions
         run: |
           python -m pip install --upgrade pip build
           python -m build
       - name: Install wheel and test dependencies
-        run: python -m pip install dist/*.whl pytest
+        run: python -m pip install dist/*.whl pytest '${{ matrix.treelite }}'
       - name: Run test suite
         working-directory: ${{ runner.temp }}
         run: python -m pytest -q -rs "$GITHUB_WORKSPACE/tests"

README.md

@@ -57,6 +57,63 @@ with CUDA 12.x:
 For other Python or CUDA versions, choose the appropriate CuPy package from the
 [CuPy installation guide](https://docs.cupy.dev/en/stable/install.html).
 
+## CPU inference with Treelite and TL2cgen
+
+Py-Boost uses Treelite 4 (`>=4.1.2,<5`) and TL2cgen 1 (`>=1,<2`) for CPU
+inference. Both dependencies are installed with Py-Boost; `treelite_runtime`
+is no longer used. NumPy 1.x and 2.x are supported, subject to your Python
+version and the other dependencies.
+
+Convert a fitted model trained with `target_splitter='Single'` (the default):
+
+```python
+from py_boost import TLPredictor, TLCompiledPredictor
+
+tl_model = TLPredictor(model)
+prediction = tl_model.predict(X_test, nthread=4)
+tl_model.dump('cpu_model')
+restored = TLPredictor.load('cpu_model')
+```
+
+Single-output and multi-output regression, binary/multilabel classification,
+and multiclass classification keep the original model's postprocessing.
+Predictions have shape `(n_samples, n_outputs)`, including single-output models.
+The underlying Treelite model stores raw scores as regression targets; the
+wrapper applies `model.postprocess_fn` once, after restoring this output shape.
+A custom CPU postprocessor can be supplied with `postprocess_fn=...`.
+Both predictor wrappers support shallow copying with `copy.copy()`; saving a
+predictor leaves the original and its copies usable.
+
+Compile the model with TL2cgen for faster CPU inference. Compilation requires a
+C compiler; prediction with the compiled library does not require a compiler:
+
+```python
+# Linux: GCC must be installed and on PATH.
+compiled = tl_model.compile('gcc', 'model.so', nthread=4)
+# Windows: run from a Visual Studio developer shell, with MSVC on PATH.
+# compiled = tl_model.compile('msvc', 'model.dll', nthread=4)
+
+prediction = compiled.predict(X_test)
+compiled.dump('compiled.pkl')
+restored_compiled = TLCompiledPredictor.load('compiled.pkl')
+```
+
+`TLCompiledPredictor.predict()` also accepts SciPy sparse matrices (CSR, CSC
+and COO) without converting them to dense arrays. Unstored entries and stored
+NaNs are treated as missing values; explicitly stored zeros remain zero.
+
+Compiled predictor dumps contain metadata and the library path. Deploy the
+`.so`/`.dll` alongside the dump. If its path changes, load it with
+`TLCompiledPredictor.load('compiled.pkl', libpath=new_path)`. Use
+`set_libpath(new_path)` to relocate an already loaded predictor.
+Use a library built for the target operating system and architecture. The
+legacy `compiler` argument accepts only its default value, `'ast_native'`.
+See the [TL2cgen migration guide](https://tl2cgen.readthedocs.io/en/latest/treelite-migration.html)
+for the underlying API changes.
+
+Treelite 3 model checkpoints and libraries compiled with `treelite_runtime`
+should be re-exported from the original Py-Boost model with this version.
+
 ## Quick tour
 
 Py-boost is easy to use since it has an interface similar to scikit-learn's. For usage examples, please see:

py_boost/gpu/losses/losses.py

@@ -111,7 +111,8 @@ def get_grad_hess(self, y_true, y_pred):
         return (y_pred - y_true), cp.ones((y_true.shape[0], 1), dtype=cp.float32)
 
     def postprocess_output(self, y_pred):
-        return cp.expm1(y_pred)
+        xp = np if type(y_pred) is np.ndarray else cp
+        return xp.expm1(y_pred)
 
     def base_score(self, y_true):
         y_true = cp.log1p(y_true)

py_boost/utils/tl2cgen_compat.py

@@ -0,0 +1,87 @@
+"""Import TL2cgen on Windows Python installations without a Conda DLL directory."""
+
+import builtins
+import os
+import sys
+from importlib import import_module
+from importlib.abc import Loader, MetaPathFinder
+from importlib.machinery import PathFinder
+
+
+class _DllDirectoryOs:
+    """An os view used only inside the dependencies' native-library loaders."""
+
+    def __init__(self, conda_bins):
+        self._conda_bins = conda_bins
+
+    def __getattr__(self, name):
+        return getattr(os, name)
+
+    def add_dll_directory(self, path):
+        if (os.path.normcase(os.path.normpath(path)) in self._conda_bins
+                and not os.path.isdir(path)):
+            return None
+        return os.add_dll_directory(path)
+
+
+class _DllDirectoryLoader(Loader):
+    def __init__(self, loader, conda_bins):
+        self._loader = loader
+        self._os = _DllDirectoryOs(conda_bins)
+
+    def create_module(self, spec):
+        return self._loader.create_module(spec)
+
+    def exec_module(self, module):
+        original_import = builtins.__import__
+
+        def module_import(name, globals=None, locals=None, fromlist=(), level=0):
+            if name == 'os' and level == 0:
+                return self._os
+            return original_import(name, globals, locals, fromlist, level)
+
+        # Give only this module a guarded os import. The process-wide os module
+        # and builtins stay unchanged, including during concurrent imports.
+        module.__dict__['__builtins__'] = dict(vars(builtins), __import__=module_import)
+        self._loader.exec_module(module)
+
+    def __getattr__(self, name):
+        return getattr(self._loader, name)
+
+
+class _DllDirectoryFinder(MetaPathFinder):
+    def __init__(self, conda_bins):
+        self._conda_bins = conda_bins
+
+    def find_spec(self, fullname, path=None, target=None):
+        if fullname not in ('treelite.core', 'tl2cgen.libloader'):
+            return None
+        spec = PathFinder.find_spec(fullname, path, target)
+        if spec is not None and spec.loader is not None:
+            spec.loader = _DllDirectoryLoader(spec.loader, self._conda_bins)
+        return spec
+
+
+def _import_tl2cgen():
+    conda_bins = {
+        os.path.normcase(os.path.join(os.path.normpath(prefix), 'Library', 'bin'))
+        for prefix in (sys.prefix, sys.base_prefix)
+    }
+    if sys.platform != 'win32' or all(os.path.isdir(path) for path in conda_bins):
+        return import_module('tl2cgen')
+
+    # TL2cgen 1.0 and older Treelite 4 versions register optional Conda directories.
+    # CPython and venv installations do not have them, so import otherwise fails
+    # before the packaged DLL can be loaded. Ignore only that missing directory
+    # inside those two modules; real DLL-loading errors still propagate.
+    finder = _DllDirectoryFinder(conda_bins)
+    sys.meta_path.insert(0, finder)
+    try:
+        return import_module('tl2cgen')
+    finally:
+        sys.meta_path.remove(finder)
+
+
+tl2cgen = _import_tl2cgen()
+# TL2cgen imports Treelite inside the guarded scope above.
+treelite = import_module('treelite')