Add NaN-aware binary loss and multilabel metrics

README.md

@@ -68,3 +68,103 @@ Py-boost is easy to use since it has an interface similar to scikit-learn's. For
   examples of parsing and inference on CPU with ONNX
 
 More examples are coming soon.
+
+## Missing binary targets
+
+For binary or multilabel targets, use `NaN` for unknown labels and `0` for
+observed negatives. `BCEWithNaNLoss` excludes missing labels from gradients
+and Hessians. Its initial probabilities are weighted frequencies over observed
+labels; zero-weight rows do not affect them. Every training target column must
+contain an observed label with positive row weight. Without `sample_weight`,
+each row has weight 1. Both all-NaN columns and columns whose observed labels
+all have zero weight raise `ValueError` listing their zero-based indices before
+initialization or tree construction. This check applies to the training
+partition of each cross-validation fold; no columns are silently removed.
+Entirely unobserved columns are allowed in evaluation sets.
+Keep `lambda_l2 > 0` so leaves without observed labels have finite updates.
+
+```python
+from py_boost import GradientBoosting
+from py_boost.gpu.losses import BCEWithNaNLoss, NaNAucMetric
+
+model = GradientBoosting(BCEWithNaNLoss(), metric=NaNAucMetric(), lambda_l2=1)
+model.fit(X_train, y_train, eval_sets=[{"X": X_valid, "y": y_valid}])
+```
+
+The loss also accepts aliases `bce_nan` and `multilabel_nan`. Its default
+metric is `NaNBCEMetric`; metric names `bce`/`logloss` and `auc`/`roc` select
+the corresponding NaN-aware metrics when used with this loss. Explicit metric
+aliases `bce_nan`, `auc_nan` and `gini_nan` are also available. Existing losses
+and their default metrics are unchanged.
+
+Import these metrics from `py_boost.gpu.losses`:
+
+| Metric | Result |
+| --- | --- |
+| `NaNBCEMetric` | Mean BCE over observed labels; `return_list=True` gives per-output losses as a CuPy array. |
+| `BCEWithNaNMetricWeighted(output_weights)` | Weighted mean of per-output BCE. |
+| `NaNAucMetric` | Macro ROC AUC; `return_list=True` gives a NumPy array. |
+| `NaNAucMetricByClass` | Per-output ROC AUC as a NumPy array, for reporting. |
+| `NaNGiniMetricByClass` | Per-output Gini (`2 * AUC - 1`) as a NumPy array, for reporting. |
+| `NaNGiniMetricAtN` | Weighted macro Gini; call with `prod_weights=` and `n=` to restrict to the first `n` target columns. |
+
+All metrics support finite, nonnegative row weights of shape `(n_samples,)` or
+`(n_samples, 1)`. NaN labels and zero-weight rows are excluded independently
+for each target. ROC AUC/Gini are undefined for outputs without positive
+weight in both classes; BCE is undefined for outputs without any positively
+weighted labels. Per-output results contain NaN for these outputs, and macro
+metrics exclude them. A scalar metric is NaN if no eligible output remains.
+Positive row weights are rescaled before aggregation so multiplying all
+weights by a common positive constant does not change the metric, subject
+to floating-point precision. All-zero weights produce an undefined metric.
+
+With early stopping enabled, an undefined (NaN) scalar metric on the last
+evaluation set raises a clear error instead of silently pruning to one tree.
+Choose an evaluation set and metric with enough observed, positively weighted
+labels, or set `es=0` to keep training and record NaN in the evaluation history.
+The per-class vector metrics are for reporting only.
+
+Observed targets must be `0` or `1`; `NaN` is the only missing-label marker.
+Training and evaluation labels are validated once in their original dtype
+before conversion to float32, including labels on zero-weight rows. A float64
+label such as `1.00000001` is rejected instead of being rounded to `1`.
+Predictions must be finite wherever a label
+has positive row weight; invalid predictions are reported as errors rather
+than treated as missing labels. BCE additionally requires probabilities in
+`[0, 1]`, while ROC AUC/Gini also accept finite decision scores.
+
+Output/product weights must follow target column order and are normalized
+over eligible, positively weighted outputs. `n` selects leading columns,
+not the outputs with the highest scores. Compared with custom implementations
+that return an unnormalized weighted sum, non-unit-sum weights therefore have
+different semantics. Standard `CrossValidation` is supported; `AdaptiveESCV`
+requires an explicit scorer for missing labels.
+
+`FilterSketch` supports zero Hessians from missing or zero-weight labels.
+Outputs with zero total Hessian are excluded from selection. If all remaining
+output importances are zero, selection uses a uniform distribution over the
+eligible outputs; if none remain, it raises an error identifying the lack of
+positive Hessian mass. The separate training-column check requires an observed
+label with positive row weight in each training output. Gradient squaring and
+per-leaf accumulation use float32 to limit GPU memory; reductions and probability
+normalization use float64. With the default BCE clipping and ordinary row
+weights, gradient magnitudes are well within the float32 range. Extreme weights
+or gradient scales can still cause overflow or underflow during squaring, and
+roundoff can affect importance estimates when leaf gradients nearly cancel.
+Float64 reductions do not remove these float32 training limitations. Zero
+gradients at an optimum are valid and do not trigger a missing-training-label
+error.
+
+For `BCEWithNaNLoss`, row weights are validated as finite and nonnegative
+before conversion. Training weights are stored as float32: overflow to infinity
+or conversion of a positive weight to zero raises an explicit representation
+error with the affected row indices. Training weights are not normalized, so
+their scale retains its effect on `lambda_l2`. Evaluation weights stay float64
+through `fit`, allowing metrics to handle weights such as `1e308` and `1e-308`.
+All-zero training weights are rejected by the per-output check; all-zero
+evaluation weights are allowed for reporting and produce an undefined metric.
+These strict validation and weighted-initialization policies apply only to
+`BCEWithNaNLoss`; existing losses retain their preprocessing and initial scores.
+
+GPU regression tests can be run with `python -m pytest -q tests`
+after installing Py-Boost, a compatible CuPy build, and pytest.

py_boost/callbacks/callback.py

@@ -1,5 +1,6 @@
 """Default callbacks"""
 import logging
+import math
 from ..utils.logging import verbosity_to_loglevel, set_stdout_level
 
 logger = logging.getLogger(__name__)
@@ -217,7 +218,11 @@ def set_verbosity_level(verbose):
 
 
 class EarlyStopping(Callback):
-    """Callback for early stopping"""
+    """Stop using the last validation set's scalar metric.
+
+    With early stopping enabled, an undefined (NaN) score raises ValueError.
+    Set num_rounds=0 to disable early stopping and this check.
+    """
 
     def __init__(self, num_rounds=100):
 
@@ -253,6 +258,13 @@ def after_iteration(self, build_info):
         num_iter = build_info['num_iter']
         # if multiple valid sets passed - use the last one
         score = build_info['iter_score'][-1]
+        if math.isnan(score):
+            raise ValueError(
+                f"Early stopping metric '{self.metric.alias}' is undefined (NaN) on "
+                f"evaluation set {len(build_info['iter_score']) - 1}. Check observed labels "
+                'and positive sample weights, choose another metric or evaluation set, '
+                'or disable early stopping with es=0.'
+            )
 
         if num_iter == 0:
             self.best_score = score

py_boost/gpu/boosting.py

@@ -5,7 +5,7 @@
 except Exception:
     pass
 from .base import Ensemble
-from .losses import loss_alias
+from .losses import Loss, loss_alias
 from .tree import DepthwiseTreeBuilder
 from .utils import pad_and_move, validate_input
 from ..callbacks.callback import EarlyStopping, EvalHistory, CallbackPipeline
@@ -282,19 +282,24 @@ def _create_build_info(self, mempool, X, X_enc, y, sample_weight, max_bin, borde
         Returns:
             DepthwiseTreeBuilder, build_info
         """
-        # quantization
+        cp.random.seed(self.seed)
 
-        y = cp.array(y, order='C', dtype=cp.float32)
+        def prepare_target(target, weights, training):
+            prepare = getattr(self.loss, 'prepare_input', None)
+            if prepare is None:
+                # Keep custom losses implementing the original interface working.
+                return Loss.prepare_input(self.loss, target, weights, training=training)
+            return prepare(target, weights, training=training)
 
-        if sample_weight is not None:
-            sample_weight = cp.array(sample_weight, order='C', dtype=cp.float32)
+        y, sample_weight = prepare_target(y, sample_weight, training=True)
+        valid = [prepare_target(x['y'], x['sample_weight'], training=False) for x in eval_sets]
+        y_val = [target for target, _ in valid]
+        w_val = [weights for _, weights in valid]
+        initialize = getattr(self.loss, 'initial_score', None)
+        self.base_score = self.loss.base_score(y) if initialize is None else initialize(y, sample_weight)
 
         X_cp = pad_and_move(X_enc)
-
         X_val = [cp.array(x, order='C') for x in eval_enc]
-        y_val = [cp.array(x['y'], order='C', dtype=cp.float32) for x in eval_sets]
-        w_val = [None if x['sample_weight'] is None else cp.array(x['sample_weight'], order='C', dtype=cp.float32)
-                 for x in eval_sets]
 
         # save nfeatures for the feature importances
         self.nfeats = X.shape[1]
@@ -313,12 +318,6 @@ def _create_build_info(self, mempool, X, X_enc, y, sample_weight, max_bin, borde
                                        max_depth=self.max_depth,
                                        max_bin=max_bin,
                                        )
-        cp.random.seed(self.seed)
-
-        y = self.loss.preprocess_input(y)
-        y_val = [self.loss.preprocess_input(x) for x in y_val]
-        self.base_score = self.loss.base_score(y)
-
         # init ensembles
         ens = cp.empty((y.shape[0], self.base_score.shape[0]), order='C', dtype=cp.float32)
         ens[:] = self.base_score

py_boost/gpu/losses/__init__.py

@@ -9,13 +9,20 @@
     'MSELoss',
     'MSLELoss',
     'BCELoss',
+    'BCEWithNaNLoss',
     'CrossEntropyLoss',
 
     'Metric',
     'RMSEMetric',
     'RMSLEMetric',
     'R2Score',
     'BCEMetric',
+    'NaNBCEMetric',
+    'BCEWithNaNMetricWeighted',
+    'NaNAucMetric',
+    'NaNAucMetricByClass',
+    'NaNGiniMetricByClass',
+    'NaNGiniMetricAtN',
     'AccuracyMetric',
     'RocAucMetric',
 

py_boost/gpu/losses/losses.py

@@ -7,7 +7,8 @@
 except Exception:
     CUDA_FOUND = False
 
-from .metrics import metric_alias, RMSEMetric, RMSLEMetric, BCEMetric
+from .metrics import metric_alias, RMSEMetric, RMSLEMetric, BCEMetric, NaNBCEMetric, NaNAucMetric
+from .metrics import _row_weights, _rescale_row_weights
 from .multiclass_metrics import multiclass_metric_alias, CrossEntropyMetric
 
 
@@ -47,6 +48,21 @@ def preprocess_input(self, y_true):
         """
         return y_true
 
+    def prepare_input(self, y_true, sample_weight=None, training=False):
+        """Prepare a dataset using the existing float32 preprocessing contract.
+
+        Losses may override this to validate values before conversion or
+        retain higher precision for evaluation weights.
+        """
+        y_true = cp.array(y_true, order='C', dtype=cp.float32)
+        if sample_weight is not None:
+            sample_weight = cp.array(sample_weight, order='C', dtype=cp.float32)
+        return self.preprocess_input(y_true), sample_weight
+
+    def initial_score(self, y_true, sample_weight=None):
+        """Initialize while preserving existing base_score(y_true) overrides."""
+        return self.base_score(y_true)
+
     def postprocess_output(self, y_pred):
         """Method defines how to postprocess sum of trees to output prediction (ex. apply sigmoid for BCELoss)
 
@@ -154,6 +170,83 @@ def get_metric_from_string(self, name=None):
         return metric_alias[name]
 
 
+class BCEWithNaNLoss(BCELoss):
+    """Binary/multilabel log loss with missing (NaN) targets.
+
+    Missing labels contribute zero gradient and Hessian. Initial log odds
+    use weighted observed-label frequencies. Every training output must have
+    an observed label with positive row weight. Evaluation outputs may be
+    entirely missing. Use positive lambda_l2 for leaves with no observed
+    labels. The default evaluation metric also ignores missing labels.
+    """
+
+    def preprocess_input(self, y_true):
+        observed = ~cp.isnan(y_true)
+        if bool((observed & (y_true != 0) & (y_true != 1)).any()):
+            raise ValueError('Observed binary targets must be 0 or 1; use NaN for missing labels')
+        return y_true
+
+    def prepare_input(self, y_true, sample_weight=None, training=False):
+        # Validate labels in their original dtype, before rounding to float32.
+        y_true = self.preprocess_input(cp.asarray(y_true))
+        if sample_weight is not None:
+            sample_weight = _row_weights(sample_weight, y_true.shape[0])
+            if training:
+                converted = sample_weight.astype(cp.float32)
+                invalid = ~cp.isfinite(converted) | ((sample_weight > 0) & (converted == 0))
+                if bool(invalid.any()):
+                    rows = cp.flatnonzero(invalid[:, 0]).get().tolist()
+                    raise ValueError(
+                        'Training sample_weight cannot be represented in float32 '
+                        'without overflow or loss of positive weights; affected rows: '
+                        f'{rows}'
+                    )
+                sample_weight = converted
+            # Evaluation weights stay float64 for stable metric aggregation.
+            sample_weight = cp.array(sample_weight, order='C')
+        return cp.array(y_true, order='C', dtype=cp.float32), sample_weight
+
+    def initial_score(self, y_true, sample_weight=None):
+        return self.base_score(y_true, sample_weight)
+
+    def base_score(self, y_true, sample_weight=None):
+        y_true = cp.asarray(y_true)
+        if y_true.ndim == 1:
+            y_true = y_true[:, None]
+        observed = ~cp.isnan(y_true)
+        if sample_weight is not None:
+            weights = _row_weights(sample_weight, y_true.shape[0])
+            observed &= weights > 0
+        counts = observed.sum(axis=0)
+        missing_outputs = cp.flatnonzero(counts == 0)
+        if missing_outputs.size:
+            raise ValueError(
+                'Training target columns have no observed labels '
+                f'(zero-based indices): {missing_outputs.get().tolist()} '
+                '(positive sample weight required)'
+            )
+        if sample_weight is None:
+            means = cp.where(observed, y_true, 0).sum(axis=0) / counts
+        else:
+            weights = _rescale_row_weights(cp.where(observed, weights, 0))
+            means = (cp.where(observed, y_true, 0) * weights).sum(axis=0) / weights.sum(axis=0)
+        means = means.astype(cp.result_type(y_true.dtype, cp.float32))
+        means = cp.clip(means, self.clip_value, 1 - self.clip_value)
+        return cp.log(means / (1 - means))
+
+    def get_grad_hess(self, y_true, y_pred):
+        missing = cp.isnan(y_true)
+        grad, hess = super().get_grad_hess(cp.where(missing, 0, y_true), y_pred)
+        return cp.where(missing, 0, grad), cp.where(missing, 0, hess)
+
+    def get_metric_from_string(self, name=None):
+        if name is None or name in ('bce', 'logloss'):
+            return NaNBCEMetric()
+        if name in ('auc', 'roc'):
+            return NaNAucMetric()
+        return metric_alias[name]
+
+
 def softmax(x, clip_val=1e-5):
 
     xp = np if type(x) is np.ndarray else cp
@@ -225,6 +318,9 @@ def get_metric_from_string(self, name=None):
     'multilabel': BCELoss(),
     'logloss': BCELoss(),
 
+    'bce_nan': BCEWithNaNLoss(),
+    'multilabel_nan': BCEWithNaNLoss(),
+
     # for multiclass
     'multiclass': CrossEntropyLoss(),
     'crossentropy': CrossEntropyLoss(),