Skip to content

fix(augur): use regression-appropriate scorers for regressor estimators - #1105

Merged
Zethson merged 1 commit into
scverse:mainfrom
Sizerta:fix-augur-regressor-scoring
Sep 21, 2026
Merged

Zethson merged 1 commit into
scverse:mainfrom
Sizerta:fix-augur-regressor-scoring

Conversation

@Sizerta

@Sizerta Sizerta commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

PR Checklist

PR Checklist

  • Referenced issue is linked
  • If you've fixed a bug or added code that should be tested, add tests!

Description of changes

Fixes #655

Augur with a regressor estimator (e.g. random_forest_regressor) could crash during predict() with

AttributeError: RandomForestRegressor has none of the following attributes: predict_proba.

when the target had more than two distinct values (for example, multiple timepoints or a continuous outcome). In this case, the multiclass branch of set_scorer was selected and returned multiclass classification scorers, including scorers using response_method="predict_proba", even though the estimator was a regressor.

The non-multiclass fallback also used a hardcoded RandomForestClassifier | LogisticRegression whitelist to select classification scorers. As a result, classifiers outside that whitelist could receive regression scorers.

set_scorer now checks whether the estimator is a regressor first, using scikit-learn's is_regressor():

  1. Regressors → regression scorers (augur_score, r2, ccc, neg_mean_squared_error, explained_variance), regardless of the multiclass argument.
  2. Classifiers with multiclass=True → multiclass classification scorers (unchanged).
  3. Other non-regressor estimators → binary classification scorers (unchanged scoring behavior, without the previous estimator whitelist).

This prevents regressor-based Augur runs from requesting class probabilities and ensures classifiers outside the previous hardcoded whitelist follow the classification-scoring path.

Technical details

  • src/pertpy/tools/_augur.py: reordered set_scorer so the is_regressor(self.estimator) branch is evaluated before the multiclass branch, and replaced the previous estimator-type ternary with the standard classification-scoring path.

  • tests/tools/test_augur.py: added two regression tests for Got an error when using 'random_forest_regressor' estimator #655:

    • test_regressor_scorer_with_more_than_two_labels — verifies that set_scorer(multiclass=True) returns the regression scorer set for a random-forest regressor.
    • test_regressor_cross_validation_with_four_timepoints — end-to-end test using four numeric timepoints, verifying that cross-validation produces a finite mean_ccc and does not produce mean_auc.

The regression-specific scorer test ensures that a regressor with multiple target values is not assigned classification scorers. The end-to-end test verifies that the corrected scorer selection avoids the predict_proba() failure during cross-validation.

Environment: Windows, Python 3.14.6, scikit-learn 1.9.1. The mypy pre-commit hook and the full hatch test matrix could not run locally because ete4 (via the tcoda extra) fails to build with Cython on Windows; this is pre-existing and unrelated to this change. CI will cover the supported test matrix.

Additional context

multiclass is only meaningful for classifiers. A regressor with more than two distinct numeric target values remains a regression problem, so the estimator type must take precedence over the multiclass flag.

The documentation checklist item from the original template was removed because this is a bug fix with no API change requiring documentation updates.

@codecov-commenter

codecov-commenter commented Sep 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.97%. Comparing base (e8abf85) to head (7b108ab).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1105   +/-   ##
=======================================
  Coverage   79.97%   79.97%           
=======================================
  Files          55       55           
  Lines        7534     7536    +2     
=======================================
+ Hits         6025     6027    +2     
  Misses       1509     1509           
Files with missing lines Coverage Δ
src/pertpy/tools/_augur.py 89.66% <100.00%> (+0.06%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Zethson Zethson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thank you!

@Zethson
Zethson merged commit a1f033e into scverse:main Sep 21, 2026
20 checks passed
@Sizerta
Sizerta deleted the fix-augur-regressor-scoring branch September 21, 2026 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Got an error when using 'random_forest_regressor' estimator

3 participants