fix(augur): use regression-appropriate scorers for regressor estimators - #1105
Merged
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1105 +/- ##
=======================================
Coverage 79.97% 79.97%
=======================================
Files 55 55
Lines 7534 7536 +2
=======================================
+ Hits 6025 6027 +2
Misses 1509 1509
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Checklist
PR Checklist
Description of changes
Fixes #655
Augurwith a regressor estimator (e.g.random_forest_regressor) could crash duringpredict()withAttributeError: RandomForestRegressor has none of the following attributes: predict_proba.when the target had more than two distinct values (for example, multiple timepoints or a continuous outcome). In this case, the
multiclassbranch ofset_scorerwas selected and returned multiclass classification scorers, including scorers usingresponse_method="predict_proba", even though the estimator was a regressor.The non-multiclass fallback also used a hardcoded
RandomForestClassifier | LogisticRegressionwhitelist to select classification scorers. As a result, classifiers outside that whitelist could receive regression scorers.set_scorernow checks whether the estimator is a regressor first, using scikit-learn'sis_regressor():augur_score,r2,ccc,neg_mean_squared_error,explained_variance), regardless of themulticlassargument.multiclass=True→ multiclass classification scorers (unchanged).This prevents regressor-based Augur runs from requesting class probabilities and ensures classifiers outside the previous hardcoded whitelist follow the classification-scoring path.
Technical details
src/pertpy/tools/_augur.py: reorderedset_scorerso theis_regressor(self.estimator)branch is evaluated before themulticlassbranch, and replaced the previous estimator-type ternary with the standard classification-scoring path.tests/tools/test_augur.py: added two regression tests for Got an error when using 'random_forest_regressor' estimator #655:test_regressor_scorer_with_more_than_two_labels— verifies thatset_scorer(multiclass=True)returns the regression scorer set for a random-forest regressor.test_regressor_cross_validation_with_four_timepoints— end-to-end test using four numeric timepoints, verifying that cross-validation produces a finitemean_cccand does not producemean_auc.The regression-specific scorer test ensures that a regressor with multiple target values is not assigned classification scorers. The end-to-end test verifies that the corrected scorer selection avoids the
predict_proba()failure during cross-validation.Environment: Windows, Python 3.14.6, scikit-learn 1.9.1. The mypy pre-commit hook and the full
hatch testmatrix could not run locally becauseete4(via thetcodaextra) fails to build with Cython on Windows; this is pre-existing and unrelated to this change. CI will cover the supported test matrix.Additional context
multiclassis only meaningful for classifiers. A regressor with more than two distinct numeric target values remains a regression problem, so the estimator type must take precedence over themulticlassflag.The documentation checklist item from the original template was removed because this is a bug fix with no API change requiring documentation updates.