fix(evaluation): load_model raises RecursionError on models holding an evaluator

[nikolas-sapa]

Problem

save_model succeeds on a BanditClassifier, but load_model raises RecursionError:

>>> learner = BanditClassifier(
...     schema=schema,
...     base_classifiers=[HoeffdingTree, NaiveBayes],
...     policy=EpsilonGreedy(epsilon=0.1, burn_in=100),
... )
>>> save_model(learner, fd)
>>> load_model(fd)
RecursionError: maximum recursion depth exceeded

The traceback is ~3000 lines of two frames repeating, reported at evaluation.py:254, ClassificationEvaluator.__getattr__.

Cause

The recursion is real, and it is not about the size of the object graph.

Python calls __getattr__ for every attribute that is not set. Unpickling begins by asking a freshly created, still empty object for __setstate__, so the lookup lands in __getattr__, which answered by calling metrics_header():

evaluation.py:254  in __getattr__        if metric in self.metrics_header():
evaluation.py:224  in metrics_header     ... = self.moa_basic_evaluator.getPerformanceMeasurements()
evaluation.py:254  in __getattr__        if metric in self.metrics_header():
evaluation.py:224  in metrics_header     ... = self.moa_basic_evaluator.getPerformanceMeasurements()
...

self.moa_basic_evaluator is not set yet at that point, so the attribute lookup came straight back to __getattr__ and repeated until the stack ran out. Two frames alternating.

Nothing is special about BanditClassifier. It holds a list of ClassificationEvaluator objects, so unpickling it unpickles them, so it hits this. Any object owning a ClassificationEvaluator does the same, and a bare ClassificationEvaluator does it on its own:

object	before	after
BanditClassifier	RecursionError	loads
ClassificationEvaluator	RecursionError	loads
ClassificationWindowedEvaluator	RecursionError	loads
PrequentialResults	RecursionError	loads
RegressionEvaluator	loads	loads
HoeffdingTree (control)	loads	loads

Fix

Three sites have this shape, so all three are fixed:

• ClassificationEvaluator.__getattr__ and ClassificationWindowedEvaluator.__getattr__ read self.moa_basic_evaluator, and both returned None for a name they did not know. They now stop when the evaluator is not in __dict__, and raise AttributeError otherwise.
• PrequentialResults.__getattr__ already raised AttributeError, but read self.cumulative on the way, so it recursed in the same way. It now stops when the cumulative evaluator is not there.

Why not raise the recursion limit

Raising the limit would only push the failure onto the next object in the graph, and would leave every future unpickling of these classes slow and fragile. The recursion limit is untouched by this change.

Second bug fixed by the same change

Returning None for an unknown metric meant hasattr was always True on these objects. That made the guard in plot_regression_results dead code:

if not hasattr(result.windowed, "rmse"):
    raise ValueError("Cannot process results that do not include regression results.")

This never fired, because hasattr(result.windowed, "rmse") returned True even for classification results. Verified before and after:

before:  classification: hasattr(windowed,'rmse') -> True   (should be False)
after:   classification: hasattr(windowed,'rmse') -> False  (correct)

Tests

• test_bandit_classifier_save_load_roundtrip — round-trips a BanditClassifier, then checks it predicts identically to the original on 20 held-out instances, and that the evaluator state came back. Identity of predictions is what makes it load-bearing: a graph that loads but loses the models or evaluators fails here.
• test_classification_evaluator_save_load_roundtrip — the smallest object that reproduces the bug, so the test pins the root cause rather than the object that happens to hold one.
• test_evaluator_getattr_does_not_recurse_before_init — __getattr__ must decline a name on an object built with __new__. No unpickling involved, so the test fails on the recursion itself instead of waiting for a RecursionError.

All three fail on the parent commit 65a3262:

FAILED tests/test_bandit_classifier.py::test_bandit_classifier_save_load_roundtrip
FAILED tests/test_bandit_classifier.py::test_classification_evaluator_save_load_roundtrip
FAILED tests/test_bandit_classifier.py::test_evaluator_getattr_does_not_recurse_before_init
E   RecursionError: maximum recursion depth exceeded
!!! Recursion detected (same locals & position)
3 failed, 1 passed

and pass on this branch: 4 passed.

Verification

uv run pytest -q tests --ignore=tests/ocl (torch is not installed in this environment, so tests/ocl fails collection):

	failed	passed	skipped
main @ 65a3262	9	265	21
this branch	9	268	21

Same 9 failures before and after, all ModuleNotFoundError: No module named 'torch'. The 3 extra passes are the 3 new tests. uv run invoke fmt is clean.