Repository navigation
Fix boosting training and recover from failed Optuna trials - #198
Open
SergeevVladislav wants to merge 1 commit into
Open
SergeevVladislav wants to merge 1 commit into
SergeevVladislav wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was done:
An exception in an Optuna trial could abort the pipeline, including when the first trial failed before a best trial existed. Tuning now continues after individual failures, stops cleanly at
fail_tolerance, retains the best completed parameters, and falls back to initial parameters when no trial completes. Invalid distribution types still fail before tuning, and keyboard interrupts propagate with progress-bar cleanup.This PR also fixes defects reproduced while auditing the three boosting adapters:
best_iteration, including when training reaches the iteration limit. Importance uses the nativeget_score()API, retains feature names, and includes unused features with zero importance. Linear-booster predictions retain their supported behavior.inplaceargument.fobjAPI and LightGBM 4's callableobjective. Explicit loss-adapter dictionaries are validated correctly. Multiclass F1 objectives retain the backend's derivative shape and apply observation weights; metric callbacks do not apply the prediction softmax to class labels.log1p(target)withexpm1predictions. RMSLE evaluation uses log-space RMSE; other requested metrics use the original scale. Quantileqand Huberareach the native parameters, and quantile callbacks honor their parameter and observation weights.Added 74 regression cases and documented the behavior. Reviewed the complete diff and affected training, tuning, scoring, inference, and serialization paths, including missing-class scoring, legacy metric input formats, and convergence on large targets.
Validation:
tox.ini.fobj.212e38e: 60 failed, 52 passed, plus one teardown error from an unused XGBoost parameter warning.cuda:0/GPUmodel configuration verified. These used a separate local harness, not CPU fallback.e27f2fb): no conflicts; 381 tests passed across the combined task, boosting, tuning, and preset suites.pre-commit run --files <changed files>andcodespell <changed files>: passed.sphinx-build -E -b html docs <build directory>: succeeded with six existing warnings; all 193 generated API pages were verified.Full-suite status: a complete Python 3.13 run before the final RMSLE scale refinement finished with 343 passed, 1 failed, 9 skipped, and 4 passing subtests. The final full-suite rerun is in progress. The nine skips require CUDA-enabled Torch, while this environment has CPU Torch; native boosting CUDA was tested separately. Other Python versions were covered by the focused suite, not full-suite runs.
The remaining failure is
TestTabularAutoMLXGB.test_fit_predict_binary, whose existing fixture requires both OOF and holdout AUC above 0.69. The controlled comparison gives OOF/holdout 0.69794 / 0.68610 with upstream all-tree predictions and 0.67353 / 0.71981 with the corrected best-iteration predictions. Thus the upstream fixture already fails on holdout, while this PR fails its OOF threshold and improves holdout AUC. The fixture and thresholds are unchanged; the full suite is not reported as green. Best-iteration prediction intentionally changes model outputs and requires quality review beyond these regression tests.Why / Goal:
Keep individual tuning failures from discarding a usable pipeline, and correct the reproduced boosting training and compatibility defects. This is a separate PR against
development; it does not contain the metrics expansion from #195.Related issues:
N/A. No issue was created for this audit.