Skip to content

Add optional tuning helpers and shared regularized defaults - #95

Merged
upadhyan merged 20 commits into
mainfrom
codex/feat-29-tuning-defaults
Nov 16, 2026
Merged

upadhyan merged 20 commits into
mainfrom
codex/feat-29-tuning-defaults

Conversation

@upadhyan

Copy link
Copy Markdown
Collaborator

Closes #29.

Add built-in search spaces so users can tune SGT trees and forest base trees without defining every hyperparameter range themselves. Install sgtlearn[tune] and use suggest_params(trial) with Optuna, or suggested_params() with sklearn grid/randomized search. Both helpers accept independent TAO and bivariate flags and omit inactive parameters and constants, except the required TAO enablement override.

Changes

  • Apply shared regularized tree defaults to classifiers, regressors, and forest base trees. Ordinary fitting uses binary ShapeCART, with TAO and bivariate branching off. Existing forest-specific defaults follow sklearn.
  • Standardize regression targets using training-only, sample-weighted statistics for each output and bootstrap tree. Predictions, exported leaf values, and later TAO inputs retain original target units.
  • Bound post-fit forest TAO target copies by active workers instead of allocating a transformed target matrix for every tree upfront.
  • Include the reproducible 512-trial, 12-dataset TabArena study, Pareto selection, per-dataset analysis, sensitivity sweeps, and reserved-test results. Preserve the preliminary five-dataset study separately.
  • Add an executed tutorial showing both tuning APIs, sklearn search examples, and all four TAO/bivariate configurations.

Validation

  • Full suite: 1,531 passed, 2 skipped.
  • New memory regression test fails before the fix and passes with one and two workers; existing TAO and target-scaling tests pass.
  • Ruff and Black passed for changed production code and new tests.
  • Tutorial executed successfully; Sphinx built with warnings treated as errors.
  • Native wheel and optional-extra installation smoke checks passed during branch development.

Limits and follow-ups

The saved benchmark results measure the study-selected configuration, including TAO, bivariate branching, and seven partitions. The final constructor defaults include subsequent explicit overrides and were not evaluated in another full study. Forest performance was not benchmarked. See docs/research/2026-10-26-tuning-defaults.md for results and limitations.

Moving TAO out of estimator constructors and renaming num_partitions to branching_factor remain separate v1.0.0 work in #93 and #94.

@upadhyan
upadhyan merged commit 4ddb684 into main Nov 16, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Suggested Hyperparameters

1 participant