From 7c314711b2322a6d540d13cb96fea31b4ba0626c Mon Sep 17 00:00:00 2001 From: Nakul Upadhya Date: Mon, 16 Nov 2026 11:41:57 -0500 Subject: [PATCH 1/4] Add a getting-started guide for coding agents --- README.md | 2 + docs/agents.md | 101 ++++++++++++++++++++++++++++++++++++++++++ docs/conf.py | 13 +++++- docs/index.rst | 1 + docs/llms.txt | 18 ++++++++ docs/requirements.txt | 1 + 6 files changed, 135 insertions(+), 1 deletion(-) create mode 100644 docs/agents.md create mode 100644 docs/llms.txt diff --git a/README.md b/README.md index 04af764..09ea99a 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,8 @@ `sgtlearn` is a Python package for learning [Shape Generalized Trees (SGTs)](https://neurips.cc/virtual/2025/loc/san-diego/poster/115950). +**Using a coding agent?** Point it at the [sgtlearn agent guide](docs/agents.md) or its [published version](https://sgtlearn.readthedocs.io/en/latest/agents.html) for fitting, tuning, and inspecting a model. + - 🌳 **Shape Generalized Trees (SGTs):** A class of decision trees where each node applies a learnable, axis-aligned shape function to one or two logical features for non-linear and interpretable splits. - 👁 **Interpretability:** Each node's shape function can be visualized directly. - ⚡ **ShapeCART Algorithm:** An efficient induction method for learning SGTs from data. diff --git a/docs/agents.md b/docs/agents.md new file mode 100644 index 0000000..d232437 --- /dev/null +++ b/docs/agents.md @@ -0,0 +1,101 @@ +# For AI coding agents + +`sgtlearn` fits Shape Generalized Trees (SGTs) with ShapeCART. Each internal +node can learn a nonlinear, axis-aligned shape function over one feature or an +opt-in pair of features. Plot these functions to interpret which inputs route +to each child. + +The recipe below runs on **sgtlearn 0.3.2** and the current development branch. +Set the model options explicitly because defaults vary across versions. +Install with `python -m pip install sgtlearn`. Wheels cover CPython 3.11–3.14 +on Linux, macOS, and Windows (x86-64 and Arm64), with no compiler needed. +Install `pandas` for the DataFrame example. + +## Fit, compare, and inspect + +```python +import matplotlib.pyplot as plt +from sklearn.datasets import load_iris +from sklearn.model_selection import GridSearchCV, train_test_split +from sklearn.tree import DecisionTreeClassifier +from sgtlearn import SGTClassifier, plot_tree + +X, y = load_iris(return_X_y=True, as_frame=True) +X_train, X_test, y_train, y_test = train_test_split( + X, y, test_size=0.25, stratify=y, random_state=42 +) +sgt = SGTClassifier( + max_depth=4, num_partitions=2, pairwise_candidates=0, + tao_n_runs=0, random_state=42, +) +search = GridSearchCV(sgt, {"max_depth": [2, 4], "inner_max_leaf_nodes": [2, 4]}, cv=3) +model = search.fit(X_train, y_train).best_estimator_ +baseline = DecisionTreeClassifier(max_depth=model.max_depth, random_state=42) +baseline.fit(X_train, y_train) + +print("SGT accuracy", model.score(X_test, y_test)) +print("CART accuracy", baseline.score(X_test, y_test)) +print("Predictions", model.predict(X_test.iloc[:3])) +tree = model.tree_export() +print("Root routing", tree["nodes"][tree["root_index"]]) +plot_tree(model, X=X_train, feature_names=X.columns.tolist()) +plt.show() +``` + +The estimators work with sklearn pipelines, `cross_val_score`, and search +estimators. Fit preprocessing within each CV fold and evaluate the selected +model on held-out data. For regression use `SGTRegressor` with a regression +metric. See the [tuning tutorial](tutorials/hyperparameter-tuning.ipynb) for +the search helpers on versions that include `sgtlearn.tune`. + +## Choose the smallest model that works + +| Step | Settings | +| --- | --- | +| Binary SGT | Start with `num_partitions=2`, `tao_n_runs=0`, and `pairwise_candidates=0`. Tune `max_depth` for outer depth and `inner_max_leaf_nodes` or `inner_max_depth` for shape complexity. | +| Binary SGT with TAO | Try `tao_n_runs=10` for up to ten refinement sweeps. | +| Ternary SGT (SGT₃) | Try `num_partitions=3`, with and without TAO. Use it when a third branch improves validation performance. | +| Bivariate SGT (S²GT) | Try `pairwise_candidates=0.5` and `num_partitions=2` with TAO off. Allow more runtime before adding TAO. | +| Bivariate ternary SGT (S²GT₃) | Combine the pair budget with `num_partitions=3`. Evaluate this last since it searches pairs and allows more branches. | + +Use the same folds and metric at each step. Fit a shallow binary model first +to measure runtime on your data. More rows, candidate features and pairs, +deeper trees, TAO sweeps, and forest size increase the work. Time a +representative fit before starting cross-validation on a large or wide dataset. + +## Data and evaluation + +- Pass dense numeric arrays or DataFrames with the same column order at fit + and predict time. The native trainer uses float32 features. NaN features + have learned routing. Replace infinities and supply targets without missing + values. For an sklearn CART comparison, ensure its input is compatible too. +- One-hot encode categoricals using vocabularies fitted on training data. + Pass each group of dummy columns to `fit` through + `feature_dict={"color": ["color_red", "color_blue"]}`. Keep the column names + and order for prediction. [Categorical and missing-value examples](tutorials/categorical-features.ipynb) +- Set `random_state` on the estimator, data split, and randomized search. + Use class labels for classification and numeric targets for regression. + +## Public API + +| Import from `sgtlearn` | Use | +| --- | --- | +| `SGTClassifier`, `SGTRegressor` | Fit an individual classifier or regressor. | +| `RandomSGForestClassifier`, `RandomSGForestRegressor` | Fit bootstrap ensembles of SGTs. | +| `plot_tree` | Plot a fitted tree and its node shape functions. Use `max_depth=1` to focus on the root split. | +| `export_text`, `export_graphviz` | Export routing rules or Graphviz DOT. Check for `export_graphviz` on older releases. | +| `tao.TAO_refine` | Refine a fitted tree or forest in place using the training data. | +| `configure_feature_dict`, `ProcessedFeatures` | Resolve and reuse logical feature groups. | +| `BaseShapeCART` | Shared estimator base. Instantiate a classifier or regressor. | +| `make_plus` | Generate a synthetic classification dataset for shape splits. | + +Confirm available options in the [API reference](api/index.rst) and match the +docs to the installed version. For runnable repository examples see +[`examples/`](https://github.com/optimal-uoft/sgtlearn/tree/main/examples) and +[`example.py`](https://github.com/optimal-uoft/sgtlearn/blob/main/example.py). +Use [the roadmap](roadmap.rst) to check which paper features are implemented. +For background and citation see +[Empowering Decision Trees via Shape Function Branching](https://arxiv.org/abs/2510.19040) +and the [README BibTeX](https://github.com/optimal-uoft/sgtlearn#citation). +For research on optimizing compact trees and simpler shape functions, see +[Literati](https://arxiv.org/abs/2609.09299), a separate optimal SGT algorithm. diff --git a/docs/conf.py b/docs/conf.py index b2e47ea..8671909 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -35,10 +35,20 @@ "sphinx.ext.mathjax", "sphinx_copybutton", "nbsphinx", + "myst_parser", ] templates_path = ["_templates"] -exclude_patterns = ["_build", "Thumbs.db", ".DS_Store", "**.ipynb_checkpoints"] +exclude_patterns = [ + "_build", + "Thumbs.db", + ".DS_Store", + "**.ipynb_checkpoints", + "adr", + "research", + "specs", + "superpowers", +] # Notebooks are pre-executed and committed with outputs; never execute them at # docs-build time (the native C++ extension is mocked and RTD has no compiler). @@ -86,6 +96,7 @@ html_theme = "furo" html_title = f"sgtlearn {version}" html_static_path = ["_static"] +html_extra_path = ["llms.txt"] html_theme_options = { "source_repository": "https://github.com/optimal-uoft/sgtlearn", diff --git a/docs/index.rst b/docs/index.rst index 63d2b9a..a14f60e 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -39,6 +39,7 @@ Highlights installation quickstart + agents .. toctree:: :maxdepth: 2 diff --git a/docs/llms.txt b/docs/llms.txt new file mode 100644 index 0000000..06ff3a7 --- /dev/null +++ b/docs/llms.txt @@ -0,0 +1,18 @@ +# sgtlearn + +> Python estimators for Shape Generalized Trees and forests with a scikit-learn API. + +## Start here + +- [For AI coding agents](https://sgtlearn.readthedocs.io/en/latest/agents.html): Fit, tune, and inspect an SGT using supported APIs. +- [Installation](https://sgtlearn.readthedocs.io/en/latest/installation.html): Install the package or build from source. +- [Quick Start](https://sgtlearn.readthedocs.io/en/latest/quickstart.html): Train and plot a tree. + +## Reference and tutorials + +- [API Reference](https://sgtlearn.readthedocs.io/en/latest/api/index.html): Estimators, TAO, plotting, exports, and datasets. +- [Shape functions tutorial](https://sgtlearn.readthedocs.io/en/latest/tutorials/shape-functions.html): See how SGT nodes learn and visualize nonlinear splits. +- [Forests tutorial](https://sgtlearn.readthedocs.io/en/latest/tutorials/forests.html): Compare a single tree, an SG forest, and a CART forest. +- [Categorical features](https://sgtlearn.readthedocs.io/en/latest/tutorials/categorical-features.html): Group one-hot columns into logical features. +- [Hyperparameter tuning](https://sgtlearn.readthedocs.io/en/latest/tutorials/hyperparameter-tuning.html): Search with scikit-learn and Optuna. +- [Roadmap](https://sgtlearn.readthedocs.io/en/latest/roadmap.html): Implemented and planned features. diff --git a/docs/requirements.txt b/docs/requirements.txt index 6618778..046887c 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -7,6 +7,7 @@ sphinx>=7.2 furo>=2024.1.29 sphinx-copybutton>=0.5 nbsphinx>=0.9 +myst-parser>=3 ipython>=8 # sgtlearn runtime dependencies (imported by autodoc when documenting the API) From 8417dfc15c50e739236813f15a729a8f301469b7 Mon Sep 17 00:00:00 2001 From: Nakul Upadhya Date: Mon, 16 Nov 2026 11:47:24 -0500 Subject: [PATCH 2/4] Point coding agents to recommended tuning spaces --- docs/agents.md | 10 ++++++++-- docs/llms.txt | 6 +++++- 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/docs/agents.md b/docs/agents.md index d232437..5d58740 100644 --- a/docs/agents.md +++ b/docs/agents.md @@ -45,8 +45,14 @@ plt.show() The estimators work with sklearn pipelines, `cross_val_score`, and search estimators. Fit preprocessing within each CV fold and evaluate the selected model on held-out data. For regression use `SGTRegressor` with a regression -metric. See the [tuning tutorial](tutorials/hyperparameter-tuning.ipynb) for -the search helpers on versions that include `sgtlearn.tune`. +metric. + +If the installed version includes `sgtlearn.tune`, start with its recommended +search spaces. Pass `suggested_params()` directly to sklearn's `GridSearchCV` +or `RandomizedSearchCV`. In an Optuna objective, pass +`suggest_params(trial)` as estimator keyword arguments. Both helpers accept +`use_tao=True` and `use_bivariate=True` to include those options. The +[tuning tutorial](tutorials/hyperparameter-tuning.ipynb) shows both workflows. ## Choose the smallest model that works diff --git a/docs/llms.txt b/docs/llms.txt index 06ff3a7..091dd3a 100644 --- a/docs/llms.txt +++ b/docs/llms.txt @@ -14,5 +14,9 @@ - [Shape functions tutorial](https://sgtlearn.readthedocs.io/en/latest/tutorials/shape-functions.html): See how SGT nodes learn and visualize nonlinear splits. - [Forests tutorial](https://sgtlearn.readthedocs.io/en/latest/tutorials/forests.html): Compare a single tree, an SG forest, and a CART forest. - [Categorical features](https://sgtlearn.readthedocs.io/en/latest/tutorials/categorical-features.html): Group one-hot columns into logical features. -- [Hyperparameter tuning](https://sgtlearn.readthedocs.io/en/latest/tutorials/hyperparameter-tuning.html): Search with scikit-learn and Optuna. - [Roadmap](https://sgtlearn.readthedocs.io/en/latest/roadmap.html): Implemented and planned features. + +## Recommended tuning spaces + +- [`sgtlearn.tune.suggested_params()`](https://sgtlearn.readthedocs.io/en/latest/tutorials/hyperparameter-tuning.html#1.-Parameter-grids-for-scikit-learn-searches): Pass the recommended binary and ternary grids to `GridSearchCV` or `RandomizedSearchCV`. +- [`sgtlearn.tune.suggest_params(trial)`](https://sgtlearn.readthedocs.io/en/latest/tutorials/hyperparameter-tuning.html#2.-Suggestions-for-an-Optuna-trial): Sample estimator parameters in an Optuna objective. Install `sgtlearn[tune]` to use Optuna. From c7550dce90510790991129f4f5c51ad2ca87b3eb Mon Sep 17 00:00:00 2001 From: Nakul Upadhya Date: Mon, 16 Nov 2026 12:28:50 -0500 Subject: [PATCH 3/4] Demonstrate shape splits on the Plus Sign dataset --- docs/agents.md | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/docs/agents.md b/docs/agents.md index 5d58740..2cba4fc 100644 --- a/docs/agents.md +++ b/docs/agents.md @@ -13,25 +13,32 @@ Install `pandas` for the DataFrame example. ## Fit, compare, and inspect +Use the synthetic Plus Sign dataset to compare shallow trees. Each SGT node +can group disjoint intervals into the same branch, while each CART node makes +one threshold split. Both searches use the same outer-depth range and folds. + ```python import matplotlib.pyplot as plt -from sklearn.datasets import load_iris +import pandas as pd from sklearn.model_selection import GridSearchCV, train_test_split from sklearn.tree import DecisionTreeClassifier -from sgtlearn import SGTClassifier, plot_tree +from sgtlearn import SGTClassifier, make_plus, plot_tree -X, y = load_iris(return_X_y=True, as_frame=True) +X, y = make_plus(n_samples=1500, grid=3, margin=0.07, random_state=42) +X = pd.DataFrame(X, columns=["x0", "x1"]) X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.25, stratify=y, random_state=42 ) sgt = SGTClassifier( - max_depth=4, num_partitions=2, pairwise_candidates=0, - tao_n_runs=0, random_state=42, + num_partitions=2, pairwise_candidates=0, tao_n_runs=0, random_state=42, ) -search = GridSearchCV(sgt, {"max_depth": [2, 4], "inner_max_leaf_nodes": [2, 4]}, cv=3) +depths = [1, 2, 3] +search = GridSearchCV(sgt, {"max_depth": depths, "inner_max_leaf_nodes": [2, 4]}, cv=3) model = search.fit(X_train, y_train).best_estimator_ -baseline = DecisionTreeClassifier(max_depth=model.max_depth, random_state=42) -baseline.fit(X_train, y_train) +cart_search = GridSearchCV( + DecisionTreeClassifier(random_state=42), {"max_depth": depths}, cv=3 +) +baseline = cart_search.fit(X_train, y_train).best_estimator_ print("SGT accuracy", model.score(X_test, y_test)) print("CART accuracy", baseline.score(X_test, y_test)) @@ -42,6 +49,10 @@ plot_tree(model, X=X_train, feature_names=X.columns.tolist()) plt.show() ``` +On this held-out split, the SGT scores **100.0%** and CART scores **87.7%**. +These results illustrate the effect of richer splits on this synthetic dataset +with a maximum outer depth of three. + The estimators work with sklearn pipelines, `cross_val_score`, and search estimators. Fit preprocessing within each CV fold and evaluate the selected model on held-out data. For regression use `SGTRegressor` with a regression From 6e66d251b3329a85c4a0afda7d772f5b10af0254 Mon Sep 17 00:00:00 2001 From: Nakul Upadhya Date: Mon, 16 Nov 2026 12:37:05 -0500 Subject: [PATCH 4/4] Refine tuning and plotting guidance for agents --- docs/agents.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/agents.md b/docs/agents.md index 2cba4fc..39bf4f4 100644 --- a/docs/agents.md +++ b/docs/agents.md @@ -64,12 +64,13 @@ or `RandomizedSearchCV`. In an Optuna objective, pass `suggest_params(trial)` as estimator keyword arguments. Both helpers accept `use_tao=True` and `use_bivariate=True` to include those options. The [tuning tutorial](tutorials/hyperparameter-tuning.ipynb) shows both workflows. +We suggest the Optuna workflow. ## Choose the smallest model that works | Step | Settings | | --- | --- | -| Binary SGT | Start with `num_partitions=2`, `tao_n_runs=0`, and `pairwise_candidates=0`. Tune `max_depth` for outer depth and `inner_max_leaf_nodes` or `inner_max_depth` for shape complexity. | +| Binary SGT | Start with `num_partitions=2`, `tao_n_runs=0`, and `pairwise_candidates=0`. | | Binary SGT with TAO | Try `tao_n_runs=10` for up to ten refinement sweeps. | | Ternary SGT (SGT₃) | Try `num_partitions=3`, with and without TAO. Use it when a third branch improves validation performance. | | Bivariate SGT (S²GT) | Try `pairwise_candidates=0.5` and `num_partitions=2` with TAO off. Allow more runtime before adding TAO. | @@ -99,7 +100,7 @@ representative fit before starting cross-validation on a large or wide dataset. | --- | --- | | `SGTClassifier`, `SGTRegressor` | Fit an individual classifier or regressor. | | `RandomSGForestClassifier`, `RandomSGForestRegressor` | Fit bootstrap ensembles of SGTs. | -| `plot_tree` | Plot a fitted tree and its node shape functions. Use `max_depth=1` to focus on the root split. | +| `plot_tree` | Plot a fitted tree and its node shape functions. If possible, run a loop to really make sure the tree is legible to users. | | `export_text`, `export_graphviz` | Export routing rules or Graphviz DOT. Check for `export_graphviz` on older releases. | | `tao.TAO_refine` | Refine a fitted tree or forest in place using the training data. | | `configure_feature_dict`, `ProcessedFeatures` | Resolve and reuse logical feature groups. |