Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

`sgtlearn` is a Python package for learning [Shape Generalized Trees (SGTs)](https://neurips.cc/virtual/2025/loc/san-diego/poster/115950).

**Using a coding agent?** Point it at the [sgtlearn agent guide](docs/agents.md) or its [published version](https://sgtlearn.readthedocs.io/en/latest/agents.html) for fitting, tuning, and inspecting a model.

- 🌳 **Shape Generalized Trees (SGTs):** A class of decision trees where each node applies a learnable, axis-aligned shape function to one or two logical features for non-linear and interpretable splits.
- 👁 **Interpretability:** Each node's shape function can be visualized directly.
- ⚡ **ShapeCART Algorithm:** An efficient induction method for learning SGTs from data.
Expand Down
119 changes: 119 additions & 0 deletions docs/agents.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# For AI coding agents

`sgtlearn` fits Shape Generalized Trees (SGTs) with ShapeCART. Each internal
node can learn a nonlinear, axis-aligned shape function over one feature or an
opt-in pair of features. Plot these functions to interpret which inputs route
to each child.

The recipe below runs on **sgtlearn 0.3.2** and the current development branch.
Set the model options explicitly because defaults vary across versions.
Install with `python -m pip install sgtlearn`. Wheels cover CPython 3.11–3.14
on Linux, macOS, and Windows (x86-64 and Arm64), with no compiler needed.
Install `pandas` for the DataFrame example.

## Fit, compare, and inspect

Use the synthetic Plus Sign dataset to compare shallow trees. Each SGT node
can group disjoint intervals into the same branch, while each CART node makes
one threshold split. Both searches use the same outer-depth range and folds.

```python
import matplotlib.pyplot as plt
import pandas as pd
from sklearn.model_selection import GridSearchCV, train_test_split
from sklearn.tree import DecisionTreeClassifier
from sgtlearn import SGTClassifier, make_plus, plot_tree

X, y = make_plus(n_samples=1500, grid=3, margin=0.07, random_state=42)
X = pd.DataFrame(X, columns=["x0", "x1"])
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.25, stratify=y, random_state=42
)
sgt = SGTClassifier(
num_partitions=2, pairwise_candidates=0, tao_n_runs=0, random_state=42,
)
depths = [1, 2, 3]
search = GridSearchCV(sgt, {"max_depth": depths, "inner_max_leaf_nodes": [2, 4]}, cv=3)
model = search.fit(X_train, y_train).best_estimator_
cart_search = GridSearchCV(
DecisionTreeClassifier(random_state=42), {"max_depth": depths}, cv=3
)
baseline = cart_search.fit(X_train, y_train).best_estimator_

print("SGT accuracy", model.score(X_test, y_test))
print("CART accuracy", baseline.score(X_test, y_test))
print("Predictions", model.predict(X_test.iloc[:3]))
tree = model.tree_export()
print("Root routing", tree["nodes"][tree["root_index"]])
plot_tree(model, X=X_train, feature_names=X.columns.tolist())
plt.show()
```

On this held-out split, the SGT scores **100.0%** and CART scores **87.7%**.
These results illustrate the effect of richer splits on this synthetic dataset
with a maximum outer depth of three.

The estimators work with sklearn pipelines, `cross_val_score`, and search
estimators. Fit preprocessing within each CV fold and evaluate the selected
model on held-out data. For regression use `SGTRegressor` with a regression
metric.

If the installed version includes `sgtlearn.tune`, start with its recommended
search spaces. Pass `suggested_params()` directly to sklearn's `GridSearchCV`
or `RandomizedSearchCV`. In an Optuna objective, pass
`suggest_params(trial)` as estimator keyword arguments. Both helpers accept
`use_tao=True` and `use_bivariate=True` to include those options. The
[tuning tutorial](tutorials/hyperparameter-tuning.ipynb) shows both workflows.
We suggest the Optuna workflow.

## Choose the smallest model that works

| Step | Settings |
| --- | --- |
| Binary SGT | Start with `num_partitions=2`, `tao_n_runs=0`, and `pairwise_candidates=0`. |
| Binary SGT with TAO | Try `tao_n_runs=10` for up to ten refinement sweeps. |
| Ternary SGT (SGT₃) | Try `num_partitions=3`, with and without TAO. Use it when a third branch improves validation performance. |
| Bivariate SGT (S²GT) | Try `pairwise_candidates=0.5` and `num_partitions=2` with TAO off. Allow more runtime before adding TAO. |
| Bivariate ternary SGT (S²GT₃) | Combine the pair budget with `num_partitions=3`. Evaluate this last since it searches pairs and allows more branches. |

Use the same folds and metric at each step. Fit a shallow binary model first
to measure runtime on your data. More rows, candidate features and pairs,
deeper trees, TAO sweeps, and forest size increase the work. Time a
representative fit before starting cross-validation on a large or wide dataset.

## Data and evaluation

- Pass dense numeric arrays or DataFrames with the same column order at fit
and predict time. The native trainer uses float32 features. NaN features
have learned routing. Replace infinities and supply targets without missing
values. For an sklearn CART comparison, ensure its input is compatible too.
- One-hot encode categoricals using vocabularies fitted on training data.
Pass each group of dummy columns to `fit` through
`feature_dict={"color": ["color_red", "color_blue"]}`. Keep the column names
and order for prediction. [Categorical and missing-value examples](tutorials/categorical-features.ipynb)
- Set `random_state` on the estimator, data split, and randomized search.
Use class labels for classification and numeric targets for regression.

## Public API

| Import from `sgtlearn` | Use |
| --- | --- |
| `SGTClassifier`, `SGTRegressor` | Fit an individual classifier or regressor. |
| `RandomSGForestClassifier`, `RandomSGForestRegressor` | Fit bootstrap ensembles of SGTs. |
| `plot_tree` | Plot a fitted tree and its node shape functions. If possible, run a loop to really make sure the tree is legible to users. |
| `export_text`, `export_graphviz` | Export routing rules or Graphviz DOT. Check for `export_graphviz` on older releases. |
| `tao.TAO_refine` | Refine a fitted tree or forest in place using the training data. |
| `configure_feature_dict`, `ProcessedFeatures` | Resolve and reuse logical feature groups. |
| `BaseShapeCART` | Shared estimator base. Instantiate a classifier or regressor. |
| `make_plus` | Generate a synthetic classification dataset for shape splits. |

Confirm available options in the [API reference](api/index.rst) and match the
docs to the installed version. For runnable repository examples see
[`examples/`](https://github.com/optimal-uoft/sgtlearn/tree/main/examples) and
[`example.py`](https://github.com/optimal-uoft/sgtlearn/blob/main/example.py).
Use [the roadmap](roadmap.rst) to check which paper features are implemented.
For background and citation see
[Empowering Decision Trees via Shape Function Branching](https://arxiv.org/abs/2510.19040)
and the [README BibTeX](https://github.com/optimal-uoft/sgtlearn#citation).
For research on optimizing compact trees and simpler shape functions, see
[Literati](https://arxiv.org/abs/2609.09299), a separate optimal SGT algorithm.
13 changes: 12 additions & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,20 @@
"sphinx.ext.mathjax",
"sphinx_copybutton",
"nbsphinx",
"myst_parser",
]

templates_path = ["_templates"]
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store", "**.ipynb_checkpoints"]
exclude_patterns = [
"_build",
"Thumbs.db",
".DS_Store",
"**.ipynb_checkpoints",
"adr",
"research",
"specs",
"superpowers",
]

# Notebooks are pre-executed and committed with outputs; never execute them at
# docs-build time (the native C++ extension is mocked and RTD has no compiler).
Expand Down Expand Up @@ -86,6 +96,7 @@
html_theme = "furo"
html_title = f"sgtlearn {version}"
html_static_path = ["_static"]
html_extra_path = ["llms.txt"]

html_theme_options = {
"source_repository": "https://github.com/optimal-uoft/sgtlearn",
Expand Down
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ Highlights

installation
quickstart
agents

.. toctree::
:maxdepth: 2
Expand Down
22 changes: 22 additions & 0 deletions docs/llms.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# sgtlearn

> Python estimators for Shape Generalized Trees and forests with a scikit-learn API.

## Start here

- [For AI coding agents](https://sgtlearn.readthedocs.io/en/latest/agents.html): Fit, tune, and inspect an SGT using supported APIs.
- [Installation](https://sgtlearn.readthedocs.io/en/latest/installation.html): Install the package or build from source.
- [Quick Start](https://sgtlearn.readthedocs.io/en/latest/quickstart.html): Train and plot a tree.

## Reference and tutorials

- [API Reference](https://sgtlearn.readthedocs.io/en/latest/api/index.html): Estimators, TAO, plotting, exports, and datasets.
- [Shape functions tutorial](https://sgtlearn.readthedocs.io/en/latest/tutorials/shape-functions.html): See how SGT nodes learn and visualize nonlinear splits.
- [Forests tutorial](https://sgtlearn.readthedocs.io/en/latest/tutorials/forests.html): Compare a single tree, an SG forest, and a CART forest.
- [Categorical features](https://sgtlearn.readthedocs.io/en/latest/tutorials/categorical-features.html): Group one-hot columns into logical features.
- [Roadmap](https://sgtlearn.readthedocs.io/en/latest/roadmap.html): Implemented and planned features.

## Recommended tuning spaces

- [`sgtlearn.tune.suggested_params()`](https://sgtlearn.readthedocs.io/en/latest/tutorials/hyperparameter-tuning.html#1.-Parameter-grids-for-scikit-learn-searches): Pass the recommended binary and ternary grids to `GridSearchCV` or `RandomizedSearchCV`.
- [`sgtlearn.tune.suggest_params(trial)`](https://sgtlearn.readthedocs.io/en/latest/tutorials/hyperparameter-tuning.html#2.-Suggestions-for-an-Optuna-trial): Sample estimator parameters in an Optuna objective. Install `sgtlearn[tune]` to use Optuna.
1 change: 1 addition & 0 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ sphinx>=7.2
furo>=2024.1.29
sphinx-copybutton>=0.5
nbsphinx>=0.9
myst-parser>=3
ipython>=8

# sgtlearn runtime dependencies (imported by autodoc when documenting the API)
Expand Down
Loading