Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 64 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ on:
branches: [main]

concurrency:
# One cancelable group per workflow+ref; covers Ubuntu matrix, macOS smoke,
# notebooks, and docs (no per-OS duplicate concurrency keys).
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

Expand Down Expand Up @@ -90,6 +92,10 @@ jobs:
fh.write(f"{name}={text}\n")
print(f"{name}={text}")

# Path gates (expensive jobs skip when false unless workflow=true):
# - test → Ubuntu pytest matrix AND macOS core smoke (test-macos)
# - notebooks → Ubuntu nbmake only (never macOS)
# - docs / lint → as named
lint = hit("src/", "tests/", "pyproject.toml", ".pre-commit-config.yaml")
test = hit("src/", "tests/", "scripts/", "pyproject.toml", "uv.lock")
notebooks = hit("examples/", "src/", "pyproject.toml", "uv.lock")
Expand Down Expand Up @@ -122,6 +128,8 @@ jobs:
uvx ruff format --check src/ tests/

test:
# Ubuntu full suite runs unmarked + marked tests (Ray when extras installed).
# macOS smoke (TASK-2142) uses: -m "not slow and not distributed and not ray"
name: test (${{ matrix.python-version }})
needs: changes
if: needs.changes.outputs.test == 'true' || needs.changes.outputs.workflow == 'true'
Expand All @@ -132,6 +140,8 @@ jobs:
include:
- python-version: "3.10"
coverage: "false"
- python-version: "3.11"
coverage: "false"
- python-version: "3.12"
coverage: "true"

Expand Down Expand Up @@ -187,7 +197,44 @@ jobs:
files: ./coverage.xml
fail_ci_if_error: false

test-macos:
# Core smoke on macOS (Q6/Q7/R8): Python 3.12, CPU torch, hard-fail.
# Shares the ``test`` path filter with Ubuntu pytest (not ``notebooks``).
# Marker expression from TASK-2141; no Ray / Dask extras (ray tests deselected).
# nbmake never runs on macOS — only notebook-plan / notebooks on Ubuntu.
name: test (macos-3.12)
needs: changes
if: needs.changes.outputs.test == 'true' || needs.changes.outputs.workflow == 'true'
runs-on: macos-latest
steps:
- uses: actions/checkout@v7

- name: Install uv
uses: astral-sh/setup-uv@v9.0.0
with:
python-version: "3.12"
enable-cache: true
cache-dependency-glob: "uv.lock"

- name: Install dependencies
run: >
uv sync --frozen
--extra dev --extra rl --extra mpc --extra symmetry
--extra lightning --extra equivariance
--extra baselines-ode --extra msm

- name: Run macOS core smoke tests
env:
OMP_NUM_THREADS: "1"
MKL_NUM_THREADS: "1"
OPENBLAS_NUM_THREADS: "1"
TORCH_NUM_THREADS: "1"
run: >
uv run --no-sync pytest tests/ -n auto
-m "not slow and not distributed and not ray"

notebook-plan:
# Ubuntu-only nbmake planning; macOS never executes tutorial notebooks.
name: notebook-plan
needs: changes
if: needs.changes.outputs.notebooks == 'true' || needs.changes.outputs.workflow == 'true'
Expand Down Expand Up @@ -241,6 +288,14 @@ jobs:
"examples/1[4-9]_*.ipynb examples/2[0-6]_*.ipynb"
),
},
{
"shard": "27-46",
"pattern": (
"examples/2[7-9]_*.ipynb "
"examples/3[0-9]_*.ipynb "
"examples/4[0-6]_*.ipynb"
),
},
]
}
matrix = smoke if os.environ["EVENT_NAME"] == "pull_request" else full
Expand Down Expand Up @@ -305,7 +360,7 @@ jobs:
# by path filters; this aggregator still reports success when skips are expected.
ci:
name: ci
needs: [changes, lint, test, notebook-plan, notebooks, docs]
needs: [changes, lint, test, test-macos, notebook-plan, notebooks, docs]
if: always()
runs-on: ubuntu-latest
steps:
Expand All @@ -321,7 +376,14 @@ jobs:
needs = json.loads(os.environ["NEEDS_JSON"])
if needs["changes"]["result"] != "success":
sys.exit("changes job did not succeed")
for job in ("lint", "test", "notebook-plan", "notebooks", "docs"):
for job in (
"lint",
"test",
"test-macos",
"notebook-plan",
"notebooks",
"docs",
):
result = needs[job]["result"]
if result not in ("success", "skipped"):
sys.exit(f"{job} failed with result={result}")
Expand Down
56 changes: 56 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,62 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [0.13.0] - 2026-08-12

Ops-and-trust release on the 0.12 stack: default safetensors checkpoints,
fit callbacks and tracking adapters, a config-driven CLI, thin Ray Tune
helpers, homogeneous representation-level explainability, and a broader CI
matrix. Homogeneous scientific defaults and the linear latent operator
contract are unchanged.

### Added

- Default checkpoint container ``safetensors_v1`` (directory or ``.kgckpt`` /
``.zip`` bundle) with ``meta.json`` / ``config.json`` /
``model.safetensors``. Core dependency ``safetensors>=0.4``. Load
auto-detects the safe layout and legacy format-1 ``.pt`` pickles. See
``SECURITY.md`` and Sphinx serialization docs.
- Optional ``.kgckpt`` zip packaging of the safetensors directory layout.
- Fit callback protocol (``FitCallback`` / ``NoOpFitCallback``) wired through
``run_fit_loop`` and ``GraphKoopmanModel.fit(..., callbacks=)``. In-tree
``CsvFitLogger`` and optional ``TensorBoardFitLogger``; W&B / MLflow remain
examples-only DIY (no cloud SDK pins).
- Console script ``koopman-graph`` with ``train`` / ``predict`` subcommands,
JSON configs in core and optional YAML via ``[cli]`` (PyYAML). Sphinx
``cli`` page and ``examples/cli/`` smoke config.
- Power-user ``koopman_graph.tuning``: ``fit_history_metrics``, lazy
``run_ray_tune``, and example search-space scaffolds. Ray Tune example
script migrated; Optuna stays examples-only.
- Homogeneous ``explain_representation`` /
``RepresentationExplanation`` (GNNExplainer by default; optional Captum
integrated gradients via ``[explain]``). Interpretive / non-causal masks;
not ResDMD and not ``ModeEnergyAttribution``. Tutorial
``examples/46_representation_explain.ipynb``. Bib keys
``Ying2019GNNExplainer``, ``Sundararajan2017IntegratedGradients``.
- CI: Ubuntu pytest on Python 3.10, 3.11, and 3.12 (coverage on 3.12);
macOS core smoke on 3.12 with
``-m "not slow and not distributed and not ray"`` (hard-fail). Notebook
push shards extended through ``46_*``.

### Changed

- **Behavior change:** ``Model.save`` / the default checkpoint path write
``safetensors_v1`` instead of a pickle-only format-1 ``.pt``. Pass
``format="legacy_pt"`` only when you intentionally need the legacy pickle
layout. Prefer loading trusted ``legacy_pt`` files only (see
``SECURITY.md``).

### Notes

- Portable inference export (TorchScript / ``torch.export`` / ONNX) is
**not** shipped in 0.13.0 (Phase 83 cut-candidate). Training checkpoints
(``safetensors_v1``) remain the supported weight-sharing path; ONNX stays
deferred.
- Windows is best-effort community support and is not in the CI matrix.
- Representation explain remains a homogeneous MVP (rejects hetero /
RelGraph / hypergraph / adaptive topology / delays / controls).
- Homogeneous training defaults are unchanged relative to 0.12.0.

## [0.12.0] - 2026-08-05

Scale-and-honesty release on the 0.11 stack: Kronecker-sum exact spectrum
Expand Down
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ authors:
repository-code: "https://github.com/tjkessler/KoopmanGraph"
url: "https://koopmangraph.readthedocs.io/"
license: Apache-2.0
version: 0.12.0
date-released: 2026-08-05
version: 0.13.0
date-released: 2026-08-12
doi: 10.5281/zenodo.21811756
identifiers:
- type: doi
Expand Down
21 changes: 17 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,9 @@ ruff format --check src/ tests/

# Tests with coverage gate (matches the CI 3.12 job; use -n auto for speed)
pytest tests/ -n auto --cov=koopman_graph --cov-report=term-missing --cov-fail-under=90

# Optional: same marker expression planned for macOS smoke (TASK-2142)
# pytest tests/ -n auto -m "not slow and not distributed and not ray"
```

With uv after `uv sync --extra dev`:
Expand Down Expand Up @@ -238,15 +241,25 @@ Branch protection on `main` is configured through a GitHub ruleset
Mark the aggregator check from [`ci.yml`](.github/workflows/ci.yml) as
**required** before merge:

- `ci` — succeeds when path-selected jobs (`lint`, `test`, `notebooks`,
`docs`) succeed or are intentionally skipped; fails if any selected job fails
- `ci` — succeeds when path-selected jobs (`lint`, `test`, `test-macos`,
`notebooks`, `docs`) succeed or are intentionally skipped; fails if any
selected job fails

Individual jobs still appear on the workflow run for diagnosis:

- `lint` — Ruff via `uvx` (no full torch install)
- `test (3.10)`, `test (3.12)` — pytest with `pytest-xdist`; coverage gate on 3.12 only
- `test (3.10)`, `test (3.11)`, `test (3.12)` — pytest with `pytest-xdist`;
coverage gate on 3.12 only. Ubuntu runs the full default suite (including
`@pytest.mark.ray` when `[ray]` / `[distributed]` extras are installed).
Opt-in multi-process DDP uses `@pytest.mark.distributed` plus
`KOOPMAN_GRAPH_DISTRIBUTED_TESTS=1`.
- `test (macos-3.12)` — macOS core smoke on `macos-latest` with
`-m "not slow and not distributed and not ray"` (hard-fail; no Ray/Dask
extras). Shares the same path filter as Ubuntu `test` (not the notebooks
filter). Windows is not in CI (best-effort community).
- `notebooks (smoke)` on pull requests — 10 representative tutorials; on pushes
to `main`, `notebooks (01-13)` / `notebooks (14-26)` run the full suite
to `main`, `notebooks (01-13)` / `notebooks (14-26)` / `notebooks (27-46)`
run the full suite on **Ubuntu only** (macOS does not run nbmake)
- `docs` — Sphinx documentation build (`sphinx-build -W`, warnings as errors)

The **Draft paper** workflow ([`draft-pdf.yml`](.github/workflows/draft-pdf.yml))
Expand Down
15 changes: 9 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,11 +135,14 @@ Featured tutorials: [01 synthetic](https://github.com/tjkessler/KoopmanGraph/blo
- [Scope and limitations](https://koopmangraph.readthedocs.io/en/latest/limitations.html) — when not to use; measured boundaries
- [Architecture](https://koopmangraph.readthedocs.io/en/latest/architecture.html) — public vs power-user API layers
- [FAQ / troubleshooting](https://koopmangraph.readthedocs.io/en/latest/faq.html) — install, imports, checkpoints
- [Installation](https://koopmangraph.readthedocs.io/en/latest/installation.html) — dependencies and install paths
- What’s new in 0.12.0: Kronecker-sum exact spectrum for eligible discrete
and continuous graph operators (three-path spectrum contract), Scale /
architecture honesty for residual dense inverse and eig-reg ceilings, and
docs-only topology-criticality honesty — see [CHANGELOG.md](CHANGELOG.md).
- [Installation](https://koopmangraph.readthedocs.io/en/latest/installation.html) — dependencies, install paths, and CI platforms
- [CLI](https://koopmangraph.readthedocs.io/en/latest/cli.html) — `koopman-graph train` / `predict` config workflow
- [SECURITY.md](SECURITY.md) — supported versions and checkpoint trust boundaries
- What’s new in 0.13.0: default `safetensors_v1` checkpoints (legacy pickle via
`format="legacy_pt"`), fit callbacks / CSV–TensorBoard tracking, config-driven
CLI, thin Ray Tune helpers, homogeneous `explain_representation` MVP, and
Ubuntu 3.10–3.12 plus macOS core-smoke CI — see [CHANGELOG.md](CHANGELOG.md).
Portable TorchScript / ONNX export is not shipped in this release.

## Related software

Expand All @@ -164,7 +167,7 @@ If you use KoopmanGraph in research, please cite:
publisher = {Zenodo},
doi = {10.5281/zenodo.21811756},
url = {https://github.com/tjkessler/KoopmanGraph},
version = {0.12.0},
version = {0.13.0},
}
```

Expand Down
46 changes: 35 additions & 11 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Security updates are provided for the **latest published release** of

| Version lineage | Security updates |
| --------------- | ---------------- |
| Latest PyPI release (currently the 0.8.x line) | Supported |
| Latest PyPI release (currently the 0.12.x line) | Supported |
| Older published majors/minors | Not regularly backported |

Check the installed version with:
Expand Down Expand Up @@ -45,14 +45,38 @@ This channel is the same maintainer contact used for

## Loading checkpoints and dataset caches

Model checkpoints, conformal calibration files, hierarchical wrappers, and
on-disk teaching caches (``*.pt``) are loaded with PyTorch
Model checkpoints use one of two on-disk containers (logical architecture
schema remains format-1 either way):

### Preferred: `safetensors_v1` directory or `.kgckpt` zip

Default `GraphKoopmanModel.save` / `save_checkpoint` writes a **directory**
containing `meta.json`, `config.json` (architecture metadata), and
`model.safetensors` (weights). When the destination path ends in `.kgckpt`
or `.zip`, the same three files are packed into a zip bundle (still no
pickle). Weight tensors load through the safetensors library rather than
pickle. Prefer this container when sharing trained models.

The JSON sidecar files are not executable code, but an untrusted checkpoint can
still carry misleading architecture metadata or adversarial weight values that
affect numerical results. Treat third-party directories or `.kgckpt` files as
untrusted inputs; inspect `config.json` before load when provenance is unclear.

### Legacy: `legacy_pt` pickle file

Single ``*.pt`` / ``*.pth`` files (and some training `checkpoint_path`
writers that still pin this format) are loaded with PyTorch
``torch.load(..., weights_only=False)``. That API deserializes Python objects
and can execute code from a malicious file.

**Trust boundary:** only load checkpoints and cache artifacts that you created
yourself or obtained from a source you trust. Do not load untrusted ``.pt`` /
checkpoint files shared over the network or from unknown mirrors. Prefer
rebuilding teaching caches with the repository download scripts (SHA256-verified
where digests are pinned) rather than accepting third-party ``traffic.pt`` /
``contact.pt`` blobs.
and can execute code from a malicious file. Pass ``format="legacy_pt"`` only
when you need a single pickle file.

**Trust boundary for `legacy_pt`:** only load pickle checkpoints that you
created yourself or obtained from a source you trust. Do not load untrusted
``.pt`` / checkpoint files shared over the network or from unknown mirrors.

### Teaching caches and other ``*.pt`` artifacts

Conformal calibration files, hierarchical wrappers, and on-disk teaching
caches (``*.pt``) remain pickle-based. Prefer rebuilding teaching caches with
the repository download scripts (SHA256-verified where digests are pinned)
rather than accepting third-party ``traffic.pt`` / ``contact.pt`` blobs.
Loading