Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 35 additions & 4 deletions .github/workflows/actions.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: Installing and Testing
on: [push]
on: [push, pull_request]

# https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs
# `contents` is for permission to the contents of the repository.
Expand Down Expand Up @@ -31,9 +31,40 @@ jobs:
# - name: mypy
# run: |
# python -m mypy --ignore-missing-imports --follow-imports=silent --no-strict-optional equitrain tests
# - name: Test with pytest
# run: |
# python -m pytest -sv tests
- name: Test with pytest
run: |
python -m pytest -sv tests

equiload-integration:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v4
with:
repository: lutfia95/RTorch
# Pin the native ABI dependency. Update this only after the referenced
# EquiLoad commit has been merged/released and pushed to GitHub.
ref: 2beef89beafa485314fc6ec92333682125b975ec
path: equiload
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- uses: dtolnay/rust-toolchain@stable
- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y build-essential libhdf5-dev
- name: Build EquiLoad against Equitrain's Torch
run: |
python -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e '.[dev,torch]'
bash equiload/scripts/install-python.sh --venv "$GITHUB_WORKSPACE/.venv"
- name: Test the optional EquiLoad integration
run: |
. .venv/bin/activate
python -m pytest -q tests/test_equiload_loader.py
build-and-install:
runs-on: ubuntu-latest
steps:
Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,11 @@ jobs:
- name: Install dependencies
run: |
pip install --upgrade pip
uv pip install build
uv pip install build twine
- name: Build package
run: python -m build
- name: Validate release metadata
run: python -m twine check dist/*
- name: Upload release distributions
uses: actions/upload-artifact@v4
with:
Expand Down
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,5 @@ _version.py
.*_cache/

# distribution
dist/
build/
dist/
10 changes: 10 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Keep source distributions focused on installable package source. Documentation,
# examples, development workflows, and tests are available from the repository.
prune .github
prune .vscode
prune docs
prune resources
prune tests
exclude environment.yml
exclude mkdocs.yml
exclude .pre-commit-config.yaml
30 changes: 30 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,36 @@ pip install -e '.[docu]'
mkdocs serve
```

## Optional EquiLoad HDF5 Loader

Equitrain's portable PyTorch/PyG loader remains the default.
[EquiLoad](https://github.com/BAMeScience/EquiLoad) is a separately installed native
extension. This integration is tested with the `v0.1.0` release. Build it against
the exact Torch installation in the Equitrain environment, because PyTorch's C++
ABI is not stable between Torch releases:

```bash
git clone --branch v0.1.0 https://github.com/BAMeScience/EquiLoad.git equiload
cd equiload
bash scripts/install-python.sh --venv /path/to/equitrain/.venv
cd ..
equitrain --backend torch --torch-loader equiload ...
```

Use the tagged release when building EquiLoad. Earlier development checkouts
also reported version `0.1.0` but lacked the global sample-index fix required
for correct multi-file loading. Rebuild those older installations from the tag.

Do not install a wheel built in another environment. EquiLoad is CPU-side; Equitrain
moves its returned batches to the active accelerator device. It currently requires
Torch 2.6 through 2.14; the standard Python loader remains available for other
supported Equitrain Torch versions.

The EquiLoad loader is currently single-process and does not support weighted resampling or
relative reaction losses. It returns the same nested PyG batch structure as the standard
loader and moves batches to Equitrain's active accelerator device. Without `equiload`
installed, the default `--torch-loader python` requires no Rust tooling.

## Installation

```bash
Expand Down
16 changes: 16 additions & 0 deletions docs/training-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,10 @@ weights such as `energy_weight`, `forces_weight`, and `stress_weight`. Missing
targets are written with zero target weights during preprocessing; global CLI
loss weights still control which losses are requested.

Torch losses apply these per-configuration weights and exclude zero-weight
targets from each component's average. Both the Python and EquiLoad loader
adapters preserve these weights.

## Optimizers

`--opt` selects the Torch optimizer. Supported names are:
Expand Down Expand Up @@ -110,6 +114,13 @@ equitrain ... --ema --ema-decay 0.999
Torch checkpoints write EMA state to `ema.bin` when EMA is enabled. JAX
checkpoints write `ema_params.msgpack`.

Torch validation uses the averaged parameters while optimization continues from
the raw parameters. Checkpoints retain raw model weights and optimizer state
for resuming. `equitrain-export` applies the separately saved moving average
when present. The Python checkpoint loader supports
`load_checkpoint(..., use_ema=True)` for inference; its default loads raw weights
for training.

## Checkpoints

Equitrain evaluates before training and after every epoch. A new best
Expand Down Expand Up @@ -183,6 +194,11 @@ Torch workflows normally use `--batch-size`. Optional graph caps
`--batch-max-nodes` and `--batch-max-edges` split large collated batches; with
`--batch-drop`, individual graphs exceeding the cap are dropped.

Torch sub-batches are normalized by each loss component's labeled element count,
including across distributed ranks. If filtering removes every graph from a
training batch on one rank, all ranks skip that batch together and log the skip
to keep distributed training synchronized.

JAX train/evaluate workflows require `--batch-max-edges`; the loader infers
fixed padded shapes from the data and edge cap. JAX prediction also accepts
`--batch-max-nodes` and can reuse cached streaming statistics when explicit caps
Expand Down
9 changes: 9 additions & 0 deletions equitrain/argparser.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,15 @@ def add_common_data_args(parser: argparse.ArgumentParser) -> argparse.ArgumentPa
type=str2bool,
default=True,
)
parser.add_argument(
'--torch-loader',
choices=('python', 'equiload'),
default='python',
help=(
'Torch HDF5 graph-loader implementation. equiload requires the separately '
'installed EquiLoad extension; python is the portable default.'
),
)
parser.add_argument('--seed', help='Random seed for splits', type=int, default=123)
return parser

Expand Down
12 changes: 10 additions & 2 deletions equitrain/backends/jax_nnx_compat.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,22 @@ def state_to_pure_dict(state: Any) -> dict[str, Any]:

from flax import nnx

def _variable_value(value: Any) -> Any:
"""Return an NNX variable value across supported Flax releases."""
# Flax <= 0.11 exposed get_value(); Flax 0.12 removed it in favor of
# the public value property. Do not access raw_value, which is an
# implementation detail and bypasses variable metadata.
getter = getattr(value, 'get_value', None)
return getter() if callable(getter) else value.value

def _extract(value: Any) -> Any:
if _MaceConfigVar is not None and isinstance(value, _MaceConfigVar):
config_val = value.get_value()
config_val = _variable_value(value)
if isinstance(config_val, dict) and not is_config_dict(config_val):
return _make_config_dict(config_val)
return config_val
if isinstance(value, nnx.Variable):
return value.get_value()
return _variable_value(value)
return value

return nnx.to_pure_dict(state, extract_fn=_extract)
Expand Down
Loading
Loading