Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
- uses: actions/checkout@v7

- name: Set up virtual environment
uses: prefix-dev/setup-pixi@v0.10.1
uses: prefix-dev/setup-pixi@v0.11.0
with:
environments: ${{ matrix.environment }}
frozen: true
Expand Down
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ repos:
- id: check-yaml
- id: debug-statements
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.15.13
rev: v0.17.0
hooks:
- id: ruff-check
- id: ruff-format
- repo: https://github.com/tox-dev/pyproject-fmt
rev: v2.21.0
rev: v2.30.3
hooks:
- id: pyproject-fmt
1 change: 1 addition & 0 deletions docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ To verify that DESCENT is installed correctly, run:

```python
import descent

print(descent.__version__)
```

Expand Down
55 changes: 12 additions & 43 deletions docs/optimization.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,7 @@ Adaptive Moment Estimation (Adam) is the default optimizer:
```python
from descent.optim import Adam

optimizer = Adam(
lr=1e-3,
betas=(0.9, 0.999),
eps=1e-8
)
optimizer = Adam(lr=1e-3, betas=(0.9, 0.999), eps=1e-8)
```

**Pros:**
Expand All @@ -48,7 +44,7 @@ from descent.optim import AdamW
optimizer = AdamW(
lr=1e-3,
weight_decay=1e-4, # L2 regularization
betas=(0.9, 0.999)
betas=(0.9, 0.999),
)
```

Expand All @@ -61,11 +57,7 @@ Stochastic Gradient Descent with momentum:
```python
from descent.optim import SGD

optimizer = SGD(
lr=1e-2,
momentum=0.9,
nesterov=True
)
optimizer = SGD(lr=1e-2, momentum=0.9, nesterov=True)
```

**Pros:**
Expand All @@ -83,11 +75,7 @@ Limited-memory BFGS for quasi-Newton optimization:
```python
from descent.optim import LBFGS

optimizer = LBFGS(
lr=1.0,
max_iter=20,
history_size=10
)
optimizer = LBFGS(lr=1.0, max_iter=20, history_size=10)
```

**Pros:**
Expand Down Expand Up @@ -116,11 +104,7 @@ Reduce learning rate at specified epochs:
```python
from torch.optim.lr_scheduler import StepLR

scheduler = StepLR(
optimizer,
step_size=30,
gamma=0.1
)
scheduler = StepLR(optimizer, step_size=30, gamma=0.1)
```

### Reduce on Plateau
Expand All @@ -131,11 +115,7 @@ Reduce when loss plateaus:
from torch.optim.lr_scheduler import ReduceLROnPlateau

scheduler = ReduceLROnPlateau(
optimizer,
mode='min',
factor=0.5,
patience=10,
threshold=1e-4
optimizer, mode="min", factor=0.5, patience=10, threshold=1e-4
)
```

Expand All @@ -146,11 +126,7 @@ Cosine decay with warm restarts:
```python
from torch.optim.lr_scheduler import CosineAnnealingWarmRestarts

scheduler = CosineAnnealingWarmRestarts(
optimizer,
T_0=10,
T_mult=2
)
scheduler = CosineAnnealingWarmRestarts(optimizer, T_0=10, T_mult=2)
```

### Exponential Decay
Expand All @@ -160,10 +136,7 @@ Exponential learning rate decay:
```python
from torch.optim.lr_scheduler import ExponentialLR

scheduler = ExponentialLR(
optimizer,
gamma=0.95
)
scheduler = ExponentialLR(optimizer, gamma=0.95)
```

## Gradient Computation
Expand Down Expand Up @@ -211,11 +184,7 @@ For problems with structured residuals, DESCENT provides Levenberg-Marquardt opt
```python
from descent.optim import LevenbergMarquardt

optimizer = LevenbergMarquardt(
lambda_init=1e-3,
lambda_factor=10.0,
max_iter=100
)
optimizer = LevenbergMarquardt(lambda_init=1e-3, lambda_factor=10.0, max_iter=100)
```

## Regularization
Expand All @@ -229,7 +198,7 @@ Add L2 penalty to loss:
optimizer = AdamW(lr=1e-3, weight_decay=1e-4)

# Or manually
loss = target_loss + 1e-4 * torch.sum(parameters ** 2)
loss = target_loss + 1e-4 * torch.sum(parameters**2)
```

### L1 Regularization
Expand Down Expand Up @@ -301,13 +270,13 @@ from itertools import product
lr_values = [1e-2, 1e-3, 1e-4]
wd_values = [0, 1e-4, 1e-3]

best_loss = float('inf')
best_loss = float("inf")
best_params = None

for lr, wd in product(lr_values, wd_values):
optimizer = AdamW(lr=lr, weight_decay=wd)
loss = train(optimizer, ...)

if loss < best_loss:
best_loss = loss
best_params = (lr, wd)
Expand Down
21 changes: 11 additions & 10 deletions docs/targets.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ target = EnergyTarget(
dataset="qm_energies.json",
weight=1.0,
denominator="std", # Normalize by standard deviation
energy_type="absolute" # or "relative"
energy_type="absolute", # or "relative"
)
```

Expand Down Expand Up @@ -69,7 +69,7 @@ target = ThermoTarget(
weight=0.5,
properties=["density", "hvap", "dielectric"],
temperatures=[298.15], # K
pressures=[1.0] # atm
pressures=[1.0], # atm
)
```

Expand Down Expand Up @@ -150,41 +150,42 @@ Create custom target functions by subclassing `BaseTarget`:
from descent.targets import BaseTarget
import torch


class CustomTarget(BaseTarget):
def __init__(self, dataset, weight=1.0, **kwargs):
super().__init__(weight=weight)
self.dataset = self.load_dataset(dataset)

def compute_loss(self, system, parameters):
"""
Compute loss for this target.

Parameters
----------
system : System
The molecular system
parameters : dict
Current force field parameters

Returns
-------
loss : torch.Tensor
Scalar loss value
"""
predictions = self.predict(system, parameters)
references = self.get_references()

loss = torch.mean((predictions - references) ** 2)
return loss

def predict(self, system, parameters):
"""Compute predictions using current parameters."""
# Your implementation here
pass

def get_references(self):
"""Get reference values."""
return torch.tensor(self.dataset['values'])
return torch.tensor(self.dataset["values"])
```

## Combining Targets
Expand All @@ -200,7 +201,7 @@ config = TrainingConfig(
{"type": "energy", "dataset": "qm.json", "weight": 1.0},
{"type": "thermo", "dataset": "exp.json", "weight": 0.5},
{"type": "dimer", "dataset": "dimers.json", "weight": 0.3},
]
],
)
```

Expand Down
59 changes: 19 additions & 40 deletions docs/training.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ config = TrainingConfig(
force_field="openff-2.0.0.offxml",
parameters=["[#6:1]-[#6:2]", "[#6:1]-[#1:2]"],
targets=[...],
optimizer_config={...}
optimizer_config={...},
)

# Run training
Expand All @@ -50,7 +50,7 @@ from descent.targets import EnergyTarget
target = EnergyTarget(
dataset="energies.json",
weight=1.0,
denominator="std" # Normalize by standard deviation
denominator="std", # Normalize by standard deviation
)
```

Expand All @@ -61,11 +61,7 @@ Fit to experimental thermodynamic properties (density, enthalpy of vaporization,
```python
from descent.targets import ThermoTarget

target = ThermoTarget(
dataset="thermo.json",
weight=0.5,
properties=["density", "hvap"]
)
target = ThermoTarget(dataset="thermo.json", weight=0.5, properties=["density", "hvap"])
```

### Dimer Targets
Expand All @@ -75,10 +71,7 @@ Fit to dimer interaction energies:
```python
from descent.targets import DimerTarget

target = DimerTarget(
dataset="dimers.json",
weight=0.3
)
target = DimerTarget(dataset="dimers.json", weight=0.3)
```

## Optimization Strategies
Expand All @@ -88,10 +81,7 @@ DESCENT uses PyTorch optimizers for parameter optimization:
```python
from descent.optim import AdamW

optimizer = AdamW(
lr=1e-3,
weight_decay=1e-4
)
optimizer = AdamW(lr=1e-3, weight_decay=1e-4)
```

Common optimizers:
Expand All @@ -107,11 +97,9 @@ The overall loss is a weighted combination of target losses:
```python
from descent.utils.loss import WeightedLoss

loss_fn = WeightedLoss(targets=[
(energy_target, 1.0),
(thermo_target, 0.5),
(dimer_target, 0.3)
])
loss_fn = WeightedLoss(
targets=[(energy_target, 1.0), (thermo_target, 0.5), (dimer_target, 0.3)]
)
```

## Training Loop
Expand All @@ -125,12 +113,7 @@ from descent.train import train
config = TrainingConfig(...)

# Run training
results = train(
config,
max_epochs=100,
batch_size=32,
validation_split=0.2
)
results = train(config, max_epochs=100, batch_size=32, validation_split=0.2)

# Access results
print(f"Final loss: {results['final_loss']}")
Expand Down Expand Up @@ -158,17 +141,17 @@ Save and resume training:
```python
# Save checkpoint
checkpoint = {
'epoch': epoch,
'parameters': parameters,
'optimizer_state': optimizer.state_dict(),
'loss': loss
"epoch": epoch,
"parameters": parameters,
"optimizer_state": optimizer.state_dict(),
"loss": loss,
}
torch.save(checkpoint, 'checkpoint.pt')
torch.save(checkpoint, "checkpoint.pt")

# Resume training
checkpoint = torch.load('checkpoint.pt')
parameters = checkpoint['parameters']
optimizer.load_state_dict(checkpoint['optimizer_state'])
checkpoint = torch.load("checkpoint.pt")
parameters = checkpoint["parameters"]
optimizer.load_state_dict(checkpoint["optimizer_state"])
```

## Best Practices
Expand All @@ -189,6 +172,7 @@ Create custom target functions by subclassing `BaseTarget`:
```python
from descent.targets import BaseTarget


class CustomTarget(BaseTarget):
def compute_loss(self, system, parameters):
# Custom loss computation
Expand All @@ -203,12 +187,7 @@ Use PyTorch learning rate schedulers:
```python
from torch.optim.lr_scheduler import ReduceLROnPlateau

scheduler = ReduceLROnPlateau(
optimizer,
mode='min',
factor=0.5,
patience=10
)
scheduler = ReduceLROnPlateau(optimizer, mode="min", factor=0.5, patience=10)
```

### Regularization
Expand Down
Loading
Loading