Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 30 additions & 21 deletions .agents/skills/pasteur-rs/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,44 +1,53 @@
---
name: pasteur-rs
description: Local Pasteur simulation and evaluation CLI. Use with the hf CLI skill for Hub download/upload. Complements hf skills — does not replace them.
description: Local Pasteur simulation and evaluation CLI (pasteur-cli) for binary, multiclass, multilabel, and regression ONNX models. Use with the hf-cli skill for Hub download/upload; pasteur-cli never touches the network.
---

# Pasteur Rust (pasteur-cli)

Use this skill together with `hf skills add`. **All Hugging Face Hub I/O goes through `hf`.** `pasteur-cli` only reads/writes local files.
`pasteur-cli` only reads and writes local files. **All Hugging Face Hub I/O goes through `hf`** ([hf-cli](../hf-cli/SKILL.md)): download inputs first, pass local paths, upload results afterwards.

## When to use which tool
## Commands

| Goal | Command |
|------|---------|
| Download dataset/model | `hf download ...` |
| Upload simulation bundle | `hf repos create ... --private` + `hf upload` |
| Run simulators (blackout/jitter/flipper) | `pasteur-cli simulate --input <local.parquet> --labels <groups.parquet>` |
| Generate README | `pasteur-cli card --input ./output --source-dataset ... --dest-repo ...` |
| Evaluate/compare models | `pasteur-cli evaluate/compare <sim_type> --sim-root ./sim --labels ... --model ...` |
| Flipper stability | `pasteur-cli evaluate flipper --sim-root ./sim --labels ... --model ... --flip-threshold 0.5` |
| Download a dataset or model | `hf download ...` |
| Simulate blackout/jitter/flipper | `pasteur-cli simulate --input <clean.parquet> --labels <groups.parquet> --output ./sim` |
| Score one model | `pasteur-cli evaluate <sim_type> --sim-root ./sim --labels <groups.parquet> --model <model.onnx>` |
| Score several models | `pasteur-cli compare <sim_type> --sim-root ./sim --labels ... --model a.onnx --model b.onnx [--predictions-out preds.parquet]` |
| Write the dataset card | `pasteur-cli card --input ./sim --source-dataset <id> --dest-repo <id>` |
| Publish a bundle | `hf repos create <id> --type dataset --private`, then `hf upload` |

## Required layout before upload
`<sim_type>` is `blackout`, `jitter`, or `flipper`. Flags and defaults: [cli/README.md](../../../cli/README.md).

After `simulate --labels <groups.parquet>`, the bundle must look like:
## Task types

Pass the same `--task` to `simulate`, `evaluate`, and `compare`; it must match the model's `metadata.json` `task_type`.

| `--task` | Truth | `--positive-group-id` | `--flip-threshold` |
|----------|-------|-----------------------|--------------------|
| `binary` (default) | `--labels` | The positive cohort | Probability cutoff, default 0.5 |
| `multiclass` | `--labels` | One per class, in `output.classes` order; cohorts must partition the rows | Unused (argmax) |
| `multilabel` | `--labels` | One per label, in `output.classes` order | Default for labels without `output.thresholds`, 0.5 |
| `regression` | `--targets <parquet> --target-col <col>` | Unused | Clinical cutoff in target units. No default; pass the same value to `simulate` and to `evaluate`/`compare` |

For classifiers, `--flip-threshold` belongs to `evaluate`/`compare` only; `simulate` rejects it. Cohorts are matched to model outputs by position; Pasteur cannot check that they line up.

## Bundle layout

```
output/
sim/
clean/clean.parquet
blackout/blackout.parquet
jitter/jitter_*.parquet
flipper/flipper.parquet
jitter/jitter_<i>.parquet
flipper/flipper.parquet # classifiers: needs --labels; regression: --targets and --flip-threshold
```

Input schemas (clean parquet, `groups.parquet` labels) are documented in [Data Contracts](../../../docs/source/data-contracts.rst).

`flipper/` is only written when `--labels` is passed to `simulate` (flipper needs labels to sample cross-label pairs; blackout/jitter don't). Its parquet also carries meta columns (`pair_id`, `step`, `t`, `source_a_row`, `source_b_row`, `label_a`, `label_b`) alongside the interpolated features — `evaluate`/`compare sim_type=flipper` strip these before scoring.

Run `pasteur-cli card` to add `README.md`, then upload each folder with `hf upload`.
Schemas: inputs and `metadata.json` in [Data Contracts](../../../docs/source/data-contracts.rst), outputs in [Output Layout](../../../docs/source/outputs.rst), metrics in [Reading the results](../../../docs/source/metrics.rst).

## Full workflows
## Publishing

See [AGENTS.md](../../../AGENTS.md) in the pasteur-core repo root for copy-paste agent workflows (simulate → publish, evaluate, compare + predictions).
Only publish simulations of public or synthetic data. Run `pasteur-cli card` to add `README.md`, then upload the bundle with `hf upload` to a private repo.

## Build

Expand Down
19 changes: 6 additions & 13 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,18 @@
# Agent Guide

Pasteur is a Rust workspace for local clinical-AI simulation and evaluation.

## Safety

- Never commit patient data, PHI, real clinical datasets, or proprietary models.
- `pasteur-cli` performs local compute only; keep it network-free.
- `pasteur-cli` is local compute only; keep it network-free. Hub I/O goes through the official `hf` CLI; download inputs first, then pass local paths.
- Create Hugging Face model and dataset repositories as **private** by default.

## Tool boundary

- Use `pasteur-cli` for simulation, evaluation, comparison, dataset cards, and its local cache.
- Use the official `hf` CLI for Hub downloads, uploads, repositories, and the Hub cache.
- Download remote inputs before passing local paths to `pasteur-cli`.
- Only publish simulations of public or synthetic data. Bundles keep each patient's row ID, so never upload one built from patient data, even to a private repo.

## Development

Git worktrees must not share a `CARGO_TARGET_DIR`: Cargo hashes workspace crates the same way in each checkout, so a shared target dir silently mixes artifacts from different checkouts.

```bash
cargo fmt --all --check
cargo clippy --workspace -- -D warnings
cargo test --workspace --exclude pypasteur-bindings
cargo clippy --workspace --locked -- -D warnings
cargo test --workspace --locked --exclude pypasteur-bindings
```

See [README.md](README.md), [CONTRIBUTING.md](CONTRIBUTING.md), and the [CLI reference](cli/README.md).
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,4 +98,4 @@ cargo test --workspace --exclude pypasteur-bindings

See [CONTRIBUTING.md](CONTRIBUTING.md). Pushing a `v*` tag runs `.github/workflows/release.yml`.

Automation: publishing simulations of public or synthetic datasets to Hugging Face is described in [AGENTS.md](AGENTS.md), with agent skills under [`.agents/skills/`](.agents/skills/). Never publish bundles made from patient data.
Automation: the rules for agents are in [AGENTS.md](AGENTS.md), and the publishing workflow is in the agent skills under [`.agents/skills/`](.agents/skills/). Never publish bundles made from patient data.
2 changes: 1 addition & 1 deletion cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ Same flags as `evaluate`, except:
- `--predictions-out` writes a local per-patient predictions parquet
- A single `--contract` applies to every model

**Predictions schema:** `source_row_id`, `variant_name`, `label`, one f64 column per model (stem of model path), `all_agree` (all models on the same side of 0.5). Regression writes `target` instead of `label`, and `all_agree` compares sides of `--flip-threshold` (null without one).
**Predictions schema:** `source_row_id`, `variant_name`, `label`, one f64 column per model (stem of model path), `all_agree` (every model makes the same decision). Binary decisions are the side of a fixed 0.5. Multiclass and multilabel write `label__<class>` and `<model>__<class>` per class; their decisions are the argmax, or the set of labels at or over each threshold. Regression writes `target` instead of `label`, and `all_agree` compares sides of `--flip-threshold` (null without one).

The `--output` directory must already exist.

Expand Down
Loading