cailmdaley · Oct 10, 2026 · Oct 10, 2026 · Oct 10, 2026 · Oct 10, 2026 · Oct 10, 2026
diff --git a/.github/workflows/deploy-image.yml b/.github/workflows/deploy-image.yml
@@ -120,9 +120,8 @@ jobs:
           docker run --rm -e HYPOTHESIS_PROFILE=ci "$IMAGE" \
             pytest -rX -m "not slow" --no-cov tests/science
 
-      # The actual test suite, run inside the shipped image — replacing the
-      # retired conda-based suite. pytest exercises the same wheels, binaries,
-      # and Python (3.12) that production runs on, not a parallel environment.
+      # Run the full suite inside the shipped image: the same wheels,
+      # binaries, and Python (3.12) that production uses.
       # pyproject's addopts add `--cov=shapepipe`; COVERAGE_FILE is set to /tmp
       # in the image so it works on read-only filesystems too. The Hypothesis
       # profile is explicit here so CI always uses the deterministic, capped

diff --git a/CLAUDE.md b/CLAUDE.md
@@ -5,15 +5,14 @@ runs the full chain from raw survey images to calibrated shear catalogues —
 object detection, PSF modelling, and shape measurement — and was used to produce
 the first UNIONS cosmic-shear release.
 
-The project is now entering a substantial rework of the shape-measurement
-pipeline, with the near-term goal of a tight loop between the pipeline and image
-simulations for validation and calibration. Development is by a small team;
+Development focuses on a tight loop between the shape-measurement pipeline and
+image simulations for validation and calibration. Development is by a small team;
 reproducibility and clarity matter more than breadth.
 
 ## Environment — the container is the source of truth
 
 ShapePipe is not a stand-alone library: it needs system tools (Source Extractor,
-PSFEx, WeightWatcher), MPI, and a specific scientific-Python stack. The supported
+PSFEx), MPI, and a specific scientific-Python stack. The supported
 way to get all of that is the container.
 
 - **Dependencies** are declared in `pyproject.toml` as **floor ranges**; exact
@@ -105,8 +104,8 @@ Full detail: `docs/source/installation.md` and `docs/source/container.md`.
   exercises exactly what ships — run them the same way, in the container.
 - **CI** (`.github/workflows/deploy-image.yml`): every PR and push builds the
   image and runs the test suite + the example pipeline + binary smokes; pushes to
-  `develop` / `main` / `master` additionally publish the images. API docs deploy
-  from `master` (`cd.yml`).
+  any branch additionally publish the images. See `cd.yml` for versioned API-doc
+  builds and deployment.
 - **Style**: PEP 8; numpydoc docstrings on public modules, classes, and methods.
   Match the surrounding code.
 

diff --git a/Dockerfile b/Dockerfile
@@ -2,8 +2,8 @@
 #
 # One image: CI tests it, canfar batch jobs run it, downstream stacks (e.g.
 # sp_validation) build FROM it. Everyday CLI tools and all extras (test,
-# lint, doc, jupyter, fitsio, …) are baked in, so there is nothing a job
-# runs that CI did not already exercise.
+# lint, doc, jupyter, fitsio, …) are baked in. See
+# .github/workflows/deploy-image.yml for the checks run before publication.
 
 # ----------------------------------------------------------------------
 # base — system deps
@@ -61,7 +61,7 @@
 # host's `mpirun` launches one container rank per slot, and the OpenMPI inside
 # the image wires the ranks together through PMIx. That handshake requires the
 # container's PMIx to be compatible with the host launcher's. Debian bookworm's
-# package is OpenMPI 4.1.4 with PMIx 2.x; modern clusters (e.g. candide) now run
+# package is OpenMPI 4.1.4 with PMIx 2.x; modern clusters (e.g. candide) run
 # OpenMPI 5.0.x with PMIx 5.x, and a PMIx 2 client cannot talk to a PMIx 5
 # server — so every rank silently degrades to a standalone "rank 0 of 1" and the
 # job runs N independent copies instead of one N-rank job. Building OpenMPI
@@ -86,7 +86,7 @@
        --disable-dlopen --disable-sphinx && \
    make -j"$(nproc)" && make install && \
    cd / && rm -rf /tmp/openmpi-*
 ENV PATH="/opt/ompi/bin:${PATH}" \
    LD_LIBRARY_PATH="/opt/ompi/lib:${LD_LIBRARY_PATH}"

 # uv — fast reproducible Python deps installer. pyproject.toml + uv.lock

diff --git a/README.rst b/README.rst
@@ -3,8 +3,8 @@ ShapePipe
 
 |CI| |CD| |python312| |release|
 
-.. |CI| image:: https://github.com/CosmoStat/shapepipe/workflows/CI/badge.svg
-  :target: https://github.com/CosmoStat/shapepipe/actions?query=workflow%3ACI
+.. |CI| image:: https://github.com/CosmoStat/shapepipe/actions/workflows/deploy-image.yml/badge.svg
+  :target: https://github.com/CosmoStat/shapepipe/actions/workflows/deploy-image.yml
 
 .. |CD| image:: https://github.com/CosmoStat/shapepipe/actions/workflows/pages/pages-build-deployment/badge.svg
   :target: https://github.com/CosmoStat/shapepipe/actions/workflows/pages/pages-build-deployment
@@ -24,8 +24,8 @@ Quickstart
 ----------
 
 ShapePipe ships as a container image, so you can run the bundled example
-pipeline — a single CFIS tile through the full chain — without installing
-anything:
+pipeline — a chain of demonstration modules — without installing the
+scientific stack on the host:
 
 .. code-block:: bash
 
@@ -37,8 +37,8 @@ anything:
 
 The image is built, tested, and published on every push to the `GitHub
 Container Registry <https://github.com/CosmoStat/shapepipe/pkgs/container/shapepipe>`_:
-``:develop`` tracks the integration branch, release tags (e.g. ``:v1.1.0``) a
-stable cut.
+``:develop`` tracks the integration branch; branch tags support testing before
+merge. See ``.github/workflows/deploy-image.yml`` for publication triggers.
 
 Documentation
 -------------

diff --git a/conftest.py b/conftest.py
@@ -3,7 +3,8 @@
 Markers, environment detection, and the candide skip policy live here so
 every test module — wherever it sits in the tree — sees the same rules.
 
-Markers (also declared in ``pyproject.toml`` so ``--strict-markers`` is on):
+See ``pyproject.toml`` for marker declarations and pytest defaults.
+Markers used by the suite include:
 
 * ``slow``    — heavy compute (minutes), not part of the fast inner loop.
 * ``candide`` — needs the candide cluster and/or its real on-disk data;
@@ -42,10 +43,7 @@
 # Candide detection
 # --------------------------------------------------------------------------- #
 
-# Candide compute / login nodes are named c01, c03, n22..n36, etc. The login
-# host this suite is most often driven from is ``c03``. We match the candide
-# node-name families rather than a fixed list so new nodes are covered, and
-# allow an explicit override for CI or odd hostnames.
+# Match Candide node-name families rather than a fixed list of hosts.
 _CANDIDE_HOST_RE = re.compile(r"^(c\d{2}|n\d{2})$", re.IGNORECASE)
 
 
@@ -54,7 +52,7 @@ def on_candide():
 
     The check is, in order: an explicit ``SHAPEPIPE_ON_CANDIDE`` override
     (``1``/``0``), then the hostname against the candide node-name families
-    (``c0x`` login, ``nXX`` compute; whole bare hostname, so ``c6.nibi.sharcnet``
+    (``cXX`` login, ``nXX`` compute; whole bare hostname, so ``c6.nibi.sharcnet``
     does not match). Cheap, import-safe, no cluster calls.
     """
     override = os.environ.get("SHAPEPIPE_ON_CANDIDE")
@@ -106,9 +104,8 @@ def pytest_collection_modifyitems(config, items):
 def artifacts_dir():
     """Directory where guardrail tests drop plots + status summaries.
 
-    The artifacts SEAM: a later GitHub Pages / status step publishes from
-    here. Created on demand so a clean checkout has nothing to commit until
-    a test actually emits.
+    Create ``tests/_artifacts`` when the fixture is requested. Tests write
+    plots and status summaries here for inspection or publication.
     """
     from pathlib import Path
 

diff --git a/docs/source/clusters.md b/docs/source/clusters.md
@@ -29,7 +29,7 @@ protocol as the host launcher (see the candide section below).
 
 ## candide (SLURM)
 
-candide uses **SLURM** (`sbatch`; the old `qsub`/PBS commands are gone). The repo
+candide uses **SLURM** (`sbatch`). The repo
 ships ready job scripts in `example/pbs/` — `candide_smp.sh` (single node,
 parallelised with joblib) and `candide_mpi.sh` (multi-node, hybrid MPI). To run
 the bundled single-tile example end to end:
@@ -88,14 +88,11 @@ post-processing — is documented in the
 [CANFAR production walkthrough](pipeline_canfar.md).
 
 ```{note}
-The CANFAR production submission scripts (`scripts/sh/job_sp_canfar*.bash`) still
-run under the pre-container environment and are slated for the same
-container-first cleanup the candide scripts received. Treat the walkthrough as
-the current-but-evolving production procedure.
+For the CANFAR production submission scripts (`scripts/sh/job_sp_canfar*.bash`),
+follow their environment requirements and the production walkthrough.
 ```
 
 ## ccin2p3
 
-ccin2p3 is not yet containerised. The `example/pbs/cc_{smp,mpi}.sh` scripts target
-the pre-container environment; a container-first path mirroring candide is future
-work.
+The `example/pbs/cc_{smp,mpi}.sh` scripts target a host-installed environment.
+See their headers for machine-specific requirements.
diff --git a/docs/source/container.md b/docs/source/container.md
@@ -7,7 +7,7 @@ CANFAR), see [Running on a cluster](clusters.md).
 
 ## One image
 
-Every tag (`:develop`, a feature branch, `:v1.1.0`, …) is the same build:
+Every published branch tag (`:develop`, a feature branch, …) uses the same build:
 everyday CLI tools (`vim`, `tmux`, `htop`, `rg`, `fd`, `jq`, `bat`, `less`,
 `git-lfs`, …) and **all** Python extras (test, lint, doc, jupyter, fitsio,
 release) on top of core deps. CI runs the test suite inside it before
@@ -110,7 +110,7 @@ This is the **single source of truth** for *what kinds of environments
 shapepipe is compatible with*. Edit when:
 
 - adding a new dependency (`uv add foo` will edit it for you)
-- bumping a minimum after crossing a major version line
+- bumping a minimum when code requires a newer API
 - adding/restructuring extras
 - changing project metadata (scripts, version, etc.)
 
@@ -129,9 +129,9 @@ uv add 'foo>=1.2'                    # adds to pyproject AND uv.lock
 ```
 
 Commit `uv.lock` to the repo — that's how reproducibility transfers
-between machines. The Dockerfile uses `uv sync --frozen`, which fails if
-`pyproject.toml` and `uv.lock` have drifted. That's the discipline
-mechanism: a stale lockfile cannot ship.
+between machines. The Dockerfile uses `uv sync --frozen` to install from the
+lockfile without resolving dependencies or checking whether it is current.
+Regenerate the lock whenever requirements change.
 
 ### `Dockerfile` — the build recipe
 
@@ -161,18 +161,9 @@ The asymmetry is deliberate: Python deps go through pyproject + lockfile
 versioning). Don't `apt install` something that has a Python wheel; don't
 `pip install` something Debian packages directly (e.g. `source-extractor`).
 
-## Why this shape
-
-- **No conda.** Earlier images double-installed packages via conda *and*
-  pip, ballooning the image and creating environment-resolution bugs.
-  Dropped in favour of plain Python + uv.
-- **`uv sync --frozen`** at build time means the image is bit-exactly
-  reproducible from a tagged commit, and impossible to ship with a stale
-  lockfile.
-- **Astromatic binaries from Debian** (`psfex`, `source-extractor`)
-  instead of source builds — Debian carries the
-  GCC-compatibility patches that the previous Dockerfile had to apply
-  inline with `sed`.
-- **One image** so a canfar batch job, a downstream `FROM` clause, and CI's
-  test run are never three different environments — `vim`, `pytest`, and
-  everything else ship on every tag.
+## Build choices
+
+The image uses plain Python and uv for Python dependencies, and Debian's
+`psfex` and `source-extractor` packages for the Astromatic binaries.
+See `Dockerfile` for the source-built OpenMPI version and host-compatibility
+requirements.
diff --git a/docs/source/dependencies.md b/docs/source/dependencies.md
@@ -16,7 +16,7 @@ Scientific stack:
 |---------|------------|
 | [Astropy](https://www.astropy.org/) | {cite:p}`astropy:2013,astropy:2018` |
 | [GalSim](https://github.com/GalSim-developers/GalSim) | {cite:p}`rowe:15` |
-| [ngmix](https://github.com/aguinot/ngmix) | {cite:p}`sheldon:15` |
+| [ngmix](https://github.com/esheldon/ngmix) | {cite:p}`sheldon:15` |
 | [MCCD](https://github.com/CosmoStat/mccd) | {cite:p}`liaudat:21` |
 | [ModOpt](https://cea-cosmic.github.io/ModOpt/) | {cite:p}`farrens:20` |
 | [python-pysap](https://github.com/CEA-COSMIC/pysap) | |
@@ -26,7 +26,6 @@ Scientific stack:
 | [Matplotlib](https://matplotlib.org/) | {cite:p}`hunter:07` |
 | [Joblib](https://joblib.readthedocs.io/en/latest/) | {cite:p}`joblib:20` |
 | [mpi4py](https://mpi4py.readthedocs.io/en/stable/) | {cite:p}`dalcin:05,dalcin:08,dalcin:11` |
-| [reproject](https://reproject.readthedocs.io/) | |
 | [h5py](https://www.h5py.org/) | |
 
 Data access &amp; infrastructure (CANFAR / UNIONS):
@@ -36,21 +35,18 @@ Data access &amp; infrastructure (CANFAR / UNIONS):
 | [vos](https://github.com/opencadc/vostools) | CADC / CANFAR VOSpace access |
 | [skaha](https://github.com/shinybrar/skaha) | CANFAR Science Platform sessions |
 | canfar | CANFAR container-job submission |
-| [astroquery](https://astroquery.readthedocs.io/) | external catalogue queries |
 | [cs_util](https://github.com/CosmoStat/cs_util) | shared CosmoStat utilities |
 | [sqlitedict](https://github.com/RaRe-Technologies/sqlitedict) | on-disk pipeline state |
 
 ```{note}
-`ngmix` is pinned to the
-[`aguinot/ngmix@stable_version`](https://github.com/aguinot/ngmix/tree/stable_version)
-fork until the fixes land upstream — do not modernise that line (see the note in
-`pyproject.toml`).
+Git-sourced packages and their selected refs are declared in
+`pyproject.toml` under `[tool.uv.sources]`; `uv.lock` pins the resolved commits.
 ```
 
 ## System Dependencies
 
-The container also provides the non-Python tools ShapePipe calls, all from Debian
-packages (no source builds), plus the MPI stack:
+The container also provides the non-Python tools ShapePipe calls, from Debian
+packages, plus a source-built MPI stack (see `Dockerfile`):
 
 | Package | References |
 |---------|------------|

diff --git a/docs/source/installation.md b/docs/source/installation.md
@@ -1,9 +1,8 @@
 # Installation
 
 ```{attention}
-ShapePipe was not designed to be a stand-alone Python library. Instead users
-are expected to install the full ShapePipe environment on the system(s) where
-data should be processed.
+Running the pipeline requires both the Python package and system executables.
+Use the container for the complete supported environment.
 ```
 
 ## Container Installation (Recommended)
@@ -35,7 +34,7 @@ docker pull ghcr.io/cosmostat/shapepipe:develop
 ```
 
 ```{attention}
-We do not currently build images for Apple Silicon/amr64; however the amd64 images should work on these systems, albeit with reduced performance.
+We do not currently build images for Apple Silicon/arm64; however the amd64 images should work on these systems, albeit with reduced performance.
 ```
 
 The image bundles the astromatic binaries (`source-extractor`, `psfex`),

diff --git a/docs/source/testing.md b/docs/source/testing.md
@@ -7,30 +7,13 @@ the bundled example pipeline.
 ## The automated test suite
 
 The test suite runs with [pytest](https://docs.pytest.org/) under a single
-discovery root, `tests/`, with one tier per subdirectory:
-
-- `tests/module/` — **unit, property-based, and integration** tests for
-  the analytic surface: pure helpers and geometry (coordinate round-trips,
-  postage-stamp shapes, the MegaCam CCD flip), tested both by example and with
-  [Hypothesis](https://hypothesis.readthedocs.io/) property tests where a
-  function has a clear invariant. Integration tests that need the astromatic
-  binaries (Source Extractor, PSFEx) build synthetic FITS on the fly rather
-  than requiring survey data.
-- `tests/unit/` — **structural** tests, one parametrized case per file: every
-  shell script parses (`bash -n`), every example config parses, every
-  `shapepipe.*` submodule imports, every `[project.scripts]` entry handles
-  `-h`, and every `*_runner.py` carries its `@module_runner` metadata. Cheap,
-  broad, and good at catching the dumb-but-real regressions (a syntax error, a
-  broken import, a renamed entry point).
-- `tests/science/` — **fast scientific guardrails** (controlled simulations
-  with a known answer) and `tests/cluster/` — **candide guardrails** that read
-  real on-disk catalogs; the latter are marked `candide` and auto-skip off the
-  cluster. See `tests/README.md` for the full layout and markers.
-
-**CI is the single gate.** On every pull request and every push to `develop`,
-`.github/workflows/deploy-image.yml` builds the image and runs the suite
-*inside it* — so the tests exercise exactly the environment that ships. A green
-check means the container actually works.
+discovery root, `tests/`. See `tests/README.md` for the tiers, markers, and
+machine-specific requirements.
+
+`.github/workflows/deploy-image.yml` builds the image and runs tests inside it
+on branch pushes and pull requests targeting the integration or release branches.
+See that workflow for the suite, science checks, workflow-DAG checks, and binary
+and example smoke runs.
 
 ### Running it yourself
 

diff --git a/docs/source/workflow.md b/docs/source/workflow.md
@@ -2,8 +2,7 @@
 
 The end-to-end CFIS production run is driven by a
 [Snakemake](https://snakemake.readthedocs.io) workflow that lives in the
-`workflow/` directory of the repository. It replaces the older bit-coded bash
-job chain (`job_sp`) that this documentation previously described.
+`workflow/` directory of the repository.
 
 The workflow owns:
 
@@ -19,6 +18,6 @@ repository.
 
 ```{note}
 The PSF-validation and post-processing configurations that the workflow does
-not yet cover are kept separately in `example/cfis/`; see the README there and
+not cover are kept separately in `example/cfis/`; see the README there and
 [Post-processing](post_processing.md).
 ```
diff --git a/example/cfis/README.md b/example/cfis/README.md
@@ -2,7 +2,7 @@
 
 These configuration files cover PSF validation (`Ms`, `Pl`, `MsPl`,
 `valjoint`) and assorted post-processing / statistics steps that are **not
-(yet) covered by the Snakemake workflow**.  They are kept here for reference
+covered by the Snakemake workflow**.  They are kept here for reference
 and manual use with `shapepipe_run -c <file>`.
 
 The per-module pipeline configurations driven by the Snakemake workflow live