spOT-NMF

Gene expression deconvolution via Optimal Transport with proximity-aware niche mapping for spatial transcriptomics.

spOT-NMF is a Python package for reference-free deconvolution of spatial transcriptomics data. It replaces the reconstruction loss of non-negative matrix factorization (NMF) with an entropic optimal-transport (OT) loss, recovering gene programs and their per-spot usage, and adds modules to annotate those programs and to map them into spatial niches.

Abdelkareem, A.O. et al. Briefings in Bioinformatics (accepted, 2026) · Documentation · Issues

Contents

• Features
• Installation
• Quick start
• Niche networks
• Command-line reference
• Outputs
• Tutorials
• Reproducing the manuscript
• Citation
• Support and license

Features

• OT-NMF deconvolution. Reference-free program discovery with an entropic OT loss, solved on GPU or CPU.
• Consensus mode. Replicate factorizations clustered into consensus programs, in the style of cNMF.
• Gene selection. Batch-aware selection of overdispersed (highly variable) genes.
• Annotation. Pathway enrichment and gene-set overlap for each program.
• Niche networks. Directed program co-incidence networks partitioned into spatial niches, with tools to calibrate and explore the thresholds.
• Visualization. Spatial maps of program usage, network plots and connection heatmaps.
• Two interfaces. A command-line tool and a Python API.

Installation

spOT-NMF requires Python 3.12 or later. PyTorch is not a hard dependency, so you can install the build (CPU or CUDA) that matches your hardware.

Using uv (recommended):

uv venv --python 3.12
source .venv/bin/activate            # Windows: .venv\Scripts\activate

# PyTorch: choose one (see https://pytorch.org for other builds)
uv pip install torch --index-url https://download.pytorch.org/whl/cpu
# uv pip install torch --index-url https://download.pytorch.org/whl/cu121

uv pip install spot-nmf

Using pip:

python -m venv .venv
source .venv/bin/activate            # Windows: .venv\Scripts\activate
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install spot-nmf

From source, for development:

git clone https://github.com/MorrissyLab/spOT-NMF.git
cd spOT-NMF
uv venv --python 3.12
uv pip install torch --index-url https://download.pytorch.org/whl/cpu
uv pip install -e ".[dev]"
uv run pytest -q

Check the installation with spotnmf --help. spOT-NMF uses a GPU when one is available and falls back to the CPU otherwise.

Quick start

Command line

Run the full pipeline (deconvolution, annotation, spatial plots and networks):

spotnmf spotnmf \
  --sample_name SAMPLE1 \
  --adata_path ./data/sample1.h5ad \
  --data_mode h5ad \
  --results_dir ./results \
  --k 5 \
  --genome GRCh38

--data_mode sets how the input is read: h5ad for an AnnData file, visium (the default) for a Space Ranger output directory, or visium_hd for Visium HD.

Each step can also be run on its own:

spotnmf deconvolve --sample_name SAMPLE1 --adata_path ./data/sample1.h5ad --data_mode h5ad --results_dir ./results --k 5
spotnmf plot       --sample_name SAMPLE1 --adata_path ./data/sample1.h5ad --data_mode h5ad --results_dir ./results
spotnmf annotate   --sample_name SAMPLE1 --results_dir ./results --genome GRCh38
spotnmf network    --sample_name SAMPLE1 --results_dir ./results

network reads the per-spot usages written by deconvolve.

Python

import spotnmf as spot

adata = spot.io.read_adata("data/sample1.h5ad", data_mode="h5ad")

results = spot.cli.run_experiment(
    adata_spatial=adata,
    k=5,                    # number of programs
    sample_name="SAMPLE1",
    results_dir="./results",
    genome="GRCh38",
)

Model hyperparameters (lr, h, w, eps, ...) have tuned defaults; override them with a model_params dictionary.

Consensus mode

By default spOT-NMF runs a single factorization. Consensus mode fits n_iter replicates (default 10), clusters them into consensus programs and refits the per-spot usages against the consensus spectra. The refit is non-negative least squares by default, as in cNMF; pass model_params={"refit": "ot"} for an optimal-transport refit. Runtime grows roughly linearly with n_iter, so a GPU is recommended for large datasets.

spotnmf deconvolve --sample_name SAMPLE1 --adata_path ./data/sample1.h5ad \
  --data_mode h5ad --results_dir ./results --k 5 --consensus --n_iter 10

In Python, pass consensus=True to run_experiment.

Niche networks

The niche module turns the per-spot usage matrix into a directed program-to-program network and partitions it into spatial niches with Infomap.

1. Presence. A program is present at a location where its usage exceeds a threshold. Two detection rules are available:
   • absolute (default): usage > usage_threshold, default 0.05.
   • percentile: usage above each program's own percentile quantile (default 0.90). This is the rule used for Figures 6d and 7d of the paper.
2. Directed co-incidence. The weight of edge p → t is the fraction of p's locations where t is also present, Laplace-smoothed as (pos + 1) / (n + 2).
3. Edge criterion. Under spatial independence this fraction equals t's prevalence, so an edge is kept when co-incidence / prevalence(t) >= edge_lift. edge_threshold replaces the lift with an absolute cutoff, as in releases before 0.2.0.
4. Niches. Infomap clusters the kept edges. A program without a kept edge is isolated and labelled -1.

Parameter (CLI / Python)	Default	Description
--detection_rule / detection_rule	absolute	absolute or percentile
--usage_threshold / usage_threshold	0.05 (absolute), 0 (percentile)	Presence threshold on usage
--percentile / percentile	0.90	Per-program quantile (percentile rule only)
--edge_lift / edge_lift	2.5 (absolute), 2.0 (percentile)	Edge criterion as a fold change over the target's prevalence
--edge_threshold / edge_threshold	unset	Absolute co-incidence cutoff; overrides the lift
--n_bins / n_bins	1000	Minimum number of locations where the source program is present

Choosing the edge criterion. On the paper's colorectal Visium HD sample (24 µm bins, 50 programs), the absolute rule gives six niches for lifts between 2.00 and 2.75. At 2.5 they match the published niches one to one (ARI 0.841). At 2.0 the vascular programs merge into the immune niche, and at 3.0 a seventh niche appears. Because the lift sets the resolution of the partition, explore it on new data rather than relying on a single value:

import pandas as pd
from spotnmf import niche_networks as nn

usage = pd.read_csv("results/SAMPLE1/topics_per_spot_SAMPLE1.csv", index_col=0)

edges, niches = nn.infer_niches(usage)        # default: usage > 0.05, lift >= 2.5
grid = nn.sweep_niche_parameters(usage, detection_rule="absolute",
                                 lifts=[1.5, 2.0, 2.5, 3.0, 3.5])
grid[["edge_lift", "n_edges", "n_niches", "n_isolated", "mean_abs_asymmetry"]]

The module also provides:

• permutation_edge_stats: a per-edge empirical p value and Benjamini–Hochberg q value against rigid toroidal shifts of the presence maps (requires array coordinates).
• spatial_association: threshold-free bivariate Moran's I and Lee's L.
• Edge direction in the edge table (reciprocal_val, asymmetry, reciprocal_kept). Under the percentile rule both directions of an edge always carry the same weight; only the absolute rule gives the network a direction.

Reproducing Figures 6 and 7 of the paper:

spotnmf network --sample_name SAMPLE1 --results_dir ./results \
  --detection_rule percentile --percentile 0.90 --usage_threshold 0 \
  --edge_threshold 0.199 --n_bins 1000

In Python: nn.plot_network_analysis(results_dir, sample, **nn.PUBLISHED_SETTINGS).

Note on python-igraph. Infomap partitions depend on the igraph version. The paper used python-igraph 0.11.8, and the package pins igraph>=0.11.8,<0.12 (with leidenalg<0.11, whose later releases require igraph 1.x). With python-igraph 1.0.0 the published settings merge the vascular and immune niches on the colorectal sample, giving five niches instead of six. The package warns when another igraph series is installed.

Command-line reference

Command	Description
spotnmf	Full pipeline: deconvolution, annotation, spatial plots and networks
deconvolve	Run OT-NMF and save the factorization
plot	Plot spatial program usage
annotate	Annotate programs by pathway enrichment and gene-set overlap
network	Build and plot niche networks

Run spotnmf <command> --help for the options of each command.

Outputs

File	Contents
topics_per_spot_{sample}.csv	Program usage per spot
genescores_per_topic_{sample}.csv	Gene scores per program
ranked_genescores_{sample}.csv	Ranked marker genes per program

Annotation, plotting and network runs also write enrichment and gene-set overlap tables, spatial plots, network plots and the niche partition.

Tutorials

The notebooks run on the bundled example dataset and are stored with their figures, so they render directly on GitHub.

• Full pipeline: data loading, gene selection, deconvolution, spatial maps, marker genes and validation against ground-truth cell types.
• Proximity (niche) networks: building the co-occurrence network, detecting niches and plotting the network and connection heatmap.

Reproducing the manuscript

The main branch contains the package. The notebooks and analysis scripts behind the manuscript figures, including the revision analyses, are on the manuscript branch:

git fetch origin
git checkout manuscript

Notebooks are in scripts/manuscript_notebooks/ and the revision analysis scripts in scripts/revision/; the README in each folder lists the data each one needs.

Citation

If you use spOT-NMF, please cite:

Abdelkareem, A.O., Gill, G.S., Manoharan, V.T., Verhey, T.B. & Morrissy, A.S. spOT-NMF: gene expression deconvolution via Optimal Transport with proximity-aware niche mapping for spatial transcriptomics. Briefings in Bioinformatics (accepted, 2026). Preprint: bioRxiv (2025), doi:10.1101/2025.08.02.668292.

@article{abdelkareem2026spotnmf,
  title   = {spOT-NMF: gene expression deconvolution via Optimal Transport with proximity-aware niche mapping for spatial transcriptomics},
  author  = {Abdelkareem, Aly O. and Gill, Gurveer S. and Manoharan, Varsha Thoppey and Verhey, Theodore B. and Morrissy, A. Sorana},
  journal = {Briefings in Bioinformatics},
  year    = {2026},
  note    = {Accepted. Preprint: bioRxiv doi:10.1101/2025.08.02.668292}
}

Support and license

Questions, bug reports and feature requests are welcome on the issue tracker.

spOT-NMF is released under the GNU General Public License v3.0; see LICENSE.