phenocal

Genome-anchored calibration of a continuous phenotype across cohorts/batches, with uncertainty-propagated probabilistic labels.

phenocal is the reference implementation of the genome-anchored phenotype-calibration method of de Ruvo et al. (PhenoCal; see Citation below). It is organism-, phenotype-, and schema-agnostic: given a continuous phenotype, a cohort/batch label per sample, and the genomes as cgMLST/wgMLST allelic profiles (or any genomic distance that can flag near-clonal pairs), it estimates one calibration offset per cohort and propagates calibration uncertainty into per-cohort credible intervals and isolate-level probabilistic labels.

Zero external dependencies; builds offline.

Idea in one line

Two near-clonal isolates from different cohorts share (approximately) the same biological phenotype, so the difference in their measured values estimates the difference in cohort/protocol offset — the biology approximately cancels (exactly at zero genomic distance, approximately for near-clones). Offsets are fitted on a comparison graph by weighted least squares; one cohort is the anchor (δ=0).

flowchart LR
    A(("labA<br/>anchor, δ = 0")) ---|"3 twin pairs at d = 0<br/>labB reads 2× labA"| B(("labB<br/>δ = log₁₀ 2"))
    A ---|"3 twin pairs<br/>labC reads 5× labA"| C(("labC<br/>δ = log₁₀ 5"))
    B ---|"3 twin pairs<br/>labC reads 2.5× labB"| C

Loading

The toy example as a comparison graph: cohorts are nodes, each edge is the median ratio between genetically identical isolates measured in the two cohorts, and the offsets δ are the values on the nodes that best explain all edges at once.

A step-by-step explanation of the method, with figures, is at https://genpat-it.github.io/phenocal/; every calculation is worked out by hand, from allelic profiles to labels, in the worked example (examples/worked_*.tsv).

Installation

Prebuilt binary (Linux x86_64/arm64, macOS Intel/Apple Silicon; static, no dependencies; the download is verified against its SHA-256):

curl -fsSL https://raw.githubusercontent.com/genpat-it/phenocal/main/install.sh | sh

It installs phenocal into ~/.local/bin; set PHENOCAL_INSTALL_DIR to change the directory and PHENOCAL_VERSION=v0.1.3 to pin a release. Windows users and anyone preferring a manual install can download the archive for their platform from the latest release.

Docker (linux/amd64 and linux/arm64; also usable from Nextflow and other workflow managers):

docker pull ghcr.io/genpat-it/phenocal:latest
docker run --rm -v "$PWD:/data" ghcr.io/genpat-it/phenocal \
    --phenotypes pheno.tsv --profiles results_alleles.tsv --out result

The image also ships the toy example in /opt/phenocal/examples.

Cargo (any platform with a Rust toolchain), from crates.io:

cargo install phenocal

From source:

git clone https://github.com/genpat-it/phenocal && cd phenocal
cargo build --release        # binary: target/release/phenocal

The code exactly as used in the AIS 2026 paper is frozen at the git tag ais2026. Later releases add packaging and input formats; on the same input they give byte-identical results (see the changelog).

Quick start (synthetic toy)

phenocal --phenotypes examples/toy_phenotypes.tsv \
         --profiles examples/toy_profiles.tsv \
         --anchor labA --min-support 2 --out /tmp/toy

--pairs examples/toy_distance_matrix.tsv or --pairs examples/toy_pairs.tsv give the same results from precomputed distances.

The toy has three labs measuring three shared strains; labB reads ~2× and labC ~5× higher than labA. phenocal recovers that:

labA   1.00x  [1.00, 1.00]   (anchor)
labB   1.99x  [1.05, 3.78]
labC   5.01x  [2.64, 9.26]

Inputs

The phenotypes, and the genomes in one of three forms: allelic profiles (--profiles), or precomputed distances as a distance matrix or a pair table (--pairs). Give --profiles or --pairs, not both. Headers are matched case-insensitively; common aliases are accepted.

Phenotypes (--phenotypes): one row per sample.

column	aliases	meaning
sample	accession, id	sample identifier
value	mic, phenotype	raw phenotype (log10 by default)
cohort	author, batch, study	cohort / batch label

Allelic profiles (--profiles), the usual case

The cgMLST or wgMLST allele calls, one row per sample and one column per locus, as written by chewBBACA (results_alleles.tsv), BIGSdb/pubMLST exports, ReporTree or cgmlst-dists input. Nothing has to be computed beforehand: phenocal computes the distances itself.

FILE         lmo0001.fasta  lmo0002.fasta  lmo0003.fasta  lmo0004.fasta
S1.fasta     1              INF-12         3              LNF
S2.fasta     1              12             4              7
S3.fasta     2              *7             ASM            7

• The distance between two samples is the number of loci called in both whose alleles differ (the allelic or Hamming distance of chewBBACA, cgmlst-dists and ReporTree); a missing call never counts as a difference.
• The chewBBACA INF- prefix is removed (INF-12 is allele 12). 0, empty cells, -, ?, N, NA and the chewBBACA classes LNF, PLOT3, PLOT5, LOTSC, NIPH, NIPHEM, ALM, ASM, PAMA, EXC, INV are missing.
• Integer alleles are compared by value, so the 32-bit CRC hashes of chewBBACA hashed profiles work at any size; any other text (sequence hashes, *7 novel alleles) is an allele compared as text.
• Only the samples with a phenotype are kept, and only cross-cohort distances are computed, on all cores (--threads to change). For 2,099 Listeria wgMLST profiles (6,785 loci) this takes under two seconds.
• Row names are matched to the phenotype ids like matrix names (below).
• examples/toy_profiles.tsv is the toy in this form.

Precomputed distances (--pairs)

If you already have the distances, pass them with --pairs, in either layout (detected from the header; --pairs-format long|matrix forces one). Within- and cross-cohort pairs are allowed; only cross-cohort pairs are used.

Distance matrix, a square TSV whose first header cell is any label and whose other header cells, and first cell of each row, are the sample names (same order in rows and columns): the output of chewBBACA DistanceMatrix (distance_matrix.tsv, distance_matrix_symmetric.tsv), ReporTree (<prefix>_dist_hamming.tsv), cgmlst-dists, cgDist, for cgMLST and wgMLST alike, passed as it is:

FILE        S1.fasta  S2.fasta  S3.fasta
S1.fasta    0         3         7
S2.fasta              0         5
S3.fasta                        0

• Symmetric matrices and triangular ones are accepted, with the other half empty (chewBBACA distance_matrix.tsv) or filled with zeros (cgmlst-dists --matrix-format lower-tri/upper-tri); each pair is read once. A matrix whose two halves disagree is rejected.
• Names are matched to the phenotype sample ids exactly, without the FASTA extension (S1.fasta → S1), or by unique id prefix (S1_shovill.fasta → S1, the assembler suffix kept by chewBBACA from the file name); the run reports how many names matched each way. Samples without a phenotype are ignored; NA or empty cells are skipped.
• The matrix is streamed, so memory does not grow with its size beyond the cross-cohort pairs kept. examples/toy_distance_matrix.tsv is the toy in this layout.

Pair table, one row per pair, each pair once (useful for SNP distances or any other ruler):

column	aliases	meaning
sample_i	acc_i, i	first sample
sample_j	acc_j, j	second sample
distance	dist, d	genomic distance

Run

phenocal \
  --phenotypes pheno.tsv \
  --profiles results_alleles.tsv \
  --out result \
  --anchor USA              # optional; default = largest cohort
  --dashboard result.html   # optional; interactive HTML report

Options

flag	meaning	default
--anchor <COHORT>	cohort fixed to offset 0	largest cohort
--already-log	value is already log10	off (raw, log10 applied)
--min-support <N>	min near-clonal pairs per edge (n_min)	5
--max-drift <D>	max median drift (κ) for τ selection	1.0
--sigma <fixed|empirical>	per-cohort resolution σ_c: fixed (log10 2) or estimated from each cohort's grid	fixed
--drift-scale <doubling|rms|mean|max>	units of the τ drift tolerance (combine the two cohort σ); not load-bearing	rms
--lambda <L>	biological-drift variance per unit genetic distance in the edge SE	log10(2)²
--robust	robust bootstrap (Student-t edge perturbations)	off (Normal)
--nu <DF>	degrees of freedom for --robust	4
--bootstrap <B>	bootstrap draws for credible intervals	10000
--thresholds <list>	comma-separated tolerance cutoffs (raw scale)	0.75,1.0,1.25,1.5,2.0
--clusters <FILE>	TSV sample<TAB>cluster of clonal-cluster ids; enables the cluster-effective inference (cluster-weighted point + cluster-resampled 95% interval), the preferred summary when near-clonal pairs are nested in clusters	off (pair-level only)
--no-cutoff	skip the (costly) data-driven cutoff estimation; still writes harmonised + labels	off
--pairs-format <auto|long|matrix>	layout of --pairs (pair table or distance matrix)	auto (from the header)
--threads <N>	threads for the distances computed from --profiles	all cores
--seed <S>	RNG seed	20260603
--dashboard <FILE>	also write the interactive HTML report (open locally)	off
--trace <FILE>	also write a step-by-step mathematical trace in Markdown (renders on GitHub, no compilation)	off

Run phenocal --help for the same list and phenocal --version for the version.

Cluster-effective inference (--clusters)

When near-clonal pairs are nested within clonal clusters (e.g. NCBI SNP clusters), pairs from one cluster are not independent and the pair-level interval is anticonservative. Pass a sample<TAB>cluster table:

phenocal --phenotypes pheno.tsv --profiles results_alleles.tsv --clusters clusters.tsv \
         --anchor USA --out out

out.offsets.tsv then gains three columns — cluster_wt_fold (median over per-cluster medians, one vote per clonal unit), cluster_eff_lo95 and cluster_eff_hi95 (bootstrap resampling clusters, not pairs). This is the preferred inferential summary when cluster ids are available.

Outputs

• <out>.offsets.tsv — per cohort: δ (log10), fold vs anchor, 95% credible interval.
• <out>.sigma.tsv — per cohort: estimated measurement resolution σ_c (log10 and in dilutions) and the σ mode. With --sigma empirical this is read off each cohort's own MIC grid (recovers log10 2 ≈ 0.30 for two-fold cohorts, smaller for finer grids).
• <out>.edges.tsv — per cohort pair: d_min, selected τ, n pairs, fold, SE, weight (transparency on what supports each edge).
• <out>.harmonised.tsv — per sample: raw and harmonised phenotype (value / 10^δ).
• <out>.labels.tsv — per sample: harmonised value + probability of tolerance P(T) at each threshold.
• <out>.cutoff.tsv — data-driven sensitive/tolerant cutoff on the harmonised scale, by two methods: KDE antimode (non-parametric density valley) and GMM crossover (Bayes-optimal boundary of a 2-component Gaussian mixture, with a bootstrap 95% interval). They bracket the cutoff; phenocal does not commit to a single value (the cutoff is treated as uncertain).

Interactive dashboard (--dashboard FILE)

A single self-contained HTML file (no dependencies, no network — opens in any browser) to inspect the whole run:

• the comparison graph drawn in SVG (nodes = cohorts, anchor highlighted, edge thickness ∝ weight);
• a forest plot of the offsets with 95% credible intervals;
• before → after harmonisation (median + IQR per cohort on the log scale);
• per-edge detail — every candidate twin pair with sample IDs, genomic distance, both MICs and the log-ratio, the rows that entered the adaptive-τ median highlighted, and the resulting edge Δ;
• the per-isolate harmonised values and probabilistic labels, with a live filter.

Mathematical trace (--trace FILE)

A step-by-step Markdown document of the actual run, with the real numbers and the formula behind each step: inputs and per-cohort σ_c, the per-edge twin tables and adaptive-τ selection, the WLS normal-equation matrices (AᵀWA, AᵀWΔ), the solution, the bootstrap credible intervals, and the data-driven cutoff. It is the didactic worked example auto-filled with this run's data. Because it is Markdown (with $...$ math and tables), it renders directly on GitHub — no compilation, no Docker, just open it.

Algorithm (per cohort pair, per ruler)

flowchart TD
    P[/"Phenotypes<br/>sample · value · cohort"/] --> X
    D[/"Genomes<br/>allelic profiles, distance matrix<br/>or pair table"/] --> X
    X["1 · Cross-cohort pairs<br/>genomic distance d<br/>log-ratio s of the two values"] --> T
    subgraph E["per cohort pair"]
        T["2 · Adaptive bound τ<br/>smallest d with enough pairs<br/>and a stable median"] --> V
        V["3 · Edge value Δ<br/>median of s over d ≤ τ<br/>weight 1 / SE²"]
    end
    V --> G["4 · Weighted least squares on the comparison graph<br/>cohort offsets δ, anchor δ = 0"]
    G --> B["5 · Bootstrap the edges and refit<br/>95% credible intervals"]
    B --> H["6 · Harmonised values<br/>value / 10^δ<br/>P(harmonised ≥ c) per threshold"]
    H --> C["Cutoff on the harmonised scale<br/>KDE antimode · GMM crossover"]

Loading

1. Collect cross-cohort pairs with distance d (computed from the allelic profiles, or read from --pairs) and signed log-phenotype difference s = y_a − y_b (oriented a over b). Every pair is used once: profiles and distance matrices guarantee it, a pair table should not repeat a pair.
2. Adaptive τ: smallest distance bound with ≥ min-support pairs and median drift ≤ max-drift · σ̄_ab from the d_min value, where σ̄_ab is the edge resolution set by --drift-scale (= log10(2) for two-fold grids).
3. Edge value Δ = median(s | d ≤ τ); SE = sqrt((σ_a² + σ_b² + λ·τ)/n) with per-cohort resolution σ_c (--sigma); w = 1/SE². (Reduces to log10(2)·sqrt((1+τ)/n) when all cohorts are two-fold.)
4. Weighted least squares on the comparison graph for cohort offsets, anchor δ=0.
5. Bootstrap (sample Δ* ~ N(Δ, SE²), refit) → pair-level 95% credible intervals. With --clusters, also resample clonal clusters (not pairs) per edge and refit → cluster-weighted point + cluster-effective interval, the preferred inferential summary when near-clonal pairs are nested in clusters.
6. Per threshold c: P(T_i(c)) = mean over draws of [ MIC_i / 10^δ ≥ c ].

The offsets are identifiable up to an additive constant (gauge); the anchor is a coordinate convention and does not affect harmonised labels (Lemma 1 of the paper). A connected graph is required.

Validation

Controlled benchmark (known truth). Simulated cohorts with planted offsets, where the estimator recovers them, the propagated intervals cover at the nominal rate and widen gracefully as the comparison graph thins, and the estimate stays unbiased where naive pooling and additive per-cohort mean-centering (what ComBat-style correction reduces to for a single label) over-correct because cohorts sample different biological populations. The stated failure mode — a cohort-directional biological shift, constant within a cohort — is read as protocol by construction and is reported as the identifiability boundary rather than hidden.

Real case (AIS 2026 paper). Benzalkonium-chloride minimum inhibitory concentrations of Listeria monocytogenes from five laboratory cohorts, with the genome as the similarity ruler (cgMLST and wgMLST used as two independent rulers). Raw pooled labels leave the cross-cohort task unevaluable; calibrated labels restore it. Robustness covers a sweep of the relatedness bound τ, leave-one-edge-out refits, and permutation and leakage controls.

The implementation ships an 11-test suite (offset recovery, anchor invariance, edge selection, cutoff, cluster-effective recovery) run under CI; cargo test passes.

Citation

de Ruvo A, Castelli P, Di Pasquale A, Radomski N. PhenoCal: Natural-Control Label Calibration for Cross-Cohort Machine Learning. Accepted at The 12th International Artificial Intelligence Symposium (AIS 2026); to appear in Springer Lecture Notes in Computer Science (LNCS). Corresponding author: A. de Ruvo (andrea.deruvo@gssi.it).

License

MIT.