Skip to content

Add PGS robustness experiments, genome-wide validation and 0.6 release packages - #3

Draft
DylanCosto wants to merge 3 commits into
mainfrom
feature/robustness-experiments
Draft

DylanCosto wants to merge 3 commits into
mainfrom
feature/robustness-experiments

Conversation

@DylanCosto

@DylanCosto DylanCosto commented Oct 11, 2026 •

Copy link
Copy Markdown
Owner

Add pgsum robustness so researchers can measure how omitted calls and alternative genotype inputs change a polygenic score. JSON experiments support seeded random masks, BED coverage/dropout masks, optional frequency filling and frozen reference distributions. Outputs include an HTML report, exact scores and deltas, regional and variant attribution, ranks, cutoff crossings and input/output fingerprints.

Every comparison uses an explicit common-callable autosomal baseline. Exclusions and conditional coverage remain visible; frequency fills are experimental reconstructions. These results do not establish phenotype prediction, clinical calibration or ancestry portability.

The public benchmarks cover four sparse CAD models and the full PGS000018 inventory across all 22 autosomes. The dense benchmark retains all 1,745,179 actual rows, preserves the upstream header discrepancy, and uses nested 8/32/128 target cohorts with 64 disjoint reference people. A separate source-genotype calculation verifies scores, masks, fills, attribution, ranks and reference comparisons. Real genotypes and scoring files are not redistributed.

The full model exposed legitimate IUPAC ambiguity in the official GRCh38 FASTA. Compilation now treats those bases as unknown, keeps affected terms unresolved and prevents ambiguous indel anchors. Invalid characters still fail; the compile-rule identity invalidates old Catalog caches.

Prepare version 0.6.0 packages with documentation, checked benchmark records, licences, a hash manifest and a complete offline synthetic example. Native release jobs cover Linux and macOS on x86-64 and ARM64. Manual dispatch builds without publishing; publication requires a matching version tag and passing jobs. This PR remains a draft and no release has been published.

Validation

  • Stable Rust and Rust 1.88: 169 tests passed on each; one optional local-data parity test ignored. Formatting and all-target Clippy passed.
  • Dense public benchmark: all 840 sample/scenario checks passed exact independent reconstruction. The 128-person baseline retains 1,712,276 terms; five scenarios completed in 119.46 seconds with 585.40 MiB peak RSS.
  • Synthetic scale matrix: 24 runs through one million terms and 128 samples, with 5,760 sample/scenario checks and identical numerical evidence across thread counts.
  • Six deliberate output-corruption regressions and 16 smaller full-attribution benchmark runs passed.
  • Linux x86-64 release archive: 44 payload hashes and 38 local Markdown links verified; extracted offline example passed; two package builds were byte-identical.
  • Timings are observations on a shared host, not controlled throughput comparisons. The interrupted 128-person attempt is excluded; the checked result is a fresh unthrottled run.

Review the experiment guide, dense public validation, synthetic scale evidence, and release notes.

Hosted CI run 38499785776 passed on 775b7898: Linux stable, Linux Rust 1.88, macOS stable, existing performance smoke and robustness scale smoke, including extracted-package examples on Linux and macOS.

@DylanCosto DylanCosto changed the title Add reproducible PGS robustness experiments and public-data validation Add PGS robustness experiments, genome-wide validation and 0.6 release packages Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant