Skip to content

Optimize cohort scoring and prepare release validation - #1

Merged
DylanCosto merged 9 commits into
mainfrom
fix/panel-comparison-and-imputation
Sep 30, 2026
Merged

DylanCosto merged 9 commits into
mainfrom
fix/panel-comparison-and-imputation

Conversation

@DylanCosto

@DylanCosto DylanCosto commented Sep 30, 2026 •

Copy link
Copy Markdown
Owner

Cohort scoring repeatedly subtracted and added contributions for each non-reference sample. Precompute exact differences once per term and skip packed groups of four homozygous-reference samples. Keep the general path for wide coefficients, different dosage-weight exponents, and missing calls, preserving score text and coverage.

On synthetic workloads of two 100,000-term scores and 257 samples, the first five-run comparison reduced scoring time by 22–24% for sparse cohorts and 48% for dense cohorts at 1 and 4 threads. A second comparison confirmed the gain and passed a 15% scoring-regression gate. These are synthetic scoring-only measurements; single-sample gVCF speed was essentially unchanged. See bench/README.md for reproduction and timings.

Add a benchmark runner with an independent integer oracle, exact-output comparisons across binaries and thread counts, wall/CPU time, and peak memory. CI now tests stable Rust on Linux/macOS and Rust 1.88 on Linux, plus a release-mode benchmark smoke test. Release packaging checks the tag against the package version, uses locked tests, smoke-tests native binaries, and includes release notes.

This PR also fixes reference-panel comparison symmetry: terms below 99% panel call rate are excluded from both sides and matched coverage. Score JSON moves to pgsum-score-v3, undefined Z-scores become null, and imputation metadata distinguishes observed scores, sample filling, and reference-panel filling. Public hand-calculated tests and migration notes cover the changes.

Validation:

  • Full suite: 87 passed, 1 ignored (external HG002 data), on Rust 1.88 and in release mode on stable Rust.
  • cargo fmt --check and cargo clippy --all-targets --locked -- -D warnings: passed.
  • Two five-run before/after benchmark comparisons passed the independent oracle and output-equivalence checks.
  • Uploaded repository tree exactly matches the locally tested files.

Large real-genome panel comparisons have not been rerun under the exclusion rule. Historical panel results are labeled accordingly. No release tag has been published.

@DylanCosto DylanCosto changed the title Fix reference-panel comparisons and label imputation Optimize cohort scoring and prepare release validation Sep 30, 2026
@DylanCosto
DylanCosto marked this pull request as ready for review September 30, 2026 12:54
@DylanCosto
DylanCosto merged commit c8acca6 into main Sep 30, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant