Skip to content

Fix ANCOM-BC estimation and qualify pinned reference comparisons - #13

Merged
hyperpolymath merged 11 commits into
mainfrom
fix/issue-23-reference-pins
Nov 7, 2026
Merged

hyperpolymath merged 11 commits into
mainfrom
fix/issue-23-reference-pins

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

The native original-ANCOM-BC implementation and its R reference disagreed on soil, while the initial reference CI also had incompatible CRAN dependencies. This change tightens the native optimizer's stopping rule, keeps coefficient estimation independent of the conservative-SE option, fixes numeric group ordering, and makes the complete reference qualification reproducible.

The numerical investigation identified an upstream bounded-simplex variance defect. Unmodified ANCOMBC 2.12.0 converges to a zero extra variance even when a positive value improves its likelihood. The focused upstream repair is ANCOMBC#344; its analytical regression and negative control pass. The native Julia estimator agrees with the explicitly pinned repaired R estimator on mock, gut and soil at the original thresholds. A separately named pristine control keeps the three known soil discrepancies visible as expected failures. This is not a claim of equivalence to unmodified ANCOMBC 2.12.0.

The optional matrix reference adapter also avoids a one-column metadata bug, fixed separately in ANCOMBC#343. R 4.5.0 / Bioconductor 3.22 are pinned along with the CRAN snapshot and CVXR API. No R fallback is added to the native implementation.

Validation includes 202 assertions on Julia 1.10 and 1.12.5 (including Aqua), the pinned nine-module Agda gate and 20 controls, all eight ALDEx2 output columns plus identical-draw p/q checks, and 27 checks against the pinned ANCOM-BC repair. Source-specific receipts and commands are recorded in proofs/PROOF-STATUS.md and docs/ANCOMBC-SPEC.md; the final branch CI must pass before this draft is considered ready.

This remains a review draft. The code, tests and proofs were developed with OpenAI Codex assistance and need human maintainer review and understanding before General registration. There is no registered v0.3 release yet. MetaManifold's candidate is tested only through a fork development environment; no upstream WebUI PR has been submitted and issue 23 is not closed. The proofs cover stated exact algebraic identities, not optimizer global optimality, Float64 refinement, biological identifiability or calibration/FDR control.

Pin the CRAN snapshot and CVXR API needed by ANCOMBC 2.12.0. Keep numerical conformance thresholds unchanged. Clarify that agreement with a reference is not calibration evidence.
Preserve numeric group ordering before string conversion; align native/reference structural-zero defaults and expose the reference pseudocount. Install the optional phyloseq reference dependency explicitly. Keep ANCOM-BC numerical conformance pending and thresholds fixed.
Use ancombc(data, aggregate_data, meta_data) from the pinned public manual, avoiding the optional phyloseq dependency. Preserve reference versions and numerical thresholds.
Keep an unused sample identifier alongside condition because ANCOMBC 2.12.0 drops a one-column metadata frame when subsetting its generic matrix input. The model formula is unchanged. Retain the oracle R library path after setup-r.
Match the reference API semantics: conserve changes the standard error, while the coefficient retains the EM bias correction. Document the distinction from the WLS numerator in paper equation 35, test coefficient invariance, and retain the fixed oracle thresholds.
Load the Statistics standard library explicitly for the oracle bias diagnostic.
Use the declared tolerance for native parameter convergence, rather than its square root. Give the slower reference EM a larger iteration budget and assert its internal convergence residual before using it as an oracle. Preserve every numerical comparison threshold; disclose AI assistance and registry review requirements.
Both reference and native fits converge but disagree on soil. Record fitted components and likelihoods without changing acceptance tolerances. Document the reference-source inspection accurately.
Keep the unmodified Bioconductor gate visible. Add an explicitly identified estimator-only comparison at the signed upstream repair SHA, without changing numerical thresholds. Record the completed nine-module proof receipt.
… conformance

Preserve the three unmodified ANCOMBC 2.12.0 soil discrepancies as explicit expected failures backed by an analytical boundary counterexample. Require all original conformance thresholds against pinned repair 6d09473. Document the source distinction, derivation and remaining upstream review.
@coderabbitai

coderabbitai Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Assign WLS variance classes by fitted mixture-proportion quantiles, with explicit lower/upper tie conventions. Add analytical WLS and variance examples and record the classification in provenance.
@hyperpolymath
hyperpolymath marked this pull request as ready for review November 7, 2026 18:11
@hyperpolymath
hyperpolymath merged commit ca18716 into main Nov 7, 2026
1 check passed
@hyperpolymath
hyperpolymath deleted the fix/issue-23-reference-pins branch November 7, 2026 18:11
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-07T18:14:28.905927Z bf84ba0 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants