[soodoku]

DoubleML currently asks users to choose a fixed repetition count. This proposal adds a scalar mean diagnostic over existing per-repetition coefficients and an opt-in sequential driver for a single-treatment, nonclustered PLR model.

The driver fits a private one-repetition model on independently shuffled partitions, checks after burn-in, never refits earlier repetitions, and returns either target_reached or budget_exhausted. Its separate mean result carries no sampling SE or p-value; the supplied model and its median-based inference are preserved. Fold-specific tuning, custom scores, clustered data, and multiple treatments are explicitly excluded.

The normal stopping rule follows Ritzwoller–Romano. Its error-rate interpretation is asymptotic. In 1,000 independent pairs on fixed synthetic data, tolerance 0.02 used 29.953 repetitions on average and had 6.0% exceedance (95% Monte Carlo interval 4.69%–7.65%); tolerance 0.01 used 122.261 repetitions and had 6.2% exceedance (4.87%–7.87%). Neither exhausted the budget. These results do not establish exact 5% calibration.

Validation completed:

• 41 feature tests, including output types and preservation of the existing summary DataFrame; the preceding 39-test run reached 100% statement coverage of the new module.
• 3 benchmark tests covering Wilson interval endpoints and model-dependent dimensions and fit counts; 44 feature/benchmark tests pass together. Reverting the benchmark changes makes all three new cases fail for invalid sample indices or incorrect fit counts; restoring them passes.
• 774 utility/PLR/default regression tests.
• Full repository pre-commit hooks (including the pinned Black and Ruff versions), configured mypy, and documentation lint.
• Source distribution and wheel build; new exports verified in the wheel.
• API documentation builds; 21 warnings concern existing dataset references and inaccessible online inventories.

Full pytest run: 11,588 passed, 66 skipped, 2 failed. Both failures are existing LPLR Optuna tuning assertions (nuisance_space-ml_a0-random and nuisance_space-ml_a0-tpe, line 93 of test_lplr_tune_ml_models.py). Both also fail in a clean snapshot of upstream 469daf260d4ea3520b8265acf17a9d29e800c5e0 using the same environment; no LPLR code is changed here. The full run used four pytest workers and unchanged upstream downloads for its two remote fixtures through a temporary local test adapter. Additional feature cases were checked separately after the full run; all 41 feature cases pass.

The benchmark source, raw results, and limitations are included under benchmarks/. Numerical diagnostics use a dictionary, consistent with existing utilities such as gain_statistics; existing inference tables remain DataFrames.

Codacy static analysis passes on the DRY follow-up commit 7a05c58bd1a7d2b980901561233308c504bf4fe1. The build workflow and CodeQL workflow have not executed: GitHub requires maintainer approval for this contribution. Local results above are not a claim that those upstream jobs passed.

The DRY follow-up replaces the handwritten Wilson interval with SciPy's maintained binomtest(...).proportion_ci(method="wilson") and derives benchmark dimensions and nuisance-fit counts from the model. Default adaptive and fixed-budget benchmark results agree before and after the refactor, excluding timing. The duplicate-code scan found no copied blocks involving the contribution. Benchmark documentation links here for human-review status rather than duplicating it.

This pull request includes code written with the assistance of AI.
Human review was confirmed for commit 19d275b6a268c60408ab1cd3a34666c1f7783fbb.
The subsequent DRY follow-up has not yet been reviewed by a human.

[SvenKlaassen]

Thanks @soodoku,
that looks interesting. But this would be a large devation from our current design and only be implemented for PLR.
We are in the process to generally redesign our package architecture and i will check with coauthors to see if such an implementation would be useful to include.

[soodoku]

Thanks, Sven—that makes sense. The PLR restriction was intended to keep the initial implementation narrow, but I agree that the fitting driver should fit the broader architecture.

I’d be happy to adapt this to the redesign once you’ve discussed it with the coauthors. One possible split is to keep the scalar diagnostic separate from the sequential fitting driver, with the latter using whatever repetition and sample-splitting interfaces the redesign provides.

A key question is whether a separate mean-aggregation diagnostic is useful alongside the existing median-based inference. If helpful, I can put together a short design note covering that distinction and the interfaces needed for broader model support.

[SvenKlaassen]

Thank you very much. I will comment again on this if we decide to go forward, but this might take a long time as we are currently updatin/redesigning a lot of parts of the package.