Repository navigation
Conversation
Author
|
Superseded by the ready-for-review upstream PR: DoubleML#429 . The feature branch is retained. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DoubleML currently asks users to choose a fixed repetition count. This proposal adds a scalar mean diagnostic over existing per-repetition coefficients and an opt-in sequential driver for a single-treatment, nonclustered PLR model.
The driver fits a private one-repetition model on independently shuffled partitions, checks after burn-in, never refits earlier repetitions, and returns either
target_reachedorbudget_exhausted. Its separate mean result carries no sampling SE or p-value; the supplied model and its median-based inference are preserved. Fold-specific tuning, custom scores, clustered data, and multiple treatments are explicitly excluded.The normal stopping rule follows Ritzwoller–Romano. Its error-rate interpretation is asymptotic. In 1,000 independent pairs on fixed synthetic data, tolerance 0.02 used 29.953 repetitions on average and had 6.0% exceedance (95% Monte Carlo interval 4.69%–7.65%); tolerance 0.01 used 122.261 repetitions and had 6.2% exceedance (4.87%–7.87%). Neither exhausted the budget. These results do not establish exact 5% calibration.
Validation completed:
Full pytest run: 11,588 passed, 66 skipped, 2 failed. Both failures are existing LPLR Optuna tuning assertions (
nuisance_space-ml_a0-randomandnuisance_space-ml_a0-tpe, line 93 oftest_lplr_tune_ml_models.py). Both also fail in a clean snapshot of upstream469daf260d4ea3520b8265acf17a9d29e800c5e0using the same environment; no LPLR code is changed here. The full run used four pytest workers and unchanged upstream downloads for its two remote fixtures through a temporary local test adapter. The final six additional feature cases were checked separately; all 39 feature cases pass.This is a draft review PR in the finite-sample fork, before upstream submission. The benchmark source, raw results, and limitations are included under
benchmarks/.This pull request includes code written with the assistance of AI.
The code has not yet been reviewed by a human.