Skip to content

Add tested MakeTables integration (continuation of #383) - #445

Open
soodoku wants to merge 5 commits into
DoubleML:mainfrom
finite-sample:feat/maketables-integration
Open

soodoku wants to merge 5 commits into
DoubleML:mainfrom
finite-sample:feat/maketables-integration

Conversation

@soodoku

@soodoku soodoku commented Nov 2, 2026 •

Copy link
Copy Markdown

MakeTables currently cannot consume DoubleML models directly. This adds the plug-in interface so fitted models inheriting from DoubleML can be passed to maketables.ETable, without adding a MakeTables runtime dependency or import.

This continues Alexander Fischer's #383 on current main and retains his implementation commit with its original authorship. The scope remains the existing DoubleML hierarchy, as requested in the review; framework/BLP and scalar classes are outside this contribution.

The interface exposes treatment labels, estimates, standard errors, t-statistics, p-values, pointwise 95% confidence intervals, the outcome name, and sample size. Confidence intervals come from confint(), preserving its aggregation across repeated sample splits. Extraction avoids repeatedly computing the summary. Before fitting or after resetting sample splits, the coefficient table is empty, consistent with summary and the original proposal.

There are 61 integration cases in doubleml/utils/tests, covering PLR, PLIV, IRM, multiple treatments, clustered data, repeated splits, HTML/LaTeX exports, model comparisons, an unfitted model, and resetting/refitting fitted models. MakeTables is a development dependency so these tests run in CI. The executed example is in the separate documentation PR DoubleML/doubleml-docs#280, targeting dev.

Independent automated review identified and verified a stale-interval edge case after resetting sample splits. This is fixed, with ten regression cases that failed before the fix and pass afterward. Independent automated review of the final combined commit b931dbfa found no remaining blocking findings.

This branch also includes the test-only repairs from #446 (consistent LPLR tuning validation) and #448 (an independently fitted sensitivity-defaults test). These can be reviewed and merged independently first. No LPLR or sensitivity production code changes.

Local validation

Python 3.12.12; NumPy 2.5.4, pandas 3.1.0, scikit-learn 1.9.1, Optuna 5.0.0, MakeTables 0.2.0.

  • Current integration tests: 61 passed.
  • Corrected LPLR tuning module: 8 passed, independently checked against scikit-learn's CV score and actual inner-fit predictions.
  • All pre-commit hooks, Ruff, repository-pinned Black, and mypy passed.
  • A separate environment without MakeTables successfully imported DoubleML, fitted a model, and accessed the plug-in data.
  • The original 11 extraction/unfitted regression cases fail on unmodified main and pass with this integration; the ten new reset/refit cases fail without the stale-interval guard and pass with it.
  • Complete-suite validation at b931dbfa: 11,613 passed, 66 skipped, and four worker-process crashes with pytest -n 4 --maxschedchunk=1. All four affected tests passed on a serial pytest --lf rerun (4 passed). Thus all 11,617 non-skipped cases passed across the full run and serial retries; the parallel invocation itself did not exit successfully. BLAS/OpenMP threads were limited to one.
  • The sensitivity-defaults test also passes independently and before the PLR-defaults test (2 passed).

The original four LPLR assertion failures came from comparing two-fold post-fit RMSE with a different five-fold tuning objective; #446 corrects the comparison and isolates sampler state. The skipped cases are intentional parameter combinations, not missing dependencies.

GitHub's build and CodeQL workflows require maintainer approval for this fork contribution. This PR is ready for review.

PR checklist

  • Title and description summarize the changes.
  • Related PRs referenced.
  • Feature has integration tests.
  • Formatting and lint checks pass.
  • Full suite completed; four worker-crashed cases passed on serial retry (details above).

@soodoku
soodoku marked this pull request as ready for review November 2, 2026 21:22
@soodoku

soodoku commented Nov 2, 2026 •

Copy link
Copy Markdown
Author

@SvenKlaassen @felixschmitz — this continuation of #383 is now ready for review. It includes the integration tests requested on #383, with the executed notebook in DoubleML/doubleml-docs#280.

Felix, following your offer to help in py-econometrics/maketables#63, could you review the MakeTables plug-in contract and table behavior? Sven, could you review the DoubleML integration and approve the pending build/CodeQL runs?

Independent automated review caught stale confidence intervals after resetting sample splits; that is fixed and covered by ten reset/refit cases. All 61 integration cases pass. The pre-existing LPLR test failures are corrected in #446, also included here; the sensitivity-test isolation fix is in #448. Complete-suite validation passed 11,613 cases with 66 intentional skips and four worker-process crashes; all four affected cases passed on serial retry. The PR description records the validation and companion test fixes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants