Proposal: complete the Information-LASSO pathway for scalable high-dimensional discovery #44

Motivation

While reviewing the high-dimensional discovery path, I noticed that information_lasso already has a public API entry and substantial documentation describing information-guided variable selection, but the current implementation still delegates directly to the ordinary LASSO implementation.

Conceptually, the documentation describes something more interesting: using conditional-mutual-information estimates to guide sparse variable selection, with the goal of retaining the information-theoretic structure of oCSE while reducing the cost of high-dimensional discovery.

I would like to propose completing this pathway as a self-contained contribution and benchmark it against both ordinary LASSO and standard oCSE.

Why this may be useful

The current oCSE forward search becomes expensive as the number of variables and candidate lags grows, because many candidate conditional-information evaluations and significance tests are required.

Information-guided sparse selection could potentially provide a substantially smaller candidate set in high-dimensional systems while still using information-theoretic relevance rather than purely linear predictive coefficients.

This also creates a possible bridge between the currently documented information_lasso method and a future scalable oCSE workflow:

all lagged candidates
        ↓
information-guided sparse selection
        ↓
smaller candidate set
        ↓
optional exact oCSE forward/backward refinement
        ↓
causal graph

I would treat the standalone Information-LASSO implementation and the optional oCSE-refinement idea as separate stages so that the initial contribution remains reviewable.

Proposed first stage

1. Define the intended Information-LASSO objective precisely from the existing documentation.

2. Compute information-based weights for candidate lagged predictors.

3. Use those weights in the sparse-selection stage rather than delegating directly to ordinary LASSO.

4. Integrate the implementation through the existing method="information_lasso" API without changing the behavior of the current standard/alternative oCSE methods.

5. Add unit and integration tests.

6. Benchmark against:

   • ordinary LASSO,
   • standard oCSE,
   • the new Information-LASSO path.

Evaluation

I would evaluate both scientific quality and computational behavior, including:

• true/false positive edge recovery;
• precision, recall, and F1;
• selected-candidate count;
• runtime;
• sensitivity to sample size and dimensionality;
• linear Gaussian and nonlinear synthetic systems;
• correlated predictors, where sparse screening can be difficult.

In particular, I would not assume that sparse screening preserves all causal parents. The benchmark should explicitly measure whether the computational gain comes with a recall cost.

Design question

There seem to be two useful scopes:

A. Standalone Information-LASSO

Complete the method currently described by the public API/documentation and return its selected causal graph directly.

B. Information-guided screening + exact oCSE refinement

Use Information-LASSO only as a conservative high-dimensional candidate screen, then run the existing oCSE procedure on the retained candidates.

The second option could potentially preserve more of the existing oCSE semantics while substantially reducing the expensive search space, but it is also a larger methodological change.

Would you prefer the initial contribution to stop at the standalone Information-LASSO implementation, or would the screening + oCSE refinement direction also be useful for the planned submission?

If this direction is useful, I would be happy to take ownership of the implementation, benchmarks, tests, and documentation.