[mimosapudical]

Summary

This PR implements Path B from #44 as the first methodology-oriented version of information-screened oCSE for scalable causal discovery. It builds on the standalone Information-LASSO pathway and corrected standard-oCSE conditioning semantics. Path A provides the information-weighted sparse proposal; Path B adds conditional-CMI rescue and restricted oCSE refinement around a parent-superset screening objective.

The method separates high-dimensional discovery into two stages:

1. construct a conservative information-guided candidate superset;
2. run the existing oCSE refinement only inside that reduced candidate universe.

The objective is not to reproduce the exact finite-sample trajectory of unrestricted oCSE. Instead, the screen targets parent containment:

P(Pa(Y) ⊆ Ŵ) → 1,

while keeping the retained working set substantially smaller than the original candidate universe:

|Ŵ| ≪ |C|.

This concentrates expensive conditional-information search on a smaller set without intentionally changing the downstream oCSE criterion.

Methodology formulation

For a target variable Y, let:

• C be the full set of external lagged candidates;
• Hᵧ be the target-history conditioning set;
• Ŵ be the retained screening set;
• Pa(Y) be the population causal-parent set.

1. Information-guided proposal

The first stage uses Information-LASSO to produce an initial sparse proposal S₀ from C.

Conceptually,

S₀ = InfoLasso(C → Y).

This stage is inexpensive relative to unrestricted combinatorial conditional-information search, but a purely sparse or marginal proposal need not be parent-sure.

2. Conditional rescue

For candidates not already strongly represented by the initial proposal, Path B evaluates conditional information of the form

rⱼ = Î(Xⱼ ; Y | Hᵧ ∪ S₀).

Candidates with large conditional information can therefore be recovered even when their marginal association or regression coefficient is weak because of suppression or nonlinear cancellation.

The retained working set is formed under a soft retention target rather than a hard candidate cap. The implementation preserves the full Information-LASSO proposal and, when candidates remain outside the proposal, leaves at least one slot for conditional rescue.

Ŵ = Budget_B(S₀ ∪ Rank(rⱼ)).

The purpose of the rescue stage is not to estimate the final graph directly. Its purpose is to make Ŵ a conservative candidate superset.

3. Parent-superset objective

The statistical target of screening is

Pa(Y) ⊆ Ŵ,

rather than equality between Ŵ and the final parent set.

The corresponding asymptotic objective is

P(Pa(Y) ⊆ Ŵ) → 1.

False positives inside Ŵ are therefore acceptable at the screening stage because they can still be removed by downstream oCSE refinement; false negatives are substantially more costly because an excluded parent cannot be recovered afterward.

4. Restricted oCSE refinement

After screening, the existing oCSE procedure operates only over Ŵ while preserving Hᵧ:

P̂a(Y) = oCSE(Y ; Ŵ, Hᵧ).

At the population level, this motivates the central methodology condition:

if the screen retains the required causal parents, restricting the search universe should preserve the target parent set under the usual oCSE discovery assumptions.

The computational objective is simultaneously to reduce the number of expensive information evaluations:

Q_screened ≪ Q_full,

where Q denotes the number of CMI / shuffle-CMI evaluations required by discovery.

The methodology therefore separates two responsibilities:

screen:
    retain the causal signal conservatively

oCSE:
    distinguish parents from extra retained candidates

Implementation

The new pathway is exposed through:

discover_network(
    data,
    method="information_screened",
    screen_retention=0.40,
    ...
)

For each target variable, the procedure:

1. preserves the target-history variables required for conditioning;
2. obtains an initial information-guided proposal from the Information-LASSO pathway;
3. uses a conditional-CMI rescue pass to recover variables that are weak marginally but informative conditionally;
4. fills the soft retention target while preserving the full proposal and at least one rescue slot when candidates remain
5. runs standard oCSE forward/backward refinement only on this reduced working set;
6. maps the retained variables back to their original lagged-feature indices for final edge reporting.

The conditional rescue step is particularly important because marginal or regression-only screening can miss parents under suppression or nonlinear cancellation.

Why this formulation

An early version of the Path B investigation considered reproducing the exact finite-sample discovery trajectory of unrestricted oCSE.

The experiments showed that this is unnecessarily strong.

Changing the candidate universe can change a greedy sequence of finite-sample conditional-information decisions even when the causally relevant variables remain available.

The more useful methodological question is therefore:

Does the screen retain the variables required for correct downstream discovery while removing enough irrelevant candidates to reduce conditional-information search?

This leads naturally to parent containment rather than exact path identity as the screening objective.

Conceptually:

all lagged candidates
        ↓
information-guided proposal
        ↓
conditional nonlinear rescue
        ↓
parent-superset working set
        ↓
oCSE refinement
        ↓
causal graph

This makes the expensive information-theoretic refinement conditional on a much smaller candidate universe.

Empirical evidence

Primary Gaussian benchmark

Five independent seeds, N=12 and T=300:

• mean retained candidates: 45.6%
• screen coverage of full-oCSE support: 100% on all 5 seeds
• mean truth F1:
  • full oCSE: 0.787
  • screened oCSE: 0.781
• aggregate end-to-end speedup: 1.57×
• total CMI evaluations: −39.2%
• shuffle-CMI evaluations: −41.4%

Scaling

N	full F1	screened F1	retained	CMI reduction	speedup
20	0.793	0.807	42.1%	39.8%	1.54×
50	0.723	0.717	40.8%	44.4%	1.46×
100	0.710	0.715	40.4%	48.7%	1.76×

The reduction in CMI evaluations increases with dimensionality in these experiments.

Controlled nonlinear cancellation

A separate 20-seed mechanism test evaluates a hidden parent whose marginal signal is weak because of nonlinear cancellation.

Hidden-parent recall:

realized candidate budget	proposed screen	forward regression	marginal KDE
~12%	100%	25%	15%
~21%	100%	30%	15%
40%	100%	55%	40%

This experiment isolates the reason for the conditional rescue stage rather than treating the method only as a runtime optimization.

Relation to recent work

A closely related recent direction is Zhao et al., “Fast Flow Matching based Conditional Independence Tests for Causal Discovery” (2026), which combines FMCIT with a two-stage guided PC procedure using screening followed by guided, budgeted refinement.

The reduction axis here is different.

GPC-FMCIT primarily improves the CI-testing / guided-PC pipeline, including accelerating individual conditional-independence tests.

This PR instead targets the candidate universe entering oCSE:

recent guided-PC direction:
    screening / guidance
        → fewer or cheaper CI decisions

Path B:
    information-weighted proposal
        → conditional rescue
        → parent-superset candidate universe
        → restricted oCSE

The intended contribution is therefore specifically an oCSE-oriented parent-superset screen with information-weighted proposal and conditional nonlinear rescue, rather than a general claim of introducing screening before causal refinement.

Validation

The clean implementation has focused regression coverage for:

• the new information_screened discovery method;
• screen-retention validation;
• conditional rescue ranking;
• local/global feature-index mapping;
• external candidate construction;
• preservation of target-history conditioning;
• avoidance of redundant target-history retesting;
• downstream backward refinement.

The full repository CI matrix passes on the clean branch.

Scope

This PR intentionally keeps the production implementation small and reviewable.

It does not include:

• the experiment-heavy research branch;
• a new conditional-independence estimator;
• changes to shuffle-test methodology;
• a claim that screening preserves the exact finite-sample unrestricted-oCSE trajectory;
• a general replacement for standard oCSE;
• a completed finite-sample parent-screening theorem.

The extended benchmark harness, negative-result audit, and theorem development remain separate from the production implementation.

This PR is therefore intended as the first implementation and empirical test of the Path B methodology proposed in #44, rather than as the final theoretical statement of the method.

The immediate question it tests is whether a parent-superset screen with conditional rescue can materially reduce the oCSE search space while retaining the causal information required by downstream discovery.

[chaewon-research]

Small methodology/implementation mismatch I noticed here: the PR describes the rescue score as I(X_j; Y | H_y ∪ S_0), with H_y being the target-history conditioning set. Here Z_rescue is only built from X[:, endpoint], so Z_init isn't included during the rescue step.

Since Z_init is explicitly preserved during the restricted oCSE refinement, should it also be included in Z_rescue? For example, Z_init when endpoint is empty, and np.hstack((Z_init, X[:, endpoint])) otherwise.

I think that would make the implementation match the stated rescue definition.

[mimosapudical]

@kslote1 I think Path B is ready for methodology review now.

I addressed the earlier conditioning issue from @chaewon-research: the rescue score now preserves target history and matches Î(Xⱼ; Y | Hᵧ ∪ S₀). I also clarified the screening objective as parent containment under a soft retention target, rather than a hard candidate budget, and added regression coverage for that contract.

The current Actions run is still awaiting approval, so no CI jobs have actually started yet.

The main design question I’d especially value your view on is whether parent containment under a reduced candidate universe is the right abstraction for Path B before developing the larger benchmark/paper direction further.

When you have a chance, could you take a look at #46?

[chaewon-research]

Thanks for addressing my earlier feedback. I checked the updated rescue step, and it now includes Z_init alongside the Information-LASSO proposal. That resolves the conditioning issue I raised.

I noticed a few other points worth considering:

1. The new screen_retention parameter is inserted before n_jobs in discover_network, which could break existing positional calls. Could we move it to the end of the signature to preserve backward compatibility?

2. This PR also changes the existing standard path by excluding target-history lags from candidate selection and retaining Z_init during backward refinement and edge reporting. I understand the motivation, and I saw the new tests covering some of this behavior. Since these changes affect the existing default method, could we clarify whether they're intended as a separate correction and document the behavioral differences?

3. For the broader methodology evaluation, it would be useful to report ground-truth parent containment separately from full-oCSE-support coverage. The nonlinear cancellation experiment is helpful, but measuring parent recall across different sample sizes and retention levels would provide stronger evidence for the screening objective. An ablation without conditional rescue could also help isolate its contribution.

Overall, I think the methodology is promising, and the original conditioning concern appears resolved. I'd especially like to clarify the API compatibility and standard-path changes before approval.

[mimosapudical]

@chaewon-research Thanks for the review. I’ve addressed all three points in f19d099: positional API compatibility, documentation/tests for the standard-path correction, and separate true-parent containment measurements with sample-size/retention sweeps and rescue ablations. All 14 upstream CI jobs now pass.

Results and reproducible audit. Could you re-review the latest revision?

@kslote1 The results support a conditional-rescue benefit under nonlinear cancellation, alongside remaining recall/F1 trade-offs in Gaussian systems. Could we discuss what additional evidence would justify developing Path B into the separate methods paper you mentioned?