White-box estimation of deep ReLU network outputs under a compute-metered score

Team EscAI entry and research corpus for the ARC White-Box Estimation Challenge 2026: estimate the expected post-ReLU outputs of random He-initialized MLPs (Phase 2: 1024 wide × 16 deep) from the weights alone, scored as final-layer MSE × max(0.1, C/B) where every floating-point operation is metered against a budget B = 2^41.

Author: Luiza Corpaci (MIT license — see LICENSE). Built in the open on top of two attributed codebases — a competitor's MIT-licensed repository (504aldo) and the organizers' reference implementation — with the exact boundaries of embedded / re-derived / original work documented in THIRD_PARTY.md.

Results

phase	submission	raw MSE	utilization	adjusted score
1 (256×32)	327443, then 327501	—	9.7%	official best 2.69e-7 (21.6× over the strongest bundled baseline)
2 (1024×16)	332675	2.19e-8	0.2308	5.0e-9, rank 42, zero failed MLPs

The Phase-2 estimator is factorized k_max=3 cumulant propagation with a memoryless fourth-cumulant channel, age-gated shared-subspace compression of old third-cumulant sources, a final-layer trim, and pooled batched Strassen products — the full provenance and gate history of every mechanism is in RUNGS.md (the engineering ladder, rungs 0–9, each with preregistered gates and measured verdicts).

What we think is worth reading even if you don't care about the score

The negative results are quantitative and, we believe, general to this problem class:

• The eps² law as a wall (RUNGS.md rungs 8–9, research/DEADENDS.md): replacing transported third-cumulant content costs extra MSE ≈ 4.2e-6·eps² (eps = relative error of the (2,1) diagonal slice). Staying within a few percent of raw therefore demands ~99.9 % slice fidelity — and every compressive construction we measured (rank truncation of frozen factors, graduated retirement, column sketches, a transport-covariant rider class, scalar laws) lands at eps 0.15–8 against a bar of 0.03–0.05. Energy/rank arguments systematically underestimate slice error 25–40× because the slices consume legs through Hadamard products. research/eps_prescreen.py screens any new candidate in minutes before it wastes a paired test.
• Cost-floor lemmas (research/astra_youngblock.md §4): a no-free-sandwich argument (any self-transported n×n object busts the budget) plus a materialization lemma (Hadamard consumers force dense legs per layer; the age-1 propagator is measured incompressible) bound the young-source block at ~0.055–0.07·B under this representation.
• Accuracy-by-cancellation, measured twice (research/astra_k4_overshoot.md): the strongest public κ₄ channel wins by cancelling its own base's truncation bias (its fitted λ is ~3× the physically measured value); transplanted onto an exact base the same channel hurts. Fitted constants can encode another stack's errors — a portability trap that our holdout gates caught on both attempts.
• A falsification chain that held — preregistered gates with stop clauses at every rung, never-touched validation nets, per-net paired statistics, and a dead-end catalogue (research/DEADENDS.md) where every kill carries its measurement.
• Oracle diagnostics of the remaining gap (research/tenfold_push/): correcting three pre-activation moment channels with reference values reduces raw MSE ~20×, locating the remaining error in joint moment propagation — but no deployable cheap approximation tested reproduces it.

Repository layout

• submission/estimator.py — the shipped estimator (single file, organizer-sandbox imports only), with the attributed V29 stack, our cost machinery, a switchable physics-validated fallback chain, and provenance comments on every fitted constant.
• RUNGS.md — the engineering ladder: what was built, gated, measured, kept or falsified, rung by rung.
• METRICS.json — the graded submission history and the designated submission's official + local gate numbers, pinned to the estimator's sha256; the release tag Phase2-submission-332675 carries the same numbers.
• research/ — the analysis corpus: exact f64 reference chains and parity harnesses (port_k3.py, port_k3_fast.py, port_k3_lean.py), probes (eps_prescreen.py, anatomy_rung7.py, aug_g_chain.py, fit_lambda_k4.py), the derivation documents (astra_*.md), the dead-end catalogue (DEADENDS.md), the competitor-repo digest (aldo_digest.md), and the push investigations (tenfold_push/, top3_push/).
• lab/ — the Phase-1 experiment harness (suite .npz files are not shipped; regenerate them with lab/generate.py).
• RESULTS.md, PHASE2_REVIEW.md — phase findings and reviews.
• methodology/ — the two lab-protocol documents every experiment in this corpus was run under (preregistration with falsifiers; the benchmark/billing gate discipline).
• research/contraction_push/ — joint-state sketch diagnostics and five tested closure configurations, including input-radius conditioning; none promoted.
• SVD complexity and prediction experiments — spectral profiles, held-out difficulty predictions, adaptive rank allocation, and output residual prediction; no candidate promoted.
• Output response and moment-preserving compression — deferred correction tests, SVD response capacity, and improved small-network fourth-order projections with verified structured diagonal transport.
• Complete streamed fourth-order recurrence — matched whole-network reference outputs with lower working memory; the target-width cost gate fails, and a separate exact cubic component is checked.
• Transported memory and correction targets — 53 paired/control/teacher network rollouts distinguish accumulated-error and innovation targets; no held-out gain, plus a checked gap in the claim that direction rankings rule out mixed compression bases.

Curation note

This repository is a curated selection from a larger working tree: the science (ledgers, derivation documents, reference chains, probe scripts, and the shipped estimator), without multi-gigabyte derived data or internal process notes. Two things are deliberately withheld until the challenge's submission window closes (2026-10-17): work-in-progress research lines, and our internal constraints-and-routes planning document (cited from research/DEADENDS.md as reports/ARC_CONSTRAINT_MAP.md). Both will be added here after that date — the goal is open research, not a live strategy feed.

Data: evaluation networks (research/mini_p2.npz, ~5.8 GB, sha256 12a386b64be56a2e693d8d138c15fe7cda8368c962bfc7ee9b870dcf10bc3810) are derived from the challenge's public dumps and are not shipped in this repository; regenerate them with the organizers' starter kit, or use the recorded hash to verify a copy. The grading harness (whest-starterkit/, flopscope) comes from the organizers and is likewise not redistributed.

Reproducing the gate numbers

Every number in RUNGS.md names its harness. The two entry points:

# f64 reference-chain gates (parity, census, MSE ladder)
cd research && python3 port_k3_lean.py

# billed submission check (MSE / C/B / residual gates, organizer env)
cd whest-starterkit && uv run python -u ../research/p2_submission_check.py 3