shadow-management journal written by TypeScript is rejected by the Python reader on symlinked runtime roots (macOS default TMPDIR): every s2c2 ladder row fails with shadow_management_state_invalid #4892

Summary

On macOS every s2c2.* row of loopx/control_plane/testing/authority_e2e_ladder.py fails with loopx todo add exited 1: error_code='shadow_management_state_invalid' whenever the runtime root lives under a path that contains a symlink, which is the default: both /tmp and $TMPDIR (/var/folders/…/T) resolve into /private/…. The TypeScript side writes the shadow-management journal with a source_root_digest of the real path, while the Python reader recomputes the digest from the lexical path and rejects the journal as "scope differs". Linux CI is unaffected because its temp directory has no symlink component, so the defect is invisible to the required checks.

Tested revision: c4314d9b0 (also reproduced on 4bed6ed3d), macOS on Apple silicon, Python 3.12, Node 22.22.3 (the Stage 2C CI runtime) and Node 26.4.0 behave identically.

Where the two sides diverge

• loopx/control_plane/coordination/shadow_management.ts:56-66: canonicalSourceRoot is realpathSync(resolve(root)); shadowSourceRootDigest (used when the journal is written, :374-375) hashes that canonical path; the TypeScript reader tolerates both spellings through sourceRootDigests (:60-68), which is a set of the lexical and the canonical digest.
• loopx/control_plane/coordination/shadow_management.py:82: read_shadow_management_state hashes os.path.abspath(str(runtime_root)) only, then raises ValueError("journal scope differs"), which require_shadow_primary_write_allowed surfaces as shadow_management_state_invalid.

So a journal the TypeScript bootstrap just wrote is unreadable to the Python primary-write path on any symlinked root. The Python reader is stricter than the TypeScript reader that owns the same journal.

Reproduction

From a clean checkout on macOS, with node on PATH:

# 1. Default temp directory (a symlink into /private): fails
python -m loopx.control_plane.testing.authority_e2e_ladder --row s2c2.outbox_prepared_then_committed_entries
# -> s2c2.outbox_prepared_then_committed_entries fail: loopx todo add exited 1: error_code='shadow_management_state_invalid'

# 2. A temp directory without any symlink component: passes
mkdir -p "$PWD/tmp-real" && TMPDIR="$PWD/tmp-real" python -m loopx.control_plane.testing.authority_e2e_ladder --row s2c2.outbox_prepared_then_committed_entries
# -> pass

# 3. Any symlink component reproduces it, not just /var or /tmp: fails
mkdir -p "$PWD/tmp-target" && ln -s "$PWD/tmp-target" "$PWD/tmp-link"
TMPDIR="$PWD/tmp-link" python -m loopx.control_plane.testing.authority_e2e_ladder --row s2c2.outbox_prepared_then_committed_entries
# -> fail, same error code

# 4. Control: rows that never read the shadow-management journal pass under the default TMPDIR
python -m loopx.control_plane.testing.authority_e2e_ladder --row s1.cli_document_decodes_through_ts_store --row s2c1.configure_enable_disable_roundtrip --row s2c1.every_writer_family_captures
# -> 3 pass

The digests themselves, for one directory created under the default temp directory:

R=$(mktemp -d)
python3 -c 'import hashlib,os,sys;print("py abspath ", hashlib.sha256(os.path.abspath(sys.argv[1]).encode()).hexdigest())' "$R"
python3 -c 'import hashlib,os,sys;print("py realpath", hashlib.sha256(os.path.realpath(sys.argv[1]).encode()).hexdigest())' "$R"
node -e 'const {createHash}=require("crypto");const {realpathSync}=require("fs");const p=require("path");console.log("ts canonical", createHash("sha256").update(realpathSync(p.resolve(process.argv[1])),"utf8").digest("hex"))' "$R"
# py abspath  differs from ts canonical; py realpath equals ts canonical

Second, independent run of this matrix on c4314d9b0 (each row once, fresh runtime roots): 1 fail, 2 pass, 3 fail, 4 pass 3/3; the full 23-row ladder under a symlink-free TMPDIR on the same head passes 22 rows with only the PostgreSQL row unverified and the soak row pending.

Expected

The Python reader should accept the same journal the TypeScript writer produces on every platform. The smallest change that restores parity is to let read_shadow_management_state accept the same digest set the TypeScript reader accepts (digest of os.path.abspath(root) or of os.path.realpath(root)), mirroring sourceRootDigests; a regression test with a runtime root reached through a symlink would keep the two readers aligned. Whether the lexical spelling should keep being accepted at all is the maintainers' call; the mismatch, not the choice, is the defect.

Impact

• Any macOS contributor running the Stage 2C2 rows locally gets eleven false failures unless they know to point TMPDIR at a symlink-free directory; the ladder README does not mention it.
• The primary-write guard (require_shadow_primary_write_allowed) fails closed, so this does not corrupt state; it blocks writes on a goal whose shadow capture was just enabled.
• Discovered while re-measuring the NoKV Stage 2A qualification for docs(rfc): record NoKV v0 envelope capacity and the 0.11.1 fence live qualification in the ledger #4889; recorded there as a LoopX condition, not a NoKV one.