Freeze the extraction runtime and native evidence; validate replay configuration before assessment #13

Problem and observed evidence

A nominal replay can change its inputs despite identical selector and converter receipts. The historical experiment recorded Python 3.12.10/PyMuPDF 1.28.2; the recent run recorded Python 3.14.3/PyMuPDF 1.27.2.2. All 11 recorded V4 source hashes and all 800 converter-stage receipts matched, but 123/200 native representations changed.

A same-process f154 intervention swapping only saved native lines exactly reproduced both assessments:

Installed PyMuPDF 1.28.0 reproduced the historical native hash for f154; the exact historical 1.28.2 environment was not rerun. This explains olmOCR's 40→39 change, not the primary 55-case regression, which occurs in both runs.

Code entry points

• Native extraction and preflight/run/replay ordering.
• method_hashes includes only V4 files; the dynamic frozen replay additionally imports baseline code.
• Baseline DATA initialization assumes a sibling data directory at this revision.
• Experiment requirements and v2 requirements already have some exact pins. Reconcile a supported replay environment rather than adding contradictory pins.

Required implementation

1. Define and document a reproducible, validated experiment runtime: Python patch version, PyMuPDF and underlying MuPDF, Pillow, relevant transitive dependencies and platform. Record lock/environment identity in receipts and verify it before selection.
2. Introduce explicitly distinct modes for exact saved-native-evidence replay and fresh extraction. Bind every case to native-evidence hashes and extractor identity; report mismatches as a different experiment or a blocked exact replay. Never silently replace frozen files.
3. Extend the code receipt to cover the actual baseline replay import closure, including configuration helpers and scoring/adapters. A changed helper must change the run identity.
4. Resolve the authorized data root once and validate agreement with baseline import-time configuration before run_cases. Fail early for missing required saved assessments/caches/source maps and incompatible runtime.
5. Integrate the already prepared local TRACE_GC_TEST_DATA_ROOT repair in src/abstract_validation/common.py and its reproduction-guide note, after inspecting/preserving existing work. Those two edits were uncommitted at diagnosis time; do not assume they are in the pinned commit. Keep the frozen baseline algorithm and prior hashes/history intact; record the new helper version separately.

Reproduction and verification

With the supported interpreter active, use a fresh output suffix for each invocation:

export TRACE_GC_TEST_DATA_ROOT=/authorized/TRACE-GC_RealPaper_Test
python -m src.parallel_source_v4.extraction preflight \
  --data-root "$TRACE_GC_TEST_DATA_ROOT" \
  --source-map validation_v3_private/parallel-v4-source-map.json \
  --output validation_v3_private/environment-preflight-UNIQUE
python -m unittest discover -s tests -p 'test_parallel_source*v4.py' -q

The environment variable is necessary for the local helper repair; the committed old helper ignores it. Until this issue lands, a preflight PASS alone does not establish that the later baseline import uses the requested root.

Acceptance criteria

• Reproducible installation and environment capture work from a clean isolated environment; no unsupported version substitution is described as the historical runtime.
• Root mismatch, missing baseline artifacts and runtime/native-hash mismatch fail before assessment, with a redacted actionable error.
• Exact replay of the chosen saved inputs is deterministic; fresh extraction has its own identity and an explicit native-change report.
• Changing a dynamically imported baseline helper invalidates the full code receipt.
• A controlled f154 saved-native swap reproduces both recorded hashes above, establishing the input mechanism; the integrated selector repair can subsequently change its own versioned outputs.
• Original source/page/image checks and immutable-output behavior still pass; authored root/mismatch tests exercise the failure paths.
• Documentation explains local-data access, mode semantics and the difference between execution configuration and extraction-quality fixes.

Private evidence: an authorized maintainer can resolve validation_v3_private/diagnosis-20260928/receipt-audit.json and ROOT_CAUSE_AND_SOLUTIONS.md beneath the external test-data root. Latest results.json SHA-256: 661964b8db10bf2bd806a95a0119f4492dcabf72e759fff521993f702f016b0d; historical: f9ee4f438c8cae7ce192970f7ff172ecbe7c24b3a383d48fe62cf75da7f4102b. Obtain authorized local access to PDFs, frozen labels, converter caches and the f001–f200 relative source map following the reproduction guide. Keep those assets and full assessments outside the public repository; publish only redacted aggregate results and hashes. Missing access is a blocked quality run, not a passing substitute.

Limits

This issue makes runs comparable. It does not claim to repair abstract boundaries, establish independent quality or authorize model charges. Keep any unavailable historical environment limitation explicit.

Tracking and completion dependencies

Parent: #12. Milestone: Paper extraction and GraphRAG qualification.

No mandatory issue prerequisites; implementation can begin using the documented inputs.

Required by: #15, #16, #18, #19, #21.