Repository navigation
Align composed paragraph proofs and recover mixed-prefix assessment runs without reconversion #91
Description
Goal
Repair the paragraph-proof mismatch that stops the full-corpus source-bound assessment producer, and safely resume its mixed imported/new prefix without repeating a completed conversion. Completion prerequisite for #53 and #22; epic #12.
Observed failure and confirmed cause
The real 10,000-document producer stopped with exit 1 after 2,333 contiguous sealed rows. Its next document has a sealed successful conversion (model_called:true) and bound document bytes, but no assessment or row receipt. Error: abstract_paragraph_group_not_unique_or_current from assessment_producer._attempt → verify_source_bound_assessment → _verify_source_owned_paragraph_groups.
trace_gc/pdf_structure_parallel_v4.py::_collect aligns converter regions using converter-supplied region_boxes. _compose_section_spans can compose these locally unique proofs into source_owned_monotone_paragraph_groups. The retrieval verifier replays each paragraph against the full native page without those boxes and demands a globally unique, current match with identical spans/alignment. Local uniqueness can therefore pass construction and fail the required global proof. Do not weaken that proof.
Failure stderr SHA-256: 34fb330c79139b4c807133c1f0969f3a3fd204ae1c6695a91455dceda2dc5692. Read-only census SHA-256: e5d4dd6ea2b5e7530f7d1f7888339c4079386bb5505c81882165bb795946ad6e. Existing frozen protocol SHA-256: b71d53e812db2835c9abfd970e0b660a35dae38c636dc20a957bafb3bf986132.
Required implementation
- Before emitting source-owned composed paragraph groups, replay the same global proof as the verifier. Require located status, no column change, exact original span positions and matching proof. Globally ambiguous paragraphs remain withheld/uncertain; no invented native spans or false proposal.
- Preserve all existing rows, historical failure receipts, source identities and pending conversion/intent/document bytes. Never rerun the completed pending converter call.
- Implement an explicit versioned migration for a mixed prefix with nested V2 imports plus ordinary new rows and one pending sealed conversion. Existing V2 refuses a nonempty next attempt; V1 requires attempts for every earlier row, which imported rows do not have. Neither is a valid direct fallback.
- Reassess cached converter output under reviewed repaired code in a new output/protocol, binding full historical lineage and current runtime/method/source gate identities. Do not hotpatch a frozen protocol or change prior sealed rows. Unknown outcomes, partial intent or changed artifacts HOLD without retry.
- Preserve run identity, original cutoff, ownership and receipts. Deadline extension is separate user authority. Prove predecessor process/tree/locks reconciled before one successor controller starts.
Required verification and acceptance
- Positive unique paragraphs remain source-bound and accepted under the unchanged verifier.
- Regression: converter boxes make a repeated paragraph locally unique but global replay is ambiguous or selects different spans; composer withholds it and a valid uncertain assessment passes verifier.
- Producer/migration fixture consumes an existing successful sealed conversion without any new converter/model call; full row is sealed under repaired protocol. Test restart/idempotence, nested imports, changed prefix/document/native/runtime, unknown pending outcomes and lock collision.
- Replay the actual failed cached conversion without a model call; verify repaired assessment honestly reports its status.
- Fresh affected configured 200-paper source/image gate preserves all approved original 91 IDs and reports precision/recall/withheld denominators; old receipt cannot certify changed method/image closure.
- Independent code review, package/release checks, verified signing, exact-head CI, reviewed merge and post-merge verification. Publish only permitted aggregate evidence.
- Demonstrate fresh useful progress of the single recovered full-corpus producer within authorized time. Keep Produce source-bound V4 assessments for the 10,000-paper index #53/Rerun the full 10,000-paper retrieval trial with bound fields and candidate-stage metrics #22/Reliable paper extraction, retrieval and real GraphRAG qualification #12 open until their full empirical criteria are met.
Current state
PR #92 is merged at 6b62693c338d222b0c415ec075a14afc543a1a53, with verified signing, exact-head CI and all three post-merge Python 3.11/3.12/3.13 CI jobs passing. Twenty-one targeted post-merge tests pass. The delivered tree is identical to the measured tree; all 85 method files and 129 image-related files rehash exactly.
The fresh 200-paper run and 24 fresh local OCR/source-review routes now have an integrated source-verified acceptance receipt and successful full replay verification: 97/111 complete abstracts verified, 14 withheld; all original 91 success IDs preserved; 7 partial and 82 absent cases retained. Every arm contains 200 audited cases. No scientifically false or unverified selected proposals; no paid calls or production graph writes. Review is attributed assistant review, with no independent-human qualification claim.
Source-verified receipt SHA-256: 984c97957067655710121760bb20128e204f1fa85a7b58ea87304c57dd7e4eae. Policy SHA-256: 010a19d1e8b28b06d1babe62464b441a2a7ac1bf47845fb82345b2c519760065. Legacy base receipt SHA-256: d514233c30eaf4f94c7be8a22f4ccda9fd52090e74f20385fd8cc7fd92a76f5f; it remains FAIL solely for the previously approved f081 Δ/∆ reference-encoding mismatch. Frozen reference bytes and historical scoring remain unchanged.
The original full-corpus producer window expired before recovery could start. No successor was launched or completed conversion retried. The reviewed V3 plan preserves the 2,333-row prefix and sealed pending conversion. Issue #91 remains open for actual useful recovery progress; #53/#22 remain open for their full-corpus and retrieval criteria. #20/#23 remain open for unseen qualification and actual graph-answer acceptance.
Activity
CompleteDotTech commentedon Sep 30, 2026
Reviewed repair and actual cached-document replay
The paragraph repair V2 passed independent code review and 42 focused/abstract-structure tests. Composition now requires source geometry and global unique matching. Where globally unique spans match exactly, global proof metadata is recorded without discarding a valid positive merely because local metadata differs. The retrieval verifier retains its strict proof.
Independent review SHA-256:
187b921812b8dccdc569563c1d2a851b2382e162d7d2b984944feab3b2ca4052.The actual failed cached document was reassessed and passed the strict verifier as uncertain, with zero new converter/model calls and unchanged original protocol/intent/conversion/document bytes. Actual replay receipt SHA-256:
e4273cb212a4c91e44026f72372451b81b27e2a2630eec0e3b9259041aecce49. This is one-document causal verification, not the full 200-paper gate or a resumed full corpus.The mixed-prefix V3 migration is still being implemented and reviewed. It must preserve all 2,333 sealed rows and consume the completed pending conversion without reconversion. Fresh final-code source/image acceptance, release/package checks and reviewed delivery remain outstanding. The producer stays stopped; the original deadline is unchanged and the separately proposed extension is not approved.
CompleteDotTech commentedon Sep 30, 2026
Signed implementation draft and measured repository checks
Draft PR #92 carries the global paragraph-proof repair, the native short-wrap regression repair, and V3 mixed-prefix/pending-conversion migration. Exact signed head:
a33675a3361e724a0de5ee178e6aa07d3570dfe3; GitHub reports the signature verified.Full offline validation passed: 1,345 tests run, 1,337 passed, eight optional skips, zero failures/errors, zero provider calls and production graph writes. The public source ZIP audit passed on 590 members, and installation from that ZIP in an isolated directory passed the
TRACE-GC 0.4.0CLI smoke.The first original-prefix audit completed successfully for all 2,333 sealed rows and the pending sealed conversion, with no model call. Its immutable plan SHA is
3267623a1ca6b424d388bb1d561e86c00d131fa772dbe4acedd924775fbdd11e. That plan binds the earlier repair closure; a separate audit is running for the final short-wrap code. The fresh four-arm 200-paper run and hosted Python 3.11/3.12/3.13 checks remain active.The PR remains draft. Integrated source/image acceptance and exact final recovery release are pending. No producer successor has launched; the original cutoff remains in force. #91, #53 and the full retrieval trial remain open.
CompleteDotTech commentedon Sep 30, 2026
Integrated repair and recovery preparation
PR #92 now contains the paragraph-proof repair, short-wrap native-chain repair, measured search optimization, V3 pending-conversion migration and delivered PR #93 phase infrastructure. Exact signed head:
8f989a0d1d59d4fb0368ebe84d50e7b703aef416.b46cd52eaf6c96b2798b66ba16df249fba3c2e968ceb23a31571cafadd6158aa.5fb864746bd008ce5cb445981bedb30dbd8833b88728b1da6073d681c13b3b2d.The stopped run, earlier failed validation, historical measurements and original authorized cutoff remain preserved. No recovery producer was launched by this plan. #91, #53 and #22 remain open pending their actual acceptance criteria; this update does not establish completion of the full retrieval trial.
CompleteDotTech commentedon Sep 30, 2026
Delivered repair and fresh integrated source gate
PR #92 is merged at
6b62693c338d222b0c415ec075a14afc543a1a53, with verified signing, exact-head CI and all three post-merge Python 3.11/3.12/3.13 CI jobs passing. Twenty-one targeted post-merge tests pass. The delivered tree is identical to the measured tree; all 85 method files and 129 image-related files rehash exactly.The fresh 200-paper run and 24 fresh local OCR/source-review routes now have an integrated source-verified acceptance receipt and successful full replay verification: 97/111 complete abstracts verified, 14 withheld; all original 91 success IDs preserved; 7 partial and 82 absent cases retained. Every arm contains 200 audited cases. No scientifically false or unverified selected proposals; no paid calls or production graph writes. Review is attributed assistant review, with no independent-human qualification claim.
Source-verified receipt SHA-256:
984c97957067655710121760bb20128e204f1fa85a7b58ea87304c57dd7e4eae. Policy SHA-256:010a19d1e8b28b06d1babe62464b441a2a7ac1bf47845fb82345b2c519760065. Legacy base receipt SHA-256:d514233c30eaf4f94c7be8a22f4ccda9fd52090e74f20385fd8cc7fd92a76f5f; it remains FAIL solely for the previously approved f081 Δ/∆ reference-encoding mismatch. Frozen reference bytes and historical scoring remain unchanged.The original full-corpus producer window expired before recovery could start. No successor was launched or completed conversion retried. The reviewed V3 plan preserves the 2,333-row prefix and sealed pending conversion. Issue #91 remains open for actual useful recovery progress; #53/#22 remain open for their full-corpus and retrieval criteria. #20/#23 remain open for unseen qualification and actual graph-answer acceptance.
CompleteDotTech commentedon Oct 1, 2026
Fresh delivered 200-paper run completed; native preservation failed
The real fresh-native four cached-arm run completed under frozen delivered main
24e1ee062103163e08d2c02669aee3bf4258fecf, in 3,477.686 seconds. Exit 1 is retained as FAILED_REGRESSION_PRESERVATION_GATE. This is not an acceptance pass or a new converter/model run.Results SHA-256:
8c82a79fc44465bf08a26ea80873304f5aaf9108fe6a4bdb2cc1cdd9e6cb875c. Separate actual readback SHA-256:ef47af4111b571c5a11855dfe39d899c0b47ef6042159debdcae886c761d4bf5; it verified all 800 assessment hashes, all 200 native records, four complete 200-case arms, all 87 method files and the current runtime lineage. Saved-native replay passed. New paid API calls and production graph writes were zero.The frozen denominator remains 111 complete, seven partial, 82 absent.
The preservation failure names f079, f081, f122 and f153. Their native text98 and legacy boundary checks pass, but native proposals remain withheld: leading source-prefix evidence is unresolved for f079/f122, and multi-column continuation evidence is unresolved for f081/f153. f195 remains a complete selected native case.
Legacy false-proposal counts do not certify scientific notation or source fidelity. This raw run lacks the integrated attributed fidelity/image overlay. The 24 new local OCR cases and source/crop visual reviews are complete, but their typed current native-failure and all-200 promotion bindings are being recomputed. The approved source-verified policy will be assessed separately against that new immutable binding, preserving this failed raw receipt and unchanged legacy scores.
No unseen qualification, full retrieval trial or cited-answer acceptance is claimed. #91 remains open for its delivered integrated empirical gate.
CompleteDotTech commentedon Oct 1, 2026
Current delivered 200-case source acceptance
The fresh native four-arm run completed all 200 cases per arm with zero paid calls and graph writes. Its saved replay passed, but the native preservation gate failed: four previously correct cases were withheld. The native result remains
FAILED_REGRESSION_PRESERVATION_GATE.The separately reviewed current image route selected 24 cases; 73 native cases and those 24 image cases form 97 source-verified proposals out of 111 complete abstracts. Fourteen complete abstracts remain withheld. All original 91 correct cases are included. The base receipt remains
FAILon itszero_false_primary_proposalsgate for f081; the source-verified receipt isPASS, with 97 reviewed-fidelity proposals and no measured scientific false proposals. The actual publicsource_verifycommand exited 0 against the exact source receipt.This is a previously examined development cohort and assistant-attributed source review. It does not establish independent human qualification, graph admission, a 10,000-paper trial, or a new model result.
Current receipt identities
Measured delivered source:
24e1ee062103163e08d2c02669aee3bf4258fecf.Source-verified acceptance SHA-256:
3f77057a0d845b71320003cfc64553cc429b762abfa75c2ec86becf19c507253.Policy SHA-256:
83d59e9738da9857c058fe9513eebd99c2ed421da1457afef03fb87c41e9d072.Unchanged failing legacy base SHA-256:
ad532faa89cf47c811afce0fca7b0cc493fc9050de574c0405021dab257c0157.Fresh native regression SHA-256:
8c82a79fc44465bf08a26ea80873304f5aaf9108fe6a4bdb2cc1cdd9e6cb875c.Scoped assistant peer readback SHA-256:
f7319787375bb90a73c7dff736cc7550fd764acaf3bd245e499a27d5183a6f09.The failing legacy comparison retains the previously approved f081 Δ/∆ reference-encoding exception. Reference bytes and historical scoring remain unchanged.
Issue #91 remains open: the original full-corpus producer window expired, and no recovered producer was started. Issues #20, #22 and #23 retain their separate actual qualification requirements. This update does not extend the execution window or authorize retries of historical unknown outcomes.
CompleteDotTech commentedon Oct 1, 2026
The genuine historical 200-paper source-verified verifier has now replayed successfully under its exact owned runtime and pinned historical source: exit 0 / PASS, with empty stderr. This rechecks the unchanged receipt
3f77057a0d845b71320003cfc64553cc429b762abfa75c2ec86becf19c507253; it is not a new extraction, OCR or inference run. All 3,408 historical input pins were checked again after deriving the mapping.Root and an independent assistant reviewed the mapping to delivered main
7f01f3389e3360ba89ac780f585c9f83c9ec2aae, treed98347f701576931a71b0c14a772662fc80bd181: all 87 measured method files are byte-identical; 128 of 129 image-related files are byte-identical. The sole difference is the reviewed closed SQLite restore reader intrace_gc/phase_transition.py, delivered by #102. The selector and image extraction/review algorithms remain unchanged. This establishes an auditable code-identity mapping for the historical evidence. It does not issue a new literal current-main V2 acceptance receipt or phase authorization.The measured result remains 97/111 complete first-page abstracts source-verified, preserving all original 91 success IDs; 14 complete cases are withheld, seven partial and 82 absent. The 24 reviewed image chains and all 200 cases in each cached arm remain accounted for. The legacy Δ/∆ reference-encoding score remains FAIL under unchanged reference/scoring bytes. Review is attributed assistant review, with no independent-human or unseen qualification claim.
Evidence SHA-256: captured old-verifier stdout
d9f8c078fe63e204dd1be03913cc8f4893ef05335634f2801bc6f3ad53b4f32e; mapping candidatec95bfb6cc626bfa1793ca259f7e0d1ee87ed2bf9e6e0cb4ffece9d1fb7547584; independent terminal/code readback2f0371e9083f98835796a099185ca5aebd279483ddc2a491f5aff08cef0b026a; root-reviewed evidence mapping10bce3617a6eac454762f003b2017d62e6f6267dd2f034968a1109d61f7b902b.#91 remains open for useful recovery of the full-corpus producer. Its preserved prefix contains 2,333 rows, and its original authorized cutoff expired; no successor, extension or converter retry is inferred. #20, #22, #23 and epic #12 remain open. This read-only replay made no OCR/model call or graph write.
CompleteDotTech commentedon Oct 3, 2026
PR #107 merged as
04acabedcfff1cef9871087fd6a6374375ab5ebfafter all six exact-head checks succeeded. Its actual merge workflow passed all three interpreter jobs. GitHub verifies the merge signature (valid); contributor heade70fe037e7b3d76f7ba36c4d6ed7a46eb80d5f7dis locally SSH-signed but still reportsunknown_key. No signing registry/settings changes were made. Local main is clean and zero ahead/behind.The public field-worker environment and test-discovery defects are fixed, with a real temporary-venv regression and refreshed 702-entry manifest. Full pytest: 1379 passed, 26 explicit skips, 780 subtests. Normal validation: 1378 passed, 26 skipped, zero failures/errors. Package and 703-file source ZIP audit passed; no private PDFs, weights or environment secrets included. Independent source/test/manifest review passed; earlier failures and generated validation evidence were retained.
This completes the scoped offline contribution. It does not complete genuine Windows Docling smoke/resource gates, the full corpus ledger/V4 retrieval map, private V6 verifier closure, domain artifact reconciliation, unseen human source review or live pilot qualification. Those issues remain open. No active/expired workload was changed or restarted, provider dispatched or production graph written.