Repository navigation
Support pinned Windows venv converter workers without weakening controller ownership #94
Description
Outcome
Repair genuine converter execution through a pinned Windows virtual environment, retain strict controller ownership checks, and make preflight failures diagnosable. Prerequisite for #20; contributes to epic #12.
Observed failure and causal evidence
At delivered commit 6b62693c338d222b0c415ec075a14afc543a1a53, all four exposed development plans passed binding validation across 52,643 unique pinned files. A genuine local CPU Docling smoke then terminated without worker-result.json. The controller retained its intent, stdout/stderr and invalidation receipt; the worker emitted only WORKER_FAILED / ValueError. No quality result was produced.
src/paper_converter_worker.py requires intent.controller_pid == os.getppid() before runtime validation and engine dispatch. The pinned Windows Python 3.12 venv executable is a redirector: it creates the interpreter as a child. A separate no-model probe reproduced controller → pinned venv redirector → interpreter. The interpreter's immediate parent is the redirector, so the direct-parent invariant necessarily fails for this launch path. This reproduces a pre-dispatch blocker; the opaque historical error log alone does not establish which exception was observed.
Private aggregate receipt identities:
- Parent-chain probe SHA-256:
be24fc8b8760489a1a9df3663d5300c0698f6e6d8ef1b8073a58a7b990e3c861. - Failed-attempt invalidation SHA-256:
f71e2cb5a1854a11518350ec1e33125bcbe95bbf1defe24ff672d43e0e5a6493. - Historical attempt remains invalidated and must not be overwritten or automatically retried.
Proposed repair
- Keep the direct-parent path. On Windows only, allow exactly one additional hop whose process image is the exact pinned venv executable and whose parent is the recorded controller.
- Verify executable hash, creation ordering and process liveness while holding process handles. Fail closed on inaccessible processes, mismatched ancestry, additional hops, replaced executable or inconsistent lifetimes.
- Check process liveness using a zero-timeout wait on a process handle opened with
SYNCHRONIZE, rather than treating exit code 259 as proof that a process remains alive. Microsoft documents the exit-code ambiguity and wait semantics. - Report a bounded contract error code alongside the exception type. Keep private paths and arbitrary exception contents out of logs.
- Reconcile the preserved failed attempt explicitly before a distinct, newly pinned successor smoke. Retain unknown historical claims where the artifacts do not prove more; do not clear the registry to bypass reconciliation.
Acceptance
- A genuine Windows venv subprocess passes ownership verification; the historical direct-parent check fails on the same authored launch shape.
- Wrong executable/hash, wrong ancestor, extra hop, dead process, reordered creation and unavailable process metadata are rejected.
- A process that terminates with exit code 259 is rejected as dead.
- Existing authored converter, immutable input, timeout and unknown-attempt protections still pass.
- Failure logging exposes a bounded contract code and excludes arbitrary private path text.
- Independent code review, signed delivery, exact-head CI, merge and post-merge checks complete.
- Original failed attempt remains immutable; any explicit reconciliation and successor plan have reviewable lineage.
- A fresh genuine local Docling smoke on the already-exposed development page completes under the delivered repair with verified artifacts. This is readiness evidence, not unseen qualification or closure of Qualify the frozen extraction configuration on an unseen internally reviewed paper cohort #20.
Scope and constraints
Keep source hashes, runtime/package pins, preparation/intent checks, offline enforcement, resource limits, zero automatic retries and application review boundaries. This issue grants no paid inference, offhost paper transfer or unseen-cohort prediction authority. Do not weaken ownership verification to an environment variable or a plausible PID alone.
Activity
CompleteDotTech commentedon Sep 30, 2026
Scoped implementation delivered in PR #96, merged at
7ec9a76759eb3e160555ef285dde4247d533fc45from signed, host-verified headb1cdcca9e2203e5f9b373c6cb31cb6258bd2c0d1. All six push/PR checks passed. Post-merge Windows checks: 24 tests, no failures/errors, one explicit optional skip; delivered source matches the reviewed head. Main-branch hosted checks are running.This issue remains OPEN. The original failed converter attempt is immutable and UNKNOWN; a supported signed manual reconciliation path and a distinct freshly pinned genuine Docling smoke remain outstanding. The changed extraction method closure requires a new empirical binding before qualification. The separate PyMuPDF 1.28.2 field CLI repair is tracked in #95 and is now undergoing full combined validation against this merge.
CompleteDotTech commentedon Oct 1, 2026
Reconciliation integration update
The reviewed four-file reconciliation implementation is integrated in an isolated checkout on delivered main. Its controller code identity includes the new reconciliation module in the transitive dependency map (86 files).
Supported-path checks passed with exact core pins (Python 3.12.10, cryptography 46.0.4, jsonschema 4.26.0, PyMuPDF 1.28.2, Pillow 12.2.0): seven scientific glyph-geometry tests and seven retrieval corpus-policy tests. Separately, all 45 converter tests completed with 44 passes and one optional symlink skip under Python 3.12 / PyMuPDF 1.28.2. These are authored checks, not a genuine converter result.
A default-environment full run is preserved as failed: 1,374 tests, 1,365 passed, six skips, two failures and two errors. It used different core/PDF versions; all 14 affected-path tests passed in the exact-pinned environment. Its failure is retained, and it is not a release PASS.
Operational recovery found a further required change: the invalidated historical attempt has no durable worker-PID receipt. A current process census cannot reconstruct that PID. The next version must bind the actual recorded launcher termination/control flow and reviewed installed launcher identity, then observe current matching processes under held application launch exclusion. It must retain the historical UNKNOWN outcome and prohibit retry of the old intent.
Public reconciliation delivery is held until that actual-evidence path and application authority are reviewed. No registry was cleared, no old intent retried, and no successor converter or unseen cohort was executed. #94 and #20 remain open.
CompleteDotTech commentedon Oct 1, 2026
Reviewed reconciliation boundary delivered
PR #100 merged as
24e1ee062103163e08d2c02669aee3bf4258fecf. The reviewed head8df3b45e879811a30a49e75cfbd54d4373d57c70and hosted merge have verified signatures and identical source trees. All six exact-head hosted checks passed across Python 3.11, 3.12 and 3.13; default-branch run 36802011453 is currently running.Final Windows validation: 1,388 run, 1,380 passed, eight explicit skips, zero failures/errors. Skips are two optional NumPy tests, five Windows symlink privilege tests, and one unavailable external pinned-core integration. Post-merge converter checks: 43 run, 42 passed, one Windows symlink skip, plus legacy package validation PASS. The source package audit and fresh installed-package CLI/import checks also passed.
The delivered API verifies reviewed old-attempt reconciliation and exact distinct-successor/application bindings; it retains historical unknown outcome and disables standalone caller-supplied effect approval. The original failed attempt remains immutable and
UNKNOWN_UNATTESTED_NO_RETRY. The scoped code/package reviews are attributed assistant reviews, not independent human empirical qualification.#94 remains open. The new exposed f097 plan is being directly validated against the delivered public source before root-owned signing, enrollment, live exclusion and any genuine Docling execution. No successor converter has run. #20/#22/#23 remain empirically incomplete.
The method closure now contains 87 pinned files. A fresh full 200-paper cached-arm regression has started against the delivered merge in a new immutable output, with the original pinned replay runtime and fresh native extraction. Previous legacy and source-verified results remain preserved; they are not substituted for this current run.
CompleteDotTech commentedon Oct 1, 2026
Actual historical reconciliation completed; genuine successor still pending
The attributed root source review, separate assistant code review, and one-time signing-only execution completed. The public reconciliation verifier accepted the actual signed receipt. No converter worker was launched by this action.
cb53e9d37f6880f79be29f73d2e12098ea86f2f4a503ab086af36dc15ee24ad4.ba9e867c9cc2b641855ed52516fcb5557d33b7dd82a5280841ed26cfa77e13c2.4160ae6d8b4f0b7588b58cbef6c775e525e2ca81bb086b7934d6b228a7f7b9cc.cddac24ea738e017a26ecac1ff438f873b050b91ed5d4942f44d6356b0e5eefe.All eleven pinned historical materials remained unchanged. Actual key correspondence and typed Windows ACL checks passed. The historical provider outcome remains UNKNOWN_UNATTESTED_NO_RETRY; missing historical interpreter identity has not been invented. Current process observations and historical launcher-source correlation support only the explicitly reviewed cooperative closure.
The distinct successor output remains absent. Its concrete application enrollment and execution/readiness launcher are being reviewed separately. The current host has approximately 10.1 GiB free RAM, below the unchanged plan requirement of 20 GiB; execution remains held. No shared workload was stopped, resource threshold relaxed, registry cleared, old retry authorized, paid inference used, or private paper transferred.
This is actual reconciliation progress, not successful Docling readiness or unseen qualification. Keep #94, #20, and #12 open.
CompleteDotTech commentedon Oct 3, 2026
Offline verification reproduced a separate public corpus field-worker defect: resolving the selected virtual-environment interpreter dispatched the base interpreter and lost PyMuPDF. PR #107 preserves the invoked interpreter while independently checking resolved identity/bytes, adds an actual temporary-venv regression, fixes current pytest discovery and refreshes the source manifest.
Full pytest1379 passed26 explicit skips780subtests; normal validation1378 passed26 skipped0failure/error,0provider calls/production graph writes. Independent source/test/manifest review and package/archive audit passed. Commit e70fe03 is locally SSH-signed; GitHub reports unknown_key, not verified registration. Exact-head hosted checks are pending.
This partial software fix does not supply genuine Windows Docling/resource qualification, a complete10000-ID corpus ledger or eligible-only V4 map, private verifier closure, unseen source review or live pilot acceptance. Historical expired/unknown attempts remain retained; no corpus/converter worker was restarted or resource threshold lowered.
CompleteDotTech commentedon Oct 3, 2026
PR #107 merged as
04acabedcfff1cef9871087fd6a6374375ab5ebfafter all six exact-head checks succeeded. Its actual merge workflow passed all three interpreter jobs. GitHub verifies the merge signature (valid); contributor heade70fe037e7b3d76f7ba36c4d6ed7a46eb80d5f7dis locally SSH-signed but still reportsunknown_key. No signing registry/settings changes were made. Local main is clean and zero ahead/behind.The public field-worker environment and test-discovery defects are fixed, with a real temporary-venv regression and refreshed 702-entry manifest. Full pytest: 1379 passed, 26 explicit skips, 780 subtests. Normal validation: 1378 passed, 26 skipped, zero failures/errors. Package and 703-file source ZIP audit passed; no private PDFs, weights or environment secrets included. Independent source/test/manifest review passed; earlier failures and generated validation evidence were retained.
This completes the scoped offline contribution. It does not complete genuine Windows Docling smoke/resource gates, the full corpus ledger/V4 retrieval map, private V6 verifier closure, domain artifact reconciliation, unseen human source review or live pilot qualification. Those issues remain open. No active/expired workload was changed or restarted, provider dispatched or production graph written.