Repository navigation
Bind external pilot qualification verifiers to their full executable dependency closure #97
Description
Outcome and dependency
Bind all external qualification verifiers and their executable dependencies to the reviewed release before accepting the real #23 pilot. Child/prerequisite of #23; epic #12. The affected evaluator remains HOLD.
Reproduction and cause
The frozen fresh pilot evaluator V6 captures and pins the top-level issue20, issue22 and recovery verifier modules. Their ordinary imports can still resolve mutable helper files. A separate reviewer reproduced an authored verifier whose helper changed from HOLD to PASS while the top-level verifier SHA remained unchanged. Thus a valid top-level pin does not bind the predicate actually executed.
Independent reproduction/HOLD receipt SHA-256: f0ac50dcc0d3291be0475605df4c10e5b3af3017b0e096400ff83419dbe2d606. This is authored mechanical evidence, not a real gate result. The V6 artifact and failure are retained; no live qualification is authorized.
Proposed solution
Version the evaluator and its release contract. Require a complete externally reviewed source/import closure for each external verifier, including helper modules and declared dynamic imports. Bind the manifest, module locations, exact bytes and supported runtime to the signed release. Verify the complete closure before executing verifier code; execute only the verified representation and recheck affected inputs before accepting its result. Reject ambiguous module resolution, unpinned helpers, symlink/path substitutions, changed bytes, and unsupported dynamic imports. Account explicitly for cached modules and bytecode so an unchanged source SHA cannot conceal another executable representation. Reuse existing source-closure and canonical-binding contracts where applicable.
Rebind the new evaluator to the exact fresh controller and stage releases. A mechanical test success does not authorize a stage or satisfy the real issue20/issue22/recovery criteria.
Acceptance
- Reproduce HOLD-to-PASS by changing only an imported helper under the old evaluator.
- The repaired evaluator rejects that change while the top-level verifier remains byte-identical.
- An unchanged complete authored verifier closure passes; genuine verifier failures remain HOLD.
- Negative tests cover helper replacement, missing dependency, import-path ambiguity, unapproved dynamic dependency and stale executable cache behavior relevant to the chosen loader.
- All ordinary and approved dynamic executable dependencies are represented in the release contract, verified before execution, and bound to the accepted readback.
- Review the exact final evaluator/controller/release binding with a separate attributed reviewer; retain V6 and its invalidation history.
- Complete required source/package validation, signed delivery, exact-head hosted checks and post-merge readback for any public implementation.
- Run the real prerequisite verifiers through the new evaluator when their receipts exist. Missing gates remain NOT_RUN/HOLD; no substituted positive fixtures.
Constraints
No private PDFs, labels, model assets, personal paths or private signing keys belong in the public issue/PR. Preserve the original numeric criteria and resource limits. Do not relax checks or count a verifier mock as qualification.
Activity
CompleteDotTech commentedon Oct 1, 2026
Research supporting the repair
Python 3.12 checks
sys.modulesbefore searching for source files, so an already loaded helper can satisfy an import without rereading its source. Its documentation also describes cached bytecode and import-path resolution. These are separate executable representations that a top-level source hash does not bind. See the official module-cache reference and bytecode invalidation reference.importlib.invalidate_caches()addresses finder caches; it is not a complete verified execution boundary. See official importlib documentation.Engineering inference for this issue: capture and validate the complete executable closure before loading it, control import resolution and executable caches, and reject unsupported dynamic dependencies. Preserve a negative test in which only a helper changes while the top-level verifier remains unchanged. V6 remains HOLD pending the versioned repair and separate review.
CompleteDotTech commentedon Oct 1, 2026
Guarded full-field rebuild repair: scoped review passed; empirical qualification remains open
The reviewed V3 successor resolves two concrete execution gaps: input JSON is now hashed and parsed from the same captured bytes, and inherited
PYTHONPATHcannot replace the intended field-builder imports. The owned Windows child receives-E -B -sand an initially empty private bytecode-cache prefix. A source-owner lock, child creation guard, bounded Windows Job and input/output rechecks bind the actual build and public rebuild-verification commands to the reviewed release.An independent assistant code reviewer verified all nine frozen file pins and ran six focused tests successfully, including a harmless real Windows Job, an inherited-import-path negative and a dirty-cache negative. This is scoped implementation evidence. The tests did not run corpus PDFs, models, a field rebuild or the retrieval experiment.
Frozen handoff SHA-256:
5ecd92821a584c101470d60f146eb7f28b842080405ff067e6183ddd8ce2261a.Scoped peer receipt SHA-256:
26607f7ca35a581d4ca11aa18b31e154967291b259164431a315115d4ff12a66.The runner remains disabled. The complete reviewed 10,000-document abstract assessment map, an accepted current #22 source gate, the exact enabling release and a genuine full positive trial remain outstanding. The existing 10,000-document/60-query native-only baseline is preserved; it does not establish the source-bound abstract, dense-model or relevance qualification required by #22. No producer was restarted and no paid inference was used. This issue and #22 remain open.