Skip to content

feat(issue): add bounded GitHub issue lifecycle (#197) - #201

Draft
danielbentes wants to merge 114 commits into
mainfrom
codex/issue-197-controller
Draft

danielbentes wants to merge 114 commits into
mainfrom
codex/issue-197-controller

Conversation

@danielbentes

@danielbentes danielbentes commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Related issue

Relates to #197.

Outcome

Adds a bounded flow issue controller for implementing one GitHub issue in a separate repository,
verifying the candidate, obtaining an independent review, publishing a draft pull request, waiting
for required hosted checks, requiring an exact merge approval, merging, and proving the post-merge
state.

The source-built controller completed this lifecycle for
digital-twin issue 6. The accepted run
published and merged digital-twin PR 105
after an unchanged private holdout, frozen local verification, a fresh read-only review with no
P1–P3 finding, and its required hosted Linux x64 check.

Controller and authority changes

  • Freeze repository, issue, base commit, plan, workflow, verifier, holdout, budget, and delivery
    identities before model execution.
  • Keep GitHub credentials, network access, Git metadata, branch publication, and merge authority
    outside the model boundary.
  • Restrict model writes to explicit project-relative prefixes and commands to the shared audited
    sandbox.
  • Persist owner-only lifecycle events, private verification evidence, effect preparations, and
    effect settlements for process-boundary recovery.
  • Bind candidate verification and independent review to the exact base, candidate, diff, criterion
    set, and command set.
  • Create a draft PR only after deterministic and private verification pass. Revalidate the candidate
    before every later external effect.
  • Accept merge authority only through a separate command bound to the PR number, exact head commit,
    and current merge-gate digest.
  • Observe required checks, merge commit, branch deletion, and issue closure before recording the
    terminal merged phase.

Model-runtime changes

  • Add OpenRouter as an additive provider while retaining OpenAI and Anthropic support.
  • Pin the embedded Pi packages to 0.84.4 and verify the offline catalog entry for
    z-ai/glm-5.3-flash.
  • Support the reviewed :nitro, :exacto, and :floor OpenRouter route variants only when their
    base model is known.
  • Add per-agent policyDecisionLimit. The compatibility default is 64 and the schema ceiling is
    128; neither value is represented as an industry standard or universal task budget.
  • Classify provider quota exhaustion separately from retryable rate limiting without persisting
    provider error text.
  • Reject empty terminal agent reports instead of treating them as successful completion.
  • Continue a supported provider failure after settled raw exec commands only when all recovery
    proofs pass. The closed model-session summary must reconcile latest-attempt flow_exec results, excluding only host-proven pre-execution
    refusals, with durable commands; process termination must be confirmed; sandbox cleanup must not have failed;
    every edit must be committed; no delegation may have run; accounting must be complete; and budget
    must remain. The next attempt consumes the recorded result and never replays the command.

External qualification evidence

The accepted run was issue-ef297140-9756-4dc5-92f0-4017a4bd5f07:

  • frozen base: 99fb83a14e589fe7137b6f6cf3eac97b8535be0f
  • accepted candidate: c54b8105ea8c79c37be1ff4280f3b9163ca4cda4
  • required GitHub Actions test: passed on the exact candidate
  • merge-gate digest: f52e4a0f298a5b87164f086267e242d06a649cba99b638f33a432fe9227f0064
  • observed merge commit: 374b16b229e187004e5915942f95c296af03298f
  • final issue state: closed; Flow-owned branch deletion requested and observed

The complete denominator contains 52 parent runs and 55 nested implementation or review runs.
Those ledgers report 247,516,376 tokens including cache tokens, $10.094325 in model cost, 9,429
turns, 10,590 tool calls, and 2,532 settled filesystem effects. All failures, controller corrections,
operator interventions, limitations, and immutable digests are recorded in
docs/field-reports/digital-twin-issue-6-alpha4.md.

Qualification boundary

The accepted external controller ran from built PR source on macOS. The target candidate passed its
required hosted Linux x64 check. The already-published 0.1.0-alpha.4 package predates flow issue
and is not qualified by this result. A new immutable prerelease still requires exact package
qualification and separate publication authorization.

One successful issue does not establish a cross-repository success rate, production service-level
objective, or provider comparison. The field report and roadmap preserve these limits.

UC-08 correction and roadmap reassessment

Historical UC-08 verification head: e967c29082a6647a1554fdc96312a93c6f94dd6d.

  • Make complete, canonical frozen verification invocations discoverable in flow_exec, including
    exact timeouts. Reject mismatches with actionable guidance without widening command authority.
  • Record host-proven refusals in the durable model-session ledger. Stop before another model or
    compaction request after three cumulative refusals; the already-issued tool batch settles first.
    Three is a provisional policy, not an industry standard or a measured optimum. Other work and
    recovery do not reset the count. Approved commands that exit nonzero are not refusals.
  • Preserve private holdouts, policy checks, sandbox boundaries, aggregate budgets, and explicit
    merge approval. Existing digest-only records remain readable. Active older issue runs must keep
    their original pinned binary; no cross-version active-run migration is claimed.
  • Reassess all 23 plugin commands and track onboarding, guided plan preparation, repair research,
    comparative measurement, and missing runnable practice demonstrations in the usable-checkpoint plan.
  • Independently review code, security, tests, dependency validation behavior, and documentation.
    All identified UC-08 P1–P3 findings are resolved. This incremental review does not replace the
    remaining exact-head review of the complete PR.

Local verification of the reviewed UC-08 working tree:

  • Complete serial unit/integration run: 459 files passed, one skipped; 6,361 tests passed, four
    skipped, zero failures. Coverage was not recomputed locally for UC-08.
  • Complete native runtime run: 44 tests passed, 42 expected platform skips, zero failures.
  • Typecheck, build, formatting, lint, generated references, documentation style, links, and clarity:
    passed. Lint retains one pre-existing informational constructor suggestion.
  • Compiled CLI smoke and clean installed-package CLI checks: passed. The package check uses an
    ephemeral archive; it does not establish retained artifact identity or hosted lifecycle proof.
  • Two duplicated library-audit counts were corrected. Host-sensitive real Git/filesystem tests
    received narrowly scoped, independently reviewed test watchdogs; production budgets and deadlines
    are unchanged. Earlier failed/interrupted runs and an unchanged runtime startup-test timeout are
    retained in .decisions/uc08-command-discovery.md, not erased by the final passing runs.
  • Separate commits: 65306cd feature/tests/usage, 4ed4a64 test watchdogs, e967c29 roadmap/evidence.

The first installed Linux attempt for
digital-twin issue 106
failed before review or PR publication: 48 of 52 exec results were frozen-command refusals.
At that checkpoint, no replacement model trial had run. Later attempts are recorded in the
current qualification status below; retain the original failed attempt in the denominator.
Keep this PR draft and unmerged until its qualification and review/check gates are complete.

Current installed qualification status

The fifth issue-106 attempt, run 34102894745,
used the approved preparation PR 112 merge and Flow source 544aebc13bfc50879de52396062a869ca975c367.
Both hosted checks passed on the same retained package bytes. Exact-artifact qualification is
established for those bytes, not for a future release or changed package.

The candidate failed candidate_holdout_failed before independent review or publication.
The strengthened check caught false success for settings inside an inaccessible parent directory.
Independent exact-source reproduction confirmed the defect despite all 22 generated public tests passing.
No review or repair child ran, and no new candidate PR was created. Both encrypted evidence snapshots
authenticated, and all three temporary Actions secrets were removed. Existing local keys remain intact.

None of the five installed attempts completed issue-to-merge qualification. UC-01, UC-05, UC-02,
and BR-06 remain open. The current repair contract starts from a valid blocked review, not a
controller-owned verification failure. The plan tracks that recovery-design gap and missing public
test coverage. No further pilot, budget increase, candidate merge, or release is authorized by this run.

The outcome and complete attempt history are recorded in
the issue-106 field report
and the usable-checkpoint plan.
Keep this PR draft pending full exact-head review, required checks, and qualification.

Historical source verification

Exact commit: 50e5e4c5c21bd1518ff6445a8cc3f93f5a93132e

  • npm run test:coverage: 458 files passed, 1 skipped; 6,317 tests passed, 4 skipped; 84.68%
    statements, 79.38% branches, 92.14% functions, and 85.27% lines.
  • npm run typecheck: passed.
  • npm run build: passed.
  • npm run format:check: passed across 931 files.
  • npm run lint: passed with one pre-existing informational suggestion in
    external-harness-adapter.ts.
  • npm run docs:ste, npm run docs:style, and npm run docs:links: passed.
  • npm run pack:check: built the archive, installed it into a clean environment, and executed the
    CLI successfully. Installed effective-policy SHA-256 (not an archive digest):
    5dfe0fbdfa1a86627e8762bfc071594c1bccbd6a467fc3f3ea12ebddf9b053b4.
  • Production dependency audit: zero known vulnerabilities.
  • Browser runtime suite: 2 tests passed.
  • Runtime suite: 44 tests passed, 42 expected platform skips.
  • Prime dependency audit and Go proof supervisor: passed.
  • git diff --check, changed-file secret scan, documentation denominator recomputation, and focused
    118-test recovery/compiler suite: passed.

Review checklist

  • External issue lifecycle reached verified merge and post-merge proof.
  • Complete attempt denominator and intervention ledger are documented.
  • Public documentation distinguishes source, package, host, and generalization boundaries.
  • UC-08 local tests, build, package installation, documentation, and changed-file audits pass.
  • No secrets, debug output, TODOs, FIXMEs, or placeholder implementations were added.
  • The exact installed package completes a previously unhandled external issue with the Flow
    controller running on hosted Linux x64.
  • Independent exact-head review confirms no P1, P2, or P3 finding.
  • Required hosted checks pass on the latest PR head, e826687141de40da13a2a4a6eb54cf1c08dd4cc1 (not met: isolation failed; other jobs running).

Historical admission correction and verification

Verified historical head: fcc97fc6251439868671acf4b0761dcd86fe268e.

  • Fix a reproduced empty-stdin pipe race without ignoring genuine input errors or changing production limits.
  • Seven real-process regression tests, 55 combined admission tests, and the targeted two-cycle lifecycle test passed. Build, type checking, lint, formatting, and documentation gates passed. Independent incremental review found no P1–P3 findings.
  • Retain the earlier sandbox-denied lifecycle attempt and its permitted timeout. The standalone rerun passed within the unchanged limit; its success does not establish the timeout cause.
  • Prior CI at c99836b completed with dependency audit and all four proof-runtime tests passing. Quality failed one of 6,687 tests during Git admission. The retained diagnostics do not prove that the corrected race caused that historical failure.
  • Full local coverage at 1a841be terminated with exit code 137 before producing a final report, after reporting five failures. The cause of termination remains unproven. A bounded covered diagnostic reproduced four test-watchdog timeouts and one pass. The correction uses the existing scoped 30-second watchdog policy without changing assertions or production limits. Its five selected covered behavioral tests passed, but that narrow run did not meet or replace repository-wide coverage thresholds. Both complete affected files subsequently passed all 47 tests, without skips, alongside formatting, lint, type checking, compilation, and documentation checks. Independent incremental review found no remaining P1–P3 findings. Commit ee371db contains this correction and evidence. Earlier failures remain recorded.
  • Hosted quality passed at 1a841be on September 7, 2026: 6,694 coverage tests across 468 files. Coverage reached 85.04% statements, 79.94% branches, 92.52% functions, and 85.64% lines. Two browser tests and 82 runtime tests passed. That job skipped four Lean tests assigned to the separate proof job.
  • Proof runtime subsequently passed all four tests without skips, with 85.49 seconds of test execution. It verified appliance image sha256:73559bb7a6d90c71601661ff13e82e59e1efa7ba05db22b5fb3e619f7e146a73. All three jobs in run 34111318839 completed successfully, including dependency audit. The hosted success does not erase the interrupted local run.
  • Reviewed commits ee371db and fcc97fc were pushed only after that run became terminal. They add the test correction and isolation-reuse documentation. Their production-source tree equals 1a841be; qualification is still checked on the exact new head.
  • Historical-head quality passed at fcc97fc: all 6,694 coverage tests across 468 files, two browser tests, and 82 runtime tests passed. Coverage reached 85.04% statements, 79.94% branches, 92.53% functions, and 85.63% lines. Documentation style, links, and prose checks were clean. Four Lean runtime tests were skipped in this quality job because they belong to the separate proof job. Dependency audit also passed.
  • Historical-head proof runtime passed all four tests without skips: 100.18 seconds of test execution and 103.10 seconds overall test duration. It independently built and verified appliance image sha256:73559bb7a6d90c71601661ff13e82e59e1efa7ba05db22b5fb3e619f7e146a73. All three jobs in run 34117070990 completed successfully at that exact historical head on September 7, 2026. This closed CI verification at that head, not installed issue-to-merge qualification, general verifier isolation, or full PR review.
  • The verification-repair design was approved on September 7, 2026, including limited feedback disclosure and mandatory isolation qualification. The user then selected hosted native Linux qualification first, followed by a tested Mac-local Linux environment. Repair remains disabled; complete observer qualification and the separately authorized live experiment remain pending.

Keep this PR draft and unmerged. No new pilot, candidate merge, or release was authorized by these verification results.

Historical hosted Linux isolation prerequisite qualification

Qualified prerequisite head: fd8cf95dd6af3d910f120308fa1ab97930f7c472.

  • Add a dedicated model-free Ubuntu 24.04 x64 verifier-isolation job. It uses pinned actions and Node, no persisted checkout credentials, read-only repository permission, and the production native sandbox.
  • Five real-process probes cover ordinary and new-session descendants, private files and inherited descriptors, host process access, and forged candidate result output. Positive controls and cleanup failures remain mandatory. Independent review corrected two probe weaknesses and found no remaining P1–P3 in this incremental slice.
  • All 40 affected scaffold tests passed after three expected test-first failures. Full type checking, compilation, public launcher smoke, lint, formatting, and documentation gates passed locally. The five Linux-targeted tests skipped on macOS; those skips are not qualification evidence. Prior native Mac descendant failures remain documented and reproducible through diagnostic mode.
  • Dedicated Linux isolation job passed all five real-process probes without skips: 1.37 seconds of tests, 3.75 seconds total test-runner duration. The completed job log confirms Ubuntu 24.04 x64, kernel 6.17.0-1022-azure, Node 26.7.0, and bubblewrap 0.9.0. Tests assert SRT 0.0.70. This is one passing prerequisite run, not complete behavioral-observer qualification or repair enablement.
  • Exact-head CI run 34128675526 completed successfully. Dependency audit, quality, and proof-runtime passed. All four proof-runtime tests and appliance verification passed. Quality passed 6,695 coverage tests across 468 files, two browser tests, and 87 runtime tests. Four Lean tests were skipped in quality because they belong to the separate proof job. This closes CI for the recorded historical source tree, not the latest PR head.
  • The isolation and quality logs identify GitHub's synthetic merge checkout 55023ba6a29fda99f951473b0d6f57e9d9912d90, not a direct checkout of the PR head. Independent commit API reads confirm both commits have source tree 79ef26b968d60d564ce8ee117be0c94a3a84e349. These results do not qualify later local commits.
  • UC-03 now requires a tested Mac-local Linux launcher/environment running the entire controller, with persistent state, protected credentials, bounded resources, recovery, evidence inspection, and exact-head approval. No virtualization product or new infrastructure permission has been silently selected.

Keep this PR draft and unmerged. No new live model pilot, retained-candidate modification, merge, or release is authorized by this qualification slice.

Current internal observer qualification slice

Current head: e826687141de40da13a2a4a6eb54cf1c08dd4cc1.

  • Add internal immutable-input ownership and Linux command-capture components. They remain disconnected from issue runs and repair selection. Raw execution evidence is preserved; command capture is not an application-result or behavioral receipt.
  • Expand the dedicated Linux job to 17 real-process tests across three suites. Job 101784477466 passed 16 and failed one: the original isolation suite passed 5/5, internal command capture passed 9/9, and fixture qualification passed 2/3. A nested user namespace read the mode-000 input after chmod and mount-bypass attempts were denied. The fixture profile remains unqualified; assertions are unchanged. Quality and proof jobs continue independently.
  • The two new local component suites passed 40 tests, and the affected scaffold suites passed 40 tests. Type checking, lint, formatting, build, generated capability-reference validation, documentation gates, and whitespace checks passed. All 17 Linux-targeted cases skipped on Mac; skips are not qualification.
  • Independent incremental reviews resolved all confirmed P1–P3 findings. Full exact-head PR review and installed lifecycle qualification remain open.
  • A later full local regression run was interrupted after an outer execution sandbox denied localhost ownership listeners. An unchanged selected case reproduced listen EPERM, a direct listener control confirmed the permission boundary, and the identical lifecycle case passed with host permission. The complete one-worker, non-live suite is now rerunning with that permission; no complete new local suite result is claimed yet.
  • Pinned upstream source shows that the native helper discards the normal-exit versus signal distinction before bubblewrap. The design compares a protected native-supervisor extension, a separate supervisor, and unsupported terminal handling. A new supervisor mechanism remains a pending user decision. Neither optional filesystem telemetry nor candidate output is accepted as application-result proof.
  • Keep the required Mac-local Linux launcher/environment on UC-03. Do not enable repairs or claim general readiness from these internal components.

Keep this PR draft and unmerged. No new live pilot, retained-candidate modification, merge, or release is authorized by this slice.

The expanded run checked out synthetic merge 3d934d6f03d259351d671fabfe93093fac5c469f, whose tree 12cb8436b4e575786df9f479aed0addfbc25545d equals the PR head's tree. Source review suggests namespaced DAC-bypass capabilities explain the denied-file read; runtime capability confirmation is still pending. An application-result protocol alone does not resolve this fixture-access gap. No stronger sandbox policy has been selected or enabled.

@danielbentes danielbentes added documentation Improvements or additions to documentation enhancement New feature or request labels Aug 31, 2026
@danielbentes danielbentes self-assigned this Aug 31, 2026
Parse bounded validate, doctor, run, inspect, events, resume, cancel, and exact merge requests behind an effect-free service boundary. Return generated idempotency identifiers for recoverable mutations.

Refs #197
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant