Repository navigation
feat(acp): continue a codex subagent from its persisted thread - #11960
Conversation
|
Intent: Let an ordinary codex subagent run on its parent's runtime instead of paying its own adapter tree, and stay continuable afterwards by loading the thread codex persisted — so |
GPT 5.6 Review — 🔴 changes requested (blocking)GPT 5.6 found at least one blocking issue that must be resolved before merging This comment is updated in place on each push. BLOCKING -- test/test_codex_session_mcp.py:1835 -- Real-adapter contract tests became skip-only (origin: validation) Adjudication (Opus 4.8) — is blocking on each finding proportionate?F1 is anchored to Harm rung: this is a blocking AUTOSDE rule; not weighed. False positive or not applicable? A repository writer can comment: |
Design Review (Fable 5) — 🔴 BLOCK (blocking)Design-level review of Design-Verdict: BLOCK Stale-base test rewrites revert two sibling pins that landed at base — one assertion is provably red at this PR's own HEAD. BlockersRewritten pins clobber #11944 and #11946, unmentioned in the description. WatchThe invariant membership rests on — [DESIGN-REVIEWED] db59c29 |
First Principles Review (Fable 5) — 🔴 BLOCKPremise-level review of First-Principles-Verdict: BLOCK The sharing change is measured and clean; riding beside it, the diff silently deletes six pinned tests and flips ratchets no source in this PR implements. Not justified as shipped
What this change shipsInventory (9 items) — 6 justifiedIntent: stop paying a dedicated adapter tree for every codex subagent by running it on the parent's runtime, with continuation served from codex's persisted thread — ADDITION.
Blockers
[FIRST-PRINCIPLES-REVIEWED] db59c29 |
Opus 4.8 Review — 🔴 changes requested (blocking)Reviewed The candidate is confirmed against source. BLOCKING — the diff's own test edit asserts a [BLOCK-MERGE] db59c29 BLOCKING — test/test_acp_capability_sets_leaf.py:175 [OPUS-REVIEWED] db59c29 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
|
self-added: no
|
|
self-added: no
|
|
self-added: yes
|
7ae7ca8 to
4bdd2e1
Compare
|
Rebased: main moved from |
|
self-added: yes
|
4bdd2e1 to
24ce11a
Compare
|
Rebased: main moved from |
|
self-added: yes
|
|
self-added: no
|
…otdev#11960) Put codex in ACP_BACKENDS_SESSION_SHARING, so an ordinary codex subagent runs on its parent's runtime instead of paying its own Node process and app-server. Teardown is unchanged: a subagent session is still closed, because one left resident holds the MCP fleet codex starts from its own config on a runtime nobody is using. What spawn_continue addresses is the thread codex persists under CODEX_HOME, restored with session/load -- measured on codex-acp 1.11.0 against codex 0.154.0, including from a restarted adapter process. Also reads the served backend's label at the shared bind instead of fixing it at kiro's, and corrects the record: the set comment, the harness docstrings, three spec rows and the RFC passages naming symbols that no longer exist. Co-authored-by: zejiangg <zejiangg@amazon.com>
kirodotdev#11960)" This reverts commit 6c21f0d.
Problem / Motivation
A codex subagent pays for a whole second agent: its own Node adapter and its own
codex app-server. Kiro subagents do not. They get a session on the parent's process.Codex was kept out of that because of one belief: its teardown verb
session/closeevicts the session, so a shared subagent "would not survive" the conversation that spawned it, andspawn_continuewould answerconversation_gone.The belief measured the wrong thing.
closedisposes the live session. It leaves the threadcodexwrote underCODEX_HOMEalone. That thread is what a continuation needs.Why it matters
Every ordinary codex subagent spawns a second adapter tree today. That is slower to start and costs memory nothing reclaims until the process is recycled.
It also left the repo asserting something untrue in six places, including two tests built to hold codex out.
What changed (motivation → approach → change)
Codex subagents now run on the parent's runtime, and a finished one can still be continued.
ACP_BACKENDS_SESSION_SHARINGsrc/kiro_crew/agent_sdk/backends.pysession/close, still evictssrc/kiro_crew/acp/harness/codex.pysession/loadon the thread underCODEX_HOMEsrc/kiro_crew/subagent_manager/run.py_bind_shared_handle_implsrc/kiro_crew/subagent_manager/run.pydocs/system-specs/modules/harness-parity.md,docs/system-specs/modules/providers.md,docs/system-specs/modules/agent-host-contract.md,docs/request-for-change/rfc-crew-agent-sdk-boundary.mdKIROCREW_LIVE_CODEX_PROMPT_TESTSsession/promptruns only when this equals1test/test_codex_session_mcp.pySharing here does not mean a session that stays alive. A resident subagent session would hold the MCP fleet codex starts from its own config — about 44 processes and 2751 MB for one session — on a runtime nobody is using. So teardown keeps closing, and the conversation is reached again by loading the record the host kept. That is what kiro already does with its transcript under
<kiro home>/sessions/cli.So membership asks one question: after this backend's teardown verb, can a
session/loadstill restore the thread? Codex answers yes. KAS answers no, because_kiro/session/deleteremoves the record, and that is the whole of its exclusion.The label read is a smaller thing than it looks, and the body of this PR is where to say so.
_bind_shared_handle_implwrote kiro's label for every backend. It is not what a continuation reads:_run_inner_implre-captures the label from the same provider right after session acquisition, and that write is what reachesstate.json. A run that completes was always labelled correctly, and the live run below confirms it — the continuation succeeds with the old constant in place. What the constant owns is the window before that re-capture. A run cancelled there leaves the identity record claiming kiro for a session another host holds, and the tombstone prune then takes_cleanup_session_files_sync's kiro branch, unlinks a path that was never going to exist, and reports cleanup succeeded — where the real label returns "no cleanup route for this provider" and the prune keeps its retry metadata.No keep-alive teardown variant exists in this diff. The idea was considered and dropped: the measurement says codex never needed one.
Backwards compatibility
Compatible: codex subagents move from a dedicated session to a shared one, which is the behaviour kiro subagents already have. Nothing is rejected, renamed or removed, no teardown verb changes, and a subagent whose spawn carries a model pin, an
allowed_toolslist orbarestill takes the dedicated arm.Tests
test_acp_codex_harness.py— codex is in the sharing set, the provider answers eligible, and the same test asserts the teardown verb is still the disposingsession/closeand that codex is inACP_BACKENDS_HARNESS_OWNED_SESSIONS, so "sharing" cannot be read as "resident".test_acp_codex_harness.py— the runtime set still does not imply sharing, now shown with KAS.test_acp_capability_sets_leaf.py,test_harness_parity.py— the set's value, its proper-subset relation toACP_BACKENDS_ACP_RUNTIME, and the opt-in ratchet, all re-pointed at KAS as the separating member.test_subagent_continuable.py::TestSharedBindIdentityLabel— a realAcpSessionProviderover a codex runtime is bound, and the published live cleanup identity, the persisted identity andinfo._session_providerall readcodex; the kiro case is pinned beside it becauseACP_BACKEND_KIROis the empty string and a truthiness test would have sent kiro down the wrong branch. Mutation-verified: restoring the old constant kills the codex test and leaves the kiro one green.test_codex_session_mcp.py::test_provider_config_only_excludes_process_configuration— the ambientconfig.tomlcopied into the throwawayCODEX_HOMEis narrowed to an allowlist of provider keys, somcp_serversin any spelling (header table, inline, dotted) plushooksandnotifycannot ride in and start the operator's own processes. Carries no skip marker, so it runs in CI where the live test does not.test_codex_session_mcp.py::test_real_codex_acp_load_after_close_restores— a gated live test against a real adapter: close evicts,session/loadon the closed id restores and recalls the first turn, the same load works from a restarted adapter process, andsession/deletearchives the thread so a later load refuses. Skips unlessKIROCREW_LIVE_CODEX_PROMPT_TESTSequals exactly1andCODEX_PATHis set and a codexconfig.tomlexists. The opt-in is separate on purpose:CODEX_PATHis configuration that anyone using the internal codex wrapper exports, so it is not consent to spend model tokens. This is the only test in the file that sendssession/prompt; the other two live tests handshake without a model call and carry no such gate.Manual verification
The capability was driven end to end against a real
codex-acp1.11.0 and a realcodex0.154.0.488, on the final commit:A parent session on the codex backend, an ordinary
spawn(no keep, no pin) that landed on the parent's runtime, the parent torn down with its process confirmed dead, thencontinue_conversation— which resumed the same sessionId and answered with the word the first turn planted.The same run was driven twice, once with the label read in place and once with the bind-site constant restored, because that is what decides whether the label read is the fix or a separate correctness point:
state.jsonproviderrun_pid == runtime_pidcodexnutmegrun_pid == runtime_pidcodexmarzipanBoth rows resume. So the shared-subagent path was never what made a codex
spawn_continueanswerconversation_gone, and the label is not what this PR fixes —ACP_BACKENDS_SESSION_SHARINGmembership is. The claim that a persistedPROVIDER_LABEL_DEFAULTbreaks the ordinary continuation does not hold:_run_inner_implre-captures the label from the provider before anything persistsstate.json, and that write is authoritative. The label read earns its place on the cancellation window described above, not on this path.The adapter-level half was measured separately with token accounting. On the restarted adapter the recall turn spent 330 input tokens against 9786 cached-read, above the planting turn's 7738, so the restored thread is in the cached prefix rather than re-sent.
Two host limits are disclosed rather than worked around. This box cannot create user namespaces (
unshare(CLONE_NEWUSER)→EPERM), so no Crew sandbox backend exists and codex's OS credential mask can never be applied; the driver patchedsandbox.credential_mask_appliestoTruein its own process only, which is the one product gate it bypassed. The user bus is also unreachable, so clearingXDG_RUNTIME_DIRput the cgroup scope on the product's own documented no-systemd-user-session path. Everything downstream of those two ran real code.One observation the run produced that this PR does not change:
spawn_releaseon a codex conversation reportsreleased, and a latercontinuestill resumes it.release_conversation_implcalls_cleanup_session_files_sync, which has no route for a non-kiro provider and returnsFalsehaving deleted nothing, whilestate.jsonkeeps the sid — so the continuation re-seeds. It is disk state rather than resident memory, holds for every backend that owns its session store, and predates this change. Tracked in #11959 with thesession/deletearchive proof.Backend gates green: mypy (1631 files), flake8, isort, black, harness-parity, agent-sdk-boundary, feature-map, docs-lint, comment-history, and the scoped backend selection at 28756 passed. Three failures in
test_app_manager.py::TestMalformedMcpUrlIsSkippedNotFatalreproduce identically on a cleanorigin/maincheckout of the same commit, so they are this host's, not this diff's.Related Issues
Closes #11274
Related: #11959 —
spawn_releasedisposes nothing on a backend that owns its session store. Disk state, not resident memory, so it is outside this line of work's leak rule; deferred deliberately rather than wired tosession/deletehere.Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)