Repository navigation
feat(acp): continue a codex subagent from its persisted thread - #12101
Conversation
|
Intent: Re-land #11960 on a current base — an ordinary codex subagent runs on its parent's runtime and stays continuable by loading the thread codex persisted, so |
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify the claimed credentialled Suggestions
[DESIGN-REVIEWED] fae998c |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS One measurement carries the whole membership and no CI lane ever re-runs it: confirm the credentialled The reversal of the base pins is the honest kind: the deleted comment itself recorded the exclusion as conditional ("out of this set until the missing pieces land — a keep-aware teardown, and a provider label the continuation path can read"), the PR shows the first piece was mismeasured and ships the second, and a runnable gated test plus the rewritten pins carry the new facts. The What this change shipsInventory (8 items) — 8 justifiedIntent: stop paying a dedicated adapter tree for every codex subagent and make a finished one continuable, on the measured fact that codex's persisted thread survives
[FIRST-PRINCIPLES-REVIEWED] fae998c |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
|
self-added: yes
|
|
self-added: yes
|
1a966aa to
8e33ff8
Compare
|
mechanism: no code change; the capability ratchet named here already runs unskipped on every shard, and the line the finding points at is the live adapter measurement rather than the ratchet
|
|
mechanism: no code change; the rides-along is documentation only, and the body already declares it
|
8e33ff8 to
4e43824
Compare
|
Rebased: main moved in No literal file overlap: none of the four commits touches any of this PR's 13 files, so Re-run on the new base before pushing: my own eight affected files at 576 passed / 1 skipped (the skip is the live codex test, unopted), and the nine overlapping test files main changed at 535 passed / 1 skipped. black, isort, flake8, comment-history and harness-parity green; |
4e43824 to
968efec
Compare
|
Rebased: main moved in Readiness had passed on the previous head No literal file overlap again — #11993 touches none of this PR's 13 files, so |
Put codex in ACP_BACKENDS_SESSION_SHARING, so an ordinary codex subagent runs on its parent's runtime instead of paying its own Node process and app-server. Teardown is unchanged: a subagent session is still closed, because one left resident holds the MCP fleet codex starts from its own config on a runtime nobody is using. What spawn_continue addresses is the thread codex persists under CODEX_HOME, restored with session/load -- measured on codex-acp 1.11.0 against codex 0.154.0, including from a restarted adapter process. Also reads the served backend's label at the shared bind instead of fixing it at kiro's, and corrects the record: the set comment, the harness docstrings, three spec rows and the RFC passages naming symbols that no longer exist. Re-land on the current base: the test additions sit on top of the one-SQLite-store subagent_persistence API and keep the real-adapter contract markers, rather than carrying the file states they were written against.
968efec to
fae998c
Compare
|
Rebased to resolve a real conflict: GitHub had this head at CONFLICTING. One of them is a genuine collision rather than proximity. #11942 ("compact opencode, and stop telling four backends they self-manage") edits the same three files this PR does: Resolved by keeping both sides' contributions, not by taking a side. This branch's comment and assert are the feature. main's trailing three-line preamble for the harness pins that follow is main's, and it is kept — dropping a comment main added is exactly the clobber that reverted #11960.
Equivalence of the kept hunks: the sorted #11942's own content is intact rather than merely conflict-free: Four re-land invariants re-checked on Tests on the new base: my eight affected files 576 passed / 1 skipped, and #11942's five test files 217 passed. black, isort, flake8, comment-history and harness-parity green; |
buluoray
left a comment
There was a problem hiding this comment.
Approving at fae998c. No blocking findings. All three bots reviewed this head with no blocking findings and no override was needed.
Harness parity is clean, and clean in the way the rule actually asks for. backends.py:781 is ACP_BACKENDS_SESSION_SHARING = frozenset({ACP_BACKEND_KIRO, ACP_BACKEND_CODEX}) — positive membership by named backend. I grepped the diff for the forbidden shape (not is_claude_backend and friends) and found none in source; the only hits are docs prose and an assert KAS not in SESSION_SHARING exclusion test, which is a legitimate use. Selection stays at agent.acp_backend with agent.provider still ["acp"], so no seam moved. Model literals: none as a default or fallback; the one claude-agent-acp hit is a backend name in a comment.
The most valuable change in this PR is a fail-open bug that the title does not mention. run.py:2921 replaces a hardcoded PROVIDER_LABEL_DEFAULT with self._manager._provider_label_of(provider). Before this, a codex session cancelled inside the pre-recapture window recorded the kiro label, so _cleanup_session_files_sync took the kiro branch, found nothing to clean, and reported cleanup as succeeded. A cleanup that fails while claiming success is the worst shape for this kind of code, and the fix turns it fail-closed. The comment at the call site is also the right instinct — it routes through a table so "a harness added later is one table row rather than one more branch here", citing H13.
The resume path refuses rather than guessing, which is the question I most wanted answered. run.py:1103: if info.conversation_key and not _resumed: raise RuntimeError("resume_failed: session/load did not restore…"). So a missing, truncated or archived thread produces a clean refusal, never a silent resume into wrong context. On cross-run isolation: the session key is conversation_key or f"subagent:{info.id}" (run.py:412), unique per run, and although codex's session/load resolves from the sessionId alone, a thread that is not there refuses and drives _resumed=False. One subagent cannot land in another's thread without forging its conversation key.
The keep/continuation split at run.py:994 is coherent: those runs take the get_or_create arm that returns a real _resumed, while the sharing arm hardcodes _resumed=False — so the refusal above can never fire spuriously on a sharing run.
One finding, minor. providers.md, harness-parity.md and agent-host-contract.md were updated, but docs/system-specs/modules/subagent.md and acp-client.md were not, even though session sharing now covers codex and continuation survives teardown. Both specs describe surfaces this changes, so the next reader of either will have a stale picture. Worth a follow-up rather than a block.
Tests fail on revert where it counts: the capability-set leaf test reds if membership changes, and test_keep_marks_continuable_and_skips_sharing reds if the not info.keep gate goes. The live codex restore test is opt-in behind KIROCREW_LIVE_CODEX_PROMPT_TESTS and never runs in CI — which the repo discloses honestly in real_adapter_gate.py rather than letting it read as coverage.
I did not run the suite, and the measured restore/evict behaviour comes from that opt-in test rather than from anything I executed.
Not merging; that is Ray's call.
Problem / Motivation
A codex subagent pays for a whole second agent: its own Node adapter and its own
codex app-server. Kiro subagents do not. They get a session on the parent's process.Codex was kept out of that because of one belief: its teardown verb
session/closeevicts the session, so a shared subagent "would not survive" the conversation that spawned it, andspawn_continuewould answerconversation_gone.The belief measured the wrong thing.
closedisposes the live session. It leaves the threadcodexwrote underCODEX_HOMEalone. That thread is what a continuation needs.Why it matters
Every ordinary codex subagent spawns a second adapter tree today. That is slower to start and costs memory nothing reclaims until the process is recycled.
It also left the repo asserting something untrue in six places, including two tests built to hold codex out.
What changed (motivation → approach → change)
Codex subagents now run on the parent's runtime, and a finished one can still be continued.
ACP_BACKENDS_SESSION_SHARINGsrc/kiro_crew/agent_sdk/backends.pysession/close, still evictssrc/kiro_crew/acp/harness/codex.pysession/loadon the thread underCODEX_HOMEsrc/kiro_crew/subagent_manager/run.py_bind_shared_handle_implsrc/kiro_crew/subagent_manager/run.pydocs/system-specs/modules/harness-parity.md,docs/system-specs/modules/providers.md,docs/system-specs/modules/agent-host-contract.md,docs/request-for-change/rfc-crew-agent-sdk-boundary.mdKIROCREW_LIVE_CODEX_PROMPT_TESTSsession/promptruns only when this equals1test/test_codex_session_mcp.pySharing here does not mean a session that stays alive. A resident subagent session would hold the MCP fleet codex starts from its own config — about 44 processes and 2751 MB for one session — on a runtime nobody is using. So teardown keeps closing, and the conversation is reached again by loading the record the host kept. That is what kiro already does with its transcript under
<kiro home>/sessions/cli.So membership asks one question: after this backend's teardown verb, can a
session/loadstill restore the thread? Codex answers yes. KAS answers no, because_kiro/session/deleteremoves the record, and that is the whole of its exclusion.The label read is a smaller thing than it looks, and the body of this PR is where to say so.
_bind_shared_handle_implwrote kiro's label for every backend. It is not what a continuation reads:_run_inner_implre-captures the label from the same provider right after session acquisition, and that write is what reachesstate.json. A run that completes was always labelled correctly, and the live run below confirms it — the continuation succeeds with the old constant in place. What the constant owns is the window before that re-capture. A run cancelled there leaves the identity record claiming kiro for a session another host holds, and the tombstone prune then takes_cleanup_session_files_sync's kiro branch, unlinks a path that was never going to exist, and reports cleanup succeeded — where the real label returns "no cleanup route for this provider" and the prune keeps its retry metadata.No keep-alive teardown variant exists in this diff. The idea was considered and dropped: the measurement says codex never needed one.
Backwards compatibility
Compatible: codex subagents move from a dedicated session to a shared one, which is the behaviour kiro subagents already have. Nothing is rejected, renamed or removed, no teardown verb changes, and a subagent whose spawn carries a model pin, an
allowed_toolslist orbarestill takes the dedicated arm.Tests
test_acp_codex_harness.py— codex is in the sharing set, the provider answers eligible, and the same test asserts the teardown verb is still the disposingsession/closeand that codex is inACP_BACKENDS_HARNESS_OWNED_SESSIONS, so "sharing" cannot be read as "resident".test_acp_codex_harness.py— the runtime set still does not imply sharing, now shown with KAS.test_acp_capability_sets_leaf.py,test_harness_parity.py— the set's value, its proper-subset relation toACP_BACKENDS_ACP_RUNTIME, and the opt-in ratchet, all re-pointed at KAS as the separating member.test_subagent_continuable.py::TestSharedBindIdentityLabel— a realAcpSessionProviderover a codex runtime is bound, and the published live cleanup identity, the persisted identity andinfo._session_providerall readcodex; the kiro case is pinned beside it becauseACP_BACKEND_KIROis the empty string and a truthiness test would have sent kiro down the wrong branch. Mutation-verified: restoring the old constant kills the codex test and leaves the kiro one green.test_codex_session_mcp.py::test_provider_config_only_excludes_process_configuration— the ambientconfig.tomlcopied into the throwawayCODEX_HOMEis narrowed to an allowlist of provider keys, somcp_serversin any spelling (header table, inline, dotted) plushooksandnotifycannot ride in and start the operator's own processes. Carries no skip marker, so it runs in CI where the live test does not.test_codex_session_mcp.py::test_real_codex_acp_load_after_close_restores— a gated live test against a real adapter: close evicts,session/loadon the closed id restores and recalls the first turn, the same load works from a restarted adapter process, andsession/deletearchives the thread so a later load refuses.KIROCREW_LIVE_CODEX_PROMPT_TESTS=1is the ONE gate that decides whether it runs, and it is the onlyskipifon it. Past that gate every precondition fails rather than skips: codex-acp absent,nodeoffPATH,CODEX_PATHunset, no codexconfig.toml, a refusedsession/new, a planting prompt that does not reachend_turn.test_real_adapter_gate.py::test_no_guarded_test_carries_a_skip_the_lane_cannot_seeis what forbids theskipif(_ENTRY is None)form in this file, on the ground that a skipif cannot fail. The opt-in is a declaration that this host holds a working credentialled codex, so a run that reports success having measured nothing is the shape the no-silent-skip rule refuses; a host that cannot meet it unsets the variable. The opt-in is separate on purpose:CODEX_PATHis configuration that anyone using the internal codex wrapper exports, so it is not consent to spend model tokens. This is the only test in the file that sendssession/prompt; the other two live tests handshake without a model call and carry no such gate.Manual verification
The capability was driven end to end against a real
codex-acp1.11.0 and a realcodex0.154.0.488, on the final commit:A parent session on the codex backend, an ordinary
spawn(no keep, no pin) that landed on the parent's runtime, the parent torn down with its process confirmed dead, thencontinue_conversation— which resumed the same sessionId and answered with the word the first turn planted.The same run was driven twice, once with the label read in place and once with the bind-site constant restored, because that is what decides whether the label read is the fix or a separate correctness point:
state.jsonproviderrun_pid == runtime_pidcodexnutmegrun_pid == runtime_pidcodexmarzipanBoth rows resume. So the shared-subagent path was never what made a codex
spawn_continueanswerconversation_gone, and the label is not what this PR fixes —ACP_BACKENDS_SESSION_SHARINGmembership is. The claim that a persistedPROVIDER_LABEL_DEFAULTbreaks the ordinary continuation does not hold:_run_inner_implre-captures the label from the provider before anything persistsstate.json, and that write is authoritative. The label read earns its place on the cancellation window described above, not on this path.The adapter-level half was measured separately with token accounting. On the restarted adapter the recall turn spent 330 input tokens against 9786 cached-read, above the planting turn's 7738, so the restored thread is in the cached prefix rather than re-sent.
Two host limits are disclosed rather than worked around. This box cannot create user namespaces (
unshare(CLONE_NEWUSER)→EPERM), so no Crew sandbox backend exists and codex's OS credential mask can never be applied; the driver patchedsandbox.credential_mask_appliestoTruein its own process only, which is the one product gate it bypassed. The user bus is also unreachable, so clearingXDG_RUNTIME_DIRput the cgroup scope on the product's own documented no-systemd-user-session path. Everything downstream of those two ran real code.One observation the run produced that this PR does not change:
spawn_releaseon a codex conversation reportsreleased, and a latercontinuestill resumes it.release_conversation_implcalls_cleanup_session_files_sync, which has no route for a non-kiro provider and returnsFalsehaving deleted nothing, whilestate.jsonkeeps the sid — so the continuation re-seeds. It is disk state rather than resident memory, holds for every backend that owns its session store, and predates this change. Tracked in #11959 with thesession/deletearchive proof.The full scoped backend selection is not re-run here: this host is memory constrained and the instruction for this branch is the affected files at
-n 2. When that wide selection was last driven on this feature it reported three failures intest_app_manager.py::TestMalformedMcpUrlIsSkippedNotFatal, which reproduced identically on a cleanorigin/maincheckout of the same commit, so they are this host's rather than this diff's. CI runs the selection.Re-land of #11960 (reverted in #12088)
#11960 was merged on a base that predated three commits, and a rebase-merge replayed
its older copies of two test files over them. #12088 reverts it. This branch is the
same feature rebuilt on the current base: the reverts are what it was developed
against, and nothing here re-applies a file state.
bc0d2b0cfc#11780 — one-SQLite-storesubagent_persistencetest/test_subagent_continuable.pywas replaced by a pre-refactor copy calling the removed_run_agent_identity_path; 25 tests failed on every shardTestSharedBindIdentityLabelis added to itbf3fccf7d5#11946 — Real Adapter Contract Tests lanetest/test_codex_session_mcp.pylost the three@pytest.mark.real_adaptermarkers and therequire_real_adapterguard, so the lane selected 1 test against a floor of 4d92b8c4162#11944 — codex session compactiontest_acp_capability_sets_leaf.pycollapsed theACP_BACKENDS_COMPACTpin from three members back to twokiro,claude,codex,opencode) — this branch carries main's value byte for byte and changes only theACP_BACKENDS_SESSION_SHARINGpinThe third row is not in #12088's text. It surfaced here because every file the old
commit touched was diffed against the revert base and each line confirmed as this
feature's own, rather than only the two failures being repaired.
The new live test deliberately carries no
real_adaptermarker. The contract laneinstalls adapters from a pinned manifest and then asserts on its own junit report that
no SELECTED case was skipped. It holds no model credential and does not set this test's
spend opt-in, so a marked case here would be selected, skip, and turn the lane red —
the exact shape that assertion exists to refuse. Selection stays at 4, the floor's
value, verified by collection. What covers this file on every shard is
test_provider_config_only_excludes_process_configuration, which carries no marker andno skip.
Verified on this branch, rebased onto
origin/mainwith #12088 already in it: 576passed, 1 skipped across the eight affected files, which include the compaction and
capability pins and the real-adapter corpus gate;
-m real_adaptercollects 4; mypy clean on 1647 files; black, flake8, isort,agent-sdk-boundary, docs-lint, comment-history, harness-parity and feature-map all
green. The re-landed
TestSharedBindIdentityLabelis still mutation-verified on thecurrent API: restoring the hardcoded label fails the codex case and leaves the kiro one
green.
Related Issues
Re-lands #11960, which was reverted in #12088. #11274 was closed by that merge and stays closed, so this PR claims no closing keyword for it.
no linked issue: every issue this touches is already in its final state. #11274 is closed and stays closed, #11960 and #12088 are pull requests, and #11959 is a deliberate follow-up this change does not close.
Related: #11959 —
spawn_releasedisposes nothing on a backend that owns its session store. Disk state, not resident memory, so it is outside this line of work's leak rule; deferred deliberately rather than wired tosession/deletehere.Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)