Repository navigation
feat(acp): gate runtime kills on session lease ownership (cap=1) - #14482
Conversation
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS A missed lease release now shields a live process from the leak sweep forever — the design has no reconciliation backstop for stale leases. WatchThe gate deliberately trades kill-too-early for never-killed: the PR itself states a skipped release means the teardown "refuses its own teardown and leaks the process," and The change introduces a new cross-cutting invariant — every post-registration kill path must release before it signals — plus a new top-level module, and the owning specs ( Suggestions
[DESIGN-REVIEWED] d977f9b |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The lease table ships cap>1 pooling — sticky placement, Not justified as shipped
What this change shipsInventory (7 items) — 6 justifiedIntent: FIX — stop one session's teardown from killing a runtime another session is still using, and attribute every kill.
[FIRST-PRINCIPLES-REVIEWED] d977f9b |
Opus 5 Review — ✅ no blocking findingsReviewed Review detailsOne stale ratchet value fails its own test. FINDING — test/test_kill_chokepoint_ratchet.py:81 — [OPUS-REVIEWED] d977f9b Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
5307f52 to
242c706
Compare
7025e09 to
14560ce
Compare
14560ce to
03a0afd
Compare
03a0afd to
6854c49
Compare
6854c49 to
a06f4d0
Compare
…d shields A process two sessions share stops being killed by the first one to leave, and every kill that does happen says who fired it. Nothing asked who owns a runtime before killing it: _sync_kill_provider is the funnel every hard kill reaches, and it signalled the whole process group without consulting anything about who was using it. Under session sharing one runtime hosts a parent session and its subagents, so one teardown could take the others with it, and the death was logged where the process is reaped -- which records that a process died without naming who ended it. The lease table, the gate, the provider's acquire and every release site are one unit. A table nothing acquires is inert machinery whose refusal can never run, and a gate wired without complete release sites refuses the force-kill paths and leaks the process instead. The lease means the same thing as membership of the session registry, which is why every pre-registration cleanup path is authorized unchanged and only two paths must release before they signal. The two pid shields had the same shape of bug for a different reason: as sets they recorded which pids were shielded, not how many holders each had, so the first holder out tore the shield off a process the second still needed.
a06f4d0 to
d977f9b
Compare
Problem / Motivation
Goal: a process two sessions are using stops being killed by the first one to leave, and every kill that does happen says who fired it.
Nothing on
mainasks who owns a runtime before killing it.session_pid._sync_kill_provideris the funnel every hard kill reaches -- all three_dispatch_hard_killimplementations resolve their killer throughget_sync_kill_provider(), and the dashboard's reset-all fallback calls it directly -- and it signals the whole process group without consulting anything about who is using it.Under session sharing one kiro-cli process hosts a parent session and its subagents, so "this provider's process" and "a process only this provider uses" are different statements. Dogfooding on 09-26 showed the gap twice: one session's teardown took 15 and then 10 other sessions with it. The death is logged where the process is reaped, which records that a process died and not who ended it, so neither loss was attributable afterwards.
Two pid shields have the same shape of bug for a different reason.
_PROTECTED_PIDSandstarting_pidsare sets, so they record WHICH pids are shielded and not HOW MANY holders each has.discardtherefore cannot tell "the last holder left" from "one of several left", and the first holder out tears the shield off a process the second is still using.Why it matters
A user whose session is torn down loses every other session sharing that runtime, with no signal that it happened. The 15-and-10 losses were invisible until someone counted the survivors.
The shields fail the same way: the sweep reaps a live runtime, or a cold start's pid is exposed mid-spawn, because a concurrent holder's exit dropped a shield that was still needed.
Not a goal
Raising the cap above one runtime per owning session.
CHAT_RUNTIME_CAPis 1 and behaviour is unchanged: release drops the lease to zero before the teardown kills, so the gate authorizes exactly what it authorizes today.Leasing a subagent's co-tenancy. A subagent is handed a runtime it did not spawn and must not kill, and at cap=1 an entry that holds a lease has no room for a second, so having subagents acquire would change which process they land on. Their co-tenancy becomes a lease in the change that raises the cap.
Park,
forget,retire_pid,drain_parkedand a generalkill_runtimeentry point. Each has no caller even with the wiring done, so each would be new surface nothing reads.What changed (motivation -> approach -> change)
PidRefcountsrc/kiro_crew/runtime_ownership.pyin, iteration,len, truthiness,set(...), set algebra)._PROTECTED_PIDScountssrc/kiro_crew/session_pid.pystarting_pidscountssrc/kiro_crew/session_allocation.py,src/kiro_crew/session.pysrc/kiro_crew/runtime_ownership.pyacquire/release, the lease reads, andauthorize_runtime_kill, which returns False while any lease is outstanding.note_runtime_killis the allow path's attribution, so there is one implementation of "who fired" and the refusal is the only thing the gate adds.src/kiro_crew/session_pid.py,src/kiro_crew/acp/runtime.py_sync_kill_providerandAcpRuntime.killask before they signal and return without signalling when refused.src/kiro_crew/acp/session_provider.pyacquire_runtime_lease/release_runtime_lease;shutdownreleases at the top of its owning arm, before the destroy and the kill.src/kiro_crew/session_allocation.py_sessions; releases on the one failure handler that spans registration, before its hard kill.src/kiro_crew/dashboard/handlers/sessions.pyshutdownbefore it reaches its own release.test/test_kill_chokepoint_ratchet.pytest/test_runtime_ownership.pyThe shields' fix is counting, and it is safe to drop in because every existing holder already pairs one register with one unregister -- which is also why no holder changes.
The attribution's root cause is placement rather than content: a line written where the process is reaped cannot name the caller, because by then the caller has returned. So it moves to the decision point, at WARNING rather than INFO, because the gateway runs at WARNING and an INFO line would be absent from the log of every deployment that has the problem.
Three choices carry the ownership design.
The release belongs to the lease holder, never inside the funnel. Releasing immediately before asking whether a holder remains always yields zero, which would make the refusal unreachable at cap=1 and the whole table a rider. A non-owning killer does not release, and that asymmetry is what makes the refusal reachable today: it is exactly the co-tenant kill above.
The lease is taken at registration, not at provider start.
AcpSessionProvider.start()is never awaited in production -- the kiro startup path constructs the provider, assigns it to the outer provider's_client, and the construction site's own comment names the session lifecycle as the owner from that point. A lease taken instart()would never exist in a running gateway.Registration is also what makes the release obligation small. Every hard-kill site was classified against it: nine of the ten
_dispatch_hard_killcall sites run before a session is registered -- a failedstart, a failed identity stamp, a warm-pool provider discarded before any session claims it -- so they hold no lease and are authorized unchanged. One site's handler spans the lock section that registers the session, so it releases first. The dashboard's timeout arm is the other.One detail is easy to get wrong and is worth naming. The session index is pruned by value, not by the keys an entry's leases name:
releaseempties the lease map before it forgets the entry, so a key-derived prune finds nothing and strands one index row -- holding the entry and its runtime -- per session key. At cap=1 every release is a last release, so that would be one stranded row per ordinary teardown for the gateway's life.Backwards compatibility
Compatible.
PidRefcountis aMutableSet[int]supporting every operation its holders use, no caller assigns a plainsetto either shield, and the counted behaviour differs from the set only in the case the set got wrong. The_sync_kill_providerandAcpRuntime.killsignatures are unchanged. At cap=1 the gate authorizes what is authorized today, because each release drops the count to zero before its kill.Tests
199 pass across
test_runtime_ownership.py(21 cases),test_kill_chokepoint_ratchet.py,test_acp_session_provider.py,test_acp_session_provider_shutdown.py,test_acp_runtime_kill.py,test_dashboard_reset_sessions.py,test_agent_lifecycle_cycle.py,test_process_identity_structural.pyandtest_review_repair_routing_skill.py.Both shield defects are RED on base: running the two shield assertions against an unmodified
origin/mainworktree fails both, withsecond holder still needs the shieldandsecond start still needs the shield.The equivalence evidence for the gate is that the ~20 cases in
test_acp_runtime_kill.py, which assume a kill is unconditional, pass unchanged. The import-cycle pin passes too:session_pidreaching the gate imports no ACP module, which is why the gate lives at the top level rather than underacp/.Six mutations, each run and reverted with the restore verified by sha256:
PidRefcount.discardto set semantics_sync_kill_providertest_the_gate_verdict_is_acted_onfails while its weaker sibling still passestest_a_post_registration_kill_releases_its_lease_firstfails on that parameterThe ordering is the invariant, not merely the presence of a call: a release after the kill is as broken as none, so the test requires the release to precede the kill in the same block. Two tests assert the registry is empty after a last release, one of them across six session keys, because that leak is per key and a single key cannot show it growing.
One guard exists because it caught a real defect during this change. Recognising a lease holder by its two methods made every
MagicMockprovider on a release path aTypeErrorat the await, because a mock answers everyhasattrand returns a non-awaitable. The holder is recognised by its lease slot's type instead, andtest_a_mock_shaped_provider_is_not_mistaken_for_a_lease_holderpins it.Gates run locally:
flake8,isort,mypy src/kiro_crew/, the agent-SDK boundary check, the comment-history gate and the repo's own baselined black gate all exit 0.Pattern harvest
Rule candidate: a gate whose refusal branch has no reachable caller is a rider, and a gate whose release sites are incomplete leaks the resource it guards -- so the table, the gate, the acquire and every release site must land in one change rather than in a stack.
Rule candidate: before placing a lifecycle hook in a method, confirm the method is called on the production path.
AcpSessionProvider.start()reads like the natural acquire point and is never awaited.Rule candidate: prune an index by value when the key set is derived from state the caller may already have emptied, so the correctness does not depend on call order.
Inherited CI failures
None. The board is green.
Four lanes were red for a while and all four were
main's own, none reachable from this diff:Backend Tests (3.12, 2)(a pidfd test intest_oneshot_supervisor.py), its downstreamCoverage Gate,Frontend Tests (2)(a Bengali-digit ceiling over its cap in the sharedbn.jsoncatalog) andElectron Shell Tests(a port-owner pin). #14518 fixed all four onmain, and this branch is rebased onto the tip that carries it, so they pass here.Worth recording for the next person who hits this, because it cost several rounds: a base fix does NOT reach an open pull request on its own. A base move fires no
pull_requestevent, so no lane re-runs; and a close-and-reopen is worse than useless, because closing DROPSrefs/pull/N/mergeand the reopen's dispatch then races the rebuild. That race was lost visibly here -- a round dispatched after the fix had merged loggedHEAD is now at 7a4ae11 Merge <head> into <old main>and faithfully reproduced the pre-fix failures. Rebasing onto the tip is what fixes it, because the base fix becomes an ancestor and no merge commit has to be built correctly for CI to see it. The check that tells the two apart is the job log's own checkout line.One baseline moved for a reason that belongs to
main. The kill-primitive ratchet's ceiling is 67 rather than 66 becausemaingained a bypass site inpdf_extract.py-- a Windows-only tree kill for a document-parsing subprocess, commita41323c654(#14424) -- between this work's original base and the tip it now sits on. An identical AST scan of a pristinemaintip worktree also reports 67, so this change adds none, and the mutation that inserts one more call still fails the ratchet at 68. Routing that site through an ownership gate would consult a table that can never hold a lease for a document parser.Manual verification
N/A -- unit coverage sufficient. Both shield defects and the index leak are reproduced as failing assertions, the gate's equivalence rests on the unconditional-kill suite passing unchanged, and every release site is pinned by a mutation.
Related Issues
Fixes #14352
Supersedes #14362, which carried the shields and the attribution line alone. The lease table, the gate, the acquire and every release site are one unit -- a table nothing acquires is inert machinery whose refusal can never execute, and a gate wired without complete release sites refuses the force-kill paths and leaks the process instead, trading a kill-too-early defect for a never-killed one. So they land here together.