Skip to content

feat(acp): gate runtime kills on session lease ownership (cap=1) - #14482

Merged
iamwhatever merged 1 commit into
mainfrom
feat/runtime-lease-gate-and-release-sites-w3a
Sep 28, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
feat/runtime-lease-gate-and-release-sites-w3a

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: a process two sessions are using stops being killed by the first one to leave, and every kill that does happen says who fired it.

Nothing on main asks who owns a runtime before killing it. session_pid._sync_kill_provider is the funnel every hard kill reaches -- all three _dispatch_hard_kill implementations resolve their killer through get_sync_kill_provider(), and the dashboard's reset-all fallback calls it directly -- and it signals the whole process group without consulting anything about who is using it.

Under session sharing one kiro-cli process hosts a parent session and its subagents, so "this provider's process" and "a process only this provider uses" are different statements. Dogfooding on 09-26 showed the gap twice: one session's teardown took 15 and then 10 other sessions with it. The death is logged where the process is reaped, which records that a process died and not who ended it, so neither loss was attributable afterwards.

Two pid shields have the same shape of bug for a different reason. _PROTECTED_PIDS and starting_pids are sets, so they record WHICH pids are shielded and not HOW MANY holders each has. discard therefore cannot tell "the last holder left" from "one of several left", and the first holder out tears the shield off a process the second is still using.

Why it matters

A user whose session is torn down loses every other session sharing that runtime, with no signal that it happened. The 15-and-10 losses were invisible until someone counted the survivors.

The shields fail the same way: the sweep reaps a live runtime, or a cold start's pid is exposed mid-spawn, because a concurrent holder's exit dropped a shield that was still needed.

Not a goal

Raising the cap above one runtime per owning session. CHAT_RUNTIME_CAP is 1 and behaviour is unchanged: release drops the lease to zero before the teardown kills, so the gate authorizes exactly what it authorizes today.

Leasing a subagent's co-tenancy. A subagent is handed a runtime it did not spawn and must not kill, and at cap=1 an entry that holds a lease has no room for a second, so having subagents acquire would change which process they land on. Their co-tenancy becomes a lease in the change that raises the cap.

Park, forget, retire_pid, drain_parked and a general kill_runtime entry point. Each has no caller even with the wiring done, so each would be new surface nothing reads.

What changed (motivation -> approach -> change)

Piece Where What it does
PidRefcount src/kiro_crew/runtime_ownership.py A pid shield that counts holders and releases on the last one, reading like the set it replaces (in, iteration, len, truthiness, set(...), set algebra).
_PROTECTED_PIDS counts src/kiro_crew/session_pid.py The sweep shield survives one of several holders leaving.
starting_pids counts src/kiro_crew/session_allocation.py, src/kiro_crew/session.py The cold-start shield survives one of two concurrent starts finishing; both files carry the type on their pass-through properties.
The lease table and the gate src/kiro_crew/runtime_ownership.py acquire / release, the lease reads, and authorize_runtime_kill, which returns False while any lease is outstanding. note_runtime_kill is the allow path's attribution, so there is one implementation of "who fired" and the refusal is the only thing the gate adds.
Both kill paths consult it src/kiro_crew/session_pid.py, src/kiro_crew/acp/runtime.py _sync_kill_provider and AcpRuntime.kill ask before they signal and return without signalling when refused.
The provider holds the lease src/kiro_crew/acp/session_provider.py acquire_runtime_lease / release_runtime_lease; shutdown releases at the top of its owning arm, before the destroy and the kill.
Acquire at registration src/kiro_crew/session_allocation.py Acquires where the session joins _sessions; releases on the one failure handler that spans registration, before its hard kill.
The reset-all release src/kiro_crew/dashboard/handlers/sessions.py The timeout arm releases before its force kill, because a timeout cancels shutdown before it reaches its own release.
The ratchet test/test_kill_chokepoint_ratchet.py A ceiling on unattributed primitive calls, plus pins that both paths consult the gate, pass a caller and a reason, act on the verdict, and release before they kill.
Behaviour tests test/test_runtime_ownership.py The counted shields, the attribution line, the lease table and the gate.

The shields' fix is counting, and it is safe to drop in because every existing holder already pairs one register with one unregister -- which is also why no holder changes.

The attribution's root cause is placement rather than content: a line written where the process is reaped cannot name the caller, because by then the caller has returned. So it moves to the decision point, at WARNING rather than INFO, because the gateway runs at WARNING and an INFO line would be absent from the log of every deployment that has the problem.

Three choices carry the ownership design.

The release belongs to the lease holder, never inside the funnel. Releasing immediately before asking whether a holder remains always yields zero, which would make the refusal unreachable at cap=1 and the whole table a rider. A non-owning killer does not release, and that asymmetry is what makes the refusal reachable today: it is exactly the co-tenant kill above.

The lease is taken at registration, not at provider start. AcpSessionProvider.start() is never awaited in production -- the kiro startup path constructs the provider, assigns it to the outer provider's _client, and the construction site's own comment names the session lifecycle as the owner from that point. A lease taken in start() would never exist in a running gateway.

Registration is also what makes the release obligation small. Every hard-kill site was classified against it: nine of the ten _dispatch_hard_kill call sites run before a session is registered -- a failed start, a failed identity stamp, a warm-pool provider discarded before any session claims it -- so they hold no lease and are authorized unchanged. One site's handler spans the lock section that registers the session, so it releases first. The dashboard's timeout arm is the other.

One detail is easy to get wrong and is worth naming. The session index is pruned by value, not by the keys an entry's leases name: release empties the lease map before it forgets the entry, so a key-derived prune finds nothing and strands one index row -- holding the entry and its runtime -- per session key. At cap=1 every release is a last release, so that would be one stranded row per ordinary teardown for the gateway's life.

Backwards compatibility

Compatible. PidRefcount is a MutableSet[int] supporting every operation its holders use, no caller assigns a plain set to either shield, and the counted behaviour differs from the set only in the case the set got wrong. The _sync_kill_provider and AcpRuntime.kill signatures are unchanged. At cap=1 the gate authorizes what is authorized today, because each release drops the count to zero before its kill.

Tests

199 pass across test_runtime_ownership.py (21 cases), test_kill_chokepoint_ratchet.py, test_acp_session_provider.py, test_acp_session_provider_shutdown.py, test_acp_runtime_kill.py, test_dashboard_reset_sessions.py, test_agent_lifecycle_cycle.py, test_process_identity_structural.py and test_review_repair_routing_skill.py.

Both shield defects are RED on base: running the two shield assertions against an unmodified origin/main worktree fails both, with second holder still needs the shield and second start still needs the shield.

The equivalence evidence for the gate is that the ~20 cases in test_acp_runtime_kill.py, which assume a kill is unconditional, pass unchanged. The import-cycle pin passes too: session_pid reaching the gate imports no ACP module, which is why the gate lives at the top level rather than under acp/.

Six mutations, each run and reverted with the restore verified by sha256:

Mutation Result
Revert PidRefcount.discard to set semantics 3 shield tests fail
Drop the attribution call from _sync_kill_provider 2 ratchet tests fail
Add one direct primitive call elsewhere in the tree the ceiling fails at 68 against 67, and the staleness check with it
Keep the gate call but delete the guard that acts on its verdict test_the_gate_verdict_is_acted_on fails while its weaker sibling still passes
Drop the release at any one of the three release sites test_a_post_registration_kill_releases_its_lease_first fails on that parameter
Prune the session index by lease-derived keys instead of by value both registry-emptiness tests fail

The ordering is the invariant, not merely the presence of a call: a release after the kill is as broken as none, so the test requires the release to precede the kill in the same block. Two tests assert the registry is empty after a last release, one of them across six session keys, because that leak is per key and a single key cannot show it growing.

One guard exists because it caught a real defect during this change. Recognising a lease holder by its two methods made every MagicMock provider on a release path a TypeError at the await, because a mock answers every hasattr and returns a non-awaitable. The holder is recognised by its lease slot's type instead, and test_a_mock_shaped_provider_is_not_mistaken_for_a_lease_holder pins it.

Gates run locally: flake8, isort, mypy src/kiro_crew/, the agent-SDK boundary check, the comment-history gate and the repo's own baselined black gate all exit 0.

Pattern harvest

Rule candidate: a gate whose refusal branch has no reachable caller is a rider, and a gate whose release sites are incomplete leaks the resource it guards -- so the table, the gate, the acquire and every release site must land in one change rather than in a stack.

Rule candidate: before placing a lifecycle hook in a method, confirm the method is called on the production path. AcpSessionProvider.start() reads like the natural acquire point and is never awaited.

Rule candidate: prune an index by value when the key set is derived from state the caller may already have emptied, so the correctness does not depend on call order.

Inherited CI failures

None. The board is green.

Four lanes were red for a while and all four were main's own, none reachable from this diff: Backend Tests (3.12, 2) (a pidfd test in test_oneshot_supervisor.py), its downstream Coverage Gate, Frontend Tests (2) (a Bengali-digit ceiling over its cap in the shared bn.json catalog) and Electron Shell Tests (a port-owner pin). #14518 fixed all four on main, and this branch is rebased onto the tip that carries it, so they pass here.

Worth recording for the next person who hits this, because it cost several rounds: a base fix does NOT reach an open pull request on its own. A base move fires no pull_request event, so no lane re-runs; and a close-and-reopen is worse than useless, because closing DROPS refs/pull/N/merge and the reopen's dispatch then races the rebuild. That race was lost visibly here -- a round dispatched after the fix had merged logged HEAD is now at 7a4ae11 Merge <head> into <old main> and faithfully reproduced the pre-fix failures. Rebasing onto the tip is what fixes it, because the base fix becomes an ancestor and no merge commit has to be built correctly for CI to see it. The check that tells the two apart is the job log's own checkout line.

One baseline moved for a reason that belongs to main. The kill-primitive ratchet's ceiling is 67 rather than 66 because main gained a bypass site in pdf_extract.py -- a Windows-only tree kill for a document-parsing subprocess, commit a41323c654 (#14424) -- between this work's original base and the tip it now sits on. An identical AST scan of a pristine main tip worktree also reports 67, so this change adds none, and the mutation that inserts one more call still fails the ratchet at 68. Routing that site through an ownership gate would consult a table that can never hold a lease for a document parser.

Manual verification

N/A -- unit coverage sufficient. Both shield defects and the index leak are reproduced as failing assertions, the gate's equivalence rests on the unconditional-kill suite passing unchanged, and every release site is pinned by a mutation.

Related Issues

Fixes #14352

Supersedes #14362, which carried the shields and the attribution line alone. The lease table, the gate, the acquire and every release site are one unit -- a table nothing acquires is inert machinery whose refusal can never execute, and a gate wired without complete release sites refuses the force-kill paths and leaks the process instead, trading a kill-too-early defect for a never-killed one. So they land here together.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 27, 2026
@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of d977f9b4a9c6d04084590465320f91bd8bcfb3e6 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] d977f9b

False positive or not applicable? A repository writer can comment:
/ai-review override gpt d977f9b4a9c6d04084590465320f91bd8bcfb3e6: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🟡 CONCERNS

Design-level review of d977f9b4a9c6d04084590465320f91bd8bcfb3e6 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

A missed lease release now shields a live process from the leak sweep forever — the design has no reconciliation backstop for stale leases.

Watch

The gate deliberately trades kill-too-early for never-killed: the PR itself states a skipped release means the teardown "refuses its own teardown and leaks the process," and leases_on_pid excludes only dead runtimes — so a live orphan whose session is gone but whose lease was never dropped is permanently protected from _sync_kill_provider, the exact sweep built to reap it. The three known release sites are pinned by mutation tests, but nothing reconciles the lease table against the session registry at runtime, so any future kill path or unclassified cancellation window degrades silently into a resident leaked kiro-cli process until gateway restart.
Clears when: a periodic reconciliation (a lease whose session key is no longer registered is dropped, at WARNING) lands, or a maintainer explicitly accepts leak-on-missed-release as the failure posture in the spec.

The change introduces a new cross-cutting invariant — every post-registration kill path must release before it signals — plus a new top-level module, and the owning specs (docs/system-specs/modules/session.md, acp-client.md) say nothing about it; AGENTS.md requires the spec update in the same commit. The invariant currently lives only in code comments and this PR body, which the next kill-path author will not read.
Clears when: the session (and/or acp-client) spec gains the ownership/lease/release-before-kill contract in this PR.

Suggestions

  • Drop the placement machinery (_last_for_session stickiness, first-fit scan, Acquisition.joined, the spawn callback): at cap=1 _pick_locked provably always returns None and no caller reads joined, so it is unreachable surface — the same test the PR applied to reject park/retire_pid — and its semantics will be re-litigated anyway when the cap-raise change arrives.

[DESIGN-REVIEWED] d977f9b

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of d977f9b4a9c6d04084590465320f91bd8bcfb3e6 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The lease table ships cap>1 pooling — sticky placement, spawn/cap/joined — dead at cap=1 and excluded by the PR's own ## Not a goal.

Not justified as shipped

  • Item 6 — one consumer, generalized: RUNTIME_OWNERSHIP.acquire has exactly one production caller (src/kiro_crew/acp/session_provider.py:158), which always passes cap=CHAT_RUNTIME_CAP (1) and a spawn callback returning the runtime it already holds; grep of \.joined|leases_on_runtime|_last_for_session finds zero production readers, and at cap=1 _pick_locked can never return a leased entry, so placement, joined, and the sticky index are unreachable — machinery for the cap raise the description defers ("Raising the cap above one runtime per owning session" is ## Not a goal).
    Subtraction: shrink acquire to minting a lease on a fresh entry (runtime, session_key); delete _pick_locked, _last_for_session, has_room, the spawn and cap parameters, and Acquisition.joined/leases_on_runtime — the sticky index is also the sole source of the stranded-row hazard two tests defend.
    Clears when: the placement machinery is deleted, or the cap-raising change that consumes joined and cap ships it.

What this change ships

Inventory (7 items) — 6 justified

Intent: FIX — stop one session's teardown from killing a runtime another session is still using, and attribute every kill.

  1. A kill of a runtime a live session still leases is refused at both kill funnels — justified
  2. Every kill (and refusal) logs one WARNING line naming pid, caller, reason at the decision point — justified
  3. The orphan sweep's pid shield survives one of several holders leaving — justified
  4. The cold-start pid shield survives one of two concurrent starts finishing — justified
  5. Sessions lease at registration; shutdown, the failure handler, and the dashboard timeout arm release before killing — justified
  6. The lease table's cap-based placement, sticky reuse, and join/spawn machinery — one consumer, generalized: dead at cap=1, deferred by ## Not a goal
  7. A repo-wide ratchet (baseline 67) pinning attributed kills and release-before-kill ordering — justified

[FIRST-PRINCIPLES-REVIEWED] d977f9b

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed d977f9b4a9c6d04084590465320f91bd8bcfb3e6 — this comment is updated in place on each push.

Review details

One stale ratchet value fails its own test.

FINDING — test/test_kill_chokepoint_ratchet.py:81 — BYPASS_BASELINE = 67 is one below the real AST count (68 direct kill-primitive calls under src/kiro_crew outside platform_compat.py), so test_bypass_baseline_is_not_stale's total == BYPASS_BASELINE and test_kill_primitive_bypass_count_does_not_grow's total <= BYPASS_BASELINE both fail on this commit → Fix: re-pin BYPASS_BASELINE = 68 and make the docstring's "66 calls exist" state the same number.

[OPUS-REVIEWED] d977f9b

Verdict parsed from the review's SHA-scoped output markers for commit d977f9b4a9c6d04084590465320f91bd8bcfb3e6.

False positive or not applicable? A repository writer can comment:
/ai-review override fable d977f9b4a9c6d04084590465320f91bd8bcfb3e6: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Sep 27, 2026
@chenmingwei23
chenmingwei23 force-pushed the feat/runtime-ownership-kill-chokepoint-w1 branch from 5307f52 to 242c706 Compare September 27, 2026 16:50
@chenmingwei23
chenmingwei23 requested a review from a team as a code owner September 27, 2026 16:50
@chenmingwei23
chenmingwei23 requested review from dodgeblaster and removed request for a team September 27, 2026 16:50
@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Sep 27, 2026
@chenmingwei23
chenmingwei23 force-pushed the feat/runtime-lease-gate-and-release-sites-w3a branch from 7025e09 to 14560ce Compare September 27, 2026 16:56
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Sep 27, 2026
@chenmingwei23
chenmingwei23 changed the base branch from feat/runtime-ownership-kill-chokepoint-w1 to main September 27, 2026 17:39
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Sep 27, 2026
@chenmingwei23
chenmingwei23 force-pushed the feat/runtime-lease-gate-and-release-sites-w3a branch from 14560ce to 03a0afd Compare September 27, 2026 17:46
@github-actions github-actions Bot added readiness: checking Automated validation is still running merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Sep 27, 2026
@chenmingwei23
chenmingwei23 force-pushed the feat/runtime-lease-gate-and-release-sites-w3a branch from 03a0afd to 6854c49 Compare September 27, 2026 18:32
@github-actions github-actions Bot removed the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Sep 27, 2026
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 27, 2026
@chenmingwei23
chenmingwei23 force-pushed the feat/runtime-lease-gate-and-release-sites-w3a branch from 6854c49 to a06f4d0 Compare September 27, 2026 19:04
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 27, 2026
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Sep 27, 2026
@chenmingwei23 chenmingwei23 reopened this Sep 27, 2026
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 27, 2026
…d shields

A process two sessions share stops being killed by the first one to leave, and
every kill that does happen says who fired it.

Nothing asked who owns a runtime before killing it: _sync_kill_provider is the
funnel every hard kill reaches, and it signalled the whole process group without
consulting anything about who was using it. Under session sharing one runtime
hosts a parent session and its subagents, so one teardown could take the others
with it, and the death was logged where the process is reaped -- which records
that a process died without naming who ended it.

The lease table, the gate, the provider's acquire and every release site are one
unit. A table nothing acquires is inert machinery whose refusal can never run,
and a gate wired without complete release sites refuses the force-kill paths and
leaks the process instead. The lease means the same thing as membership of the
session registry, which is why every pre-registration cleanup path is authorized
unchanged and only two paths must release before they signal.

The two pid shields had the same shape of bug for a different reason: as sets
they recorded which pids were shielded, not how many holders each had, so the
first holder out tore the shield off a process the second still needed.
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 27, 2026
@chenmingwei23
chenmingwei23 force-pushed the feat/runtime-lease-gate-and-release-sites-w3a branch from a06f4d0 to d977f9b Compare September 27, 2026 23:03
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 27, 2026
@iamwhatever
iamwhatever merged commit 68c6476 into main Sep 28, 2026
108 checks passed
@iamwhatever
iamwhatever deleted the feat/runtime-lease-gate-and-release-sites-w3a branch September 28, 2026 00:16
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nothing asks who owns a runtime before killing it: one session's teardown can kill a process another session is using

2 participants