Skip to content

fix(acp): a fresh chat joins the process a restarted founder mounted its tree on - #37639

Merged
chenmingwei23 merged 2 commits into
mainfrom
fix/chat-share-tree-founder-key
Nov 15, 2026
Merged

chenmingwei23 merged 2 commits into
mainfrom
fix/chat-share-tree-founder-key

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: after a gateway restart, a team lead and the workers it opens share one KAS process again under ACP runtime sharing.

On a live gateway after a restart, the lead ran alone (spawned ... tenants=1) while two workers it opened afterwards shared one process between themselves (joined ... tenants=2).

Why it matters

The lead is usually the longest-lived chat, so after every restart the tree pays for one extra KAS process and its MCP servers for as long as the lead lives, and the memory saving the preview promises does not show for the most common tree shape.

Not a goal

  • Making a fresh chat that arrives while a restarted founder is still launching wait for that launch (it founds its own, as it did before this PR).
  • Removing the founded-start INFO diagnostic; it stays until the live restart confirms the fix, then can be dropped.

What changed (motivation → approach → change)

  • Symptom: after a restart the lead and its workers land on two processes.
  • Root cause: a chat resumed after a restart rejoins the scratch tree it was recorded in, so its sharing key carries that tree (shared_scratch). When the lead is the tree's first start, it founds a process that mounts that tree as its own work_scratch_dir, but the process was published under the tree-carrying key. Workers opened afterwards are fresh (no shared_scratch), so their key never matched and they founded a second process. fix(acp): rejoin the shared chat process a restarted chat's tree lives on #37579 covered the reverse order (a restarted chat finding a fresh founder's process through join_on_tree).
  • Change 1: RuntimeOwnership.acquire takes an optional published_key, read off the spawned runtime. ChatSharePlacement publishes a founded runtime under the founder's empty shared_scratch only when the runtime exposes exactly that tree, which is the key a fresh founder publishes. A runtime that does not expose the tree keeps the full key.
  • Change 2 (Opus and GPT finding): two chats of one tree restarting together both carry the tree key; the second waits on the first one's launch and, re-picking by its own key, missed the entry published under the empty key and founded a second process. The founding now records the entry it published and a woken waiter takes it, when it has room and is alive, before re-picking.
  • Change 3: a founded start logs one INFO line per live process of its tree naming the key fields that differ, as field name and empty/set only (no paths, env or account era), or says there is no live process in its tree.
  • The acquire contract and the waiter rule are recorded in docs/system-specs/modules/runtime-ownership.md.

Security: can a worker read what it should not in the lead's directory? No more than without a restart. The directory is the session tree's shared scratch work directory ($KIROCREW_SCRATCH), which every chat on a shared process already sees: when a fresh lead founds the process, its workers join and use the directory that process allocated. This PR only makes the restarted case behave the same. A chat can join only a process whose key matches on every other field, including the tree root (only chats of the same session tree), the Bot member and its memory store (a Bot never shares with another Bot or with an ordinary chat), the account era, sandbox tier and environment. The recorded tree is the restarted chat's own, read from agent_scratch_trees.recorded_tree for its session key, and the spawner still checks it is a plain directory under the scratch root. The scratch tree is agent-writable by design and holds no credentials.

Backwards compatibility

Compatible: published_key is optional and the other two acquire callers do not pass it; without it an entry is published under the matching key exactly as before.

Removes nothing: no key, record, path or public function is removed; one optional parameter and one read-only accessor (live_keys) are added.

Tests

Proof: PROVEN test/test_chat_runtime_sharing_cleanup.py::TestChatSharePlacement::test_a_fresh_chat_joins_the_process_a_restarted_founder_mounted_its_tree_on (head d82000e)

  • test_a_fresh_chat_joins_the_process_a_restarted_founder_mounted_its_tree_on: a restarted lead founds a process exposing its tree; a fresh worker then joins it.
  • test_two_restarts_of_one_tree_at_once_share_the_process_the_first_founds: the second restart waits on the first one's launch and joins what it published. Fails with the waiter change reverted ("the waiter founded a second process").
  • test_a_founded_start_logs_which_key_fields_kept_it_off: the line names shared_scratch(ours=empty,theirs=set) and never the path.
  • Existing restart tests unchanged and passing, including "a restart never joins a process that does not expose its tree".
  • test_chat_runtime_sharing_cleanup.py, test_chat_runtime_sharing.py, test_chat_runtime_tree_sharing.py, test_runtime_ownership.py: 317 passed, serial; black and flake8 clean.

Manual verification

Not yet re-run on a live gateway restart. On the next restart, the founded-start INFO line and the joined placement lines for the lead's workers confirm it.

Related Issues

Follow-up to #37466 and #37579.

Render Benchmark (counter budgets) is red on this PR and on main for a runner reason (the ubuntu-26.04 image has no Playwright chromium), tracked in #37645; it is not a required check and this PR does not change the workflow.

Sibling sweep

Swept: git grep -n "RUNTIME_OWNERSHIP.acquire(" -- src/
Sibling: src/kiro_crew/providers/chat_share.py -- changed here
Sibling: src/kiro_crew/runtime_ownership.py -- not affected -- registration leases key by the runtime object itself and never carry a tree

Pattern harvest

Not generalizable: the defect is one key field that a founder mounts as its own state; no other key field is both a spawn input and something the spawned process re-exposes.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

…its tree on

A chat resumed after a gateway restart keys the scratch tree it was
recorded in (shared_scratch). When it is the first start of its tree it
founds a process that mounts that tree as its own work_scratch_dir, yet
the process was published under the tree-carrying key, so a fresh chat
of the same tree (no shared_scratch) never matched it and founded a
second process. Seen live: a team lead resumed at boot stayed alone
while the workers it opened afterwards shared among themselves.

The founded runtime is now published under the founder's empty
shared_scratch when it exposes that tree, the same key a fresh founder
publishes; a runtime that does not expose it keeps the full key.

A founded start also logs at INFO which key fields differ from each live
process of its tree, by field name and emptiness only.
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 15, 2026
@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 02d0ec1483b2d8cbaf0fb10c080f0964bcabe22c touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 02d0ec1483b2d8cbaf0fb10c080f0964bcabe22c — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

The fix sits at mechanism level and its defect is pinned by tests. One declared-temporary diagnostic adds a public accessor that only that diagnostic uses.

Notes

  • Item 3 — oversized: the description says the INFO diagnostic "stays until the live restart confirms the fix, then can be dropped", and it also says the existing "joined placement lines for the lead's workers confirm it". So the existing lines can already do the confirmation. Subtraction: delete _log_why_not_joined and its call in place, delete RuntimeOwnership.live_keys (1 consumer, from grepping live_keys( in src/), and delete test_a_founded_start_logs_which_key_fields_kept_it_off.

What this change ships

Inventory (5 items) — 3 justified

Intent: after a gateway restart, a lead and the workers it opens share one KAS process again. FIX.

  1. A restarted chat that founds its tree's first process publishes it under the empty-tree key, so fresh workers of that tree join it — justified
  2. A second restart waiting on the first one's launch takes the entry that launch published instead of founding a second process — justified
  3. A founded start logs one INFO line per live process in its tree, naming which key fields differ — rides along; declared temporary, has a removal plan
  4. New public RuntimeOwnership.live_keys accessor — one consumer, the diagnostic in item 3
  5. The runtime-ownership spec records the published_key contract and the waiter rule — justified

[FIRST-PRINCIPLES-REVIEWED] 02d0ec1

Description read

[DESCRIPTION-READ] a63d4d581384d3b45f1b65b8469c753943691ff6088b28fa852e06be288f244b

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 02d0ec1483b2d8cbaf0fb10c080f0964bcabe22c — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the founder's entry is published under the key a fresh founder of the same tree would publish, only when the runtime exposes that tree. The waiter only takes over a founding that matched its own key.

Notes

  • Nobody has yet checked the fix on a live gateway restart. The founded-start INFO line that is meant to show it worked should be removed once that check is done; otherwise it stays as permanent log noise on every founded start.

[DESIGN-REVIEWED] 02d0ec1

Description read

[DESCRIPTION-READ] fb6740e405583d6efe8f2fe97e8986707d9691b0ca5c8ca22d26b39ce5ea0d31

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 02d0ec1483b2d8cbaf0fb10c080f0964bcabe22c — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 02d0ec1

Verdict parsed from the review's SHA-scoped output markers for commit 02d0ec1483b2d8cbaf0fb10c080f0964bcabe22c.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 02d0ec1483b2d8cbaf0fb10c080f0964bcabe22c false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 02d0ec1483b2d8cbaf0fb10c080f0964bcabe22c and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 02d0ec1

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 02d0ec1483b2d8cbaf0fb10c080f0964bcabe22c false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Nov 15, 2026
@KiroCrewCaptain

KiroCrewCaptain commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: Approve

Why: the decider estimates a 12% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes.

A process started by a restarted chat is now listed under the key a fresh founder would use (empty shared_scratch), but only when the process really exposes the recorded tree (work_scratch_dir == tree). A runtime that fell back to its own directory keeps the full key. Matching still uses the caller's own key, and the emptied key is the same one _join_own_tree already uses. The waiter change takes the entry the launch published, and only for waiters whose key equals the founder's. It still checks admits(), liveness and the closed flag, so a waiter can only land on a process for the same tree. The only other acquire caller (the registration lease, cap 1) passes no published_key and never has waiters. The trust boundary is unchanged: every other key field still has to match, including tree root, member and store, account era, sandbox tier and env. A joiner gets the founder's scratch tree exactly as it does when a fresh lead founds the process. The new tests use the real lease table and wait on events, not sleeps, and each fails without its part of the change. Existing restart tests still pass.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 02d0ec1483 · 221 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 15, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Nov 15, 2026
…ished

Opus and GPT: two chats of one tree restarting at once both carry the
tree key; the second waits on the first one's launch, which publishes
under the founder's empty shared_scratch, so the waiter's re-pick by its
own key missed it and founded a second process on the tree. The
founding now records the entry it published and a woken waiter takes
it (room and liveness permitting) before re-picking. The acquire
contract is recorded in the runtime-ownership spec.
@chenmingwei23
chenmingwei23 force-pushed the fix/chat-share-tree-founder-key branch from d82000e to 02d0ec1 Compare November 15, 2026 16:56
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 15, 2026
@chenmingwei23
chenmingwei23 enabled auto-merge (squash) November 15, 2026 17:58

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. On head 02d0ec1 the review checked four things. Placement still matches on the caller's full key. A founder publishes the emptied-tree key only when its runtime really exposes that tree as its own scratch dir, and otherwise keeps the full key. A woken waiter takes the published entry only when that entry is still tracked, has room and is alive. Mutation runs showed that each half of the fix has a test that fails without it: dropping published_key fails the fresh-chat test, and disabling the waiter takeover fails the two-restarts test.
Done well: the fix stays inside the existing seams and fails toward the full key, and the concurrent-restart race is reproduced deterministically with Event barriers and bounded waits instead of sleeps.
Optional:

  • Removing the self._alive(published) guard or the entry-membership guard in runtime_ownership.py (around line 369) fails no test. Two small founding tests would pin them: the founder's runtime dies after it publishes, and the founder releases before the waiter resumes.
  • The acquire docstring (runtime_ownership.py:327) and runtime-ownership.md:146 still say a woken waiter only re-picks. They should say it first takes the published entry when that entry is valid.
  • _log_why_not_joined also runs for _place_alone starts, where it always logs "no live process in this tree". If the diagnostic stays after the live check, skip it when the start is not joinable, or link an issue for removing it.

@chenmingwei23
chenmingwei23 merged commit 5132aec into main Nov 15, 2026
99 checks passed
@chenmingwei23
chenmingwei23 deleted the fix/chat-share-tree-founder-key branch November 15, 2026 18:51
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants