Skip to content

fix(session): a send during an in-flight eager start joins it instead of spawning a second process - #27035

Merged
bolichen97 merged 1 commit into
mainfrom
fix/join-inflight-eager-start
Nov 2, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/join-inflight-eager-start

Conversation

@iamwhatever

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A message sent while a just-opened chat's agent is still starting uses that same agent, so each chat runs one agent process instead of two.

When a sidebar chat opens, the gateway starts its agent early (the eager spawn). If the user sends while that start is still running, the send finds no registered session yet and starts a second agent process for the same chat. Both finish; the later one loses the registration race and is shut down. One full start (about 109 MB of process) is wasted every time.

Why it matters

Opening a chat and typing right away is the common case. Each time, the host briefly carries a second agent process and pays a second full start in CPU. On a loaded host that is extra memory and start work for nothing.

Not a goal

  • Making the send faster. The send still waits for one start; the gain is one process per chat, not time.
  • Two concurrent real (non-eager) sends on the same new key. They still race as they do on main.
  • Any change to the eager spawn's admission, caps, or binding checks.
  • Joining a BACKGROUND eager start from a FOREGROUND claim, or promoting a queued start's priority. Such a claim keeps today's behaviour (its own start, possibly a duplicate).

What changed (motivation → approach → change)

A send during an eager start started its own process → the allocation had no record that a start for the key was already running → now it has one.

  • src/kiro_crew/session_allocation.py: new SessionRegistryState.speculative_starts map (key -> event and the eager call's start priority).
  • src/kiro_crew/session_allocation_parts/get_or_create.py, get_or_create (front door): an eager (speculative=True) allocation on a key with no registered session adds an announcement event to SessionRegistryState.speculative_starts, in the same lock hold. On every exit (registered, refused, failed, cancelled) it removes the announcement and sets the event.
  • _get_or_create_impl: a real claim that finds no live session but finds an announcement waits for the event, then claims again through the front door. Exception: a FOREGROUND claim (a person waiting) does not join a BACKGROUND eager start, because that start waits in every start queue at BACKGROUND; the claim cold-starts at its own priority as on main. Equal priorities always join: the claim's own start would queue behind the eager one in the same FIFO anyway. It finds the eager start's session and claims it. If the eager start failed or was refused, there is no session and no announcement, so the claim cold-starts exactly as on main.

The gain is one agent process per chat (about 109 MB) and one start not wasted, not latency: in-process the send-to-session time is about the same (median 1959 ms vs 2007 ms below, inside a stubbed 2.0 s start).

Backwards compatibility

Compatible: no API, config key, or stored format changes; a real claim only waits on a start that is already running for its own key and then takes the normal path.

Removes nothing: the change adds one in-memory map and one wait; no path, key, record, or validator is removed or tightened.

Tests

  • test/test_session_get_or_create.py::TestRealClaimJoinsAnInFlightEagerStart::test_a_send_during_an_eager_start_spawns_one_process[fg|bg] (both priorities equal, FOREGROUND as on new-chat open and BACKGROUND as on focus): eager start parked in start(), real claim arrives, both resolve to one provider; asserts one factory call. Red on main (assert 2 == 1, "the send spawned a second process"), green here.
  • ...::test_a_foreground_send_does_not_wait_on_a_background_eager_start: a BACKGROUND announcement does not hold a FOREGROUND claim; the claim cold-starts. Mutation: dropping the priority check makes it hang to its 10 s bound.
  • ...::test_a_send_cold_starts_when_the_eager_start_fails: the eager start raises; the joined claim cold-starts a fresh provider (two factory calls, the claim holds the second, one session registered, announcement cleared).

Proof: PROVEN test/test_session_get_or_create.py::TestRealClaimJoinsAnInFlightEagerStart::test_a_send_during_an_eager_start_spawns_one_process[fg], test/test_session_get_or_create.py::TestRealClaimJoinsAnInFlightEagerStart::test_a_send_during_an_eager_start_spawns_one_process[bg] (head 45452cf)

Determinism: 20/20 green at -n0; 10/10 green at -n 2 --dist loadgroup beside test/test_session.py and test/test_eager_spawn.py (-n 2, not 4, under this host's load rule); one shuffled run of the whole file, seed=17351, 56 passed.

Covering files run locally (related set only, -n 2): test_session_get_or_create.py, test_session_allocation_bindings.py, test_session.py, test_eager_spawn.py, test_eager_spawn_resume_prefetch.py, test_session_compaction.py, test_session_control_create_eager_spawn.py, test_eager_spawn_admission.py, test_eager_spawn_shutdown.py: 487 passed (with test_start_priority.py added).

Manual verification

In-process timing on this machine (in-process numbers, not a live gateway): a throwaway pytest bench builds a SessionManager whose provider start() spawns one real child process and then waits 2.0 s (stand-in for the KAS spawn + handshake). Each run opens with an eager get_or_create(speculative=True), sends 50 ms later, and times the send from its get_or_create call to holding a session it can prompt (the next step is the prompt to KAS). n=7 each, main 0f0097d297 vs this branch:

processes spawned per chat send → session (ms) median
main 2 2 2 2 2 2 2 2007 2007 2007 2007 2007 2007 2007 2007
branch 1 1 1 1 1 1 1 1971 1960 1959 1959 1958 1959 1960 1959

Related Issues

Fixes #27016

Found while measuring first-token latency; noted in #26970's body.

Sibling sweep

Swept: git grep -n "speculative=True" -- src/
Sibling: src/kiro_crew/dashboard/chat_turn/eager_spawn.py -- not affected -- it is the announcing caller; the fix lives in the allocation it calls, its own code is unchanged
Sibling: src/kiro_crew/session_compaction.py -- not affected -- its successor start runs while the held session is still registered, so no announcement is added and real claims behave as on main

Pattern harvest

Not generalizable: the defect is one missing "start already in flight" record in the single allocation path; the won-race arm already existed to clean up after it.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@iamwhatever
iamwhatever requested review from a team and KiroCrewCaptain as code owners November 1, 2026 12:08
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 1, 2026
@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 45452cfe4a716a6400652002b93c222b3ca393d1 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 45452cfe4a716a6400652002b93c222b3ca393d1 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the announcement is added and removed in the same lock holds that own the key's allocation, and a failed eager start falls back to main's cold start.

Notes

  • When a claim joins an eager start, it calls owner.get_or_create again while its outer reservation token is still held. That means two reservations on one key for one caller. If an ending fence goes up during the join wait, it invalidates the outer token, and adopt_turn runs twice. Returning the joined result from the outer front door, without nesting a second reservation, would keep one token per caller.

[DESIGN-REVIEWED] 45452cf

Description read

[DESCRIPTION-READ] 9d4446d89f942d2df1b12e246876905e22497dcc7f654849de495fd6fd249e50

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 45452cfe4a716a6400652002b93c222b3ca393d1 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

This fixes the cause (linked #27016) at the right level. The [fg|bg] tests fail on main, and nothing in the change already exists elsewhere.

Notes

  • Item 5 — inherited scope: the missing "start in flight" record applies to any start, but the announcement is only added when speculative is set, so two real sends still race. ## Not a goal excludes that case on purpose, so this is deferred, not a defect. Subtraction: none.

What this change ships

Inventory (5 items) — 5 justified

Intent: when a person types into a chat they just opened, the chat should use the one agent already starting, not launch a second one. FIX.

  1. A send made while the chat's eager start is still running uses that start's process, so there is one process per chat — justified
  2. That send now waits for the eager start to finish instead of starting its own process — justified
  3. A FOREGROUND send does not wait on a BACKGROUND eager start; it cold-starts as on main — justified
  4. If the eager start fails or is refused, the waiting send cold-starts as on main — justified
  5. New in-memory per-key map SessionRegistryState.speculative_starts (event + priority) — justified

[FIRST-PRINCIPLES-REVIEWED] 45452cf

Description read

[DESCRIPTION-READ] 3458464d3492a752df1511edd3c7e019232ccf34ea6b3a41fd29d14e5baa8585

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 45452cfe4a716a6400652002b93c222b3ca393d1 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 45452cf

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 45452cfe4a716a6400652002b93c222b3ca393d1 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 45452cfe4a716a6400652002b93c222b3ca393d1 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 45452cf

Verdict parsed from the review's SHA-scoped output markers for commit 45452cfe4a716a6400652002b93c222b3ca393d1.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 45452cfe4a716a6400652002b93c222b3ca393d1 false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 1, 2026

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Kiro Crew Auto-Pipeline [operator: buluoray]:

Approving. A send that arrives while the chat's eager start is still running now waits for that start and claims its session, so a new chat runs one agent process instead of two.

What I checked:

  • The announcement is added in the same lock hold as the eager call's reservation (get_or_create.py:368-375) and is cleared and set in an inner finally (:398-406) that runs before the SessionEndingError and BaseException handlers, so a refused, failed, fenced or cancelled eager start always wakes the joined claim. The claim then re-enters the front door and takes the normal path, which is the same outcome as main's "eager start registered first" case. Turn execution, the first-turn observation and the semaphore hand-off are unchanged, so no message is lost or run twice.
  • Ending fence: the waiting claim's reservation is invalidated by begin_ending like any other, and the re-entry takes a fresh reservation after the lift, so the request is held rather than dropped. Traced from source; no test covers it.
  • Mutations, run locally on this head: replacing joined_start = entry[0] with None reddens test_a_send_during_an_eager_start_spawns_one_process[fg] and [bg] (assert 2 == 1). Removing started.set() from the finally reddens both of those plus test_a_send_cold_starts_when_the_eager_start_fails (the claim never resolves). Both restored.

Follow-ups, not blocking:

  1. A joined claim pays the eager start's whole duration before cold-starting when that start is refused late, for example SpeculativeResumeRefused after a full spawn and session/load (:1175). On main the claim's own start ran in parallel. The FOREGROUND-vs-BACKGROUND rule keeps the dashboard focus-then-send path out of this, so only a BACKGROUND real claim on a key with a resume prefetch in flight can hit it. A short comment at the join noting that trade would help the next reader.
  2. The join re-entry passes _won_race_retries unchanged (:611), so hops are not bounded by that counter. Each hop needs a new eager start to announce in between, so this is theoretical.
  3. docs/system-specs/modules/session.md, the get_or_create row, still describes speculative=True without the announcement or the join rule. A sentence there would keep the spec matching the allocation path.

@CrysisDeu CrysisDeu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve at 45452cfe. The eager start posts its per-chat marker and event inside the same lock hold in which it saw no session, so there is no check-then-act gap and no missed wake-up; a send that finds the marker waits on the event and then re-claims with its own settings. Every exit of the eager start (success, refusal, failure, cancel, stop fence) releases the waiter, and on failure the send spawns its own process. Existing race-loser cleanup is unchanged. CI green, no overrides.

Nits: the test reads a private asyncio.Event attribute; no test for a binding change, a cancel or a stop fence during the wait; an always-failing start now runs twice before the user sees the error; the rejoin loop has no retry limit.

Posted by code-reviewer (Kiro Crew member) on behalf of @CrysisDeu. Full report on file.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 45452cf. No blocking findings; LGTM.

Non-blocking notes (follow-up is fine):

  • rc/kiro_crew/session_allocation_parts/get_or_create.py:599 - The joined claim calls owner.get_or_create again while its outer reservation token is still held. That gives one caller two tokens, runs adopt_turn twice (idempotent, so harmless), and starts a new ending-fence wait budget (fence_deadline) on the neste
  • est/test_session_get_or_create.py - No test covers an ending fence rising while the claim waits on the join, or a joiner with wait_if_busy=False. Both paths look correct on reading, but they are the two edges where the new wait meets existing machinery.
  • rc/kiro_crew/session_allocation_parts/get_or_create.py:565 - A joined claim now always takes the eager start's session, even when the slot's bindings changed mid-handshake. This still looks safe. eager_spawn checks the bindings and calls sessions.remove() synchronously after release(), so it should take _lock be

@bolichen97
bolichen97 merged commit ddef7ee into main Nov 2, 2026
97 of 98 checks passed
@bolichen97
bolichen97 deleted the fix/join-inflight-eager-start branch November 2, 2026 03:39
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A send during an in-flight eager start spawns a second agent process

4 participants