Repository navigation
fix(acp): give interactive chat starts priority over subagent starts - #16020
Conversation
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify the grown startup watchdog deadline (two added [DESIGN-REVIEWED] a1db36f Description read[DESCRIPTION-READ] c9616e8309aff006e4309ab4cfce4e35f3ccb6f4fd68e8afbeb69544d5504960 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Verify the one capacity change: the outer cold-start ceiling rises 4→5 ( Not justified as shipped
What this change shipsInventory (10 items) — 9 justifiedIntent: a FIX — a session start a person is waiting on no longer queues FIFO behind subagent/cron/pool-fill starts (pinned by
Lens 4/5 evidence: no prior priority primitive exists (grepped [FIRST-PRINCIPLES-REVIEWED] a1db36f Description read[DESCRIPTION-READ] 6aca50c3a3607c8cc0549bad695775cb5f0e709c3a968807305820230cad4d4b That sha256 names the PR description this verdict read. Recompute it with |
Opus 5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
7607243 to
87ecb80
Compare
87ecb80 to
6c87ac7
Compare
|
This push (6c87ac7), rebased onto origin/main 0777838, still one commit:
The previous red lanes were not caused by this PR: Backend Lint (main's F811, fixed by #16032) and Backend Tests shard 4 / Coverage Gate (the autonudge lock flake, fixed by #16040). Both fixes are included through the rebase. |
|
self-added: yes
|
|
self-added: yes
|
|
self-added: yes
|
|
self-added: yes
|
|
self-added: yes
|
6c87ac7 to
d68ba49
Compare
|
Rebased: main moved in
|
|
self-added: yes
|
|
self-added: yes
|
A start a person was waiting on queued FIFO behind every subagent, cron and pool-fill start at three bounded queues: SessionManager's cold-start semaphore, the gateway-wide spawn admission and the session/new gate. A subagent fan-out held the person typing behind work nobody was waiting on. All three are now one PrioritySemaphore (kiro_crew.start_priority): FIFO within a priority, FOREGROUND first, and a background waiter is passed by at most _FOREGROUND_BYPASS_LIMIT foreground grants. Every start is BACKGROUND unless its caller claims FOREGROUND where a person waits: the dashboard runner for a turn a person sent, eager spawns from the owner's slot actions, channel messages a transport received from a person, side chat, threads, the CLI, the optimizer, Issue Radar, Refine, Plan, owner workflow authoring and STT endpointing. Nothing is derived from a session key, because automation runs on interactive keys too. _start_sem gains an additive person reserve, so background keeps its width and the first concurrent person start never waits behind background holders stuck in the inner queues. The identity sweep drains it ahead of both priorities. A subagent's startup clock now pauses while queued at any of the queues, and the paused total is capped by _START_QUEUE_MAX_SECS.
d68ba49 to
a1db36f
Compare
|
Pushed a1db36f (one commit, rebased onto origin/main c0671f5). It replaces d68ba49, which conflicted with main.
Gate: black (touched files), isort, flake8, mypy (clean), docs-lint, harness parity. The targeted tests (every test file that touches the start-priority and companion paths, plus the test files main changed since the old base) gave 11831 passed and 33 skipped. |
|
self-added: yes
|
|
self-added: yes
|
|
self-added: yes
|
Problem / Motivation
Goal: A session start that a person is waiting on never queues first-in-first-out behind subagent, cron or pool-fill starts.
Three in-process queues bound every agent start: the
SessionManagercold-start semaphore, the gateway-wide spawn admission and thesession/newgate. All three were FIFO. When a subagent fan-out was running, a person who opened a chat, typed into Slack or pressed Refine waited behind every start already queued, and nobody was waiting on any of those.Why it matters
Fan-outs are routine. One parent can queue dozens of starts, and the gate admits two at a time. Behind that queue, a person's first message, or a Slack reply, sat for the whole fan-out, and from the person's side it looked like a hang ("chats throttling each other"). It hits everyone who runs subagents and uses the dashboard or a channel at the same time.
Not a goal
SpawnGate, which only exists whenmcp_gateway.stub_serversis configured. Tracked as fix(mcp-gateway): order gatewayd SpawnGate waiters by start priority #15830 (G11b)._pool_fill_lock. That lock is only held withsession.pool_size > 0, and the hold already existed. Residual tracked as fix(session): pool refill should not hold _pool_fill_lock across the start-permit wait #15831._bgruntime when a person starts waiting on_bg_runtime_lock. The person's ownsession/newis ordered; the respawn already under way runs at its starter's priority. Listed as a known residual inacp-client.md.agent.session_start_concurrency, or the collector headroom. The one capacity change is deliberate:_start_semgains one person-only permit (outer ceiling 4 -> 5,new_cold_start_semaphore). Background still holds at most 4, and the extra start is a person's chat that was going to start anyway. The gateway-wide spawn admission (_COLD_START_MAX_CONCURRENT = 2) is unchanged, so on a memory-tight host no more than two runtimes spawn and initialize at once, as before.What changed (motivation → approach → change)
PrioritySemaphorein the neutralkiro_crew/start_priority.py(application code may not importkiro_crew.acp). FIFO within a priority, FOREGROUND first, direct hand-off to the woken waiter. A queued BACKGROUND waiter is passed by at most_FOREGROUND_BYPASS_LIMITFOREGROUND grants, so background is never starved.release(priority)rejects a release naming a class that holds nothing.drain()is the identity sweep's barrier: it outranks both priorities and hands back what it holds, even when cancelled.person_origin); Slack events and interactions unless a trusted bot posted; the CLI chat; STT endpointing throughrun_bg_oneliner->get_bg_session; and every dashboard handler, which reads who asked throughowner_start_priority(an app token is BACKGROUND, because the person-only reserve must not go to an app). Issue Radar's label-recommendation route now reuses_run_oneshot_model, so its claim is made in one place.person_origin(queue_drain.person_tag; absent reads False), and a drain ORs the flags of the entries it collapses, so a replay never takes the finished turn's opener's flag._start_semis held across the provider's inner queue waits, so it gets one additive person-only permit. Background keeps its width of 4._exec_startedis re-stamped._startup_deadlinenow budgets two handshake rounds ofinitializeplussession/new, and the paused total is capped by_START_QUEUE_MAX_SECS, past which a start ends as "Never started: start queues saturated".priority=, per-class depth and the bypass count. A queue wait underSTART_QUEUE_LOG_MIN_MS(100 ms) is no longer logged: the subagentsession/newline used to fire on any wait above zero, and an uncontended acquire measures a few microseconds, so every start logged a wait nobody had. The start clock still receives every wait; only the log line is gated.run_bg_oneliner'stimeoutbounds the drive only, because cancelling the acquisition kills the shared_bg(re)spawn and leaks a session whosesession/newwas already sent. A test pins this.start_priority.py,acp/runtime.py,acp/runtime_start.py,providers/acp.py,providers/base.pysession/newgate take a prioritysession.py,session_allocation.py,session_background.py,session_lifecycle.py,session_pool.py_start_semreserve, priority threading,drain()sweep barriersubagent.py,subagent_manager/run.py,subagent_manager/monitoring.py,subagent_manager/terminal.pydashboard/request_priority.py,dashboard/chat_runner.py,dashboard/chat_handlers.py,dashboard/chat_threads.py,dashboard/handlers/*,dashboard/stt_stream.py,llm_helpers.pysrc/kiro_crew/apps(builtins/issue_radar,builtins/meetings,builtins/code_review_sage)messaging/transport.py,messaging/dispatch.py,messaging/queue_drain.pyperson_origin,ChannelTurn.start_priority, queued-entry flagsrc/kiro_crew/slack,src/kiro_crew/discord,src/kiro_crew/teams,src/kiro_crew/telegram,src/kiro_crew/webex,src/kiro_crew/feishu,src/kiro_crew/imessage,src/kiro_crew/wecom,src/kiro_crew/weixin,src/kiro_crew/whatsappreceivesetsperson_origin; dispatchers and drains pass it oncli_chat.py,task_planner.py,src/kiro_crew/taskrunner.py,workflows/service.py,config/sections.py,config/loader.py,src/kiro_crew/constants.pyINITIALIZE_TIMEOUT_SECSSpecs updated in this commit:
acp-client.md§ Start priority (claimers, guarantee, residuals, clock rule),session.md,subagent.md,messaging.md,taskrunner.md,prompt-optimizer.md,dynamic-subagent-sizing.mdandresource-protection.md(deadline formula), plusconfig-baseline.json.Backwards compatibility
Compatible. Every new parameter defaults to BACKGROUND, which is how starts behaved before except that a later FOREGROUND start can now pass them, within the bounded bypass. Widths, config keys and wire formats are unchanged.
The start-queue callbacks (
on_gate_queued/on_gate_acquired) now also receive the queue's name, and only internal callers use them.SessionManager._start_semmust now be aPrioritySemaphore; only tests assign it.Tests
test/test_start_priority.py(new):held().drain(): it outranks both priorities, gives back what it holds when cancelled, is not re-entrant, and the identity sweep completes under a foreground stream.receive, and every channel dispatcher.test_session_start_gate.py: with 2 permits held by child starts and 3 more child starts queued, a chatcreate_sessionis the nextsession/new.test_acp_provider.py: the priority reaches both runtime queues, including the resume-died respawn.test_subagent_startup_pressure.py/test_subagent_startup_watchdog.py: the clock pauses and accumulates across all three queues, and a stall is reaped at the right total. A start parked past the cap gets the saturation terminal. The companion acquisition brackets only its lock and admission waits, and a hung companion spawn is reaped at the startup deadline.test_subagent_startup_budget.py: the deadline formula, including the two-round case.test_session_sharing.py: a task run a person opened that falls back to the companion runtime (no bootstrap provider) spawns that runtime FOREGROUND, both through_get_or_bootstrap_run_runtimeandget_subagent_runtimedirectly.test_webex_dispatch.py,test_teams_midturn.py, Discord, Telegram): a drained replay round-trips through the real enqueue both ways (a person's message starts FOREGROUND, a queued wake BACKGROUND) under any opener.run_bg_onelinerdoes not cancel the acquisition; the startup window covers theinitializebudget; and every site that re-stamps_exec_startedclears the paused-clock fields.Manual verification
N/A. The behaviour is queue ordering inside one process, and tests drive it through the real semaphores, the real
create_sessiongate and the realSessionManagercold-start path. A live repro needs a concurrent subagent fan-out and a chat, and that adds timing noise rather than evidence.Related Issues
Follow-ups: #15830 (gatewayd
SpawnGatepriority, G11b). #15831 (pool refill lock) was fixed on main by #15857; this PR's rebase moves its four newTestFillLockReleasedAcrossStarttests onto aPrioritySemaphoreand drops the residual fromacp-client.md.Pattern harvest
Rule candidate: review-prompt
Pattern: "a bounded queue that both people and automation wait in is FIFO, so a person queues behind work nobody is waiting on. Bounded start queues take an explicit, caller-claimed priority, never one derived from a session key."
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement