Repository navigation
fix(metrics): split the sharing tenants bucket past 4 and count unshared turns' start - #37980
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks. Both gaps are fixed with new closed values on existing counters, and the only value removed ( Notes
[DESIGN-REVIEWED] 5d008ea Description read[DESCRIPTION-READ] e5e416df30fbe9c8b12a2bb5c62421e89c78ef92bd685190da3f78110127146c That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. Each item answers one of the two launch questions in #37979. The per-client turn clock is a test-only seam that ships in production code. Notes
What this change shipsInventory (6 items) — 5 justifiedIntent: let whoever decides whether sharing goes on by default see how full a shared process gets, and compare turn-start delay with sharing on and off. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] 5d008ea Description read[DESCRIPTION-READ] 9fdfefed6d7579a5f24df0eba723554c589609dfad7f1966f914966f70b7d089 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
3813626 to
2c06427
Compare
|
Disposition for the Opus 5.5 BLOCKING finding on 3813626 (
|
2c06427 to
ac9d556
Compare
|
Disposition for the Opus 5.5 BLOCKING finding on 2c06427 (src/kiro_crew/acp/client_turn.py:80, a Slack/Discord/Telegram chat read tenants=0 on the AcpClient arm but 1 on the handle arm): fixed in ac9d556.
|
|
Pod run on head ac9d556 (systemd pod, loopback fake model for goose; a KAS lead opens 8 goose children with session_create; ACP runtime sharing and the session harness preview on; the goose switch on for one tree and off for another, pod restarted between; 2 rounds of all 8 children asking at once). Totals from the pod's local metrics JSONL, summed over DELTA exports:
|
ffa4108 to
1deb8a7
Compare
1deb8a7 to
ba77aa2
Compare
…red turns' start tenants_bucket now reads 0/1/2/3/4-7/8-10/11+ so a process at the default cap of 10 reads apart from one with 4. turn_started gains a closed shared=true/false attribute and is now also counted on AcpClient.stream_events, the arm a claude, goose or deepseek chat takes when it does not share, so the preview's on and off arms compare on one series. Closes #37979
ba77aa2 to
5d008ea
Compare
Code change approver recommendation: ApproveWhy: the decider estimates a 10% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes. Telemetry-only change: splits the tenants bucket past 4 into 4-7/8-10/11+ and adds turn_started on the AcpClient arm with a closed An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Tier 1: every tier gate came back clean and the automated review approved this head. Approved at the hand-over of PR review to the Code Change Approver (2026-11-16), for a head decided just before it took over.
Problem / Motivation
Goal: Make the chat runtime sharing metrics able to answer two launch questions: how many chats a shared process really carries, and whether a turn starts later with sharing on than with it off.
Two gaps in
kirocrew.chat_runtime_share.*(src/kiro_crew/metrics/chat_runtime_share.py, #37189), found in a pod run with a KAS lead and 8 goose / 8 deepseek child chats (#37979):tenantstopped out at4+. The default per-process cap (agent.chat_runtime_sharing_max_sessions) is 10, so a process with 8 chats read the same as one with 4.turn_startedat all. Only KAS turns did.Why it matters
These numbers decide whether sharing goes on by default. Without gap 1 nobody can see how full a process gets near the cap, or how many chats one crash takes down (
process_diedandrss_sampleduse the same bucket). Without gap 2 there is no "off" arm for first-event delay, so "does a turn start later when it shares a process" cannot be read from one chart.Not a goal
turn_endedon theAcpClientarm. It stays handle-arm only; see Other suggestions below.What changed (motivation -> approach -> change)
Gap 1:
tenants_bucketnow returns0/1/2/3/4-7/8-10/11+.8-10holds the default cap;11+is only reachable with a raised cap. The old4+is split into4-7,8-10and11+; from the release carrying this change,4+no longer appears.Gap 2, symptom -> root cause: the counter was emitted only in
acp/session_handle_turn.py::_run_turn, theAcpRuntime+AcpSessionHandlearm. claude, goose and deepseek (ACP_BACKENDS_SHARED_RUNTIME_ONLY_FOR_CHATS) take that arm only when a chat got a shared placement; otherwiseproviders/acp_lifecycle.py::startkeeps them onAcpClient, whosestream_eventsemitted nothing. Now:acp/client_turn.py::stream_eventscounts the same span (prompt written -> first non-synthetic event),shared=false.tenantsis1for a session the gateway registered (a chat on any channel, a cron or task session) and0for an unregistered one (a subagent, a worker-pool client), the same split the handle arm gets from its registration lease.AcpClientholds no lease, soruntime_ownership.acquire_session_lease(the registry's own call inget_or_create) now marks its_registered_sessionslot where a runtime-backed session would take the lease (src/kiro_crew/acp/client_class.py).turn_startedgains a closedsharedattribute (true/false). On the handle arm it is the newRuntimeOwnership.placed_by_sharing(runtime)(src/kiro_crew/runtime_ownership.py): a live runtime placed under aChatRuntimeKey, the same rule theprocessesgauge uses. A KAS chat with sharing off holds only a registration lease keyed by the runtime itself, so it readsfalse.turn_started{harness=goose}split bysharedis then the on-vs-off comparison. Export allowlist (metrics/product_metrics.py): still a counter, every value closed, no key, pid or path added.docs/system-specs/modules/metrics.mdanddocs/guides/telemetry-otlp-export.mdare updated.Backwards compatibility
Breaking: the
tenantsvalue4+is no longer emitted onplaced,rss_sampled,process_died,turn_started,turn_endedandkas_queued; counts from 4 up now read4-7,8-10or11+. Accepted: the preview has been out two days and no decision reads that series yet.turn_startedgains asharedattribute, which a query that ignores it still sums the same. Writer sweep on origin/main @ 529da1e:git grep -n '"4+"' -- src testfinds onlychat_runtime_share.py(changed here) and an unrelatedtest/test_cli_doctor_install.pystring.Reader: src/kiro_crew/metrics/chat_runtime_share.py:tenants_bucket -- release -- the only producer of the value; every caller passes a count, none compares to
4+Tests
test/metrics/test_chat_runtime_share.py:test_bucket_edges_are_inclusive, newtest_the_tenant_buckets_split_around_the_default_cap(reads the shipped cap fromAgentConfig).test_a_turn_counts_its_start_delay_and_how_it_endedassertssharedand has an 8-lease case reading8-10.test_only_a_process_sharing_placed_reads_shared: a registration lease readsfalse, a keyed placementtrue.test_a_turn_on_a_process_of_its_own_is_counted_unshared: drivesclient_turn.stream_eventsfor registered goose, deepseek and claude sessions (tenants=1) and an unregistered one (tenants=0); every case is red with theclient_turn.pyhunk reverted, and the registered cases are red with theacquire_session_leasehunk reverted.test_registration_marks_a_client_that_owns_its_process: a realAcpClientkeyed on dashboard, Slack, Telegram and cron is marked byacquire_session_lease; aMagicMockprovider is not. Red with that hunk reverted.-n0, and the whole file in a shuffled order (seed 17589). The-n 4repeat was not run (serial-only on this host).test/metrics/test_product_metrics.py,test/test_acp_client_prompt_turn.py,test/test_acp_client_agent_started_turns.py,test/test_runtime_ownership.py,test/test_chat_runtime_sharing.py,test/test_acp_bindings.py,test/test_client_bindings.py,test/test_flake_pattern_ratchet.py; black, isort, flake8, mypy on the changed files;semgrep scan --config semgrep/against the merge base;scripts/check_file_line_cap.py.Proof: PROVEN test/metrics/test_chat_runtime_share.py::test_bucket_edges_are_inclusive, test/metrics/test_chat_runtime_share.py::test_the_tenant_buckets_split_around_the_default_cap, test/metrics/test_chat_runtime_share.py::test_a_turn_counts_its_start_delay_and_how_it_ended[2-dashboard:chat-0-1-12.0-end_turn-kas-want0], test/metrics/test_chat_runtime_share.py::test_a_turn_counts_its_start_delay_and_how_it_ended[8-dashboard:chat-0-1-0.5-end_turn-kas-want1], test/metrics/test_chat_runtime_share.py::test_a_turn_counts_its_start_delay_and_how_it_ended[2-subagent:sub-1-0.0-end_turn-kas-want2] (head ac9d556)
Manual verification
Pod run on head ac9d556 (systemd pod, loopback fake model, KAS lead + 8 goose children, goose switch on and then off, 2 rounds of 8 concurrent turns): switch on recorded
turn_started{harness=goose, shared=true, tenants=8-10}= 16; switch off recordedturn_started{harness=goose, shared=false, tenants=1}= 16 (0 before this PR). Details in the PR comment with the pod totals.Related Issues
Closes #37979
Sibling sweep
Swept: git grep -n "record_turn_started|_sent_at = time.monotonic" -- src/
Sibling: src/kiro_crew/acp/session_handle_turn.py -- changed here
Sibling: src/kiro_crew/acp/client_turn.py -- changed here
Pattern harvest
Not generalizable: a per-arm metric emit missing on one of two turn paths; the doc row that claimed "every ACP turn" is corrected, and the new test pins the second arm.
Other suggestions
turn_endedis still emitted only on the handle arm, so on theAcpClientarmturn_startedhas no matchingturn_ended. Worth a follow-up if the outcome mix should be compared on vs off too.Checklist