Repository navigation
fix(acp): wait for the session/set_model reply on the shared runtime - #18248
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Before merge, check one thing: an adapter that never answers What this change shipsInventory (4 items) — 4 justifiedIntent: when someone switches models on a shared-runtime session and the adapter refuses, the switch is reported as refused and the conversation is kept. This is a FIX (Refs #18169; new tests fail on base).
[FIRST-PRINCIPLES-REVIEWED] 03f6973 Description read[DESCRIPTION-READ] 64fd6b2d9bbe65666df8e507007634e3f41c7cdfb0d764fddcf6a931a6b0661f That sha256 names the PR description this verdict read. Recompute it with |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that a transient error reply (AcpPromptBusy, throttle, 5xx) caught by [DESIGN-REVIEWED] 03f6973 Description read[DESCRIPTION-READ] 46edad38492ac44728ab7b724a5eb84ef1001a5d32b9a2ee997b20d858876e69 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
644e3fe to
af4353e
Compare
AcpSessionHandle.set_model sent session/set_model without waiting for the answer, then recorded the new model unconditionally. An adapter's error reply reached no consumer (the pre-turn drain discarded it), so a refused switch read as a success: the dashboard and usage log showed the pick while the session kept serving the previous model. The request now goes through _send_awaited + _wait_for_response like the set_config_option branch. An explicit error reply is recorded in model_pin_refused and the session keeps its model, the contract the config-option branch already follows, so the dashboard answers the pick with a 4xx instead of resetting the conversation. No reply within the bounded wait is read as accepted and recorded, as before.
af4353e to
03f6973
Compare
Problem / Motivation
Goal: A model switch on a shared-runtime session that the adapter answers with an error is reported as refused, with the conversation kept, instead of being recorded as if it had worked.
AcpSessionHandle.set_modelsendssession/set_model(the kiro-cli path, and the path every backend outside the config-option set and KAS falls through to) without waiting for the reply, then records the new model unconditionally. An error reply reaches no consumer; the pre-turn drain discards it (pre-turn drain discarded N leftover frame(s) ... terminal-shaped responses). The dashboard and the usage log then show the pick while the session keeps serving the previous model, and_try_live_model_switchnever falls back.Why it matters
A model pick that silently does nothing misreports what serves every following turn: cost, quality and context-window math are all attributed to a model that is not running, and nothing tells the user. The
set_config_optionand KAS branches of the same method already wait for the adapter's answer; only thesession/set_modelbranch did not.Not a goal
session/set_config_optionfor harnesses registered fromharnesses.json. That registration mechanism is not onmain(feat(acp): harness descriptors, registry, and capability gates as config data #8255, feat(backends): config-defined AI backends, per-chat and per-subagent selection #11863, feat(acp): add a Custom ACP backend the operator names in config #12166 and feat: add experimental custom acp backend setup #13320 were all closed unmerged), so there is nothing onmainto apply the proposed predicate to.AcpClient.set_model(dedicated runtime). Its_send_requestalso writes the frame without waiting, and the comment atclient.py:1407still describes kiro-cli as usually not answering; aligning that path is a separate change.What changed (motivation → approach → change)
else:branch ofAcpSessionHandle.set_modelcallsself._runtime.send_request(METHOD_SET_MODEL, ...), which only writes the frame. The config-option and KAS branches of the same method both wait for the reply._send_awaited+_wait_for_response(req_id, timeout=_SET_MODEL_REPLY_TIMEOUT)(3s), the pairset_config_optionuses.model_pin_refused, and_model/_resolved_model_id/ the meter keep the serving model. This is the contract the config-option branch already follows, so_try_live_model_switchreads it back and answersAcpModelUnavailable(a 4xx) with the conversation still alive, and every substitute/startup caller (served-default fallback, cold start,new_conversation) stays on its model without raising.AcpProcessDied) still raises.docs/system-specs/common/model-selection.mdrecords the wait and the refusal contract.Backwards compatibility
Compatible: an accepted switch and an unanswered one are both recorded as before; only an explicit error reply changes outcome, from "recorded as switched" to "refused, session keeps its model", which is what every caller already does for a config-option refusal.
Removes nothing: the change only adds a wait for a reply that was already being sent.
Tests
test/test_acp_runtime.py::test_handle_set_model_refusal_is_recorded_and_keeps_the_serving_model(new): an error reply setsmodel_pin_refusedand leaves_model/_resolved_model_idon the old model.test/test_acp_runtime.py::test_handle_set_model_without_a_reply_records_the_switch(new): an adapter that never answers still has the switch recorded.test/test_acp_runtime.py::test_refused_served_default_switch_does_not_fail_a_kiro_session_start(new): a refused served-default switch right aftersession/newdoes not raise and the session stays onauto.test/test_codex_effort_pair_model_switch.py::TestTheDashboardPickReadsTheRefusalBack::test_a_refused_set_model_on_kiro_answers_unavailable_without_a_reset(new): a dashboard pick on kiro that the adapter refuses raisesAcpModelUnavailablerather than returning to the reset path, and the session keeps its model.session/set_model(which is how the missing await went unnoticed):test_handle_set_model,test_set_model_syncs_resolved_model_id,test_set_model_rebases_context_stats,test_load_session_moves_a_resumed_session_off_an_unserved_default(via a new_replying_sendhelper),test_acp_session_handle_capabilities.py::test_kiro_still_uses_the_set_model_request, andtest_codex_effort_pair_model_switch.py::...::test_auto_is_never_reported_as_refused.-n0: the changed test files plustest_model_served_default,test_acp_session_provider,test_kas_model_select_capture(735 passed). black / isort / flake8 / mypy on the changed files andscripts/check_comment_history.pypass.Manual verification
Probed a live
kiro-cli acp2.28.0 on Linux (initialize→session/new→session/set_model): it answered an advertised model in 0.04s and an unknown idno-such-model-xyzin 0.03s, both with{"result": {}}. So the wait costs kiro-cli a few tens of milliseconds, and kiro-cli 2.28 does not itself return an error for an unknown id. The refusal path is for adapters that do answer with an error, as in the issue'sclaude-agent-acpreport; that adapter was not run here. Not verified on macOS or Windows.Related Issues
Refs #18169
Pattern harvest
Rule candidate: review-prompt
Pattern: a JSON-RPC request sent with
send_requestand never paired with_wait_for_responselets its error reply be dropped while the caller records success.Decision for the owner
The decision is whether an explicit error reply to
session/set_modelon the shared runtime should be reported as a refused pick, rather than recorded as a switch. This draft records it inmodel_pin_refusedlike the config-option branch (a 4xx, no reset), still records the switch when the adapter does not answer, and leaves the issue'sharnesses.jsonpredicate out because that mechanism is not onmain.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.