Skip to content

fix(acp): wait for the session/set_model reply on the shared runtime - #18248

Merged
buluoray merged 1 commit into
mainfrom
fix/await-set-model-18169
Oct 9, 2026
Merged

buluoray merged 1 commit into
mainfrom
fix/await-set-model-18169

Conversation

@buluoray

@buluoray buluoray commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A model switch on a shared-runtime session that the adapter answers with an error is reported as refused, with the conversation kept, instead of being recorded as if it had worked.

AcpSessionHandle.set_model sends session/set_model (the kiro-cli path, and the path every backend outside the config-option set and KAS falls through to) without waiting for the reply, then records the new model unconditionally. An error reply reaches no consumer; the pre-turn drain discards it (pre-turn drain discarded N leftover frame(s) ... terminal-shaped responses). The dashboard and the usage log then show the pick while the session keeps serving the previous model, and _try_live_model_switch never falls back.

Why it matters

A model pick that silently does nothing misreports what serves every following turn: cost, quality and context-window math are all attributed to a model that is not running, and nothing tells the user. The set_config_option and KAS branches of the same method already wait for the adapter's answer; only the session/set_model branch did not.

Not a goal

What changed (motivation → approach → change)

  • Symptom: a refused switch reads as a success. Root cause: the else: branch of AcpSessionHandle.set_model calls self._runtime.send_request(METHOD_SET_MODEL, ...), which only writes the frame. The config-option and KAS branches of the same method both wait for the reply.
  • Change: that branch now uses _send_awaited + _wait_for_response(req_id, timeout=_SET_MODEL_REPLY_TIMEOUT) (3s), the pair set_config_option uses.
    • Explicit error reply: recorded in model_pin_refused, and _model / _resolved_model_id / the meter keep the serving model. This is the contract the config-option branch already follows, so _try_live_model_switch reads it back and answers AcpModelUnavailable (a 4xx) with the conversation still alive, and every substitute/startup caller (served-default fallback, cold start, new_conversation) stays on its model without raising.
    • No reply within the wait: read as accepted, and the model is recorded exactly as before, so an adapter that never answers this request loses nothing but the bounded wait.
    • A dead runtime (AcpProcessDied) still raises.
  • docs/system-specs/common/model-selection.md records the wait and the refusal contract.

Backwards compatibility

Compatible: an accepted switch and an unanswered one are both recorded as before; only an explicit error reply changes outcome, from "recorded as switched" to "refused, session keeps its model", which is what every caller already does for a config-option refusal.

Removes nothing: the change only adds a wait for a reply that was already being sent.

Tests

  • test/test_acp_runtime.py::test_handle_set_model_refusal_is_recorded_and_keeps_the_serving_model (new): an error reply sets model_pin_refused and leaves _model / _resolved_model_id on the old model.
  • test/test_acp_runtime.py::test_handle_set_model_without_a_reply_records_the_switch (new): an adapter that never answers still has the switch recorded.
  • test/test_acp_runtime.py::test_refused_served_default_switch_does_not_fail_a_kiro_session_start (new): a refused served-default switch right after session/new does not raise and the session stays on auto.
  • test/test_codex_effort_pair_model_switch.py::TestTheDashboardPickReadsTheRefusalBack::test_a_refused_set_model_on_kiro_answers_unavailable_without_a_reset (new): a dashboard pick on kiro that the adapter refuses raises AcpModelUnavailable rather than returning to the reset path, and the session keeps its model.
  • Mutation check: restoring the unawaited send reddens the three refusal tests; raising on the error reply instead of recording it reddens the same three; no longer catching the timeout reddens the no-reply test.
  • Updated doubles that never answered session/set_model (which is how the missing await went unnoticed): test_handle_set_model, test_set_model_syncs_resolved_model_id, test_set_model_rebases_context_stats, test_load_session_moves_a_resumed_session_off_an_unserved_default (via a new _replying_send helper), test_acp_session_handle_capabilities.py::test_kiro_still_uses_the_set_model_request, and test_codex_effort_pair_model_switch.py::...::test_auto_is_never_reported_as_refused.
  • Ran by path with -n0: the changed test files plus test_model_served_default, test_acp_session_provider, test_kas_model_select_capture (735 passed). black / isort / flake8 / mypy on the changed files and scripts/check_comment_history.py pass.

Manual verification

Probed a live kiro-cli acp 2.28.0 on Linux (initialize → session/new → session/set_model): it answered an advertised model in 0.04s and an unknown id no-such-model-xyz in 0.03s, both with {"result": {}}. So the wait costs kiro-cli a few tens of milliseconds, and kiro-cli 2.28 does not itself return an error for an unknown id. The refusal path is for adapters that do answer with an error, as in the issue's claude-agent-acp report; that adapter was not run here. Not verified on macOS or Windows.

Related Issues

Refs #18169

Pattern harvest

Rule candidate: review-prompt
Pattern: a JSON-RPC request sent with send_request and never paired with _wait_for_response lets its error reply be dropped while the caller records success.

Decision for the owner

The decision is whether an explicit error reply to session/set_model on the shared runtime should be reported as a refused pick, rather than recorded as a switch. This draft records it in model_pin_refused like the config-option branch (a 4xx, no reset), still records the switch when the adapter does not answer, and leaves the issue's harnesses.json predicate out because that mechanism is not on main.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 8, 2026
@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 03f69739016b86acaed61c5371667320f6c777d5 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 03f69739016b86acaed61c5371667320f6c777d5 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Before merge, check one thing: an adapter that never answers session/set_model now waits up to 3s at session start (ensure_served_default) and on every switch.

What this change ships

Inventory (4 items) — 4 justified

Intent: when someone switches models on a shared-runtime session and the adapter refuses, the switch is reported as refused and the conversation is kept. This is a FIX (Refs #18169; new tests fail on base).

  1. A refused session/set_model now shows up as a refused pick (a 4xx), and the session keeps the model it is serving — justified
  2. Each model switch on this branch now waits for the adapter's reply, bounded at 3s — justified
  3. No reply within the wait still records the switch, as it did before — justified
  4. The model-selection.md spec records the wait and how a refusal is handled — justified

[FIRST-PRINCIPLES-REVIEWED] 03f6973

Description read

[DESCRIPTION-READ] 64fd6b2d9bbe65666df8e507007634e3f41c7cdfb0d764fddcf6a931a6b0661f

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 03f69739016b86acaed61c5371667320f6c777d5 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 03f6973

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 03f69739016b86acaed61c5371667320f6c777d5: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 03f69739016b86acaed61c5371667320f6c777d5 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check that a transient error reply (AcpPromptBusy, throttle, 5xx) caught by except AcpError should really be reported to the user as "model unavailable".

[DESIGN-REVIEWED] 03f6973

Description read

[DESCRIPTION-READ] 46edad38492ac44728ab7b724a5eb84ef1001a5d32b9a2ee997b20d858876e69

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 03f69739016b86acaed61c5371667320f6c777d5 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 03f6973

Verdict parsed from the review's SHA-scoped output markers for commit 03f69739016b86acaed61c5371667320f6c777d5.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 03f69739016b86acaed61c5371667320f6c777d5: <one-sentence reason>

@buluoray
buluoray force-pushed the fix/await-set-model-18169 branch from 644e3fe to af4353e Compare October 8, 2026 22:42
AcpSessionHandle.set_model sent session/set_model without waiting for the
answer, then recorded the new model unconditionally. An adapter's error
reply reached no consumer (the pre-turn drain discarded it), so a refused
switch read as a success: the dashboard and usage log showed the pick while
the session kept serving the previous model.

The request now goes through _send_awaited + _wait_for_response like the
set_config_option branch. An explicit error reply is recorded in
model_pin_refused and the session keeps its model, the contract the
config-option branch already follows, so the dashboard answers the pick
with a 4xx instead of resetting the conversation. No reply within the
bounded wait is read as accepted and recorded, as before.
@buluoray
buluoray force-pushed the fix/await-set-model-18169 branch from af4353e to 03f6973 Compare October 8, 2026 23:20
@buluoray
buluoray marked this pull request as ready for review October 9, 2026 02:12
@buluoray
buluoray requested review from a team and KiroCrewCaptain as code owners October 9, 2026 02:12
@buluoray
buluoray requested a review from bolichen97 October 9, 2026 02:12
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 9, 2026

@CrysisDeu CrysisDeu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at 03f6973. Read the full diff plus the owning spec; no findings.

@buluoray
buluoray enabled auto-merge (squash) October 9, 2026 05:28
@buluoray
buluoray merged commit 1a21871 into main Oct 9, 2026
104 checks passed
@buluoray
buluoray deleted the fix/await-set-model-18169 branch October 9, 2026 05:29
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants