Skip to content

fix(kas): keep the model list retrying after a timed-out kiro-cli check - #36249

Merged
buluoray merged 1 commit into
mainfrom
crew/slack/kas-model-list-keeps-retrying
Nov 13, 2026
Merged

buluoray merged 1 commit into
mainfrom
crew/slack/kas-model-list-keeps-retrying

Conversation

@buluoray

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: After a restart, a timed-out kiro-cli check no longer leaves the model list stuck on "Auto"; the dashboard keeps its last list and reads again until the check passes.

On Kiro Agent signed in through Kiro Crew, the gateway answers the model-list read from its own recording. After a restart it has no recording, so it answered 200 with "Auto" alone. That is right when kiro-cli is signed out. It is wrong when the kiro-cli check only timed out on a fresh process: the dashboard saved "Auto" over the list it held, treated the list as live, and stopped reading. Nothing read the list again until a chat started.

Why it matters

A person with a working setup opens Settings > Chat or a model picker after a restart and sees only "Auto", with no notice and no retry. Before #35890 the same moment answered 503: the dashboard kept its list, showed the timeout notice and read again every 8 seconds, so the list came back by itself. #35890 fixed the signed-out case and, in this narrow case, took that recovery away.

Not a goal

What changed (motivation → approach → change)

Symptom: the model list stays on "Auto" after a restart, with no notice.

Cause: api_models_kas_crew_relay serves the recording whenever kiro-cli's gate refuses on this relay. It treats a timed-out check like a signed-out one, and a cold recording then answers 200 with "Auto".

Change: the handler now passes the gate's own refusal to api_models_kas_crew_relay. When that refusal is the kiro_probe_timed_out answer and nothing is recorded, the 503 stands. The dashboard then keeps its list, shows the timeout notice and reads again. is_probe_timeout_refusal in kiro_readiness.py reads the code from the answer the gate gave, not from the service again.

Situation Before After
Timed-out check, nothing recorded 200 "Auto" 503 kiro_probe_timed_out
Timed-out check, list recorded the list the list
Signed-out kiro-cli, nothing recorded 200 "Auto" 200 "Auto"
The recording turns cold mid-read (account switch) 200 "Auto" 503 if the check timed out, else 200 "Auto"

What this leaves: a person signed in through Kiro Crew only still sees "Auto" until the first chat after a restart. A person whose kiro-cli check hangs sees the timeout notice until the check passes.

The two spec paragraphs that describe this answer (model-selection.md, acp-client.md) say so.

builder-security checked: this edits a handler on a route that already serves the dashboard. It adds no route, bind, listener or access rule, and changes no sign-in check. It answers 503 instead of 200 in one case, and returns less, not more.

Backwards compatibility

Compatible: the one answer that changes is 200 "Auto" to 503 kiro_probe_timed_out, for a timed-out check with nothing recorded. That is what main answered before #35890. No stored value, route or setting changes.

Reader: website/src/providers/adapters/acp.ts:fetchAvailableModels -- chat, Settings > Chat -- a 503 keeps the cached list, marks the list degraded and reads again every 8 seconds; the case this PR restores.
Reader: website/src/components/JobForm.tsx:modelList -- crew page -- a failed read leaves the list empty, as for the gate's other 503s.
Reader: website/src/apps/mochi/panel/panelBridge.ts:getModels -- app -- a non-OK answer returns [].
Reader: src/kiro_crew/instances/ssh_tunnel_manager_peer.py:1005 -- crew page, remote Crews -- already handles a 503 from /api/models.

Tests

test/test_api_models_kas_bundle.py::TestTheCrewSignInRelay, three new tests:

  • test_a_timed_out_check_with_nothing_recorded_keeps_the_gates_503: fails on main (200).
  • test_a_timed_out_check_keeps_its_503_when_the_account_switches_mid_read: fails on main (200).
  • test_a_timed_out_check_still_serves_a_recorded_list: passes before and after; it holds the behaviour that must stay.

The existing tests in the class still hold the signed-out and launcher cases. Related-test run: 25877 passed, 17 failed. The same 17 fail on unchanged main code on this computer, so they are not from this change.

Proof: PROVEN test/test_api_models_kas_bundle.py::TestTheCrewSignInRelay::test_a_timed_out_check_with_nothing_recorded_keeps_the_gates_503, test/test_api_models_kas_bundle.py::TestTheCrewSignInRelay::test_a_timed_out_check_keeps_its_503_when_the_account_switches_mid_read (head 0d49c23)

Manual verification

Not run on a live gateway. The state needs a Kiro Crew sign-in and a kiro-cli check that times out on a fresh process, which the tests set up through agents.api_models with the gate's own timed-out answer.

Related Issues

Closes #36208

Sibling sweep

Swept: git grep -n "api_models_kas_crew_relay|_serve_recording" -- src
Sibling: src/kiro_crew/dashboard/handlers/models_kas_bundle.py:145 -- not affected -- the bundle route runs no kiro-cli check, so it has no timed-out answer to keep
Sibling: src/kiro_crew/dashboard/handlers/models_kas_bundle.py:217 -- tracked in #36247 -- a signed-out kiro-cli still gets "Auto" while cold

Pattern harvest

Not generalizable: one answer treated a timed-out check like a signed-out one; no other site reads the gate's refusal.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

On a Kiro Agent relay that starts on Kiro Crew's sign-in alone, a
refused kiro-cli gate with no recorded model list answers 200 with
"auto" alone. That is right for a signed-out kiro-cli. It is wrong
when the kiro-cli check only timed out on a fresh process: the
dashboard saved "auto" over the list it held and stopped reading, so
the picker stayed on "Auto" until a chat started.

A timed-out check with nothing recorded now keeps the gate's 503. The
dashboard keeps its last list, shows the timeout notice and reads
again, as it did before the relay answer existed. A recorded list is
still served on a timeout, and a signed-out kiro-cli still gets
"auto" while cold.

Closes #36208
@buluoray
buluoray requested review from a team and KiroCrewCaptain as code owners November 13, 2026 00:37
@buluoray
buluoray requested a review from mbriones98 November 13, 2026 00:37
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 0d49c23191c1280576579249f4a7a42657b62e20 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 0d49c23191c1280576579249f4a7a42657b62e20 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: this is a narrow 503-vs-200 change on one cold path, restores how main behaved before #35890, and a revert undoes it cleanly.

Notes

  • is_probe_timeout_refusal decides by re-parsing the gate's JSON body. It would be sturdier for reject_if_kiro_unverified to expose a structured reason, so a wording or shape change to the body cannot quietly turn this back into "Auto".
  • If the kiro-cli check keeps timing out (a broken install), a person signed in through Kiro Crew only now sees the timeout notice indefinitely instead of "Auto". The fix for that is Model list: a cold answer after a restart is saved as the list; mark it cold and keep reading #36247, so it should stay linked.

[DESIGN-REVIEWED] 0d49c23

Description read

[DESCRIPTION-READ] 8e031028f7b0fa4534c56bb7defc7ad3605459a8c526eafa493dcadd3d7e0f98

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 0d49c23191c1280576579249f4a7a42657b62e20 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks: this restores the 503 retry path that #35890 removed for one case, at mechanism level, and the change adds no surface past one helper with one caller.

What this change ships

Inventory (4 items) — 4 justified

Intent: after a restart, a person whose kiro-cli check timed out gets their model list back on its own instead of being stuck on "Auto". FIX.

  1. A timed-out kiro-cli check with nothing recorded now answers 503 kiro_probe_timed_out instead of 200 "Auto", so the dashboard keeps its list and retries — justified
  2. The same 503 stands when the recording turns cold mid-read after an account switch — justified
  3. New is_probe_timeout_refusal helper and exported KIRO_PROBE_TIMED_OUT_CODE, read from the gate's own answer — justified
  4. The model-selection and acp-client spec paragraphs now describe the timed-out exception — justified

[FIRST-PRINCIPLES-REVIEWED] 0d49c23

Description read

[DESCRIPTION-READ] cfe7728d96e7ae57c5bf6eff9db0586c767dcdcac66cab047357c2663c6cceeb

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 0d49c23191c1280576579249f4a7a42657b62e20 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 0d49c23

Verdict parsed from the review's SHA-scoped output markers for commit 0d49c23191c1280576579249f4a7a42657b62e20.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 0d49c23191c1280576579249f4a7a42657b62e20 false-positive: <reason, 20 to 500 characters>

@buluoray
buluoray enabled auto-merge (squash) November 13, 2026 00:40
@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 0d49c23191c1280576579249f4a7a42657b62e20 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 0d49c23

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 0d49c23191c1280576579249f4a7a42657b62e20 false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Nov 13, 2026

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: fix (6 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: a timed-out kiro-cli check keeps its 503 on a cold KAS model list read, with tests. Spec files changed as a ride-along (a minority of the diff on both file count and changed lines), not reviewed as a design decision: docs/system-specs/common/model-selection.md, docs/system-specs/modules/acp-client.md.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I read the whole diff on this head, and all checks are green.

  • The change is narrow. On KAS's relay started on Crew's sign-in alone, a cold read now keeps the gate's own 503 only when the gate's refusal is kiro_probe_timed_out. A sign-in wall still gets the recording or auto, as on main, and a recorded list is still served on a timeout (test_a_timed_out_check_still_serves_a_recorded_list).
  • This is not added strictness. The 503 is the answer main's gate already gives on every other harness for a timed-out check. On main, the cold auto here is saved over the list the dashboard holds and stops the retry, which is the bug in the linked issue. The dashboard already keeps its cached list while /api/models fails (website/src/components/useChatPanePickers.ts).
  • is_probe_timeout_refusal reads the code from the answer the gate already gave, so a probe that lands between two reads cannot change the decision. A body that is not JSON returns False, which keeps main's behaviour.
  • The account-switch path returns the same 503, and a test covers it.
  • No regex. The docs in model-selection.md and acp-client.md match the code.

Approving. Auto-merge is already armed by its holder, so I leave it as is.

@buluoray
buluoray merged commit 7e27c51 into main Nov 13, 2026
151 of 155 checks passed
@buluoray
buluoray deleted the crew/slack/kas-model-list-keeps-retrying branch November 13, 2026 07:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: passed Eligible automated validation passed for the current revision

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Model list shows only Auto after a gateway restart until a chat starts (Kiro Agent, Kiro Crew sign-in)

3 participants