Skip to content

fix(dashboard): reset the fallback walk on an agent switch - #19729

Merged
iamwhatever merged 1 commit into
mainfrom
fix/agent-switch-fallback-reset
Oct 13, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
fix/agent-switch-fallback-reset

Conversation

@iamwhatever

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Switching a dashboard slot to another agent starts it on that agent's own model configuration, not on the previous agent's model-fallback walk.

Switching a dashboard slot to another agent kept the previous agent's fallback recovery state (_fallback_candidate_idx, _fallback_walked, _active_fallback_model, _fallback_primary_model, _fallback_slot_model, _fallback_pick_gen). The replacement session could resume that stale walk and send a fallback model id that belongs to the old binding.

Why it matters

The next turn after an agent switch can start on an incompatible model instead of the target agent's default configuration.

Not a goal

  • Clearing the model on provider-reported in-turn agent switches; they keep the active pin.
  • Changing which model an agent defaults to.
  • Rewriting the stored model pin on a switch (the read side already scopes it).

What changed (motivation → approach → change)

Symptom: a stale fallback walk survives an agent switch. Root cause: api_chat_slot_agent commits the new agent and resets the session but never touches the provider-scoped fallback fields. Change: a new clear_slot_model_binding(slot) resets the walk when the resolved agent changes (name, or resolved member/template namespace on a name-only pick). The rollback path restores the prior walk only while this request still owns the reset, using the same identity-token ownership test the agent/workspace rollback already uses. The stored pin is deliberately kept. Same-agent resets leave model state unchanged.

This is a mechanical port of @Premshay's fix from #9115 onto current main; the commit keeps the original authorship.

Backwards compatibility

Compatible: only in-memory fallback state is reset on an agent change; no API field, key or persisted record changes.

Removes nothing: the change resets transient in-memory recovery state and removes no field, key, function or record.

Tests

  • test_api_chat_slot_agent_resets_fallback_and_persists_bindings: an agent change resets the walk and persists bindings.
  • test_api_chat_slot_agent_same_name_kind_switch_clears_the_binding: a name-only pick whose namespace moved counts as a rebind.
  • test_rebind_during_reset_restores_the_fallback_walk_it_reset: a refused switch restores the walk it reset, only while it owns it.
  • test_api_chat_slot_agent_same_agent_keeps_model_state: a same-agent reset leaves model state unchanged.
  • test_rolled_back_owner_switch_persists_the_restored_model: a rolled-back switch persists the restored model.
  • test_provider_reported_switch_preserves_the_pin (test_acp_tool_identity.py): a provider-reported in-turn switch keeps the pin.
  • The three new tests fail on main and pass here. test/test_dashboard_chat.py, test/test_acp_tool_identity.py, test/test_chat_slot_switch_atomicity.py: 1032 passed. black, isort, flake8 clean on the touched files.

Manual verification

N/A — unit coverage sufficient: the handler transaction, rollback and persistence paths are exercised directly by the dashboard suite.

Related Issues

Fixes #19725

Replaces #9115 (original fix by @Premshay, authorship kept in the commit).

Pattern harvest

Rule candidate: review-prompt
Pattern: any binding switch that retains provider-specific state must clear or revalidate that state before the next provider session starts.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

Reset provider-scoped fallback state when the resolved agent name or namespace changes. Preserve the stored model pin and restore the prior walk on refusal only while this transaction still owns the reset.

Rebase onto current upstream main without restoring its deleted remote-pick path or obsolete remote tests. Remove the unused frontend model response handoff.

Co-authored-by: Premshay <28099628+Premshay@users.noreply.github.com>
@iamwhatever
iamwhatever requested review from a team and KiroCrewCaptain as code owners October 13, 2026 06:31
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 4761273f4f824ede2aacb00d45dcdc44be870b15 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — 🟡 CONCERNS

Premise-level review of 4761273f4f824ede2aacb00d45dcdc44be870b15 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

clear_slot_model_binding is a second way to clear sticky fallback state. _clear_fallback_sticky_state already exists for this, and the two will drift apart.

Not justified as shipped

  • Item 1 — duplicate of src/kiro_crew/dashboard/chat_turn/model_fallback.py:299: the new helper rewrites the same three sticky fields that _clear_fallback_sticky_state already blanks, one by one: _active_fallback_model, _fallback_primary_model and _fallback_slot_model. I grepped for writes of _active_fallback_model = "" and _fallback_slot_model = "" in src/: there are 2 sites, this PR's helper and that function. A sticky field added to one later will be missed by the other. The name "clear_slot_model_binding" also says the opposite of its docstring: "the pin itself is kept."
    Subtraction: delete the three sticky-field writes in clear_slot_model_binding and send them through _clear_fallback_sticky_state, keeping only the walk counters and the ownership token here.
    Clears when: one function owns the list of sticky fields, or the PR names why _clear_fallback_sticky_state cannot serve an agent switch.

What this change ships

Inventory (5 items) — 4 justified

Intent: switching a dashboard slot to another agent starts that agent on its own model configuration, without carrying over the old agent's fallback walk. FIX (#19725).

  1. Switching agent (by name, or same name with a different member/template kind) clears the old fallback walk — duplicate of chat_turn/model_fallback.py:_clear_fallback_sticky_state
  2. A refused or rolled-back switch restores the walk it cleared, only while this request still owns it — justified
  3. Re-selecting the same agent leaves fallback state untouched — justified
  4. The model pin survives the switch and is persisted on rollback (pinned by tests) — justified
  5. A new test pins that a provider-reported in-turn switch keeps the pin — justified

[FIRST-PRINCIPLES-REVIEWED] 4761273

Description read

[DESCRIPTION-READ] a2c938c694697a4a32c3fc536ed98eea2265d54115a4b9ef150f7d1263ea6acc

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 4761273f4f824ede2aacb00d45dcdc44be870b15 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check that the refusal-fallback record (_refusal_fallback_*), which this switch does not reset, is dropped once the session moves off its candidate.

Suggestions

  • Put clear_slot_model_binding in chat_turn/model_fallback.py next to _clear_fallback_sticky_state. That module owns the fallback fields, and a new field would otherwise have to be added to two separate reset sites.

[DESIGN-REVIEWED] 4761273

Description read

[DESCRIPTION-READ] 635fccbcc4063792e1de8ffeb0389b7be4d0aa7e379b4c585cf2c24f5c56d9c9

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 4761273f4f824ede2aacb00d45dcdc44be870b15 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 4761273

Verdict parsed from the review's SHA-scoped output markers for commit 4761273f4f824ede2aacb00d45dcdc44be870b15.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 4761273f4f824ede2aacb00d45dcdc44be870b15: <one-sentence reason>

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 4761273f4f824ede2aacb00d45dcdc44be870b15 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 4761273

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 4761273f4f824ede2aacb00d45dcdc44be870b15: <one-sentence reason>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 13, 2026

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE at 4761273f4f

What it fixes

After a throttle fallback, switching a slot to another agent kept the old agent's fallback walk (candidate index, walked list, active fallback, primary). The replacement session could then resume or "restore" models from the wrong provider's chain. The walk is now reset whenever the resolved agent name or namespace changes. The stored model pin is kept.

What I verified

  • The reset is clear_slot_model_binding (chat_handlers.py:5072-5086). It runs only when agent_changed (:5827-5842), so a same-agent switch leaves model state as it was. A same-name pick that resolves to a different member/template counts as a change.
  • Rollback (:5903-5912) restores the snapshot only while the slot agent is still this commit's and _active_fallback_model is still the fresh _CommitToken("") the reset wrote. A later fallback activation (model_fallback.py:194) or sticky clear (:325) writes a different object, so rollback leaves that newer state alone.
  • _fallback_client_pick_epoch is not reset. That is harmless: it is re-snapshotted on the next activation (model_fallback.py:178-190).
  • Harness parity: no backend-specific branch.
  • Tests: test_dashboard_chat.py, test_chat_slot_switch_atomicity.py and test_acp_tool_identity.py give 1032 passed.
  • Mutations:
    • Removing the reset call reddens three tests: test_api_chat_slot_agent_resets_fallback_and_persists_bindings, ..._same_name_kind_switch_clears_the_binding and test_rebind_during_reset_restores_the_fallback_walk_it_reset.
    • Removing the kind term reddens test_api_chat_slot_agent_same_name_kind_switch_clears_the_binding.
    • Disabling the rollback restore reddens test_api_chat_slot_agent_reset_raise_before_pop_propagates and test_rebind_during_reset_restores_the_fallback_walk_it_reset.

Blocking

None.

Non-blocking follow-ups

  • Removing the ownership test (slot._active_fallback_model is committed_active_fallback_model) stays green, so that guard is not pinned by the suite. A test where a fallback activates between the reset and a refused switch would cover it.
  • Ownership is keyed on _active_fallback_model only. Several chat_runner.py sites (for example :17999, :18902) zero _fallback_candidate_idx/_fallback_walked without touching it. A rollback after one of those would put the older index/walk back. The window is only the 409/500 rollback arms, but this is worth a note or a per-field token.
  • No spec text describes what an agent switch does to fallback state. One sentence near the fallback budgets table in session.md would help.

Not verified

  • CI on this head and a live gateway run.
  • That the provider-side fallback marker is dropped together with the old session. I inferred this from the session reset; I did not trace it.

@iamwhatever
iamwhatever merged commit 019bde9 into main Oct 13, 2026
101 checks passed
@iamwhatever
iamwhatever deleted the fix/agent-switch-fallback-reset branch October 13, 2026 17:10
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agent switch keeps the old agent's model-fallback walk

3 participants