Skip to content

fix(dashboard): let an empty chat change member until its first send - #14702

Closed
fabiohof wants to merge 1 commit into
kirodotdev:mainfrom
fabiohof:fix/dashboard-default-member-pin
Closed

fabiohof wants to merge 1 commit into
kirodotdev:mainfrom
fabiohof:fix/dashboard-default-member-pin

Conversation

@fabiohof

@fabiohof fabiohof commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A new dashboard chat can choose another agent before its first message and sends that first message to the selected Crew Member.

Goal: A new dashboard chat can switch to any agent until its first message, regardless of how its Crew Member is chosen, and runs on the chosen member's private memory from that first message.

A plain empty dashboard chat keeps its private member choice reversible until the first message. The first send publishes the chosen member's execution identity.

Why it matters

A member choice stays editable while the chat has no message or provider context. The first send binds the conversation to the exact private member identity selected for it.

Not a goal

  • Changing the 409 member_session_pinned response after a session is bound.
  • Changing member threads, linked tabs, or channel-born tabs.
  • Changing the dashboard create payload or agent picker.

What changed (motivation → approach → change)

  • Plain empty dashboard chats created with a private member through the configured default, create request, picker, or /agent <name> persist member_choice without publishing the member execution record. Another agent can be selected before the first send.
  • The first send validates the recorded member id and store before the user row, binds that identity, and clears the pending choice. Runner-side validation covers app, taskrunner, and queued-prompt entry points without publishing their bindings before persistence.
  • Slot metadata uses the shared codec for full saves, empty-window merges, open-slot restore, recent restore, and History resume. Template and member namespaces remain distinct through eager spawn and dispatch.
  • Agent switches withdraw incompatible template selections for empty private-member picks and restore agent, namespace, choice, and prior execution metadata on rollback or cancellation.
  • /api/chat and /v1/chat/completions return 503 store_unavailable before appending a user row when the recorded member is missing, downgraded, or replaced.
  • OpenAI-compatible first sends repeat the slot-busy check after member validation with no await before append and dispatch. Concurrent requests dispatch one turn; the other returns 409 slot_busy.
  • The dashboard system spec describes the empty-chat selection and first-send binding contract.

Backwards compatibility

Compatible: request shapes are unchanged, member_choice is optional persisted metadata, and established member-bound sessions retain their pin.

Removes nothing: existing member threads, linked tabs, channel tabs, agent controls, and response contracts remain available.

Tests

  • Added concurrent OpenAI first-send coverage that coordinates the validation yield and asserts one dispatch plus one 409 slot_busy response.
  • Covered create/default/menu selection, persistence modes, restart and History restore, same-name replacement, unavailable choices, namespace collisions, cancellation, rollback, caller identity, and runner entry points.
  • 3,953 tests pass across 68 targeted test files, including every changed test file and test/test_openai_compat*.py.
  • Backend lint and every Fast Gate step pass.

Manual verification

Not performed against a running gateway. Verify a New chat with a member default can switch agents before sending, that the first message uses the selected member, and that a later member switch returns 409.

Screenshots / video

Why no screenshot: The change is backend-only and preserves the existing dashboard controls and payloads.

Related Issues

Fixes #14701

Pattern harvest

Rule candidate: review-prompt
Pattern: persist immutable selection identity when publication is deferred, validate it at publication, and preserve the explicit selection namespace across every resolver.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

@fabiohof
fabiohof requested a review from a team as a code owner September 28, 2026 13:39
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 28, 2026
@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — 🔴 changes requested (blocking)

Reviewed 6aa0df7468243bdd6df3ad37c3a04d124d799bc6 via the fork AI-review pipeline; updated in place on each push.

1 of 2 blocking finding(s) are security-class and were withheld from adjudication, so the blocking verdict stands.

BLOCKING -- src/kiro_crew/dashboard/chat_handlers.py:1553 -- Restored transcript metadata can manufacture private-store authority

expected_member=None if fresh_choice else chosen_member,
Agent-written empty transcript naming a peer in agent and member_choice, without an execution carrier -> History resume restores both -> owner sends without reselecting -> pin_private_agent_store publishes with vouch=True, authorizing the peer’s private memory.
Anchor: residual/security
Fix: Require gateway-protected evidence of the pending selection before vouching a restored choice; otherwise require explicit owner re-selection. Matching writable metadata against config does not establish authorization.

BLOCKING -- test/test_openai_compat.py:1032 -- Unbounded race barriers hang when dispatch never occurs

await dispatched.wait()
First request returns without dispatching -> second validation waits forever for dispatched -> the unbounded request gather never reaches assertions, leaving worker termination as the failure mechanism.
Anchor: tests-are-deterministic
Fix: Bound the event waits and request/background-task gathers with asyncio.wait_for so a missed signal fails locally.

[BLOCK-MERGE] 6aa0df7
[GPT-REVIEWED] 6aa0df7

Adjudication (Opus 5.5) — is blocking on each finding proportionate?

F2 harm: LOW to MEDIUM. If the second request's validation never sees a dispatch, a test hangs and CI times out, but the cost is a lost CI run, not anything that ships. The unbounded awaits are confirmed in the diff at patch lines 3747, 3749, 3751, 3772 and 3776 (test/test_openai_compat.py around line 1032). The fix is cheap: wrap those awaits in asyncio.wait_for.
It is anchored to tests-are-deterministic, which is blocking: true in .review-base-rules/AUTOSDE.yaml:361-362, and its test/**/*.py pattern covers this file. That rule's flag overrides my weighing.

F1 (fenced): I did not verify every condition at file:line, including whether the agent can write the transcript fields that History resume restores. Without that, the evidence a FLAG requires is incomplete.

[ADJUDICATION] 6aa0df7468243bdd6df3ad37c3a04d124d799bc6 total=1 uphold=1 downgrade=0
UPHOLD F2 test/test_openai_compat.py:1032 reason=autosde-blocking-rule
[GPT-ADJUDICATED] 6aa0df7468243bdd6df3ad37c3a04d124d799bc6

[ADJUDICATION-FENCED] 6aa0df7468243bdd6df3ad37c3a04d124d799bc6 fenced=1 flagged=0
UPHOLD-FENCED F1 src/kiro_crew/dashboard/chat_handlers.py:1553 -- An agent-written transcript naming a peer member is a plausible input, and it would get a vouch for that peer's private memory store, so the governance-ceiling claim stands without a complete rarity record.
[GPT-ADJUDICATED-FENCED] 6aa0df7468243bdd6df3ad37c3a04d124d799bc6

@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — 🟡 CONCERNS

Premise-level review of 6aa0df7468243bdd6df3ad37c3a04d124d799bc6 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

Delaying the member bind until the first send adds a new persisted member_choice state and two first-send refusal paths. Re-pinning the empty chat at switch time was never weighed.

Not justified as shipped

  • Item 2 — oversized: the fix (dashboard: New chat with a member default_agent cannot change agent #14701) only needs an empty chat to accept a second pick. To get that, the PR adds a durable member_choice key, a codec field and an entry in SLOT_OWNED_META_KEYS. Items 3, 4 and 6 exist only because the bind now waits for the first send. The description never says why the empty chat couldn't swap its member record at switch time instead. The PR's own withdraw_new_agent_selection already removes a member record by exact identity.
    Subtraction: on an empty plain chat, withdraw the prior member record and pin the new member at switch time. Delete member_choice, its codec field, member_choice_refusal, _may_run_unbound_member/_runs_unbound_member and the OpenAI refusal and busy recheck.
    Clears when: the description names the invariant that forbids replacing a member record on an empty chat, or the deferral is removed.
  • Item 7 — one consumer, generalized: selection_changes: list[SelectionChange] has exactly one caller (api_chat), which reads only [-1].
    Subtraction: return the single SelectionChange instead of filling a list.
    Clears when: the list parameter is replaced by a single value.

What this change ships

Inventory (8 items) — 4 justified

Intent: let a user change their mind about a new chat's Crew Member until they send the first message. FIX.

  1. An empty chat with a member pick can switch agents; it no longer gets 409 member_session_pinned — justified
  2. The member choice is saved unbound and bound on the first send — oversized
  3. A first send through /api/chat or /v1/chat/completions returns 503 when the chosen member is missing or replaced — justified
  4. An OpenAI-compatible first send checks again that the slot is free after the new await — justified
  5. A template pick stays a template through eager spawn and dispatch — rides along
  6. The chat runner refuses a turn whose member differs from the recorded choice — justified
  7. The pin helper passes the change back through a list parameter — one consumer, generalized
  8. The pin-at-switch block is deleted (about 80 lines) — justified

[FIRST-PRINCIPLES-REVIEWED] 6aa0df7

Description read

[DESCRIPTION-READ] 50ed944092d9d86eea23f059cf189f1d110b4f154291d7156c9861c2311af0a0

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — 🟡 CONCERNS

Design-level review of 6aa0df7468243bdd6df3ad37c3a04d124d799bc6 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

The rewritten tests reverse a pinned behaviour: an empty chat used to bind its member at create or pick time, and now the first send binds it. No decision record covers either rule.

Watch

  • A pinned behaviour is reversed with only the issue as authority. test_owner_second_member_pick_on_pinned_empty_chat_is_refused and test_owner_agent_pick_on_empty_chat_pins_private_memory (whose docstring reads "The agent menu on an empty chat is a grant door, like create-with-agent") pinned binding at pick time. Both are rewritten to assert the opposite. Nothing in docs/decisions/ records either rule, so the issue is the only thing that authorizes the change. If the old rule was deliberate, this silently undoes it.
    Clears when: a maintainer confirms on #14701 that pick-time binding was not a product decision, or a decision entry records the new first-send binding contract.
  • One contract is now enforced in three places, each a little differently. The first-send member-choice check lives in api_chat (the binds / revalidates / fresh_choice / chosen_member / owner_request lattice), in openai_compat._unbound_member_refusal (which also writes slot.member_choice as a side effect), and in the _run_chat mismatch raise. Today they agree only by inspection. A new entry point, or a change to one site, can let a replaced same-name member run on its private store, or refuse a valid chat, with no shared test to catch it.
    Clears when: both endpoints call one helper that validates and records the choice, and a parametrized test runs the same replaced/unavailable/fresh cases through /api/chat, /v1/chat/completions and the runner.

Suggestions

  • Keep _unbound_member_refusal free of writes. Have it return the resolved choice and let the caller commit it after the post-await slot.running recheck, so a refused or busy request leaves no trace on the slot.

[DESIGN-REVIEWED] 6aa0df7

Description read

[DESCRIPTION-READ] be26584b6181d3c6236a7f61ec79f81ead0bae9a0dd2471199a073ad13da4af0

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 28, 2026
@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 6aa0df7468243bdd6df3ad37c3a04d124d799bc6 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 6aa0df7

@fabiohof
fabiohof force-pushed the fix/dashboard-default-member-pin branch from f35b8a5 to c5211bb Compare September 28, 2026 14:41
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 28, 2026
@fabiohof
fabiohof force-pushed the fix/dashboard-default-member-pin branch from c5211bb to 3251ef7 Compare September 28, 2026 15:35
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 28, 2026
@fabiohof
fabiohof force-pushed the fix/dashboard-default-member-pin branch from 3251ef7 to 5f6f86c Compare September 28, 2026 16:27
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 28, 2026
@fabiohof
fabiohof force-pushed the fix/dashboard-default-member-pin branch from 5f6f86c to 7245496 Compare September 28, 2026 18:02
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 28, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: checking Automated validation is still running labels Sep 30, 2026
@fabiohof
fabiohof force-pushed the fix/dashboard-default-member-pin branch from c568222 to 7a42019 Compare October 3, 2026 15:06
@github-actions github-actions Bot added merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Oct 3, 2026
@fabiohof
fabiohof force-pushed the fix/dashboard-default-member-pin branch from 7a42019 to d507d28 Compare October 4, 2026 20:09
@github-actions github-actions Bot added readiness: checking Automated validation is still running merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Oct 4, 2026

@cruisercohen cruisercohen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes on head d507d28d3acc60be4cb8afc240e4d9b78d175a16. The targeted backend tests pass locally against this branch's own base (513 tests across the touched test files plus test_openai_compat*), but the PR can't land as it is:

  1. Merge conflicts with main. The branch is about 770 commits behind, and chat_handlers.py, chat_persistence.py, openai_compat.py, chat_api/resume.py, slot_persistence/metadata_line.py, the dashboard system spec and four of the touched test files all conflict. Please rebase, then re-run the suites above on the rebased head.
  2. Upheld blocking finding in openai_compat.api_completions. The new _unbound_member_refusal awaits two asyncio.to_thread calls on every first send to an empty slot. Those awaits sit after the slot.running busy check and before slot.append("user", ...) and the turn dispatch, so two concurrent first sends can both pass the busy check and both drain the same pending queue. Re-check slot.running after the refusal check and return 409 slot_busy, with no await between that check and the dispatch, and add a test that runs two concurrent first sends.
  3. Behaviour change needs a maintainer decision. This replaces bind-at-pick with first-send binding and rewrites test_owner_second_member_pick_on_pinned_empty_chat_is_refused, which asserted the old behaviour on purpose. A maintainer has to confirm the new rule on #14701 before this can merge.

Backend Tests (Windows) (7) is also red on this head, but its four failures are in test_run_bg_oneliner.py, which this PR doesn't touch, so they look unrelated. Re-running after the rebase should show whether that's true. On scope: this fixes #14701 (an empty chat whose default is a member can't change agent). It doesn't add the crew picker to the New Chat flow that #4560 asks for, so #4560 should stay open.

Defer private member binding for plain dashboard chats until the first
send. Validate private choices at create and switch time, record each
choice as `member_choice`, and preserve empty chats across restart.
Withdraw template selections when a member is picked and restore
agent-menu metadata together on rollback.

Use `slot_defers_member_binding` as the shared empty-chat predicate.
Keep `_may_run_unbound_member` config-only and perform named-store path
checks through `asyncio.to_thread` in the locked first-send path.

Apply `member_choice_refusal` to first sends through `/api/chat` and
`/v1/chat/completions`. Reject missing, removed, downgraded, or replaced
choices before the user row with `503 store_unavailable`. Record a member
adopted by the first send so the runner validates the same identity for
app, taskrunner, and queued-prompt dispatch.

Recheck OpenAI slot busyness after member validation and before append or
dispatch. Concurrent first sends dispatch one turn and return `409
slot_busy` for the other request.

Publish one private execution record before the first owner user row and
withdraw it if the slot rebounds. Drain first-send publication under
cancellation and restore its exact prior selection before releasing the
send locks. Preserve explicit member and template namespaces through
eager spawn and turn dispatch.

Persist `member_choice` through the shared slot metadata codec and cover
create, switch, restart, concurrency, rollback, caller identity,
unavailable selections, same-name replacement, History resume, namespace
collisions, event-loop isolation, adopted members, and non-endpoint first
turns.

Black-format `test/test_chat_slot_create_folder.py` and remove it from
`.github/black-baseline.txt`.

Tests: 3,953 passed across 68 targeted test files. Backend lint and Fast
Gate pass.
@fabiohof
fabiohof force-pushed the fix/dashboard-default-member-pin branch from d507d28 to 6aa0df7 Compare October 12, 2026 03:07
Copilot AI balanced review requested due to automatic review settings October 12, 2026 03:07

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision 6aa0df7468243bdd6df3ad37c3a04d124d799bc6 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions github-actions Bot added merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Oct 12, 2026

@cruisercohen cruisercohen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed on 6aa0df7. Thanks for the slot_busy re-check and its concurrency test; that ask is met. Three gaps remain.

  1. Trust boundary (chat_handlers.py:1553). A member choice restored from the transcript is enough, on the first send, for pin_private_agent_store to pin that member's private memory store to the session. The transcript is agent-writable, and main's pin deliberately refuses a member name recovered from history for that reason. Please require an explicit owner re-selection before the first send binds, or check that the restored member is one this chat was legitimately bound to, and add a test that a transcript-only member choice does not get a private-store pin.
  2. Merge conflict: the branch conflicts with main again and needs a rebase.
  3. The new selection_kind argument breaks test_eager_spawn_config_load_off_loop.py, which is red on Linux, Windows and macOS on this head.

Our earlier ask for a maintainer ruling on first-send binding (#14701) also still stands.

@fabiohof

Copy link
Copy Markdown
Contributor Author

Closing this because #18342 changed a New chat without an agent to start on the default template, so current main no longer binds it immediately to the default Crew Member and this fix is no longer needed.

@fabiohof fabiohof closed this Oct 20, 2026
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dashboard: New chat with a member default_agent cannot change agent

3 participants