Skip to content

fix(dashboard): hand off the queue when a turn holder never starts - #23029

Merged
iamwhatever merged 1 commit into
kirodotdev:mainfrom
rubencu:fix/turn-holder-exit-queue-drain
Oct 29, 2026
Merged

iamwhatever merged 1 commit into
kirodotdev:mainfrom
rubencu:fix/turn-holder-exit-queue-drain

Conversation

@rubencu

@rubencu rubencu commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A message queued behind a dashboard turn that never starts still runs, and its chat gets the end-of-turn frame.

A message sent to a dashboard chat while a turn holds the slot waits in the queue, and the end of that turn starts it. Two holders end without starting a turn, and neither starts the queued message:

  • a monitor wake refused at its authorization before the runner (nudge_dashboard_fire._run_dashboard_turn): slot replaced or closing, crew/member mode refused, memory not persistent, or the wake's claim revoked;
  • an unattended (app-owned) turn whose wait for a background-turn permit gives up after 30 minutes, or is cancelled (DashboardState.run_background_turn).

The queued message then waits for the next send. On an app-owned session with no other sender, it never runs. The chat gets no chat_done for that cycle either. The abandoned wait also shows the turn-ceiling card ("This turn hit the 4-hour limit and was stopped ... raise Turn Time Limit") for a turn that never started.

Why it matters

App-owned sessions (crews, app workers) are the ones with no person to send again, so their queued work stops for good when the fleet sits at the background-turn cap or a monitor wake is refused. The wrong card points the operator at the turn-limit setting, which cannot help.

Not a goal

  • Changing what refuses a wake. Every slot, mode, memory and claim check stays as it is, and a refused wake stays refused.
  • Changing the background-turn cap, its 30-minute wait, or which turns it covers. A drained successor runs as every queue successor does, outside the cap.
  • The Issue Radar crew runtime, which starts its turns without spawn_guarded_turn and renders its own no-permit card.
  • Stop on a turn that is still being prepared (fix(dashboard): settle orphaned stop tasks #9118).

What changed (motivation → approach → change)

The queue is handed off only by _end_turn_tail, which every _run_chat exit runs (#16279). These two holders return or raise before _run_chat, so no tail runs, and turn_dispatch.finish_turn_task only reports the task's exception, treating the abandonment's plain TimeoutError as the turn ceiling.

  • turn_dispatch.mark_turn_unstarted(slot) records the generation of the turn a dispatch did not start (_ChatSlot._unstarted_generation). The refused-wake return and both run_background_turn exits (timeout and cancellation) call it.
  • spawn_guarded_turn's done callback, after writing the dispatch's own card, calls queue_cycle._hand_off_unstarted_turn, the one owner of this drain. It acts only while the ended task still holds slot.task under the marked generation, no send has reserved the floor, the slot is still the registered, open, local one, and the gateway is not shutting down. It takes slot.task with a one-shot cycle-end task, as _launch_synthesis does, so a send arriving meanwhile queues behind it. That task runs the tail's no-outcome hand-off: the memory-preparation and sign-in holds, then _hand_off_queue. The queue drains, or the cycle ends with its done row and chat_done. It consumes no reset, retires no wait and does not re-arm AutoNudge: the fire settles its own result.
  • run_background_turn raises BackgroundTurnAbandoned, a TimeoutError subclass with the same message, so the structured fire still settles BUSY. finish_turn_task reports it with its own card: the turn never started.
  • _authorize_dashboard_turn logs one INFO line per refused wake naming the reason. A revoked claim and a replaced slot logged nothing.

The exits mark themselves rather than the callback inferring "no tail ran": an inferred trigger also fires for every _run_chat test double, since a double skips the tail, and it changed the outcome of three unrelated suites.

docs/system-specs/modules/learn-cron-dashboard.md documents the hand-off, the abandonment card and the refusal log.

Backwards compatibility

Compatible: the abandoned wait still raises a TimeoutError with the same message, so every caller that catches one keeps doing so, and nothing refused today is admitted.

Removes nothing: the change adds a hand-off after two exits and replaces one card with an accurate one; no path, record, key or function is removed.

Tests

test/test_unstarted_turn_queue_handoff.py; the first four fail on main:

  • test_a_wake_refused_before_its_runner_hands_off_the_queued_send: a send queued while the wake authorizes runs once the claim is refused, and the INFO line names the revoked claim.
  • test_a_wake_refused_for_a_replaced_slot_logs_why: the replaced-slot refusal is logged.
  • test_a_turn_that_never_gets_a_permit_hands_off_the_queued_send: with a 0.05 s permit wait, the entry queued behind the abandoned turn runs.
  • test_a_chat_send_abandoned_at_the_cap_is_not_a_turn_timeout: an app send through /api/chat abandoned at the cap shows the never-started card and no turn-ceiling card, and its cycle ends with done and chat_done.
  • test_the_hand_off_is_owed_only_to_a_holder_that_never_started (unmarked, reserved, replaced, closing, remote, successor) and test_the_hand_off_takes_the_floor_before_it_yields: the gate, and the floor taken before the first await.

test/test_slot_running_readers.py lists the new reader and slot.task publisher, and test/test_state_parts_facade.py the new slot default.

Manual verification

N/A — both exits need a full background-turn cap or a refused monitor claim to reach; the tests drive the real fire path, run_background_turn, spawn_guarded_turn and the /api/chat route against a real DashboardState.

Related Issues

Closes #23015

Pattern harvest

Rule candidate: review-prompt
Pattern: a task published as slot.task that can end without running _run_chat must hand off the slot's queue, or a send queued behind it is stranded until the next send.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

@rubencu
rubencu requested review from a team and KiroCrewCaptain as code owners October 22, 2026 05:08
@rubencu
rubencu requested a review from iamwhatever October 22, 2026 05:08
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 22, 2026
@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision 0fc41cb6fb92549acd22ada75939e7f11a95683f touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 0fc41cb6fb92549acd22ada75939e7f11a95683f via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: each exit that never starts its turn marks itself, and one owner hands off the queue. It is gated on the same floor and slot checks as the synthesis launch.

Notes

  • Any future holder of slot.task that can exit before _run_chat has to remember to call mark_turn_unstarted, and nothing makes it. The PR's own pattern-harvest rule names this; a pinning test that lists which spawn_guarded_turn callers can exit before the runner would turn that rule into something the suite enforces.

[DESIGN-REVIEWED] 0fc41cb

Description read

[DESCRIPTION-READ] da556ecbe8627ca08ab9e25ae0fcb299e93e27a6d12f107e626dae8389d5cb96

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ✅ PASS

Premise-level review of 0fc41cb6fb92549acd22ada75939e7f11a95683f via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

The fix carries two small riders, the refusal log and the reworded card. Both have a stated harm, and neither needs a recorded product decision.

Notes

  • Item 4 — rides along: the Goal is "A message queued behind a dashboard turn that never starts still runs". The INFO line for a refused wake is diagnostics outside that Goal, and the queue hand-off works without it. Subtraction: remove _refuse_dashboard_turn and its logger.info and keep the original single if (...): _settle_admission(UNAVAILABLE) condition. That also undoes the rewrite of the refusal chain into five branches.

What this change ships

Inventory (5 items) — 3 justified

Intent: when a dashboard turn never starts, a message queued behind it still runs and its chat still gets the end-of-turn frame. FIX (closes #23015; the new tests fail on base).

  1. A send queued behind a monitor wake that was refused before its runner now runs, and the cycle ends with chat_done — justified
  2. A send queued behind an unattended turn that never got a background-turn permit (timed out or cancelled) now runs — justified
  3. An abandoned permit wait shows a "this turn never started" card instead of the 4-hour turn-limit card — rides along
  4. Each refused dashboard wake writes one INFO log line naming the reason — rides along, not needed for the hand-off
  5. New slot field _unstarted_generation and the mark_turn_unstarted marker, set at three exits (all through spawn_guarded_turn's callback) — justified

[FIRST-PRINCIPLES-REVIEWED] 0fc41cb

Description read

[DESCRIPTION-READ] bbe914fe0a6947575d1453d902adcf1768ec0789c772416db1187489fad6d3a7

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 0fc41cb6fb92549acd22ada75939e7f11a95683f via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 0fc41cb

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 0fc41cb6fb92549acd22ada75939e7f11a95683f via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 0fc41cb

@rubencu

rubencu commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Could a maintainer re-run Fork Security Scope Review run 39728732843? It was cancelled before any leg reported, and PR Readiness is waiting on that lane. Every other check on 44e2768 has finished green.

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Oct 24, 2026
A send that arrives while a dashboard turn holds the slot queues behind
it, and that turn's tail hands the queue off. Two holders end without
entering _run_chat: a monitor wake refused at its pre-runner
authorization, and an unattended turn whose wait for a background-turn
permit gives up or is cancelled. Neither ran a tail, so the queued
message waited for the next send, indefinitely on an app-owned slot,
and the cycle sent no chat_done. The abandoned wait also showed the
turn-ceiling card for a turn that never started.

Both exits mark the turn unstarted, and spawn_guarded_turn's done
callback hands the queue off in its place: a one-shot task takes
slot.task and runs the tail's no-outcome hand-off, so the queue drains
or the cycle ends with done and chat_done. The cap's wait raises
BackgroundTurnAbandoned, a TimeoutError reported with its own card. A
refused dashboard wake logs one INFO line naming its reason.

Closes kirodotdev#23015
@rubencu
rubencu force-pushed the fix/turn-holder-exit-queue-drain branch from 44e2768 to 0fc41cb Compare October 29, 2026 01:33
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Oct 29, 2026

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommendation: APPROVE at 0fc41cb6fb92549acd22ada75939e7f11a95683f.

The fix is sound. The two exits that hold slot.task without entering _run_chat (a monitor wake refused before its runner, and an unattended turn whose background-permit wait times out or is cancelled) now mark the turn unstarted (fire, permit wait). spawn_guarded_turn's done callback then hands the queue off through one owner (hand-off gate). The gate requires that the ended task still owns slot.task under the marked generation, no send holds the admission reservation, the slot is still the registered, open, local one, and the gateway is not shutting down. The cycle-end task takes slot.task synchronously, so a concurrent send queues behind it instead of racing. BackgroundTurnAbandoned stays a TimeoutError with the same message, so the nudge fire still settles BUSY and the Issue Radar catch still renders its own card; finish_turn_task checks it before the ceiling branch (card). The new tests in test_unstarted_turn_queue_handoff.py drive the real fire path, run_background_turn and /api/chat, and assert the queued send runs and chat_done is sent.

All five AI lanes PASS on this head; every check is green (newest run per name); base is main; the head merges cleanly with current main; no change requests.

AI findings judged non-blocking

  • Design Review (note): nothing forces a future slot.task holder to call mark_turn_unstarted. This is a suggestion for a future pinning test. Both current exits are marked and tested (mark).
  • First Principles (note): the refusal INFO log is a rider outside the goal. It is diagnostics only, does not change which wakes are refused, and is small (refusal).

— crew-pr-reviewer (r134)

@iamwhatever
iamwhatever merged commit febf7fe into kirodotdev:main Oct 29, 2026
102 checks passed
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A message queued behind a dashboard turn that never starts waits for the next send

2 participants