Skip to content

feat(crew): release a crewmate's stale workers and resume them on send - #23574

Merged
buluoray merged 1 commit into
mainfrom
feat/crewmate-stale-workers-v2
Oct 24, 2026
Merged

buluoray merged 1 commit into
mainfrom
feat/crewmate-stale-workers-v2

Conversation

@buluoray

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A crewmate can open as many workers as its work needs, and a worker that went idle or was archived picks up again on the crewmate's next message and still reports back.

With the crewmate conductor/worker split (#23010), a crewmate hands each job to a worker session it opens. Three things go wrong once those workers age:

  • A worker idle past session.timeout_secs has its process stopped by the idle sweep. Nothing tells the crewmate (or the user) that the worker is stopped rather than gone, so it opens a fresh worker instead of resuming the one that holds the context.
  • A worker idle past dashboard.idle_slot_sweep_days is archived like any tab. The crewmate's next session_send to it answers target_not_found, so the conversation and its kit are abandoned.
  • After a gateway restart or a revive, the in-memory link (_lineage_minted / _reports_to_creator) is gone, so a restored worker's turn end no longer wakes the crewmate's thread: the report is silently lost.

Why it matters

Every crewmate that runs longer than an hour hits the idle sweep. Without this, a crewmate either piles up duplicate workers or loses the report of a worker that finished after a restart, and the user sees a crewmate that stopped answering about work it delegated.

Not a goal

  • No cap or quota on how many workers a crewmate opens; memory stays bounded by the existing cgroup max_memory_mb.
  • No change to the idle sweep's timing, the archive sweep's timing, or which sessions they cover.
  • No new revive path for callers other than a crewmate's own conductor; every other session_send to a closed session keeps target_not_found.
  • No change to session_revive's ownership fence.

What changed (motivation → approach → change)

  • Revive on send. handlers/session_control.api_session_control_send first calls session_revive.revive_own_archived_worker. It acts only when the caller's attested record is a crewmate conductor (mate_worker.is_conductor_execution) and the target is not open, and it runs the existing revive_session, so revive's own fence still decides: the caller must have created the session AND the gateway-written lineage must name it as parent. A refusal keeps the target as given, so the send answers exactly as before.
  • A restored worker still wakes its creator. mate_thread_wakes.reporting_creator accepts the crew log's session tree (_lineage_names, via session_tree._slot_tree_parent, gateway-written at the child's first turn) as the witness when the in-memory one is gone. A _created_by read from transcript metadata alone still wakes nobody.
  • process: live | released. session_status.process_state reads SessionManager.has_session on the slot's effective key. created_session_status rows carry it, and slots_frame.serialize_slot adds it to worker tabs (slots with created_by).
  • Dashboard badge. SessionRow shows a moon marker (session-row-released, "Asleep: wakes on the next message") on a released worker, also on a row with no activity time.
  • Specs updated in the same commit: crew-mode.md (stale workers released, not capped; the lineage fallback), session-control.md (revive on send; the process field), session.md (idle cleanup of a worker session).

Backwards compatibility

Compatible: process is a new optional field on session_status rows and slot payloads; session_send only changes for a conductor whose own archived worker passes revive's fence, where it previously answered target_not_found.

Removes nothing: the change adds a field, a marker and a revive path; no field, route, record or reader is narrowed.

Tests

  • test/test_crewmate_stale_workers.py
    • test_an_idle_worker_is_released_then_resumed_in_the_same_conversation: the idle sweep releases the worker's process, process_state reads released, the conversation pointer is kept, the next turn cold-starts the same key asking to resume that sid, and the worker's kit still comes from the same execution record.
    • test_a_conductor_messaging_its_archived_worker_revives_it_first: the conductor's send revives its own archived worker first.
    • test_a_refused_revive_and_any_other_caller_keep_target_not_found: a refused revive and a non-conductor caller keep target_not_found.
    • test_a_restored_worker_reports_when_the_lineage_names_its_crewmate: the crew-log lineage stands in for the in-memory witness.
    • test_transcript_metadata_alone_still_wakes_nobody: transcript metadata alone still wakes nothing.
  • website/src/test/ChatSidebar.releasedWorker.test.tsx: the marker shows on a released worker with its accessible name, not on a live worker or an ordinary session, and also on a row with no activity time (reddens when the meta-group gate drops s.process === 'released').

Manual verification

Rendered the built SPA (stubbed API) with a creator, a live worker, a released worker and an ordinary session, in light and dark: only the released worker carries the moon marker, beside its timestamp.

Screenshots / video

Light and dark, flat sidebar: the third row is a released worker.

Released worker marker, light
Released worker marker, dark

Related Issues

Follows #23010.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

@buluoray
buluoray requested review from a team and KiroCrewCaptain as code owners October 23, 2026 22:32
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 23, 2026
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 1fbf8ed789dde06ec8df6639464f505d0fbd4ea5 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

The fix items resume or wake the worker on their own. The dashboard moon marker has no named person it helps.

Notes

  • Item 4 — inherited: the description says "Nothing tells the crewmate (or the user)", but the conductor already gets process on session_status, and the next send resumes a released worker anyway. A human who sees the moon learns nothing they can act on. Subtraction: delete the payload["process"] branch in slots_frame.serialize_slot, the Moon marker in SessionRow.tsx, both process? type fields and the 13 process_released locale strings.
  • Item 3 — symptom-level: the report is lost because the creator link lives only in memory, and the fallback reads the crew log instead. Per the spec hunk, "with the crew log off … that worker's report is not delivered". The description's statement that the restart case is fixed should say this residual remains.

What this change ships

Inventory (5 items) — 3 justified

Intent: a crewmate's idle or archived workers pick their conversation back up and still report back to it. This is an ADDITION (a fix shape on top of #23010).

  1. A conductor's session_send to its own archived worker revives the worker instead of answering target_not_found — justified
  2. session_status rows tell the conductor whether a worker's process is live or released — justified
  3. After a restart, a restored worker's turn end still wakes its crewmate when the crew-log tree names the crewmate as parent — symptom-level, the report is still lost with the crew log off
  4. Worker tabs in the sidebar show a moon "Asleep" marker when released, which adds process to slot payloads — oversized, the user can't act on it
  5. The crew-mode, session-control and session specs now describe the released-worker lifecycle — justified

[FIRST-PRINCIPLES-REVIEWED] 1fbf8ed

Description read

[DESCRIPTION-READ] 72c689082fe953c48fd7a182b70f0c3bf2014a03c834e119db5e18baf4357226

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 1fbf8ed789dde06ec8df6639464f505d0fbd4ea5 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: each of the three stale-worker failures gets a fix that goes through existing code (revive_session's fence, gateway-written lineage, has_session), and screenshots of the new badge are attached.

Notes

  • The lineage fallback says no when the crew log is off. In that setup a restored worker's report is still lost without any message. The spec admits this, but nothing tells the conductor or the user. Logging or surfacing that miss would make it diagnosable.

[DESIGN-REVIEWED] 1fbf8ed

Description read

[DESCRIPTION-READ] df7bcaf67f509bedfa2bd3500e43bed45b41b9f810baa63be8c6ec053f43ab35

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of 1fbf8ed789dde06ec8df6639464f505d0fbd4ea5 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

Nothing blocks. The one new element, the moon marker, appears in both screenshots, and the blind reader got its meaning right ("asleep, paused or inactive"), though they rated that only a guess.

Notes

  • The moon has no visible label. Its meaning, "Asleep: wakes on the next message", exists only in a title tooltip that no screenshot shows. The reader "don't know whether the moon means it's broken, finished or waiting." Attach a hover screenshot showing the tooltip, so the reassurance that the worker will resume can be checked.

[UX-REVIEWED] 1fbf8ed

Description read

[DESCRIPTION-READ] c1798f2fdde9a0d1b2889a22b16470858fd0b93f68e027d850ddca77f4b15340

That sha256 names everything this verdict read: the PR description, plus the 5 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 1fbf8ed789dde06ec8df6639464f505d0fbd4ea5 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 1fbf8ed

Verdict parsed from the review's SHA-scoped output markers for commit 1fbf8ed789dde06ec8df6639464f505d0fbd4ea5.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 1fbf8ed789dde06ec8df6639464f505d0fbd4ea5 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings (all downgraded on adjudication)

GPT 6.1 flagged blocking issues on 1fbf8ed789dde06ec8df6639464f505d0fbd4ea5; Opus 5.5 adjudication downgraded every one of them. Downgraded: the remedy each one needs is disproportionate to the harm it prevents. Adjudication downgraded all 1 blocking finding(s): the remedy each one requires is disproportionate to the harm it prevents.

This comment is updated in place on each push.

Review details

BLOCKING -- src/kiro_crew/dashboard/state_parts/slots_frame.py:821 -- Idle expiry leaves the worker’s sidebar state stale

payload["process"] = process_state(self, slot)
Trigger: Leave the dashboard connected while a worker exceeds session.timeout_secs. SessionCleanup._expire_idle calls reset, removing the live session without publishing a slots update.
Outcome: The browser retains process: live, so the released worker lacks its asleep marker until an unrelated slots broadcast or reconnect. Periodic dashboard frames do not update slot rows, and lineage patches carry only parent fields.
Anchor: defect
Fix: Publish the worker’s updated process state when idle expiry successfully releases its session.

[BLOCK-MERGE-DOWNGRADED] 1fbf8ed
[GPT-REVIEWED] 1fbf8ed

Adjudication (Opus 5.5) — is blocking on each finding proportionate?

F1 (LOW): the only effect is that the sidebar does not show the asleep (moon) icon for a while. The worker still resumes correctly on the next send whatever the icon shows. Conditions I confirmed: the idle sweep removes the session at src/kiro_crew/session_cleanup.py:1625-1817 (its only callback, on_session_expire at :1808, goes to the consolidator wired at slack/gateway_runtime/service_boot.py:352, not to a dashboard publish). process is computed only when a slot is serialized (dashboard/state_parts/slots_frame.py:821, via process_state at session_status.py:252), and the icon is the only thing that reads it (website/src/pages/chat-sidebar/SessionRow.tsx:1574). The icon fixes itself on the next slots broadcast or reconnect. No data is lost and no behaviour goes wrong.
Real fix cost: a new channel from SessionCleanup back to the dashboard's slot broadcast, crossing modules, used only by the idle-expiry path and kept up forever. That costs more than a cosmetic marker that is late for a while. The finding is not tied to any AUTOSDE rule marked blocking: true.

[ADJUDICATION] 1fbf8ed789dde06ec8df6639464f505d0fbd4ea5 total=1 uphold=0 downgrade=1
DOWNGRADE F1 src/kiro_crew/dashboard/state_parts/slots_frame.py:821 reason=disproportionate-remedy
[GPT-ADJUDICATED] 1fbf8ed789dde06ec8df6639464f505d0fbd4ea5

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 1fbf8ed789dde06ec8df6639464f505d0fbd4ea5 false-positive: <reason, 20 to 500 characters>

@buluoray
buluoray force-pushed the feat/crewmate-stale-workers-v2 branch from 3d225cc to be47107 Compare October 23, 2026 23:12
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: checking Automated validation is still running labels Oct 24, 2026
No cap on how many workers a crewmate opens; a stale one is handled:
- session_send from a crewmate's conductor to an archived worker of its own
  revives it first, under session_revive's ownership fence, then sends.
  A refused revive, and every other caller, keeps target_not_found.
- After a restart or a revive, the crew log's gateway-written lineage
  naming the crewmate thread lets the worker report back
  (mate_thread_wakes._lineage_names); transcript metadata alone wakes nobody.
- session_status rows carry process: live | released, read from
  SessionManager.has_session; a worker tab shows a released one with a
  marker, even on a row with no activity time.
Updates session.md, crew-mode.md and session-control.md.
@buluoray
buluoray force-pushed the feat/crewmate-stale-workers-v2 branch from be47107 to 1fbf8ed Compare October 24, 2026 01:39
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Oct 24, 2026
@buluoray
buluoray merged commit 9e252d2 into main Oct 24, 2026
107 checks passed
@buluoray
buluoray deleted the feat/crewmate-stale-workers-v2 branch October 24, 2026 05:21
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants