Skip to content

fix(crew): prune generated crewmates never chatted with on their page - #14897

Merged
chenmingwei23 merged 1 commit into
mainfrom
fix/crewmate-prune-widen
Sep 29, 2026
Merged

chenmingwei23 merged 1 commit into
mainfrom
fix/crewmate-prune-widen

Conversation

@buluoray

@buluoray buluoray commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: a crewmate the user never chatted with on the Crewmates page is pruned at startup, so the roster stops showing a wall of agent templates.

The one-time startup prune from #12798 runs and writes its marker, but on installs whose agents come from packages it removes nothing. Every row the old agent sync wrote carries the spec's own source (package, aim), and the prune only accepted builtin. On one real install all 28 generated rows were skipped. Rows it did look at were often kept anyway, because a subagent, cron or app slot that ran the agent, or a DM thread that was only opened, counted as "chatted".

Why it matters

Anyone who upgraded from a build that ran POST /api/agents/sync keeps a Crewmates roster full of templates they never used, and the v1 marker means the prune never tries again.

Not a goal

  • Changing how agents resolve as templates for spawn, cron or apps.
  • Re-adding or changing the agent sync endpoint.
  • Pruning crewmates the user created by hand.

What changed (motivation → approach → change)

In src/kiro_crew/crewmate_prune_migration.py, a generated crewmate is now pruned unless you actually talked to it on its own Crewmates page.

Before After
Candidate rows source == "builtin" only bound to an installed spec of the user's own or of a package, with a string source stamp matching that spec's (package and aim are one source); never a Kiro Crew-owned spec or a kirocrew stamp, a private copy, a spec that is gone, a crewmate on a team, or a row with a member_id
"Chatted" any session that ran the agent, activity log, an opened DM binding a row past the metadata line in its dashboard_member-<slug> transcript, archived compaction segments included
Marker crewmate_prune_migrated.json crewmate_prune_v2_migrated.json

This PR amends §03 of docs/request-for-change/rfc-crewmates-launch.md to record this rule: package-installed specs are covered, the runtime's own, private and uninstalled specs are left alone, and "chatted with" means a turn in the crewmate's own Crewmates-page thread. A description edit does not protect a never-chatted generated row, as the RFC states; the delete re-reads the spec under the locks and refuses if its name, source, private or Kiro Crew-owned status changed. Crewmates whose names map to the same Crewmates-page thread are judged by that shared thread. The transcript read goes through jsonl_util.strict_raw_records; a row too long to buffer still counts as a turn, and any other unreadable row, or an archive directory that cannot be listed, keeps the crewmate. The locking, the refusal checks and the rule that any doubt keeps the row are the same. The new marker name gives installs the first pass left untouched one more run. The comments in dashboard/server.py, the crew-mode spec and the feature map describe the new rule.

Backwards compatibility

Compatible: the pass only deletes rows whose every field still equals what the sync wrote from the spec and that have no DM turn; hand-made, owner-edited and unreadable rows are kept, and the pass removes roster rows only, never agent spec files.

Tests

test/test_crewmate_prune_migration.py is rewritten around the new rule (79 tests): user and package rows are candidates; Kiro Crew-owned, kirocrew, spec-less, source-mismatched, non-string-source and created (member_id) and teamed rows are kept (including one teamed mid-pass); an edited-description row with no DM turn is removed; a spec renamed, made private, re-sourced or made Kiro Crew-owned mid-pass keeps its row; only a DM transcript turn counts as chatted (not subagent, cron, app or plain-chat sessions, not an empty or metadata-only thread); archived segments count; a torn or over-cap row still counts; an unlistable archive directory keeps the row; the v2 marker re-runs after v1. Each guard (source type, source match, spec-less skip, over-cap row, archive listing) was broken on purpose one at a time and each broke a test.

Manual verification

Checked on a real install with 29 generated crewmates, none of which except default had a Crewmates-page DM turn. An earlier revision of the pass ran there and removed 19. Under this revision the 4 rows whose description had drifted from the spec are removed too, and the 5 rows whose spec is no longer installed stay.

Related Issues

no linked issue: follow-up to #12798, whose filter missed package-sourced rows.

Pattern harvest

Not generalizable: the defect is one migration matching a single literal source value that the historical writer never stamped.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@buluoray
buluoray requested a review from a team as a code owner September 29, 2026 01:48
@buluoray
buluoray requested a review from kyleseaman September 29, 2026 01:48
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 29, 2026
@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🟡 CONCERNS

Design-level review of 6f136097140bf18749402c605a22fb38ea39fe93 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

The PR itself authors the RFC revision it relies on: pinned keep-behaviours (description edits, agent-run sessions) now delete rows with no named human decider.

Watch

The old test test_every_field_including_the_description_is_the_owners_signal pinned "a description edit keeps the row"; this PR flips that assertion (assert shape({**raw, "description": "rewritten"})), so a never-chatted row whose description the owner rewrote is now irreversibly deleted with that text ("the 4 rows whose description had drifted from the spec are removed too"). The same revision narrows "chatted" past two other pinned keeps (activity records, sessions that ran the agent). The reconciliation is defensible — the pre-existing RFC prose already contradicted the code on edited rows — and the rationale is recorded in the same commit, but the "revised on 2026-09-28" line in §03 is written by this diff itself and, unlike the 2026-09-23 entry, names no decider; under this repo's own rule that a pin is a decision until a human says otherwise, the ratification should be explicit, not self-recorded.
Clears when: a maintainer or the product owner confirms the 2026-09-28 §03 revision (PR approval/comment, or the RFC line naming the decider as the 2026-09-23 entry does).

[DESIGN-REVIEWED] 6f13609

Description read

[DESCRIPTION-READ] 3a4cbda2df3ac20376c520d50f4b91bccf75e66e6d619cc651d6d1db703f4cc7

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ human override accepted

@buluoray overrode this lane for 6f136097140bf18749402c605a22fb38ea39fe93 via /ai-review override (this commit only).

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 6f136097140bf18749402c605a22fb38ea39fe93 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 6f13609

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 6f136097140bf18749402c605a22fb38ea39fe93: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 29, 2026
@buluoray
buluoray force-pushed the fix/crewmate-prune-widen branch from b333f83 to 5edc75d Compare September 29, 2026 02:11
@buluoray

buluoray commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor Author
  • span=eced8e08c12f Overbroad sync detection deletes user-owned crewmates — fixed in 5edc75d

kirocrew is dropped from SYNC_ROW_SOURCES: it is the record default any non-sync writer leaves, so such a row is never a candidate.
_is_fresh_sync_shape compares the row's description to the bound spec's current description again, at discovery and inside the delete's lock; a mismatch keeps the row.
A spec-less row is a candidate only with an empty description. Mutation-checked: re-adding kirocrew or exempting description each turns a test red.

@buluoray

buluoray commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor Author
  • span=6946eb7ff1a7 Description drift claimed never to be an owner edit — fixed in 5edc75d

The description comparison is reinstated: a row whose description differs from its spec's is the owner's and is kept, which covers PUT /api/agents/{name}. The PR body no longer makes the claim.

@buluoray

buluoray commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor Author
  • span=370c3d79b750 Record removed rows' raw dicts in the marker — rebutted

Every removed row now equals, field for field, what the sync writes from its spec (source stamp, spec description, all other fields default, no extra keys), so the row carries nothing the spec does not already hold.
Recreating it is re-enrolling the same spec; a marker copy would duplicate the spec, not restore owner data. Owner-edited rows are no longer removed.

@buluoray

buluoray commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor Author
  • span=079482577a81 Deleted description pin reversed without justification — fixed in 5edc75d

The arm is reinstated: test_a_row_whose_description_differs_from_the_spec_is_kept pins that a description mismatch keeps the row, and removing the comparison fails it.

@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Sep 29, 2026
@buluoray

buluoray commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor Author
  • span=7a930084afce kirocrew rows as sync output is unverified — fixed in 5edc75d

kirocrew is no longer a candidate source; the docstring now names it as the default stamp non-sync writers leave, matching config/sections.py.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 29, 2026
@buluoray
buluoray force-pushed the fix/crewmate-prune-widen branch from 5edc75d to 752aa58 Compare September 29, 2026 02:32
@buluoray

Copy link
Copy Markdown
Contributor Author
  • span=eced8e08c12f Unreadable archive silently deletes a chatted crewmate — fixed in 752aa58

_chatted lists the archive with os.scandir; only FileNotFoundError reads as an empty archive, and every other listing or iteration OSError raises HistoryUnreadable, which keeps the crewmate and names it in doubted.
test_an_archive_directory_that_cannot_be_listed_keeps_that_crewmate pins it; swapping back to Path.glob turns it red.

@buluoray

Copy link
Copy Markdown
Contributor Author
  • span=2f6082500a24 Feature map falsely claims runtime agents are pruned — fixed in 752aa58

The feature-map row now names only the user's own and package-installed agents, and states that a row whose description no longer matches its spec is never removed.

@buluoray

Copy link
Copy Markdown
Contributor Author
  • span=cd46d1750738 Name description-mismatch keeps in the marker — rebutted

A row kept for a description mismatch stays in config.json unchanged, so the field state that explains it is on disk; comparing a row with its spec reproduces the decision without a marker entry.
The marker lists what the pass changed (removed) or could not judge (doubted, refused); a kept row changed nothing.

@buluoray

Copy link
Copy Markdown
Contributor Author
  • span=3e742863041f DM-turn-only chatted rule reverses v1 on argument alone — rebutted

The rule is a maintainer decision: the repository owner asked for it directly while scoping this fix ("chatted counts only when it happened on the crew member page").
On that install every generated row the v1 rule would have kept was one the owner had only clicked open or that a subagent, cron or app slot had run; none held a DM turn except default.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 29, 2026
@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed 6f136097140bf18749402c605a22fb38ea39fe93 — this comment is updated in place on each push.

Review details

Strict fork-sidecar read turns an unreadable agent_state.json into a permanent refusal, so the prune never records a pass.

FINDING — src/kiro_crew/crewmate_prune_migration.py:369 — fork_info = agent_state.get_fork_info(info.name, strict=True) inside the except (OSError, ValueError): return None block makes a present-but-unreadable sidecar (hard-linked by a rsync --link-dest-style restore so st_nlink != 1, or past STATE_MAX_BYTES) refuse every candidate, while _synced_candidates admitted those same rows through list_agents, which swallows the identical read (forks = {}) — so _prune_locked returns at if report.refused: before _write_marker, and the pass re-judges and re-arms the crewmate_prune_settled request barrier on every boot for as long as the sidecar stays unreadable, which the module's own "never loops boot after boot on one bad file" contract forbids → Fix: distinguish "sidecar unreadable" from "spec identity changed" — read the lineage strictly once in _prune_locked and fold a failure into the all-candidates-doubted branch teams_doubt already uses, so the rows are kept and the marker is still written (needs the doubt channel threaded through remove_never_chatted/_mutate, which this diff does not carry).

[OPUS-REVIEWED] 6f13609

Verdict parsed from the review's SHA-scoped output markers for commit 6f136097140bf18749402c605a22fb38ea39fe93.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 6f136097140bf18749402c605a22fb38ea39fe93: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 29, 2026
@buluoray
buluoray force-pushed the fix/crewmate-prune-widen branch from f2eea79 to d811cde Compare September 29, 2026 05:31
@buluoray

Copy link
Copy Markdown
Contributor Author
  • span=eced8e08c12f Disappearing archive can delete a chatted crewmate — fixed in d811cde

An archive segment already seen in the directory listing is opened with missing_is_doubt=True: if it is gone by the open, HistoryUnreadable keeps the crewmate as doubted. A missing live transcript still reads as no turn.
The listing keeps only names that start with this crewmate's own stem prefixes, so retention is bounded by its own segments. Reverting the disappearance branch to False fails the new test.

@buluoray
buluoray force-pushed the fix/crewmate-prune-widen branch from d811cde to 3958dd4 Compare September 29, 2026 06:01
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 29, 2026
@buluoray

Copy link
Copy Markdown
Contributor Author
  • span=ed8bd24b80da Strict fork read at delete time can repeat the pass — rebutted

The strict read fails closed: an unreadable agent_state.json refuses the delete and keeps every crewmate, which is the safe outcome for a destructive migration. A repeat on the next boot is a harmless re-check, not data loss, and it stops once the sidecar is readable. Relaxing it to the tolerant list_agents read would let a delete proceed without knowing whether the spec was forked.

@buluoray

Copy link
Copy Markdown
Contributor Author

/ai-review override first-principles 3958dd4: Product owner decision: RFC crewmates-launch §03 is amended in this same PR to cover package-installed specs, define chatted as a Crewmates-page DM turn, and keep created, teamed and runtime-owned crewmates.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@buluoray marked the first-principles AI finding as false positive, not applicable, or explicitly accepted for 3958dd4d76acb37d76e1331582e83532d6e68e06.

Product owner decision: RFC crewmates-launch §03 is amended in this same PR to cover package-installed specs, define chatted as a Crewmates-page DM turn, and keep created, teamed and runtime-owned crewmates.

This decision applies only to this commit. A new push requires a new judgment.

The startup prune only accepted rows stamped source "builtin", but the
old agent sync stamped each row with its spec's own source, so installs
whose agents came from packages had every row skipped and kept a roster
full of templates. Its "chatted" rule also counted any session that ran
the agent (subagent, cron, app slot) and a DM thread merely opened.

A row bound to an installed user or package spec, whose source stamp
matches that spec's, is now a candidate, and it counts
as chatted only when its own Crewmates page DM transcript holds a turn
past the metadata line, archived compaction segments included. The
marker moves to crewmate_prune_v2_migrated.json so installs the first
pass left untouched get one more run.

The crewmates launch RFC's section 03 records the rule: package-installed
specs are covered, the runtime's own, private and uninstalled specs are
left alone, and "chatted with" means a turn in the crewmate's own
Crewmates-page thread.
@buluoray
buluoray force-pushed the fix/crewmate-prune-widen branch from 3958dd4 to 6f13609 Compare September 29, 2026 07:11
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Sep 29, 2026
@buluoray

Copy link
Copy Markdown
Contributor Author
  • span=ed8bd24b80da Strict fork read at delete time can repeat the pass — rebutted

The strict read fails closed: an unreadable agent_state.json refuses the delete and keeps every crewmate, which is the safe outcome for a destructive migration. A repeat on the next boot is a harmless re-check, not data loss, and it stops once the sidecar is readable. Relaxing it to the tolerant list_agents read would let a delete proceed without knowing whether the spec was forked.

@buluoray

Copy link
Copy Markdown
Contributor Author

/ai-review override first-principles 6f13609: Product owner decision: RFC crewmates-launch §03 is amended in this same PR to cover package-installed specs, define chatted as a Crewmates-page DM turn, and keep created, teamed and runtime-owned crewmates.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@buluoray marked the first-principles AI finding as false positive, not applicable, or explicitly accepted for 6f136097140bf18749402c605a22fb38ea39fe93.

Product owner decision: RFC crewmates-launch §03 is amended in this same PR to cover package-installed specs, define chatted as a Crewmates-page DM turn, and keep created, teamed and runtime-owned crewmates.

This decision applies only to this commit. A new push requires a new judgment.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Sep 29, 2026
@chenmingwei23
chenmingwei23 merged commit c5c307c into main Sep 29, 2026
99 of 100 checks passed
@chenmingwei23
chenmingwei23 deleted the fix/crewmate-prune-widen branch September 29, 2026 15:06
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 29, 2026
bolichen97 added a commit that referenced this pull request Sep 30, 2026
…15576)

* fix(crew): prune generated crewmates never chatted with on their page (#14897)

The startup prune only accepted rows stamped source "builtin", but the
old agent sync stamped each row with its spec's own source, so installs
whose agents came from packages had every row skipped and kept a roster
full of templates. Its "chatted" rule also counted any session that ran
the agent (subagent, cron, app slot) and a DM thread merely opened.

A row bound to an installed user or package spec, whose source stamp
matches that spec's, is now a candidate, and it counts
as chatted only when its own Crewmates page DM transcript holds a turn
past the metadata line, archived compaction segments included. The
marker moves to crewmate_prune_v2_migrated.json so installs the first
pass left untouched get one more run.

The crewmates launch RFC's section 03 records the rule: package-installed
specs are covered, the runtime's own, private and uninstalled specs are
left alone, and "chatted with" means a turn in the crewmate's own
Crewmates-page thread.

(cherry picked from commit c5c307c)

* fix: prune generated crewmates bound to skill-view aliases (#15103)

* fix: prune generated crewmates bound to skill-view aliases

An older agent sync walked the agents directory before discovery learned
to leave skill-view aliases out, so it enrolled one crewmate per alias
file. The startup prune only judges rows whose spec discovery lists, and
discovery never lists an alias, so every such row was kept as if its
agent were uninstalled and the roster stayed full of them.

A row bound to an alias is now judged on its own sync stamp and
fresh-sync shape. The member_id, overlay, team and chatted rules still
apply, the delete re-checks the row under the same locks, and the marker
moves to v3 so installs the earlier passes left untouched run once more.

* fix: judge only exact alias names on the row alone

A row is judged as a skill-view alias row only when its name is exactly
what the projection writes: the prefix plus 24 lowercase hex digits. A
crewmate named with the prefix and any other tail is an ordinary row
and needs an installed spec, so with none it is kept.

The pre-delete docstrings now say the spec re-read applies to rows
bound to an installed spec; an alias row has no spec to re-read.

(cherry picked from commit 6fb9523)

---------

Co-authored-by: Ray Xu <38930179+buluoray@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants