Skip to content

fix(skill-view): backport recording a view's source agent before its sidecar is deleted to release/0.8.0 - #16074

Merged
bolichen97 merged 2 commits into
release/0.8.0from
backport/0.8.0-skill-view-sidecar-ledger
Oct 3, 2026
Merged

bolichen97 merged 2 commits into
release/0.8.0from
backport/0.8.0-skill-view-sidecar-ledger

Conversation

@bolichen97

@bolichen97 bolichen97 commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: On the 0.8 line, a chat stored under a generated kirocrew-skill-view-<digest> agent name keeps resuming after the boot drain retires that view's alias and ownership sidecar.

Backport of #16073 to release/0.8.0. Both commits were cherry-picked cleanly with -x. #16073 is not merged yet, so this PR must not merge ahead of it.

A conversation recorded while the native skill projection was on can carry a view name as its agent. source_agent_name maps it back through the view's ownership sidecar, then the view ledger (view-sources.json). The ledger is written only at publication, and only for the aliases that run publishes, so a view published before the ledger existed has the sidecar as its only record. The boot drain deletes exactly that record:

  • the orphan-sidecar sweep (_sweep_orphan_sidecars) retires a sidecar whose alias a cleanup outside the core, or an operator, already removed;
  • the prune (_reclaim_prune_candidate) retires the alias and its sidecar together, both for an intact pair and for a re-serialized one.

None of the three paths writes the ledger first. After the next boot the stored chat is refused with Crew Member 'kirocrew-skill-view-…' is unavailable; restore it or choose a member, even though the agent it was built from is installed. A pod smoke test reproduced this on a real tree: alias removed, then drain_stale_aliases(), then the resume refused.

Why it matters

Every conversation recorded under a view name before the ledger existed stops resuming at the first boot that drains it. The refusal points at a "Crew Member" the user never created and cannot restore, so there is no way forward from the message.

Not a goal

  • Recovering mappings that a drain already deleted. Nothing else records them, and the view name is a one-way digest over values that are no longer reproducible (see Manual verification). This PR only makes the refusal say what to do.
  • Changing the ledger's bound (_VIEW_SOURCES_MAX), its format, or what the drain deletes.
  • Changing which aliases or sidecars count as reclaimable.

What changed (motivation → approach → change)

Symptom: a stored view name stops resolving once the drain runs. Root cause: every sidecar deletion drops the last record of the view's source agent without copying it into the ledger. Change: record the mapping before deleting.

  • _record_view_ledger_entries(metadata_dir, views) is now the single ledger writer. It takes alias → agent, because a retired backlog holds many views of one agent. It reuses the existing rules: the same admission, an atomic owner-only write, and no rewrite when nothing changed. Each write makes its whole batch the newest entries, including pairs the ledger already holds, so the same write cannot evict a pair whose sidecar is about to go. Past the bound (_VIEW_SOURCES_MAX), entries a sidecar still answers for go first, oldest first. After that the oldest retired names go; they stop resolving, and the loss is counted in a warning. A full ledger therefore never stops the sweep from retiring sidecars. A batch larger than the bound cannot all be held, so none of its sidecars go. _record_view_ledger (agent → alias, for a publication) delegates to it.
  • _ViewLedgerWrites keeps the ledger for the duration of one locked prune or sweep. Every path that deletes a sidecar calls it before unlinking:
    • sweep: collects its batch, records every pair in one write, then unlinks. The time budget covers the unlinks too. If the ledger refuses the write, the whole batch is kept for a later run.
    • prune, both pair paths (intact and re-serialized): retain() runs before _unlink_alias_if_unchanged. The first pair that needs recording also records the next 64 candidates' sidecars (this home only) in the same write. A drain over thousands of pairs therefore costs one write per window instead of one per pair, and a walk that reclaims nothing does no extra reads. If the ledger refuses, the alias still goes. The sidecar stays and answers for the view on its own, so a full disk or a read-only metadata directory does not stop alias reclamation. After one refused write, the rest of that prune or sweep keeps sidecars without retrying the write for each candidate. _reclaim_prune_candidate now takes the ledger as a required argument.
  • All of this runs under the publication lock that each deletion already holds.
  • Impact. This change is defensive. At the core that the internal 0.8.0.1/0.8.0.2 builds shipped, no real path stored a view name, because every recorder refuses an unresolvable name first (see the impact comment on this PR). The change keeps any stored view name resolvable through the drain.
  • For a view that nothing records, the refusal now says so and gives the remedy that works: chat_runner.unavailable_agent_message() reads "This chat was recorded under a generated skill view from an earlier version (…), and the agent it was built from is not recorded. Pick the agent for this chat again, or start a new chat." The RetiredSkillView text that the ACP spawn path raises gets the same "or start a new chat".

Backwards compatibility

Compatible. The ledger file keeps its format and its bound, and older builds read it unchanged. A deletion that would previously have happened is deferred only when the ledger write fails. _record_view_ledger keeps its signature. The only other change is the message text for a non-view name, which is identical.

Tests

  • test_resume_retired_agent_names.py::test_a_view_the_real_boot_drain_retired_still_resumes[alias-removed|pair-kept|alias-rewritten]: publishes a real view and deletes the ledger, as a pre-ledger tree would look. It then runs the real drain_stale_aliases(), asserts that the alias and the sidecar are really gone, and resumes the stored name through resolve_session_agent_bindings. Each parametrization drives one of the three deletion paths. All three FAIL on main.
  • test_skill_view_mode_recovery.py::test_the_boot_drain_records_a_backlog_in_one_ledger_write: a backlog of 7 views of one agent, half with their alias already removed. The drain retires them all with exactly 2 ledger writes (one for the prune, one for the sweep), and every name still resolves.
  • test_skill_view_mode_recovery.py::test_a_sidecar_the_ledger_cannot_take_is_kept_and_its_alias_goes[True|False] and test_a_reserialized_pair_the_ledger_cannot_take_keeps_its_sidecar: when a ledger write raises, the sidecar is kept and the alias is removed, and the name still resolves through the sidecar.
  • test_the_bound_drops_redundant_entries_then_the_oldest_retired_ones: redundant entries go first, then the oldest retired name (with a warning), new pairs are always held, and an over-bound file is trimmed.
  • test_a_batch_rewrites_its_held_pairs_so_the_bound_cannot_evict_them: a held pair in a batch is not evicted by that batch's own write, and a batch larger than the bound is refused.
  • test_a_refused_write_is_not_retried_for_every_candidate.
  • test_the_orphan_sweep_unlinks_nothing_past_its_deadline: a deadline that runs out during the ledger write stops the unlinks; the batch stays recorded.
  • test_resume_retired_agent_names.py::test_an_unrecoverable_view_tells_the_user_to_pick_again_or_start_new.
  • test_skill_view_mode_recovery.py::test_the_look_ahead_records_only_this_homes_views: one look-ahead write covers its window, and another home's view takes no ledger slot.
  • test_skill_view_mode_recovery.py::test_a_ledger_that_already_holds_the_views_is_not_rewritten: views the ledger already holds, and records with no admissible agent, cause no write.

Regression proof, with each site disabled in turn:

Mutation Result
Sweep skips the ledger 3 FAIL
Intact-pair prune skips it 3 FAIL
Re-serialized-pair prune skips it 1 FAIL
Look-ahead removed (one write per pair) the write-count test FAILs
Whole fix reverted 6 FAIL
Bound refuses new pairs instead of evicting the bound test FAILs
Batch writes only its missing pairs the batch test FAILs
No latch after a refused write the latch test FAILs
Sweep unlinks past its deadline the deadline test FAILs
A refused write keeps the alias as well the refused-write test FAILs

Gate on release/0.8.0: flake8 and black are clean on the touched files, and mypy src/kiro_crew reports 0 issues in 1863 files. The projection, recovery, resume, agent-selection and session-control suites pass (515 passed). The full backend lane ran on #16073.

Manual verification

  • Tested on an isolated kirocrew pod tree. I published 11 real views and deleted the ledger, so the tree looked like it predated the ledger. A distribution's boot cleanup then removed only the aliases (11 aliases, 11 sidecars left), and the real drain_stale_aliases() ran after it. On the base: 0 sidecars and 0 ledger entries remained, and 0 of 11 stored names resolved through resolve_session_agent_bindings. With this branch: 0 sidecars, 11 ledger entries, and 11 of 11 resolved to their source agent.
  • Checked whether a mapping that was already deleted can be recovered. It cannot. The session record's template_id and selection_name both hold the view name. kiro-cli's sessions/cli/<sid>.json stores only session_state.agent_name, which is the alias. Lease records hold alias names only. The digest covers the agent, the home, the source path and the full view, including MCP env values that changed on every launch at the time, so it cannot be recomputed from installed agents.

Related Issues

Part of #16545

Follows #16024 and #16033, which map a stored view name back through source_agent_name. The deleting sweep came from #15150 (via #15295).

Pattern harvest

Rule candidate: review-prompt
Pattern: "deleting a record that a fallback resolver reads, without first writing the fallback": every path that unlinks a file which a lookup reads must write the lookup's durable fallback first, in the same locked section.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

@bolichen97
bolichen97 requested a review from a team as a code owner October 2, 2026 06:06
@bolichen97
bolichen97 requested review from buluoray and removed request for a team October 2, 2026 06:06
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 2, 2026
@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of 16f184a0379f4f92fbec98254be080390c9224b2 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Parent #16073 is still OPEN — verify it merges first, since the release branch must not land ahead of the main-branch original.

[DESIGN-REVIEWED] 16f184a

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ PASS

Premise-level review of 16f184a0379f4f92fbec98254be080390c9224b2 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

The description's own constraint — "#16073 is not merged yet, so this PR must not merge ahead of it" — is the one thing to verify before merging.

What this change ships

Inventory (8 items) — 8 justified

Intent: FIX — a chat stored under a generated kirocrew-skill-view-* name keeps resuming on the 0.8 line after the boot drain retires the view's last on-disk record (3 added tests fail on base).

  1. A stored view name still resolves after the drain deletes its alias and sidecar — justified
  2. An unrecoverable view's refusal says "pick the agent again, or start a new chat" instead of naming a restorable Crew Member — justified
  3. The ACP spawn path's RetiredSkillView error gains the same "or start a new chat" remedy — justified
  4. Ledger eviction order changed: sidecar-backed entries drop first, then oldest retired names — justified
  5. A full ledger that drops retired view names now warns once — justified
  6. On a refused ledger write, the sweep keeps its sidecar batch; the prune frees the alias but keeps the sidecar — justified
  7. A drain batch pays one ledger write per 64-candidate window (matches _PRUNE_MAX_RECLAIMS_PER_RUN = 64) instead of one per pair — justified
  8. _reclaim_prune_candidate now requires a ledger argument; the owning spec (acp-client.md) updated in the same commit — justified

Counts run: ownership-sidecar deletion sites (_unlink_projection_lease_if_unchanged with what="ownership sidecar") = 3, all covered; "is unavailable; restore it" message sites = 1, fixed; the rewritten eviction test matches a behavior change the description declares, not a silently reversed pin.

[FIRST-PRINCIPLES-REVIEWED] 16f184a

@bolichen97
bolichen97 force-pushed the backport/0.8.0-skill-view-sidecar-ledger branch from 73c5881 to dfb6151 Compare October 2, 2026 06:16
@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 16f184a0379f4f92fbec98254be080390c9224b2 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 16f184a

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 16f184a0379f4f92fbec98254be080390c9224b2: <one-sentence reason>

@bolichen97

Copy link
Copy Markdown
Contributor Author

Impact, traced against the core the internal 0.8.0.1/0.8.0.2 builds shipped (0.8.0-rc.1, which predates #15202 and 74cefff): this fix is defensive.

At that core, no real code path persists a kirocrew-skill-view-* name as any of the following: a session's agent, slot.agent, an execution record's template_id/selection_name, agent_override, or a crew-log/cron/crewmate record.

  • An untranslated alias can reach the client. The shared runtime rebuilds its projection before set_mode, while kiro-cli keeps its spawn-time alias, and frame translation only knows the current projection's aliases.
  • Every recorder refuses it first. chat_runner calls record_provider_agent_switch before it sets slot.agent. That call raises UnknownMemoryStore for an unresolvable name, because alias files never resolve (iter_agent_spec_files drops them). The handler then restores the prior selection.
  • Member slots veto the switch outright.
  • /agent is refused while the projection is on.
  • set_mode records the authored name.

So users of that build saw failed session starts or failed switches, not stored view names. The pod repro in the description wrote its session record by hand.

This PR still closes a real hole. Any stored view name, whether from a hand edit, an older record, or a future path, stays resolvable after the boot drain retires its sidecar. The refusal for a name nothing records now gives the remedy that works. It is not a P0, so please merge it on the normal schedule.

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed 16f184a0379f4f92fbec98254be080390c9224b2 — this comment is updated in place on each push.

Review details

No findings that block.

FINDING — src/kiro_crew/acp/skill_projection.py:434 — once the ledger sits at _VIEW_SOURCES_MAX with no surviving sidecars (exactly the state the boot drain of a pre-ledger backlog leaves), the new eviction pass if alias not in adding and _sidecar_answers_for(metadata_dir, alias, updated[alias]) never satisfies its len(updated) <= _VIEW_SOURCES_MAX break, so every ledger write — each sweep batch in _sweep_orphan_sidecars, each prune window, and _record_view_ledger on the publication path at line 2950 — performs up to 2048 os.stat calls (_read_json_regular_file → pinned_fs.lstat_by_name) under the held publication lock and deletes nothing, where the pre-PR eviction was updated.pop(next(iter(updated))) with no I/O at all → Fix: bound the redundancy pass to the number of entries that actually have to go (or skip it once it has examined that many without a hit) and let the existing second pass do the rest.

FINDING — src/kiro_crew/acp/skill_projection.py:449 — log = logger.debug if _LEDGER_EVICTION_WARNED else logger.warning latches a module global for the life of the process, so a drain that overflows the ledger across many batches states only its first dropped count out loud (≤256) while every later batch silently drops up to 256 more retired view names that nothing else can resolve, leaving the operator a warning that understates the loss by orders of magnitude → Fix: accumulate dropped across the drain, or rate-limit by time as _UNLINK_WARNING_INTERVAL_SECS already does in this module, and warn with the running total instead of latching after the smallest loss.

[OPUS-REVIEWED] 16f184a

Verdict parsed from the review's SHA-scoped output markers for commit 16f184a0379f4f92fbec98254be080390c9224b2.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 16f184a0379f4f92fbec98254be080390c9224b2: <one-sentence reason>

@bolichen97
bolichen97 force-pushed the backport/0.8.0-skill-view-sidecar-ledger branch from dfb6151 to 52d969e Compare October 2, 2026 06:49
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 2, 2026
@bolichen97
bolichen97 force-pushed the backport/0.8.0-skill-view-sidecar-ledger branch from 52d969e to bfeced8 Compare October 2, 2026 07:38
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 2, 2026
@bolichen97
bolichen97 force-pushed the backport/0.8.0-skill-view-sidecar-ledger branch from bfeced8 to 6a94910 Compare October 3, 2026 15:46
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 3, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author

Review findings on 2319a1feb2 (#16073) and bfeced8061 (#16074) are addressed in e6590c01d4 (#16074: 6a94910747):

  • Opus 5 (blocking): the ledger latched once full. The bound now degrades instead of refusing. Past _VIEW_SOURCES_MAX, entries a sidecar still answers for go first. Then the oldest retired names go, counted in a warning (once, then debug). A full ledger never stops the sweep or the prune.
    • This also closes GPT 5.6's earlier finding on bb5a2a75: retiring a sidecar can no longer leave its mapping unwritten while the sidecar is deleted.
    • The remaining trade-off is that names older than the newest 2048 retired views stop resolving. This is the bounded loss the Not a goal section accepts by not changing the bound, and it is now logged.
  • GPT 5.6 (blocking, fix(skill-view): backport recording a view's source agent before its sidecar is deleted to release/0.8.0 #16074): partial admission. retain_many now writes and verifies the whole admissible batch, not only its missing pairs. A pair the ledger already held becomes newest in the same write, so that write cannot evict it while its sidecar is about to go. A batch larger than the bound is refused as a whole.
  • Opus advisory: a refused write was retried for every candidate. _ViewLedgerWrites latches the failure for the rest of the prune or sweep.
  • Opus advisory: docstrings and spec. The writer docstring, the _VIEW_SOURCES_MAX comment and acp-client.md now state the actual retention order.

Each new rule has a test, and reverting it fails that test. All added lines are covered.

…eleted

A conversation stored under a kirocrew-skill-view-<digest> name maps back to
its agent through the view's ownership sidecar, then the view ledger. The
ledger is written only at publication, for the aliases that run publishes, so
a view a pre-ledger build published is resolvable through its
sidecar alone. The boot drain deletes exactly that record: the orphan-sidecar
sweep retires a sidecar whose alias an outside cleanup or an operator already
removed, and the prune retires the alias and sidecar together. The stored
chat is then refused as "unavailable" although its agent is installed.

Every sidecar deletion now records the view's alias -> agent pair in the view
ledger first, under the publication lock the deletion already holds, through
the one bounded, atomic ledger writer. The sweep records its whole batch in
one write; the prune records a window of upcoming candidates the first time a
pair needs it, so a drain of thousands of pairs costs one write per window.
A sidecar the ledger cannot take is kept; the prune still removes its alias,
since the sidecar then answers for the view on its own. Both sweep halves stay
inside the time budget. Each write makes the whole batch the newest entries;
past the ledger's bound, entries a sidecar still answers for go first, then
the oldest retired names (counted in a warning), so a full ledger never stops
the sweep. A write the filesystem refuses is not retried per candidate.

The tests run the real drain over a real published view with no ledger, in
each of the three deletion paths, and resume the stored name through
resolve_session_agent_bindings.

The RetiredSkillView text the ACP spawn path raises for a view nothing
records also names "or start a new chat" as a remedy.

(cherry picked from commit 221aa85)
… view

A resumed chat whose recorded skill-view name nothing maps back was refused
as "Crew Member 'kirocrew-skill-view-...' is unavailable; restore it or choose
a member". A generated view is not a member anyone can restore, and the view
name is a one-way digest, so the agent cannot be recovered. Say what works
instead: pick the agent for this chat again, or start a new chat.

(cherry picked from commit 9816ff6)
@bolichen97
bolichen97 force-pushed the backport/0.8.0-skill-view-sidecar-ledger branch from 6a94910 to 16f184a Compare October 3, 2026 16:11
@bolichen97

Copy link
Copy Markdown
Contributor Author

The GPT 5.6 finding on 6a94910747 ("Unreadable sidecars can evict the only valid mapping") is fixed in #16073 at 9816ff65e2 and in #16074 at 16f184a037.

Before this fix, a ledger entry counted as redundant whenever a file existed at its sidecar path. It now counts only when _sidecar_answers_for holds: the sidecar is a regular file with the managed marker, and it names the same agent the ledger entry names. This is the same reading _recorded_view_source uses. A sidecar that is unreadable, foreign, or names a different agent no longer makes its entry evictable.

test_the_bound_drops_redundant_entries_then_the_oldest_retired_ones now places an unreadable sidecar beside the oldest entry. If the check goes back to _path_exists, the test fails.

@bolichen97
bolichen97 merged commit b6850b2 into release/0.8.0 Oct 3, 2026
22 of 23 checks passed
@bolichen97
bolichen97 deleted the backport/0.8.0-skill-view-sidecar-ledger branch October 3, 2026 16:40
@github-actions github-actions Bot removed the readiness: checking Automated validation is still running label Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant