Repository navigation
fix(update): copy memory before the dashboard and unattended appliers rewrite the install - #21873
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that the dashboard Update request, which now waits for the copy of every memory store before it answers, finishes in reasonable time on an install with large stores. [DESIGN-REVIEWED] f1f4ed0 Description read[DESCRIPTION-READ] c32f7d07ff86da2540780a6454124973f2cd1b3725c72b7775fffaf276045720 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS When #21898 moves the policy copy to the provider seam, it must also delete the caller-level copy at What this change shipsInventory (7 items) — 6 justifiedIntent: one Update click or unattended update can no longer lose up to a day of
[FIRST-PRINCIPLES-REVIEWED] f1f4ed0 Description read[DESCRIPTION-READ] 872364252cbfed216bbcfa7000df8987386c798d878d8a564b8bd72924a96a80 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
bolichen97
left a comment
There was a problem hiding this comment.
Close to ready. The copy is in the right place on both appliers, and it matches what kirocrew update does. One should-fix in the new placement test.
Head reviewed: 64fe1d1.
What I checked on this head:
- Dashboard apply: the copy runs after every refusal that leaves the tree alone (dirty tree, fetch, divergence, upstream pin, interpreter floor) and before
_applyis scheduled. A failed copy returns 500 with no worker. The policy branch uses the samecan_apply()gate as the CLI. - Unattended apply: the copy runs after step 5 and before the update ownership step and
git reset --hard. A failed copy redacts and caps the text, sends one notice keyed on the target, and returns without a restart. - The copy runs through
asyncio.to_thread, so it does not block the event loop. - Issue #12440 is tier T1. No file is on the security path denylist. One commit,
fix:title, Pattern harvest present.
Should-fix (inline): the placement pin is satisfied in api_update_apply by the policy-branch copy, so deleting the git-path copy keeps it green. I confirmed this with a mutation.
Open items from the automated lanes, for the author to answer:
- The two
provider.apply()calls inupdate_schedule.pystill run with no copy. The description defers them to a follow-up. Please link that follow-up issue in the description. - The copy now runs inside the
POST /api/updaterequest, so a large store makes the request wait for the copy. Please confirm this stays inside the dashboard request timeout for a large store.
CI: Build, CI and CodeQL show as cancelled on this head, and readiness is still checking. A rerun is needed before merge.
64fe1d1 to
790fcd9
Compare
790fcd9 to
8b5627a
Compare
|
|
|
|
|
8b5627a to
7b640ab
Compare
|
|
|
|
|
|
|
|
|
6180ae8 to
5c73076
Compare
|
|
|
bolichen97
left a comment
There was a problem hiding this comment.
Head reviewed: 5c73076 (rebased onto a newer main, plus a real change since fa2777a).
What changed since my last review: the unattended git applier now runs every pre-reset refusal (commits ahead, exec config, tracked edits, hidden edits, untracked collisions, interpreter floor) through one _refused_before_reset() helper, called once before the memory copy and once after it. That is a good answer to both concerns at once. A checkout that keeps refusing no longer spends a copy every check and rotates backups out of memory.backup_keep, and an edit made while the copy runs is still caught before reset --hard. The new test_every_refusal_runs_before_any_memory_copy cases and test_an_edit_made_during_the_copy_refuses_the_reset pin both orders.
One blocker on this head: Backend Lint & Type Check is red in the mypy step, and it comes from this refactor. Moving the checks into a nested function drops the if _git is None narrowing from the outer scope, so mypy sees str | None at three call sites. I reproduced it locally with mypy on update_apply.py: errors at lines 401, 480 and 600. Details inline.
Verified locally on this head: test_slack_gateway_update_reset.py, test_update_memory_copy_placement.py, test_dashboard_updates_coverage_apply.py, test_facade_mirror_guards.py and test_spawn_audit.py, 174 passed.
The min_version question is unchanged by this push: wheel_apply.py and the dashboard handler carry the same behaviour as fa2777a, so a store that never copies still holds back a forced min_version update. That stays a maintainer decision, so this review is a comment, not an approval.
|
|
bolichen97
left a comment
There was a problem hiding this comment.
Incremental review of 52145b8 against 5c73076 (same base, the only changes are in update_apply.py and test/link_screen_sites.py).
The mypy failure from my last review is fixed. _git is now narrowed once into git_bin: str right after the is None return, and the three uses inside _refused_before_reset (git status --porcelain, git diff --name-only --diff-filter=A, and the incoming_python_floor_breach call) read git_bin, so the nested helper no longer sees str | None. Backend Lint & Type Check is green on this head. The link_screen_sites.py entry now names the helper at its new nesting depth (_auto_apply_update._refused_before_reset._obstructions), which matches where _obstructions lives after the refactor. Both edits are behaviour neutral: same binary, same arguments.
No new findings. What I verified: the interdiff between the two heads, the narrowing site and every git_bin use in the file, and the check rollup (some backend test shards were still running when I looked).
One thing remains open and is not mine to decide: whether the update min_version floor should move with this change. That is a maintainer call, so I am leaving this as a comment rather than an approval.
|
Review-ready at 52145b8: PR Readiness passed ( Open for the maintainer:
Not merged, approved, or set to auto-merge; that is the maintainer's call. |
… rewrite the install
|
Rebased onto main 6c133fa to clear a conflict: main split |
|
|
|
Review-ready at f1f4ed0: PR Readiness passed ( Open for the maintainer:
Not merged, approved, or set to auto-merge; that is the maintainer's call. |
chenmingwei23
left a comment
There was a problem hiding this comment.
Approved: readiness passed, no conflicts, checks green.
Problem / Motivation
Goal: The dashboard's Update button and the gateway's unattended update take the same fresh copy of every memory store that
kirocrew updatetakes, and never rewrite the install when that copy did not land.kirocrew updatecopies every memory store right before it rewrites the install, and refuses when the copy fails (#12428). Two other paths rewrite the same install with no copy at all:api_update_apply(the dashboard's Update button) andGatewayOrchestrator._auto_apply_update(the unattended update, including the one a policymin_versionfloor forces). A bad upgrade on either path could cost up to a day of memory, the gap since the last scheduled copy.Why it matters
memory.dbis the one file in the crew home that nothing else can rebuild. The unattended path runs when nobody is watching, so it is the path where a lost store is noticed last.Not a goal
reset --hardcalls undersrc/kiro_crew/apps/builtins/auto_improvement/: they reset a scratch clone, not this install (the issue's own out-of-scope list).slack/gateway_runtime/update_schedule.py(provider.apply()). The issue does not name it, and it is a separate applier; it is raised for a follow-up instead.before_promotehook (wheel_apply.memory_snapshot_hook).What changed (motivation → approach → change)
Both appliers rewrote the install with no copy, because only the CLI called the copy step. Each now calls that same step,
wheel_apply.snapshot_memory_before_update, before it rewrites anything. It runs on a worker thread, since it copies SQLite files. The refusal text now has one spelling,wheel_apply.snapshot_refusal(src/kiro_crew/platform/wheel_apply.py), shared by the wheel hook, both new sites and the CLI (src/kiro_crew/cli_server_update.py). The memory spec (docs/system-specs/modules/memory-skills-hooks.md) now says every applier copies first and how each one refuses.src/kiro_crew/dashboard/handlers/updates_apply.py). A failed copy refuses the request with HTTP 500, codememory_snapshot_failed, and the same "Not updating … kirocrew memory backups" message the CLI prints. No fast-forward is started. A shutdown that begins during the copy answers 503shutting_down, so no installer starts after the shutdown sweep. A policy-defined update command gets the same gate when it can apply, matching the CLI.src/kiro_crew/slack/gateway_runtime/update_apply.py). A failed copy skips this update. It logs a warning, shows one dashboard notice and a failed progress line, and tries again on the next check. It never resets without a copy and never exits the gateway. Every one of its refusals (local commits, repo exec config, a dirty tree, hidden edits, untracked collisions, the interpreter floor) runs once before the copy, so a checkout that keeps being refused takes no copy and rotates no backup out. All of them then run again after the copy, directly beforereset --hard, so a change made during the copy is still caught.A new structural test reads the syntax tree of the three applier files. Each call that rewrites the install must have a copy call after the previous rewrite and before it, so one update path cannot borrow another path's copy, and a new rewrite added without one fails CI, as the Design lane asked on #12428.
Backwards compatibility
Compatible: an install whose stores copy cleanly updates exactly as before. An install with no stores yet copies zero and goes on, as the CLI does.
Removes nothing: the change only adds a refusal on a failed copy, the same one
kirocrew updatealready makes. No key, route, field or function is removed or renamed.Tests
test/test_dashboard_updates_coverage_apply.py::TestApplyRefusals::test_memory_is_copied_before_the_fast_forward: the copy runs once, beforegit merge --ff-only.…::test_a_failed_memory_copy_refuses_the_apply: a failed copy answers 500memory_snapshot_failed, schedules no worker and runs no merge.…::test_a_policy_apply_copies_memory_first_and_refuses_without_it: a policy apply copies first, and a failed copy stops it.test/test_slack_gateway_update_reset.py::TestAutoApplyUpdateResetPath::test_memory_is_copied_before_the_reset: the copy runs once, beforegit reset --hard.…::test_a_failed_memory_copy_skips_the_update_and_says_so: no reset, no restart, one notice, one failed progress line.test/test_update_memory_copy_placement.py: every rewrite in the three applier files has its own copy since the previous rewrite, the known appliers are all found, and a second path that only has the first path's copy above it is flagged. Deleting the dashboard git path's copy now fails it.All six fail with the source change reverted and the tests kept (5 failed, and the placement test fails for both appliers). Determinism: 10 repeated runs green, plus one shuffled order green. The existing applier tests (743 across 13 files) still pass.
Manual verification
N/A — unit coverage sufficient: both appliers are driven end to end through their real handler and orchestrator code with git and the copy step stubbed.
Related Issues
Closes #12440
Follow-up for the unattended policy-provider applier: #21898. This PR fixes the gap at the caller level (each applier takes the copy itself); #21898 is the cause-level fix, moving the policy path's copy to the one seam every policy apply goes through.
Pattern harvest
Rule candidate: review-prompt
Pattern: a new path that rewrites the install without calling the pre-update memory copy.
test/test_update_memory_copy_placement.pynow enforces it for the three applier files.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)