Repository navigation
fix(memory): keep the shared -wal when another process closes the memory store - #25852
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS No design-level problem blocks this. The fix targets the real mechanism, the close that deletes the Notes
[DESIGN-REVIEWED] d18c770 Description read[DESCRIPTION-READ] 717a3228e8da718b1e51f9446ecd1187c871bdeec1c049f99ac50f7a38c61830 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS The fix works at the mechanism level and states its gaps. One read-write sibling on Notes
What this change shipsInventory (6 items) — 5 justifiedIntent: stop a second process's close of the memory store from deleting the
[FIRST-PRINCIPLES-REVIEWED] d18c770 Description read[DESCRIPTION-READ] de190d5b1c5fcad770399cfa54e9b51dc433c45904580e00b44c5756c399afae That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
6cb5758 to
ab53e26
Compare
c53d5fd to
c7b1223
Compare
c7b1223 to
a922631
Compare
|
a922631 to
2cdd5b1
Compare
2cdd5b1 to
68581ed
Compare
68581ed to
d18c770
Compare
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Approved on behalf of the watch holder (bolichen). Design and First Principles pass on this head, UX skipped or pass, no conflict, no open change request.
Backports these main PRs to release/0.11.0, in pick order. Each was a `git cherry-pick -x` of its squash commit on main: - fix(memory): read memory.db off the writer lock, split background pool (#24290) (cherry picked from commit fed830c) - fix(dashboard): bound the pre-prompt build's wait to enter the memory pool (#24414) (cherry picked from commit 520cf2c) - fix(memory): keep the shared -wal when another process closes the memory store (#25852) (cherry picked from commit dc63bb6) - fix: read the page-cache prefetch in a child to keep sqlite locks (#26302) (cherry picked from commit 81dde23) - perf(memory): score facts from a term index and prebuild after restart (#25328) (cherry picked from commit 6824348) - fix(desktop): retry a failed netstat before Windows recovery gives up (#25792) (cherry picked from commit 043dd9f) - fix(cli): say a gateway still serves the port when restart's replacement is refused (#25366) (cherry picked from commit 2e13127) - fix(cli): refuse to stop or restart a gateway the desktop app supervises (#25705) (cherry picked from commit bbf0910) - fix(hooks): approve kiro-cli read-only file searches after the deny tiers (#25150) (cherry picked from commit a0c0bd4) - fix(settings): refuse a config save that config.local.json overrides (#26216) (cherry picked from commit 7f8688e) - fix(workflows): write non-finite floats as null so runs open (#26085) (cherry picked from commit 3591282) - fix(approvals): give each background tool approval its own id (#25905) (cherry picked from commit 51ef3bb) - fix(chat): record file changes for kiro-cli v3 write tools (#25650) (cherry picked from commit 5369270) - fix(sandbox): stop blaming a host-wide OOM kill on a per-tree limit (#25647) (cherry picked from commit 791d2a2) - fix(acp): replay history when a KAS resume restores no turns (#25521) (cherry picked from commit 47c71cb) Conflicts resolved by hand: - #26302: test/spawn_audit_helpers.py adds only the page_cache.py entry (the embeddings_child.py entry names a module 0.11.0 lacks); test/test_page_cache_prefetch.py takes main's file. - #25328: test/test_dashboard_server_composition_contract.py lazy-import count 93 -> 96 (the three names it adds). - #25705: test/test_cli_restart_refused_message.py takes main's file. - #26085: dashboard/handlers/workflows.py adds import math, keeps the release typing import. Left out: #23920 (already on release/0.11.0), #26144 (needs sandbox_launch_mark from #24709), #26088 (fences a main-only port).
Backports these main PRs to release/0.11.0, in pick order. Each was a `git cherry-pick -x` of its squash commit on main: - fix(memory): read memory.db off the writer lock, split background pool (#24290) (cherry picked from commit fed830c) - fix(dashboard): bound the pre-prompt build's wait to enter the memory pool (#24414) (cherry picked from commit 520cf2c) - fix(memory): keep the shared -wal when another process closes the memory store (#25852) (cherry picked from commit dc63bb6) - fix: read the page-cache prefetch in a child to keep sqlite locks (#26302) (cherry picked from commit 81dde23) - perf(memory): score facts from a term index and prebuild after restart (#25328) (cherry picked from commit 6824348) - fix(desktop): retry a failed netstat before Windows recovery gives up (#25792) (cherry picked from commit 043dd9f) - fix(cli): say a gateway still serves the port when restart's replacement is refused (#25366) (cherry picked from commit 2e13127) - fix(cli): refuse to stop or restart a gateway the desktop app supervises (#25705) (cherry picked from commit bbf0910) - fix(hooks): approve kiro-cli read-only file searches after the deny tiers (#25150) (cherry picked from commit a0c0bd4) - fix(settings): refuse a config save that config.local.json overrides (#26216) (cherry picked from commit 7f8688e) - fix(workflows): write non-finite floats as null so runs open (#26085) (cherry picked from commit 3591282) - fix(approvals): give each background tool approval its own id (#25905) (cherry picked from commit 51ef3bb) - fix(chat): record file changes for kiro-cli v3 write tools (#25650) (cherry picked from commit 5369270) - fix(sandbox): stop blaming a host-wide OOM kill on a per-tree limit (#25647) (cherry picked from commit 791d2a2) - fix(acp): replay history when a KAS resume restores no turns (#25521) (cherry picked from commit 47c71cb) - fix(kas): auto-approve granted built-ins after Crew's own checks (#25339) (cherry picked from commit dba4de5) - fix(kas): mount web_search as the engine's remote_web_search (#26435) (cherry picked from commit 65c21b5) - fix(kas): answer a spec's shell.allowedCommands at the gate, after Crew's checks (#25539) (cherry picked from commit 39d07d9) - fix(kas): relay an MCP server's boolean env flags, never its secrets (#25331) (cherry picked from commit b734a68) - fix(security): read the -c script past options and redirections (#26087) (cherry picked from commit fea0920) - fix(kas): refuse a shell command a denyByDefault spec does not allow (#26501) (cherry picked from commit 9f0d02f) - fix(kas): name a deniedCommands deny aimed at a redirect as not carried (#26489) (cherry picked from commit 4951c0c) - fix(security): delete the curl-upload deny rule (#26469) (cherry picked from commit 70e206e) Conflicts resolved by hand: - #26302: test/spawn_audit_helpers.py adds only the page_cache.py entry (the embeddings_child.py entry names a module 0.11.0 lacks); test/test_page_cache_prefetch.py takes main's file. - #25328: test/test_dashboard_server_composition_contract.py lazy-import count 93 -> 96 (the three names it adds). - #25705: test/test_cli_restart_refused_message.py takes main's file. - #26085: dashboard/handlers/workflows.py adds import math, keeps the release typing import. - #25339: platform/governance.py keeps the release rule for a missing ceiling (main's enterprise-baseline clause is not on 0.11.0). - #26435: harness-parity.md H6 row edited in place on the release text. - #25539: its acp-client.md and agent-spec-fields.md lines are left out; the paragraphs they edit are not on 0.11.0 (security.md carries it). - #25331: the test and two doc mentions use hoist_managed_servers / resume=, the 0.11.0 names of main's renamed hoist_servers_for_session. - #26087: security package line budget 32,552 -> 32,714 (re-measured). - #26501: kas_permissions.py comment keeps 0.11.0's shapes (main's subagent rules are absent); its acp-client.md line is left out, and the agent-spec-fields.md toolsSettings row says denyByDefault is gate-enforced. - #26489: its agent-spec-fields.md sentence is added to the 0.11.0 row. - #26469: test_denied_commands_security.py keeps 0.11.0's comment; count 113 -> 112. Test fixes (host-dependent tests made hermetic; none loosens an assertion): - tmpdir_helpers.short_tmp_path: a tmp_path override for modules whose code judges a path by its components (a .kiro/crew data-home part); an agent session's TMPDIR under ~/.kiro/crew made every test path the data home. Used by the file-explorer, design-tweak (both), spawn-offload, launch-copy, home-pin, native-skill and auto-improvement-members tests; the sage ancestor-chain test creates its dir under /tmp for the same reason. - pi-gate tests and the remote-session Python 3.9 test take the first real binary on PATH (playwright_cli_installer_helpers._native_tool_on_path): a mise shim under a scratch HOME hangs or writes to stdout. - snapshot-merge ACL test names its own uid; nobody (65534) is the overflow uid inside a user namespace and cannot be named in an ACL. - installer-step and push_guard interrupt tests spawn with SIGINT at its default; a background pytest run starts with SIGINT ignored, which a shell trap and Python both keep. Left out: #23920 (already on release/0.11.0), #26144 (needs sandbox_launch_mark from #24709), #26088 (fences a main-only port), #25346 (11 of its tests fail on 0.11.0: the kiro-cli skill scope it compares against differs there), #26397 (needs #26152's new download route, not on 0.11.0), #26428 (not merged).
…26487) Backports these main PRs to release/0.11.0, in pick order. Each was a `git cherry-pick -x` of its squash commit on main: - fix(memory): read memory.db off the writer lock, split background pool (#24290) (cherry picked from commit fed830c) - fix(dashboard): bound the pre-prompt build's wait to enter the memory pool (#24414) (cherry picked from commit 520cf2c) - fix(memory): keep the shared -wal when another process closes the memory store (#25852) (cherry picked from commit dc63bb6) - fix: read the page-cache prefetch in a child to keep sqlite locks (#26302) (cherry picked from commit 81dde23) - perf(memory): score facts from a term index and prebuild after restart (#25328) (cherry picked from commit 6824348) - fix(desktop): retry a failed netstat before Windows recovery gives up (#25792) (cherry picked from commit 043dd9f) - fix(cli): say a gateway still serves the port when restart's replacement is refused (#25366) (cherry picked from commit 2e13127) - fix(cli): refuse to stop or restart a gateway the desktop app supervises (#25705) (cherry picked from commit bbf0910) - fix(hooks): approve kiro-cli read-only file searches after the deny tiers (#25150) (cherry picked from commit a0c0bd4) - fix(settings): refuse a config save that config.local.json overrides (#26216) (cherry picked from commit 7f8688e) - fix(workflows): write non-finite floats as null so runs open (#26085) (cherry picked from commit 3591282) - fix(approvals): give each background tool approval its own id (#25905) (cherry picked from commit 51ef3bb) - fix(chat): record file changes for kiro-cli v3 write tools (#25650) (cherry picked from commit 5369270) - fix(sandbox): stop blaming a host-wide OOM kill on a per-tree limit (#25647) (cherry picked from commit 791d2a2) - fix(acp): replay history when a KAS resume restores no turns (#25521) (cherry picked from commit 47c71cb) - fix(kas): auto-approve granted built-ins after Crew's own checks (#25339) (cherry picked from commit dba4de5) - fix(kas): mount web_search as the engine's remote_web_search (#26435) (cherry picked from commit 65c21b5) - fix(kas): answer a spec's shell.allowedCommands at the gate, after Crew's checks (#25539) (cherry picked from commit 39d07d9) - fix(kas): relay an MCP server's boolean env flags, never its secrets (#25331) (cherry picked from commit b734a68) - fix(security): read the -c script past options and redirections (#26087) (cherry picked from commit fea0920) - fix(kas): refuse a shell command a denyByDefault spec does not allow (#26501) (cherry picked from commit 9f0d02f) - fix(kas): name a deniedCommands deny aimed at a redirect as not carried (#26489) (cherry picked from commit 4951c0c) - fix(security): delete the curl-upload deny rule (#26469) (cherry picked from commit 70e206e) Conflicts resolved by hand: - #26302: test/spawn_audit_helpers.py adds only the page_cache.py entry (the embeddings_child.py entry names a module 0.11.0 lacks); test/test_page_cache_prefetch.py takes main's file. - #25328: test/test_dashboard_server_composition_contract.py lazy-import count 93 -> 96 (the three names it adds). - #25705: test/test_cli_restart_refused_message.py takes main's file. - #26085: dashboard/handlers/workflows.py adds import math, keeps the release typing import. - #25339: platform/governance.py keeps the release rule for a missing ceiling (main's enterprise-baseline clause is not on 0.11.0). - #26435: harness-parity.md H6 row edited in place on the release text. - #25539: its acp-client.md and agent-spec-fields.md lines are left out; the paragraphs they edit are not on 0.11.0 (security.md carries it). - #25331: the test and two doc mentions use hoist_managed_servers / resume=, the 0.11.0 names of main's renamed hoist_servers_for_session. - #26087: security package line budget 32,552 -> 32,714 (re-measured). - #26501: kas_permissions.py comment keeps 0.11.0's shapes (main's subagent rules are absent); its acp-client.md line is left out, and the agent-spec-fields.md toolsSettings row says denyByDefault is gate-enforced. - #26489: its agent-spec-fields.md sentence is added to the 0.11.0 row. - #26469: test_denied_commands_security.py keeps 0.11.0's comment; count 113 -> 112. Test fixes (host-dependent tests made hermetic; none loosens an assertion): - tmpdir_helpers.short_tmp_path: a tmp_path override for modules whose code judges a path by its components (a .kiro/crew data-home part); an agent session's TMPDIR under ~/.kiro/crew made every test path the data home. Used by the file-explorer, design-tweak (both), spawn-offload, launch-copy, home-pin, native-skill and auto-improvement-members tests; the sage ancestor-chain test creates its dir under /tmp for the same reason. - pi-gate tests and the remote-session Python 3.9 test take the first real binary on PATH (playwright_cli_installer_helpers._native_tool_on_path): a mise shim under a scratch HOME hangs or writes to stdout. - snapshot-merge ACL test names its own uid; nobody (65534) is the overflow uid inside a user namespace and cannot be named in an ACL. - installer-step and push_guard interrupt tests spawn with SIGINT at its default; a background pytest run starts with SIGINT ignored, which a shell trap and Python both keep. Left out: #23920 (already on release/0.11.0), #26144 (needs sandbox_launch_mark from #24709), #26088 (fences a main-only port), #25346 (11 of its tests fail on 0.11.0: the kiro-cli skill scope it compares against differs there), #26397 (needs #26152's new download route, not on 0.11.0), #26428 (not merged). Co-authored-by: w <w@k.local>
Problem / Motivation
Goal: A CLI command or second process that closes the memory store no longer deletes the write log the gateway is still using, so the memory database cannot split.
The gateway keeps
memory.dbopen in WAL mode. A POSIX process loses all its record locks on a file when it closes any descriptor of that file. After that, when another process (for examplekirocrew learn listorkirocrew memory export) closes its last connection, SQLite sees no lock, checkpoints, and deletesmemory.db-walandmemory.db-shm. The gateway keeps committing into the unlinked-wal. Other processes never see those commits, and pages are later written from two logs: lost rows,database disk image is malformed, and a gateway that fails to start.Why it matters
This is silent data loss in the user's memory. On one install it cost 8 rows and an index page, broke memory writes for about 8 hours, and every backup taken after the split carried the damage.
Not a goal
Connection.setconfig. On Linux x86_64 the store usespysqlite3, which has nosetconfig, so its close keeps SQLite's default and those hosts stay exposed.What changed (motivation → approach → change)
Symptom: another process's close deletes the
-walthe gateway writes through. Root cause: SQLite deletes the-walon the last close whenever it cannot see another process's lock, and a lost POSIX lock is invisible.src/kiro_crew/sqlite_wal.py: newkeep_wal_on_closesetsSQLITE_DBCONFIG_NO_CKPT_ON_CLOSE, and newtruncate_wal_before_closerunswal_checkpoint(TRUNCATE).src/kiro_crew/vector_memory_lifecycle.pyandsrc/kiro_crew/vector_memory.py: the memory store's write connections usekeep_wal_on_close, so the store's close never deletes the-walor-shm. Before closing, the store runstruncate_wal_before_close. So a stopped store still leaves no WAL content, only an empty-wal. The checkpoint goes through the-shmlocks, which the gateway keeps, so it is safe while the gateway runs.memory.dbwith the store's driver instead of the stdlibsqlite3. Wherepysqlite3is installed, the stdlib module is a second copy of SQLite in the gateway, and its close also dropped the store's locks.Known gaps: Linux x86_64 (
pysqlite3, nosetconfig) is not covered, and the first lock-loss cause is unknown.Backwards compatibility
Compatible: committed data is unchanged. A closed store now leaves an empty
memory.db-waland amemory.db-shmbeside the file instead of deleting them, and SQLite reads or resets both on the next open.Removes nothing: the change only stops a close from deleting files; no API, key, row or path is removed.
Tests
test/test_vector_memory_wal_split.py: a store that lost its locks keeps its-wal(same inode) after another process opens and closes the store, and its later commits are visible to a fresh reader with anokintegrity check. A closed store leaves no WAL content.keep_wal_on_closereports a driver withoutsetconfig. The probe uses the store's driver.test/test_sqlite_wal.py: the two new helpers on a plain stdlib connection, so they are covered on every driver: a kept connection's close leaves its-wal, a refused option reports False, the truncate empties the-walin place, and it never raises.test/test_bench_round9.py: the leaked-store check no longer reads leftover sidecars, which a closed store now keeps. It probes for an open connection instead: leaving WAL mode needs an exclusive lock..github/black-baseline.txt:test/test_bench_round9.pyis now black-clean, so its baseline row is removed.test/test_memory_backup.py: two assumptions updated: a closed store's-walis empty rather than absent, and a restore sets the emptied-wal/-shmaside with the displaced file.Proof: PROVEN test/test_vector_memory_wal_split.py::test_another_process_close_keeps_the_wal_a_lockless_writer_uses (head e227912)
Determinism: the new tests use only
tmp_pathand one bounded child process, with no timing or ordering dependence. 10/10 repeated runs and 5/5 runs in shuffled order passed.Manual verification
Reproduced on a scratch database (Linux, SQLite 3.50.4): process A writes in WAL mode, a plain
open()/close()drops its lock, process B opens read-write and closes. On main the-walis deleted and a fresh reader getsdisk I/O error. WithNO_CKPT_ON_CLOSEon B the-walsurvives and both readers see every row.Related Issues
Closes #25846
Sibling sweep
Swept:
git grep -nE "^\s*import sqlite3\b"insrc/kiro_crew, intersected with files that openmemory.dbSibling: src/kiro_crew/diagnose_probes.py:593 -- changed here
Sibling: src/kiro_crew/vector_memory_member.py:123 -- not affected -- provisioning opens a new member database nobody else holds yet
Sibling: src/kiro_crew/memory_stores.py:1335 -- not affected -- one-time upgrade of an older member file at startup, before the store opens it
Pattern harvest
Rule candidate: review-prompt
Pattern: a connection to a shared WAL database whose close may delete the
-walanother process still writes throughChecklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)