Repository navigation
fix(sqlite): open each database with one SQLite library - #18833
iamwhatever merged 1 commit into
Conversation
Symptom: the gateway mixed the stdlib sqlite3 with the bundled pysqlite3 that kiro_crew._sqlite_compat selects, on files the other library also had open. - The V1 store check (memory_stores._require_legacy_store_files) read a named store's memory.db with the stdlib driver. Its close released the open store's write lock, so another process could write inside the store's transaction. The knowledge inventory gauge did the same with knowledge.db. - The aws_control backup's conversation export and the usage API read kiro-cli's data.sqlite3 with the stdlib driver, while kiro_cli and kiro_prerequisite read it through the shim. A shim close released the export's read lock, so kiro-cli could reset the WAL mid-copy. - The lesson handlers' member-store check caught the stdlib sqlite3.Error around a shim call, so a damaged member database escaped as a 500 instead of the 503 store_unavailable. Root cause: two SQLite libraries in one process keep separate lock tables, and on POSIX closing any descriptor on a file drops every lock the process holds on it. Their exception classes are unrelated as well. Change: each of these modules takes sqlite3 from kiro_crew._sqlite_compat. On hosts without the bundled pysqlite3 the shim is the stdlib module and nothing changes.
UX Review (Opus 5.5, fork) — ⏭️ skippedRevision |
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS On a host with [DESIGN-REVIEWED] 8fe4cf4 Description read[DESCRIPTION-READ] 2bce405ae3c194c0546cc7a9fb4b4d4e2cbe7f35fe309d6ddcc9aee296548474 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Before merging, check that the two lock tests fail on base on a Linux x86_64 runner with What this change shipsInventory (7 items) — 7 justifiedIntent: open every database that two gateway code paths share through one SQLite library, so that one connection closing cannot drop another's POSIX locks, and so errors are caught by the class that is actually raised. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] 8fe4cf4 Description read[DESCRIPTION-READ] 142e53ca41028ffa43e8706c4a08a46707f42c7e55143cbb1613d76323c4e808 That sha256 names the PR description this verdict read. Recompute it with |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
iamwhatever
left a comment
There was a problem hiding this comment.
This PR fixes the root cause: five sites opened a database with the stdlib sqlite3, or caught its errors, while the rest of the process opens the same file through kiro_crew._sqlite_compat (pysqlite3). With two SQLite copies, a close drops the other copy's POSIX locks, and except sqlite3.Error misses the shim's exceptions. The diff only changes which library each site binds (memory_stores._require_legacy_store_files, metrics/inventory_gauges.py, aws_control/backend/backup.py, dashboard/handlers/kiro_usage_api.py, _prepare_member_lesson_store in dashboard/handlers/cron.py). Queries are unchanged, and I confirmed every sqlite3. use left in backup.py and kiro_usage_api.py is on kiro-cli's data.sqlite3 or its temp copy (connect with uri=True, Connection types, sqlite3.Error), which pysqlite3 supports the same way. The tests prove the fix: two child-process lock probes on real files, a 503 test fed the production reader's real error, and an import guard. The body shows all four red on origin/main and green on the branch. Base is main, checks are green, and GPT, Opus, Design and First Principles all pass on head 8fe4cf4 with no open change requests. On hosts without pysqlite3 the shim is the stdlib module, so behaviour there does not change. — crew-pr-reviewer
Problem / Motivation
Goal: Every database the gateway shares between two code paths is opened, and its errors caught, with one SQLite library.
kiro_crew._sqlite_compatselects the bundledpysqlite3where it is installed (Linux x86_64), and the memory, knowledge and kiro-cli readers open their databases through it.Two SQLite copies in one process keep separate lock tables, and on POSIX closing any descriptor on a file releases every lock the process holds on that file.
Their exception classes are unrelated as well (
issubclass(shim.Error, sqlite3.Error)is False).Three places still used the stdlib
sqlite3on a file the shim also opens:memory_stores._require_legacy_store_filesreads a named V1 store'smemory.dbwith the stdlib driver. Its close drops the open store's locks, including the write lock of a transaction in progress, so another process can take the write lock too. The knowledge inventory gauge (metrics/inventory_gauges.py) does the same withknowledge.db._export_cli_conversations) and the usage API (dashboard/handlers/kiro_usage_api.py) read kiro-cli'sdata.sqlite3with the stdlib driver, whilekiro_cli._open_state_db_readonlyandkiro_prerequisite._open_identity_db_readonlyread it through the shim. A shim close drops the export's read lock, so kiro-cli can reset the WAL the export is still copying from._prepare_member_lesson_store(dashboard/handlers/cron.py) catches the stdlibsqlite3.Erroraround opening a member database, which raises the shim's errors. A membermemory.dbthat is not a database escapes the check.Why it matters
store_unavailablethat names the reason. Nothing is half done: the check runs before any read or write.pysqlite3. On a host without it (macOS, Linux aarch64) the shim is the stdlib module, and this change makes no difference there.Not a goal
apps/builtins/auto_research(campaign storage, handlers),apps/builtins/personal_shopper/backend/store.py, andtaskq/store.pyandtaskq/migrate.py(tasks.db, also read by the doctor throughTaskStore). The snapshot engine opens.dbfiles through the shim, but only inside its component trees, and neithertasks/norapps/is a component.pod/runtime_home.py(copies the runtime's.sqlite3and.sqliteauth databases when a pod is staged) andonboarding_scan.py,onboarding_sources/*andonboarding_import.py(snapshots and other installs' files): not traced to a file the shim holds open in the same process at the same time.apps/builtins/aws_control/crew/runtime/container/supervisor/kiro_login.py,builtin_skills/security-conductor/scripts/*,eval/bench/run.py.vector_memory.pyandcli_commands.py, which import the stdlibErrorasStdlibSQLiteErrorto catch both drivers on purpose.What changed (motivation → approach → change)
sqlite3where every other reader of the same file uses the shim. Merged fix(sqlite): probe pysqlite3 for connect() at every import site #12135 moved the module-level import sites to the shim; these were missed.memory_stores._require_legacy_store_filesimportssqlite3fromkiro_crew._sqlite_compat, as the module's other two connect sites (_legacy_member_database_identity,_complete_legacy_member_database) already do.metrics/inventory_gauges.pybinds its module-levelsqlite3from the shim, so the knowledge-documents probe opensknowledge.dbwith the knowledge store's library.apps/builtins/aws_control/backend/backup.pyanddashboard/handlers/kiro_usage_api.pybind their module-levelsqlite3from the shim. Each module's every connect is on kiro-cli'sdata.sqlite3or a temporary copy of it, so nothing else they open changes driver.kiro_cli.pyandkiro_prerequisite.pyalready used the shim._prepare_member_lesson_storeimportssqlite3from the shim, asexecution_context.pyandmemory_stores.require_memory_storealready do around the same member-database read. A damaged member store answers 503store_unavailablewith its reason.except sqlite3.Errornow names the error class of the library that raises.import sqlite3andfrom sqlite3undersrc/was classified, and everyexceptclause naming a sqlite3 error class (102, by AST scan) was checked against the driver that raises; the ones left alone are under Not a goal with the reason._sqlite_compatmodule docstring).Backwards compatibility
Compatible: every changed site runs the same query and answers the same way; only the SQLite library behind the connection changes, to the one the file's other readers already use, and the lesson check now answers its documented 503 where it answered a 500.
Removes nothing: no function, setting or behaviour is removed.
Tests
test/test_memory_stores.py::TestTheV1StoreCheckKeepsTheOpenStoresLock::test_another_process_still_cannot_write_after_the_check(POSIX): atmp_pathV1 store with one synthetic episode starts a write transaction; a child process cannot take the write lock; the production_require_legacy_store_filesruns once; the child process still cannot take it.test/test_aws_control_backup_conversations.py::TestTheExportKeepsItsReadLock::test_a_kiro_cli_read_during_the_export_keeps_the_exports_lock(POSIX, needs descriptor pinning like the other export tests): the real_export_cli_conversationsruns over a synthetictmp_pathkiro-cli store with committed rows left in the WAL. Inside the export's read transaction, a child process triesPRAGMA wal_checkpoint(TRUNCATE), the productionkiro_cli._open_state_db_readonlyopens the file, queries and closes, and the child tries again. Both attempts must find the WAL busy.test/test_cron_handler_json_contract.py::test_a_damaged_member_store_answers_503_with_its_reason: the error the productionread_member_database_identityraises for atmp_pathfile of synthetic bytes is handed to_prepare_member_lesson_storethroughContextBuilder.ensure_store(as its V2 arm re-raises it); the check answers 503{"error": <reason>, "code": "store_unavailable"}.test/test_sqlite_compat_resolver.py::test_a_store_database_is_never_opened_with_the_stdlib_driver: a guard in the style oftest_the_idiom_lives_in_exactly_one_file, failing if any of the seven modules above (the five changed, pluskiro_cli.pyandkiro_prerequisite.py) regainsimport sqlite3orfrom sqlite3 import. The three newly bound modules are added toIMPORT_SITES.test/metrics/test_inventory_gauges.py::test_knowledge_probe_opens_read_onlyinterceptsconnecton the gauge's own driver (ig.sqlite3), so it still checks the probe's own connection.A child process is used for the lock tests because POSIX locks are per process: a connection in the test's own process cannot see them.
Red on unchanged
origin/mainat d32c084 (the five test files copied into a detached checkout oforigin/main,pytest -n 0): 4 failed, 208 passed.Green on this branch: the five test files together, 212 passed in each of 3 serial runs. The two child-process probes (
test_aws_control_backup_conversations.py,test_memory_stores.py) decode their output withencoding="utf-8".Gates exit 0: mypy, flake8, isort --check-only, black --check,
scripts/check_black_formatting.py,COMMENT_HISTORY_BASE_REF=origin/main scripts/check_comment_history.py,scripts/docs-lint.sh,scripts/check_memory_store_seam.py.Manual verification
N/A — unit coverage sufficient: each lock test drives the production code against a real SQLite file and observes the lock from a second process, and the lesson test drives the production check with the error the production reader raises.
Related Issues
Related: merged #12135 (probe pysqlite3 for
connect()at every import site), which moved the other import sites to the shim.No existing issue found (GitHub search for
pysqlite3,two copies of SQLite,sqlite3.Error pysqlite3,memory.db corruption,_prepare_member_lesson_store,store_unavailable lessons,data.sqlite3 backup).Open PRs that touch the changed files (#9815, #10365, #11397, #11831, #13408, #14423, #15661, #15953, #16886, #17526, #17554, #18159, #18406) edit other functions or other lines; none touches the lines this diff changes.
Pattern harvest
Rule candidate: semgrep
Pattern: a stdlib
sqlite3connect, or a stdlibsqlite3.Errorhandler, on a database that another code path in the same process opens throughkiro_crew._sqlite_compat.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)