Skip to content

fix(sqlite): open each database with one SQLite library - #18833

Merged
iamwhatever merged 1 commit into
kirodotdev:mainfrom
javenciu:fix/memory-store-probe-uses-sqlite-shim
Oct 10, 2026
Merged

iamwhatever merged 1 commit into
kirodotdev:mainfrom
javenciu:fix/memory-store-probe-uses-sqlite-shim

Conversation

@javenciu

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Every database the gateway shares between two code paths is opened, and its errors caught, with one SQLite library.

kiro_crew._sqlite_compat selects the bundled pysqlite3 where it is installed (Linux x86_64), and the memory, knowledge and kiro-cli readers open their databases through it.
Two SQLite copies in one process keep separate lock tables, and on POSIX closing any descriptor on a file releases every lock the process holds on that file.
Their exception classes are unrelated as well (issubclass(shim.Error, sqlite3.Error) is False).
Three places still used the stdlib sqlite3 on a file the shim also opens:

  • memory_stores._require_legacy_store_files reads a named V1 store's memory.db with the stdlib driver. Its close drops the open store's locks, including the write lock of a transaction in progress, so another process can take the write lock too. The knowledge inventory gauge (metrics/inventory_gauges.py) does the same with knowledge.db.
  • The aws_control backup's conversation export (_export_cli_conversations) and the usage API (dashboard/handlers/kiro_usage_api.py) read kiro-cli's data.sqlite3 with the stdlib driver, while kiro_cli._open_state_db_readonly and kiro_prerequisite._open_identity_db_readonly read it through the shim. A shim close drops the export's read lock, so kiro-cli can reset the WAL the export is still copying from.
  • _prepare_member_lesson_store (dashboard/handlers/cron.py) catches the stdlib sqlite3.Error around opening a member database, which raises the shim's errors. A member memory.db that is not a database escapes the check.

Why it matters

  • A named V1 store: another process (the CLI, a cron command, a second gateway or pod) can write inside the store's transaction, which risks a corrupted store ("database disk image is malformed") or a lost write.
  • The backup: the export holds one read transaction so that "a writer committing mid-copy cannot make two tables disagree". With its read lock gone, the archive can mix two points in time while the run reports success.
  • The Lessons panel: list, create and delete answer aiohttp's generic 500 with a traceback in the log, instead of the 503 store_unavailable that names the reason. Nothing is half done: the check runs before any read or write.
  • All three need the bundled pysqlite3. On a host without it (macOS, Linux aarch64) the shim is the stdlib module, and this change makes no difference there.

Not a goal

  • Modules that open only their own database, and only with the stdlib driver, so one library holds every connection to that file: apps/builtins/auto_research (campaign storage, handlers), apps/builtins/personal_shopper/backend/store.py, and taskq/store.py and taskq/migrate.py (tasks.db, also read by the doctor through TaskStore). The snapshot engine opens .db files through the shim, but only inside its component trees, and neither tasks/ nor apps/ is a component.
  • pod/runtime_home.py (copies the runtime's .sqlite3 and .sqlite auth databases when a pod is staged) and onboarding_scan.py, onboarding_sources/* and onboarding_import.py (snapshots and other installs' files): not traced to a file the shim holds open in the same process at the same time.
  • Code that runs outside the gateway process: apps/builtins/aws_control/crew/runtime/container/supervisor/kiro_login.py, builtin_skills/security-conductor/scripts/*, eval/bench/run.py.
  • vector_memory.py and cli_commands.py, which import the stdlib Error as StdlibSQLiteError to catch both drivers on purpose.

What changed (motivation → approach → change)

  • Root cause, all three: a module bound the stdlib sqlite3 where every other reader of the same file uses the shim. Merged fix(sqlite): probe pysqlite3 for connect() at every import site #12135 moved the module-level import sites to the shim; these were missed.
  • memory_stores._require_legacy_store_files imports sqlite3 from kiro_crew._sqlite_compat, as the module's other two connect sites (_legacy_member_database_identity, _complete_legacy_member_database) already do.
  • metrics/inventory_gauges.py binds its module-level sqlite3 from the shim, so the knowledge-documents probe opens knowledge.db with the knowledge store's library.
  • apps/builtins/aws_control/backend/backup.py and dashboard/handlers/kiro_usage_api.py bind their module-level sqlite3 from the shim. Each module's every connect is on kiro-cli's data.sqlite3 or a temporary copy of it, so nothing else they open changes driver. kiro_cli.py and kiro_prerequisite.py already used the shim.
  • _prepare_member_lesson_store imports sqlite3 from the shim, as execution_context.py and memory_stores.require_memory_store already do around the same member-database read. A damaged member store answers 503 store_unavailable with its reason.
  • In every changed module the queries are unchanged, and each except sqlite3.Error now names the error class of the library that raises.
  • Sibling sweep: every import sqlite3 and from sqlite3 under src/ was classified, and every except clause naming a sqlite3 error class (102, by AST scan) was checked against the driver that raises; the ones left alone are under Not a goal with the reason.
  • Rejected: removing the V1 check. It is the guard that refuses to open a member database through the V1 API.
  • Rejected: moving everything to the stdlib driver. The stores need the FTS5 build the shim selects (_sqlite_compat module docstring).

Backwards compatibility

Compatible: every changed site runs the same query and answers the same way; only the SQLite library behind the connection changes, to the one the file's other readers already use, and the lesson check now answers its documented 503 where it answered a 500.

Removes nothing: no function, setting or behaviour is removed.

Tests

  • test/test_memory_stores.py::TestTheV1StoreCheckKeepsTheOpenStoresLock::test_another_process_still_cannot_write_after_the_check (POSIX): a tmp_path V1 store with one synthetic episode starts a write transaction; a child process cannot take the write lock; the production _require_legacy_store_files runs once; the child process still cannot take it.
  • test/test_aws_control_backup_conversations.py::TestTheExportKeepsItsReadLock::test_a_kiro_cli_read_during_the_export_keeps_the_exports_lock (POSIX, needs descriptor pinning like the other export tests): the real _export_cli_conversations runs over a synthetic tmp_path kiro-cli store with committed rows left in the WAL. Inside the export's read transaction, a child process tries PRAGMA wal_checkpoint(TRUNCATE), the production kiro_cli._open_state_db_readonly opens the file, queries and closes, and the child tries again. Both attempts must find the WAL busy.
  • test/test_cron_handler_json_contract.py::test_a_damaged_member_store_answers_503_with_its_reason: the error the production read_member_database_identity raises for a tmp_path file of synthetic bytes is handed to _prepare_member_lesson_store through ContextBuilder.ensure_store (as its V2 arm re-raises it); the check answers 503 {"error": <reason>, "code": "store_unavailable"}.
  • test/test_sqlite_compat_resolver.py::test_a_store_database_is_never_opened_with_the_stdlib_driver: a guard in the style of test_the_idiom_lives_in_exactly_one_file, failing if any of the seven modules above (the five changed, plus kiro_cli.py and kiro_prerequisite.py) regains import sqlite3 or from sqlite3 import. The three newly bound modules are added to IMPORT_SITES.
  • test/metrics/test_inventory_gauges.py::test_knowledge_probe_opens_read_only intercepts connect on the gauge's own driver (ig.sqlite3), so it still checks the probe's own connection.

A child process is used for the lock tests because POSIX locks are per process: a connection in the test's own process cannot see them.

Red on unchanged origin/main at d32c084 (the five test files copied into a detached checkout of origin/main, pytest -n 0): 4 failed, 208 passed.

FAILED test/test_aws_control_backup_conversations.py::TestTheExportKeepsItsReadLock::test_a_kiro_cli_read_during_the_export_keeps_the_exports_lock - AssertionError: kiro-cli could reset the WAL mid-export: ['BUSY', 'RESET']
FAILED test/test_cron_handler_json_contract.py::test_a_damaged_member_store_answers_503_with_its_reason - Failed: the store check let pysqlite3.dbapi2.DatabaseError escape
FAILED test/test_memory_stores.py::TestTheV1StoreCheckKeepsTheOpenStoresLock::test_another_process_still_cannot_write_after_the_check - AssertionError: another process took the write lock the store still holds: 'ACQUIRED'
FAILED test/test_sqlite_compat_resolver.py::test_a_store_database_is_never_opened_with_the_stdlib_driver - AssertionError: assert ['memory_stor...usage_api.py'] == []
================= 4 failed, 208 passed, 31 warnings in 10.01s ==================

Green on this branch: the five test files together, 212 passed in each of 3 serial runs. The two child-process probes (test_aws_control_backup_conversations.py, test_memory_stores.py) decode their output with encoding="utf-8".
Gates exit 0: mypy, flake8, isort --check-only, black --check, scripts/check_black_formatting.py, COMMENT_HISTORY_BASE_REF=origin/main scripts/check_comment_history.py, scripts/docs-lint.sh, scripts/check_memory_store_seam.py.

Manual verification

N/A — unit coverage sufficient: each lock test drives the production code against a real SQLite file and observes the lock from a second process, and the lesson test drives the production check with the error the production reader raises.

Related Issues

Related: merged #12135 (probe pysqlite3 for connect() at every import site), which moved the other import sites to the shim.
No existing issue found (GitHub search for pysqlite3, two copies of SQLite, sqlite3.Error pysqlite3, memory.db corruption, _prepare_member_lesson_store, store_unavailable lessons, data.sqlite3 backup).
Open PRs that touch the changed files (#9815, #10365, #11397, #11831, #13408, #14423, #15661, #15953, #16886, #17526, #17554, #18159, #18406) edit other functions or other lines; none touches the lines this diff changes.

Pattern harvest

Rule candidate: semgrep
Pattern: a stdlib sqlite3 connect, or a stdlib sqlite3.Error handler, on a database that another code path in the same process opens through kiro_crew._sqlite_compat.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Symptom: the gateway mixed the stdlib sqlite3 with the bundled pysqlite3
that kiro_crew._sqlite_compat selects, on files the other library also
had open.
- The V1 store check (memory_stores._require_legacy_store_files) read a
  named store's memory.db with the stdlib driver. Its close released the
  open store's write lock, so another process could write inside the
  store's transaction. The knowledge inventory gauge did the same with
  knowledge.db.
- The aws_control backup's conversation export and the usage API read
  kiro-cli's data.sqlite3 with the stdlib driver, while kiro_cli and
  kiro_prerequisite read it through the shim. A shim close released the
  export's read lock, so kiro-cli could reset the WAL mid-copy.
- The lesson handlers' member-store check caught the stdlib
  sqlite3.Error around a shim call, so a damaged member database
  escaped as a 500 instead of the 503 store_unavailable.

Root cause: two SQLite libraries in one process keep separate lock
tables, and on POSIX closing any descriptor on a file drops every lock
the process holds on it. Their exception classes are unrelated as well.

Change: each of these modules takes sqlite3 from
kiro_crew._sqlite_compat. On hosts without the bundled pysqlite3 the
shim is the stdlib module and nothing changes.
@javenciu
javenciu requested review from a team and KiroCrewCaptain as code owners October 10, 2026 11:05
@javenciu
javenciu requested a review from bolichen97 October 10, 2026 11:05
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention labels Oct 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision 8fe4cf475fd3c9ede3586add4e0e54e97b2a0bc9 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 8fe4cf475fd3c9ede3586add4e0e54e97b2a0bc9 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

On a host with pysqlite3 (Linux x86_64), check that both new child-process lock tests go red when the stdlib import is put back, so they actually exercise the shim.

[DESIGN-REVIEWED] 8fe4cf4

Description read

[DESCRIPTION-READ] 2bce405ae3c194c0546cc7a9fb4b4d4e2cbe7f35fe309d6ddcc9aee296548474

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ✅ PASS

Premise-level review of 8fe4cf475fd3c9ede3586add4e0e54e97b2a0bc9 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Before merging, check that the two lock tests fail on base on a Linux x86_64 runner with pysqlite3; anywhere else the shim is the stdlib, so they pass trivially.

What this change ships

Inventory (7 items) — 7 justified

Intent: open every database that two gateway code paths share through one SQLite library, so that one connection closing cannot drop another's POSIX locks, and so errors are caught by the class that is actually raised. This is a FIX.

  1. Checking a named V1 store no longer releases that store's in-progress write lock to other processes — justified
  2. The knowledge-document gauge probe no longer drops the knowledge store's locks on knowledge.db — justified
  3. The backup's conversation export keeps its read lock while the gateway's own kiro-cli readers open and close the file — justified
  4. The usage API reads kiro-cli's data.sqlite3 with the same library as the other readers — justified
  5. The Lessons panel answers 503 store_unavailable with the reason for a damaged member store, instead of a generic 500 — justified
  6. Three modules are added to the import-site test that checks each site survives an empty pysqlite3 install — justified
  7. A new guard test fails if any of the seven modules that open a shared database regains the stdlib import sqlite3 — justified

[FIRST-PRINCIPLES-REVIEWED] 8fe4cf4

Description read

[DESCRIPTION-READ] 142e53ca41028ffa43e8706c4a08a46707f42c7e55143cbb1613d76323c4e808

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 8fe4cf475fd3c9ede3586add4e0e54e97b2a0bc9 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 8fe4cf4

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 8fe4cf475fd3c9ede3586add4e0e54e97b2a0bc9 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 8fe4cf4

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 10, 2026

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR fixes the root cause: five sites opened a database with the stdlib sqlite3, or caught its errors, while the rest of the process opens the same file through kiro_crew._sqlite_compat (pysqlite3). With two SQLite copies, a close drops the other copy's POSIX locks, and except sqlite3.Error misses the shim's exceptions. The diff only changes which library each site binds (memory_stores._require_legacy_store_files, metrics/inventory_gauges.py, aws_control/backend/backup.py, dashboard/handlers/kiro_usage_api.py, _prepare_member_lesson_store in dashboard/handlers/cron.py). Queries are unchanged, and I confirmed every sqlite3. use left in backup.py and kiro_usage_api.py is on kiro-cli's data.sqlite3 or its temp copy (connect with uri=True, Connection types, sqlite3.Error), which pysqlite3 supports the same way. The tests prove the fix: two child-process lock probes on real files, a 503 test fed the production reader's real error, and an import guard. The body shows all four red on origin/main and green on the branch. Base is main, checks are green, and GPT, Opus, Design and First Principles all pass on head 8fe4cf4 with no open change requests. On hosts without pysqlite3 the shim is the stdlib module, so behaviour there does not change. — crew-pr-reviewer

@iamwhatever
iamwhatever enabled auto-merge (squash) October 10, 2026 19:07
@iamwhatever
iamwhatever merged commit 03a77a9 into kirodotdev:main Oct 10, 2026
103 checks passed
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 10, 2026
@javenciu
javenciu deleted the fix/memory-store-probe-uses-sqlite-shim branch October 11, 2026 04:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants