Skip to content

fix(memory): build a new store's schema in one transaction - #21034

Closed
jeeshofone wants to merge 1 commit into
kirodotdev:mainfrom
jeeshofone:fix/13732-schema-one-transaction
Closed

jeeshofone wants to merge 1 commit into
kirodotdev:mainfrom
jeeshofone:fix/13732-schema-one-transaction

Conversation

@jeeshofone

@jeeshofone jeeshofone commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: a memory store's first open builds its whole schema in one transaction, so a fresh store pays one durable commit instead of one per DDL statement, and a failure part-way leaves no half-built schema.

On Windows CI, xdist workers are killed by the 180 s pytest-timeout during fixture SETUP, while a fresh per-test store runs its schema DDL (VectorMemoryStore.init → _init_database → executescript): 15 Windows PR runs in 7 days, per the issue. The cost is commits: DDL runs in autocommit, so every CREATE TABLE/CREATE INDEX is its own FULL-sync commit. On the parent commit a fresh v1 file takes 19 durable commits and a fresh crew file takes 23.

Why it matters

A busy Windows shard that opens a fresh store can lose its whole xdist group. In production a crash mid-setup can leave tables with no schema_version row.

Not a goal

What changed (motivation → approach → change)

  • Plan, then build only if needed. _init_database asks _schema_steps() what the file still needs, with NO transaction open: missing schema_version rows, the revision-metadata tables and indexes (record_meta.schema_is_current), and the crew lineage/store-name stamps. It also binds the lineage. An up-to-date file needs nothing, so the schema step issues no BEGIN and takes no write lock. The metadata pass after it still writes on a store holding revisions, as on main (Opening a populated memory store takes the write lock even when nothing needs trimming #21056).
  • When something is pending: BEGIN IMMEDIATE, then _schema_steps() runs AGAIN under the lock and only what is still missing runs. Another opener may have built the schema between the two reads. One COMMIT, rollback() on any exception, migration log lines after the commit.
  • executescript COMMITs an open transaction first, so migration SQL goes through memory_schema.py's new execute_script(db, sql). It splits where sqlite3.complete_statement does (a ; in a literal, comment or trigger body does not split) and runs each statement in the caller's transaction.
  • A step that commits would silently break atomicity, so after each step init() checks in_transaction and raises. A migration that committed is not recorded as applied, so the next open runs it again. The rule is documented beside _MIGRATIONS and MIGRATIONS_CREW.
  • isolation_level is restored in a finally. journal_mode=WAL stays outside (SQLite refuses it inside a transaction). reconcile and limit_v1_accepted_history keep their own transaction, so the backfill scan never holds the schema write lock.
  • WAL switch. PRAGMA journal_mode=WAL upgrades a read lock to a write lock, which SQLite does not cover with busy_timeout, so concurrent first opens of a fresh file raised database is locked (on main too). _enable_wal skips a file already in WAL and otherwise retries within the same 5 s, with busy_timeout set first.
  • memory_record_metadata.py's four additive objects live in one _SCHEMA_OBJECTS list used by ensure_schema (build) and schema_is_current (read), so the check cannot drift from the build.

Reader: _SCHEMA_OBJECTS is read only by ensure_schema and schema_is_current; _schema_steps only by _init_database, twice.

Why not BEGIN IMMEDIATE on every open: an empty up-to-date file would wait out busy_timeout behind any writer and fail, where main opens it at once.
Why not a "BEGIN IMMEDIATE;" + sql script: v1 runs a Python step between SQL migrations.

Other schema paths, checked: create_member_database and member provisioning already use one transaction; the FTS index is one statement; private member opens run no DDL.

Backwards compatibility

Compatible: schema, version rows and migration order are unchanged. The schema step of an up-to-date file writes nothing. A file needing v2/v3 gets it in one transaction, or keeps its prior schema, rows and version. Removes nothing public; VectorMemoryStore gains four private methods (composition-contract pins updated).

Tests

test/test_vector_memory_schema_transaction.py (new, 21 tests, v1 and crew where it applies). Its connections use the store's SQLite library (kiro_crew._sqlite_compat, per #18833): a second library keeps its own lock table, so the store would not see their locks.

  • test_a_fresh_store_builds_its_schema_in_one_commit: exactly 1 commit frame in the WAL, the full schema_version set, the metadata tables.
  • test_reopening_a_built_store_commits_nothing: 0.
  • test_the_schema_step_of_an_up_to_date_store_takes_no_write_lock: an empty store opens under another connection's BEGIN IMMEDIATE, with no BEGIN traced.
  • test_a_populated_up_to_date_store_opens_while_another_connection_holds_the_write_lock: xfail(strict=True) for Opening a populated memory store takes the write lock even when nothing needs trimming #21056.
  • test_a_first_open_waits_for_another_openers_lock_on_the_wal_switch: the lock is released at the first retry, no real sleep; test_four_first_opens_of_a_fresh_v1_file_all_succeed: 20 rounds.
  • test_two_openers_racing_on_a_fresh_file_build_its_schema_once: another opener builds everything between one opener's plan and its lock; the re-plan is empty and each migration is logged once.
  • test_a_failed_upgrade_leaves_the_store_at_its_prior_version (v1 and v1+v2 files with rows): schema and rows unchanged.
  • Also: a migration step or a metadata step that commits is refused; a crew file that lost its lineage stamp gets it back; a failure after the first CREATE leaves no tables; execute_script edge cases.

On main: 19/23 commits, the failed-upgrade tests leave v2/v3 half-applied, both WAL tests raise database is locked, and a committing step is not refused (prove.py: PROVEN).

Run locally (Linux, Python 3.12). Stdlib driver: the new file 19 passed, 2 xfailed; the 139 files naming a touched module (-n 4): 10312 passed, 65 skipped, 2 xfailed. pysqlite3 (the Linux x86_64 driver): test_vector_memory*, test_memory_*, test_member_memory*: 2752 passed, 21 skipped, 2 xfailed. All *ratchet* files, refactor facade, sqlite-compat, host-isolation: 431 passed. black gate, isort, flake8, mypy src/kiro_crew/ (0 issues), comment-history, file-line-cap, Semgrep 1.78 (0 findings).

Manual verification

200 fresh VectorMemoryStore.init() calls across 32 processes on a Linux host, timing only init(), two runs each:

commits per fresh store median p95 max
parent commit 19 186–241 ms 456–462 ms 475–480 ms
this PR 1 25–28 ms 44–73 ms 129–132 ms

Not verified: Windows. The effect on the stall is inferred from the commit count; the Windows shards are the check.

Rebase note (#20806)

#20806 moves _init_database into vector_memory_lifecycle.py. Whoever lands second moves this block and its helpers with it. vector_memory.py grows here (a reported-only line-cap row) and the split absorbs it.

Related Issues

Closes #13732

Pattern harvest

Rule candidate: semgrep
Pattern: executescript(...) or a run of DDL in autocommit on a store's open path. Each statement is a separate FULL-sync commit, and executescript silently commits any transaction it runs inside. Also: a write lock taken to find nothing to write, and journal_mode=WAL with no retry.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable): N/A, no documented behaviour changes
  • No secrets, credentials, or internal references in the diff

@jeeshofone
jeeshofone requested review from a team and KiroCrewCaptain as code owners October 16, 2026 01:18
@jeeshofone
jeeshofone requested a review from CrysisDeu October 16, 2026 01:18
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Oct 16, 2026
@jeeshofone
jeeshofone force-pushed the fix/13732-schema-one-transaction branch from 9362195 to 1b43b5c Compare October 16, 2026 02:19
@jeeshofone

Copy link
Copy Markdown
Contributor Author

Disposition for the independent review of 9362195, answered against the new head 1b43b5c (still one commit, same base 5d899b0).

MUST-FIX 1 (opening an up-to-date store took the write lock): fixed. _init_database now asks _schema_steps() what is pending, with no transaction open. It checks for missing schema_version rows, the revision-metadata tables and indexes (memory_record_metadata.schema_is_current, which reads the same _SCHEMA_OBJECTS list ensure_schema builds from) and the crew lineage/store-name stamps. If nothing is pending, init returns without any BEGIN. Otherwise it takes BEGIN IMMEDIATE, plans again under the lock and runs only what is still missing.

  • test_an_up_to_date_store_opens_while_another_connection_holds_the_write_lock[v1|crew]: the reviewer's probe as a regression test. A second connection holds BEGIN IMMEDIATE; init() succeeds and the store's traced statements contain no BEGIN. It is bounded by the store's own 5 s busy_timeout, so a regression raises database is locked instead of hanging. It fails on 9362195 with exactly that error.
  • test_two_openers_racing_on_a_fresh_file_build_its_schema_once[v1|crew]: one opener plans, the other builds the full schema in between, then the first takes the lock. Its second plan is empty, both open, and each migration is logged once.
  • Mutations: BEGIN on every open turns 4 tests red; reusing the first plan under the lock turns 14 red.

SHOULD-FIX 1 (no rollback test for an existing store mid-upgrade): taken. test_a_failed_upgrade_leaves_the_store_at_its_prior_version[v1|v1+v2], ported from the review. Files with a semantic and an episodic row, with a failure injected after the migrations: schema and every row are identical to before, and store._db is None. Both cases fail on main 5d899b0 (v2/v3 left half-applied) and pass here.

NIT 1 (isolation_level restored only on success): fixed. It is restored in a finally around the whole setup block.

NIT 2 (a migration step could commit mid-transaction): fixed. After every step, init() checks in_transaction and raises A memory schema step committed the setup transaction. Inside a migration the check runs before the schema_version row is written, so a step that committed is never recorded as applied and the next open runs it again. The rule is documented beside _MIGRATIONS and MIGRATIONS_CREW. Pinned by test_a_migration_step_that_commits_is_refused; with the guard removed it goes red.

NIT 3 (WAL counter ignores frame salts): no code change. The test helper's docstring now says the counter assumes a WAL written since the file was created. Every test using it starts from a fresh file.

Also added: test_reopening_a_crew_store_that_lost_its_lineage_stamp_writes_it_back. It pins that the read-only plan still schedules a missing crew stamp, and goes red if that check is skipped.

Kept and re-measured: a fresh store is still 1 commit (19 for v1 and 23 for crew on main), and reopening a built store is 0. The 200-store probe (32 processes, timing only init(), two runs each): parent commit median 186–241 ms, p95 456–462 ms; this head median 25–28 ms, p95 44–73 ms. The PR body is updated with these numbers.

Gates on 1b43b5c (Linux, Python 3.12). The new file passes 15/15. Every test file that names vector_memory, memory_record_metadata or memory_schema, plus the memory/member suites, the spawn member-memory tests and every ratchet/facade test (180 files, -n 8): 11060 passed, 65 skipped. The core files with this commit cherry-picked onto current main: 441 passed. Also green: black gate, isort, flake8, mypy src/kiro_crew (0 errors in 2510 files), comment-history against the merge-base, subprocess-encoding, memory-store seam, file-line-cap (0 failures; vector_memory.py grows a reported-only row, which #20806's split absorbs), Semgrep 1.78 with semgrep/ on the changed files (0 findings), and both PR-description modes.

Matched AUTOSDE rules: memory-store-seam (no new read path, since the store still opens only its own file; the seam script passes) and recorded-decisions-are-not-regressions (no entry under docs/decisions covers memory schema setup).

First-pass check: R1-R8 OK. One note on R5.1: the new migrated list holds at most one int per migration (3 for v1).

The fork workflows on the previous head were still awaiting maintainer approval, so no CI or AI lane has run on this PR yet.

@jeeshofone

Copy link
Copy Markdown
Contributor Author

Disposition for the independent delta review of 1b43b5c (APPROVE-WITH-FIXES, no must-fix), answered on 73f6a1d. Also rebased onto current main (b9d1104); main touched no memory files.

  • SHOULD-FIX 1 (the "opens while another connection is writing" claim): fixed by narrowing. The _init_database comment, the commit message and the PR body now claim only that the schema step of an up-to-date file takes no write lock. The test is renamed test_the_schema_step_of_an_up_to_date_store_takes_no_write_lock and its docstring says the store is empty. The new test_a_populated_up_to_date_store_opens_while_another_connection_holds_the_write_lock (v1 and crew) is xfail(strict=True, raises=OperationalError) and names Opening a populated memory store takes the write lock even when nothing needs trimming #21056. That issue covers the cause: limit_v1_accepted_history's open-time DELETE begins a write even when it deletes nothing. It suggests selecting the over-limit records first. reconcile is not changed in this PR. The test shortens the busy wait to 0.2 s, so it stays bounded.
  • SHOULD-FIX 2 (concurrent first opens fail at PRAGMA journal_mode=WAL): fixed in this PR. Moving busy_timeout above the pragma is not enough: Python's connect() already sets a 5 s busy timeout, and the probe still failed 8 rounds in 60 with the pragma moved. The switch upgrades a read lock to a write lock, and SQLite does not call the busy handler for that upgrade. A held lock raises at 0.0 s. The new _enable_wal reads PRAGMA journal_mode first, so a file already in WAL takes no lock. Otherwise it retries the switch on "locked" within the same 5 s budget (_BUSY_TIMEOUT_SECS); any other error, such as "file is not a database", raises as before. All three open paths now set busy_timeout before anything else: the legacy path, create_member_database and the private member path. The 4-thread probe now fails 0 of 60 rounds, and a lock that is never released still fails, after 5.0 s. Tests: test_four_first_opens_of_a_fresh_v1_file_all_succeed (20 rounds × 4 threads) and test_a_first_open_waits_for_another_openers_lock_on_the_wal_switch, which releases the lock at the first retry sleep with no real time passing. Both fail on 1b43b5c and pass here (3 of 3 full-file runs).
  • NIT 1 (commit-then-BEGIN evades the guard): the _require_setup_transaction docstring now names the limit. A total_changes check would not be airtight either, so the code is unchanged.
  • NIT 2 (names only) and NIT 3 (ensure_schema outside _db_lock): no change. Both behave as on main, as the review notes.
  • NIT 4 (WAL frame salts): no change. The _wal_commits docstring already states the assumption.

Gates on this head: the new file 17 passed, 2 xfailed. test_vector_memory*, test_memory_* and test_member_memory* at -n 8: 2747 passed, 21 skipped, 2 xfailed. Every *ratchet* file plus the sqlite-compat tests: 266 passed. Also passing: mypy src/kiro_crew (0 issues), the black gate, isort, flake8, comment-history, file-line-cap (0 failures), and Semgrep 1.78 with semgrep/ on the changed files (0 findings). The PR is one commit.

First-pass check: R1-R8 OK.

A vector store's first open ran its schema DDL in autocommit, so every
CREATE TABLE/INDEX was its own FULL-sync commit: 19 durable commits for a
fresh v1 file and 23 for a crew file. Under heavy parallel I/O on Windows
CI that setup outran the 180 s per-test timeout and killed the xdist
worker during fixture setup.

init() now plans the schema work first, with no transaction: missing
schema versions, the shared revision metadata tables and indexes, and
the crew lineage stamps. An up-to-date file needs nothing, so the schema
step takes no write lock. When something is pending, init() takes BEGIN
IMMEDIATE, plans again under the lock (another opener may have built it
meanwhile) and runs only what is still missing, in one transaction
rolled back on any error. A fresh store pays one commit, and a failed
build or upgrade leaves the file at its prior schema and version.

The open-time metadata pass after the schema step still writes on a
store holding accepted revisions; kirodotdev#21056 tracks that, and a strict xfail
test pins it.

executescript commits an open transaction before it runs, so migration
SQL goes through memory_schema.execute_script, which splits statements
with sqlite3.complete_statement and runs them inside the caller's
transaction. A migration step that commits the setup transaction is
refused and not recorded as applied. isolation_level is restored in a
finally. journal_mode stays outside (SQLite refuses it inside a
transaction), synchronous stays FULL, and the metadata backfill keeps
its own transaction so its full-table scan never holds the write lock.

Concurrent first opens of a fresh store file do not fail at the WAL
switch. That switch upgrades a read lock to a write lock, which SQLite
does not cover with busy_timeout, so a second opener raised "database is
locked" at once. The store's open sets busy_timeout first, skips the
switch when the file is already in WAL, and otherwise retries it within
the same 5 s budget.

Closes kirodotdev#13732
@jeeshofone
jeeshofone force-pushed the fix/13732-schema-one-transaction branch from 73f6a1d to 8ba2ed6 Compare October 16, 2026 13:14
@jeeshofone

Copy link
Copy Markdown
Contributor Author

Head 8ba2ed6: rebased onto main c55294e, still one commit. No CI lane had run on this PR before, so this head carries what a full first-pass self-check found.

Fixed: the new tests opened their own connections with a second SQLite library. The store takes sqlite3 from kiro_crew._sqlite_compat, which is the bundled pysqlite3 where it is installed (the Linux x86_64 dependency in setup.cfg). The test module used the stdlib sqlite3. Two libraries in one process keep separate lock tables and unrelated exception classes (#18833), so a stdlib lock holder is invisible to the store. With pysqlite3 built locally, the file at 73f6a1d fails 3 tests: both test_a_populated_up_to_date_store_opens_while_another_connection_holds_the_write_lock cases XPASS(strict), and test_a_first_open_waits_for_another_openers_lock_on_the_wal_switch sees no retry (assert 0 >= 1). The module now takes sqlite3 from kiro_crew._sqlite_compat: 19 passed, 2 xfailed under the stdlib driver and under pysqlite3.

Fixed: unbounded waits in test_four_first_opens_of_a_fresh_v1_file_all_succeed (determinism rule D8). Its Barrier and join() had no timeout, and it recorded only OperationalError. Both waits are bounded by kiro_crew.testing.wait.default_timeout(), the threads are daemons, every exception is recorded, and an opener that fails aborts the barrier so the others stop waiting.

Pinned: the commit check after each setup step (vector_memory.py:1623). Deleting it reddened nothing, because only migration steps were tested and _apply_migration checks on its own. test_a_metadata_step_that_commits_is_refused[v1|crew] patches ensure_schema to commit. With the check deleted, both cases fail with cannot commit - no transaction is active.

Removed: two hunks that changed nothing a test can observe. These were busy_timeout plus _enable_wal in create_member_database, and moving the private member path's busy_timeout above its identity check. sqlite3.connect() already sets a 5 s busy timeout on every connection, so moving the PRAGMA changes nothing. The member-database WAL retry answers a concurrent first open that no test, issue or review has shown. Both lines are back to main's. _enable_wal has one caller, the store's open, and its two tests pin it.

The points from the independent review of 9362195 are all still in place on this head:

  • An up-to-date open takes no write lock. _schema_steps plans with no transaction open (vector_memory.py:1614), and BEGIN IMMEDIATE plus a re-plan run only when something is pending (:1621).
  • Failed-upgrade tests: test_a_failed_upgrade_leaves_the_store_at_its_prior_version[v1|v1+v2].
  • isolation_level is restored in a finally (:1629).
  • Each migration step's commit check runs before its version row is written (:1730).

prove.py: reverting the production change reds the new tests (PROVEN). Reverting one hunk at a time leaves only the two comments beside _MIGRATIONS and MIGRATIONS_CREW uncaught. The definition hunks break the import when reverted alone.

Repository rules checked against this diff:

  • tests-are-deterministic: waits are bounded, with no sleep as a barrier and no wall-clock upper bound. The WAL test swaps the module's own time binding for a delegating namespace.
  • no-test-side-effects: everything is under tmp_path and the per-test data home.
  • no-blocking-call-on-event-loop: _enable_wal's retry sleep runs only inside init(), which async callers offload by the contract at vector_memory.py:1648.
  • an-invariant-line-needs-a-mutation-pin: see above.
  • a-refusal-is-not-a-commit: a refused step is never recorded as applied.
  • memory-store-seam, no-sync-store-call-from-a-coroutine, a-ratchet-may-only-tighten: no new read path, call site or baseline.

Gates on this head:

  • Stdlib driver: the 139 test files that name a touched module (-n 4): 10312 passed, 65 skipped, 2 xfailed.
  • pysqlite3: test_vector_memory*, test_memory_*, test_member_memory*: 2752 passed, 21 skipped, 2 xfailed.
  • After the rebase: the store and memory-store files, 706 passed, 18 skipped, 2 xfailed.
  • Every *ratchet* file, the refactor facade, the sqlite-compat and host-isolation tests: 431 passed.
  • black gate, isort, flake8, mypy src/kiro_crew/ (0 issues), comment-history, file-line-cap (0 failures), Semgrep 1.78 with the repository's rules (0 findings).

First-pass check: R1-R8 OK.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision 8ba2ed67e76da25f15c71e8dc3e046b5419533eb touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — 🟡 CONCERNS

Premise-level review of 8ba2ed67e76da25f15c71e8dc3e046b5419533eb via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The runtime check that refuses a committing schema step guards against a migration that doesn't exist yet. None of today's steps commit.

Not justified as shipped

  • Item 6 — inherited: "A step that commits would silently break atomicity, so after each step init() checks in_transaction and raises." No step in _MIGRATIONS or MIGRATIONS_CREW commits today: _migrate_v2 is a single ALTER TABLE, and SQL now goes through execute_script. The only failure on base is the test this PR adds for it.
    Subtraction: delete _require_setup_transaction and its two call sites. Keep the rule comments beside _MIGRATIONS/MIGRATIONS_CREW.
    Clears when: a shipped or pending migration step that commits is named, or the check is deleted.
  • Item 5 — rides along: the WAL-switch retry fixes a separate defect ("raised database is locked (on main too)"). It sits outside the Goal of one transaction with no half-built schema. It has its own provenance (base-failing WAL tests). Six other PRAGMA journal_mode=WAL sites in src/ still have no retry; that is deferred by "Not a goal".

What this change ships

Inventory (8 items) — 6 justified

Intent: make a fresh memory store's first open build its schema in one durable commit, so Windows test setup stops timing out and a crash can't leave a half-built schema. This is a FIX.

  1. A fresh store's tables, version rows and metadata are built in one commit instead of 19–23 — justified
  2. An upgrade that fails part-way leaves the store at its prior version — justified
  3. Opening a store that is already up to date takes no write lock — justified
  4. Two processes opening one fresh file at the same time build its schema once — justified
  5. Simultaneous first opens wait out the WAL switch instead of failing — rides along
  6. A schema step that commits is refused at open — inherited
  7. Migration SQL runs statement by statement inside the open transaction (execute_script) — justified
  8. A crew store missing its lineage or store-name stamp gets it back inside the setup transaction — justified

[FIRST-PRINCIPLES-REVIEWED] 8ba2ed6

Description read

[DESCRIPTION-READ] d73d3e90af64b40ade55fbbf48b625b1485e4d952041ccd06c3e2dede76802c5

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 8ba2ed67e76da25f15c71e8dc3e046b5419533eb via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Confirm the Windows CI shards stop timing out in fixture setup. The description says Windows was not verified and the fix is inferred from the commit count.

Suggestions

  • Add the new migration contract (a step must not commit and must not use executescript) to the memory-skills-hooks spec section that covers _MIGRATIONS. That spec is where a migration author looks, and right now the contract is only in code comments.

[DESIGN-REVIEWED] 8ba2ed6

Description read

[DESCRIPTION-READ] baaa0a2edc12d62d10e8a02af7cd12314cc73931c5036dfb68e299e9e97c0c91

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 8ba2ed67e76da25f15c71e8dc3e046b5419533eb via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 8ba2ed6

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 8ba2ed67e76da25f15c71e8dc3e046b5419533eb via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 8ba2ed6

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 16, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Hold at 8ba2ed67e76da25f15c71e8dc3e046b5419533eb: current-head AI review items are unanswered.

The change itself reads sound: the schema plan is read without a transaction, rebuilt under BEGIN IMMEDIATE, and committed once, with execute_script replacing executescript so migrations stay inside the transaction. I found no defect in the source diff, and CI on this head is green, including all eight Windows shards.

The fork review lanes posted after the last author comment, and nothing answers them yet:

  • First Principles CONCERNS, item 6: delete _require_setup_transaction and its two call sites, or name a shipped/pending migration step that commits.
  • Same lane, item 5: the WAL-switch retry rides along outside the stated goal. Justify keeping it here or split it out.
  • Design PASS check (Windows fixture-setup timeouts) and its suggestion to put the migration no-commit contract in the memory spec section.

This clears when each item has its own target=first-principles / target=design disposition for this head, and any deferred item links an issue.

— crew-pr-reviewer (r107)

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Oct 16, 2026
@buluoray

Copy link
Copy Markdown
Contributor

🤖 Kiro Crew Auto-Pipeline [operator: buluoray#ce473fca]: Taking this over in #22866 since it has been idle; you are credited as co-author on the commits. Thanks for the groundwork.

The fix itself is unchanged. It is ported onto vector_memory_lifecycle.py after #20806 moved _init_database there. Per the First Principles items on 8ba2ed6, _require_setup_transaction is dropped (a committing step still fails the open at the closing COMMIT, and a test pins that), and the WAL-switch retry moves to its own issue, #22865.

iamwhatever pushed a commit that referenced this pull request Oct 22, 2026
A fresh store's first open ran every DDL statement in autocommit, so each
CREATE was its own FULL-sync commit: 19 for a fresh v1 file, 23 for a crew
file. On a busy Windows runner that is enough for a per-test store's setup
to outrun the 180 s pytest-timeout (#13732).

_init_database now plans what the file still needs with no transaction
open (missing schema_version rows, the revision-metadata objects, the crew
stamps). An up-to-date file needs nothing and takes no write lock.
Otherwise it takes BEGIN IMMEDIATE, plans again under the lock, runs only
what is still missing, and commits once, rolling back on any error.
Migration SQL goes through memory_schema.execute_script, because
executescript commits a pending transaction first.

Ported from #21034 onto the vector_memory_lifecycle split (#20806). The
WAL-switch retry and the runtime "step committed" check from #21034 are
left out; a committing step already fails the open at the closing COMMIT.

Closes #13732

Co-authored-by: Will Laws <167437+jeeshofone@users.noreply.github.com>
@iamwhatever

Copy link
Copy Markdown
Contributor

Thank you for this fix, @jeeshofone. The same change has already landed on main: #22866 "fix(memory): build a new store's schema in one transaction" (merge commit be78b6c) carried this work forward, rebased onto the #20806 split into vector_memory_lifecycle.py. It closed #13732 on 2026-10-22. main now has memory_schema.execute_script and the single-transaction schema build, and the WAL-switch retry followed in #23920. This branch conflicts with main and has nothing left to add, so I am closing it. Please reopen if you see something here that did not make it into #22866.

@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tests: memory_cli_store_scope worker killed by 180s timeout inside SQLite schema DDL during setup (Windows)

3 participants