Skip to content

fix: gatewayd orphan reap, knowledge close seam; tenth hygiene pass - #12820

Merged
bolichen97 merged 1 commit into
kirodotdev:mainfrom
bolichen97:test/hygiene-sweep-2026-09-22
Sep 25, 2026
Merged

bolichen97 merged 1 commit into
kirodotdev:mainfrom
bolichen97:test/hygiene-sweep-2026-09-22

Conversation

@bolichen97

@bolichen97 bolichen97 commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Five full runs of the backend suite on one commit were red every time, and not with the same test. test/test_dev_fleet_app.py::test_trusted_bin_pins_the_resolved_target_not_the_symlink failed 5/5 on a host whose TMPDIR sits under $HOME. Three other tests each failed exactly once: a wall-clock linearity ratio in test/test_argv_floor_inline_and_brace_scope.py, a stale crew-log write lease in test/test_eventlog_hooks.py left by test/test_crew_log_edge_exhaustion.py 870 tests earlier on the same worker, and a second WARNING in test/test_taskq_runner_adapter.py from a sandbox sweep another test's SessionManager had armed. Underneath them the per-test probe found real gh, ssh, aws configure, systemctl and mise spawned from unit tests, 52 tests leaking descriptors, eight tree-scan ratchets holding 200+ MiB, and a teardown that SIGKILLed pid numbers it had already proven dead.

Why it matters

A single CI run would have shipped each of the three flakes as green. One of them is a production bug: mcp_gateway/gatewayd.py let a private MCP backend outlive the daemon because its reap was cancelled by the daemon's own teardown. (The third, crew_log/emit.py pinning a unit's write lease through a returned exception's traceback, was found by this pass and landed on main first through the tenth-pass PR, which returns the failure's type; this PR carries no crew-log change.) The descriptor class had no fix available to tests at all: KnowledgeStore could not close a connection another thread opened, so 52 tests leaked three descriptors per store.

What changed (motivation → approach → change)

Effect Mechanism
A private backend never outlives run_gatewayd mcp_gateway/gatewayd.py reaps a disconnected stub's backend through pool.spawn_shutdown (made public in mcp_gateway/pool.py), which shutdown_all joins
A test teardown can release a store's descriptors on every thread knowledge/store.py gains the test-only _close_all_for_tests() (generation bump; under a module flag the test conftest flips, connections open check_same_thread=False through a factory that keeps the thread guard in Python for everything but close() and are registered for the teardown close; production keeps SQLite's native guard and registers nothing); vector_memory.py::init() closes the handle it replaces
The staged node launcher runs from its managed leaf browser_cli/install.py::_run takes cwd=; _staged_node_runs passes candidate.parent
Tree ratchets stop holding the source text test/source_corpus.py streams (read, normalise, filter, parse, drop) and caches only the path list

Each red was a victim, not a culprit, so every fix lands at the seam the product already reads rather than in the assertion that went red. The dev-fleet test pins Path.home to a sibling of its fixture and a mirror test plants inside the patched home. The linearity guard counts the characters the translator reads at 3k and 12k braces (at most 6x for 4x) instead of timing a ratio, and keeps the base's 30 s and 1 s ceilings on thread CPU. _one_warning filters by logger name, and test/conftest.py gains _no_boot_sandbox_sweep, which pins kiro_crew.session.cleanup_stale_sandbox_profiles to a no-op (138 real sweeps on mc-maint threads across test_session.py without it, 0 with it).

The quiet classes follow the same rule. Real binaries are stopped at their resolvers: in src/kiro_crew/apps/builtins/auto_improvement/tests/test_dogfood_learnings.py the conftest _public_dns fixture instead of a class-local copy that dropped the _gh_prefers_ssh pin (and test_suite_scope.py beside it runs its clone's git from the clone), an autouse pin on the cached ssh -V probe in six sandbox files, platform_compat.trusted_*_bin and code_fingerprint.trusted_git_bin pinned to their no-spawn arm, monkeypatch.chdir(tmp_path) where production spawns git -C <scratch>. Every inline VectorMemoryStore, SkillsLoader and SubagentManager goes through an opened register-and-close fixture; store fixtures call _close_all_for_tests() (production has no moment at which every thread is provably idle short of exit, so the seam gets no production consumer). test/test_pid_lifecycle.py::_reap signals only while process_start_time still matches the identity pinned at spawn, and test/test_platform_compat.py probes a pid above pid_max and, on Windows where no pid ceiling is readable, asserts the same contract through a stubbed taskkill instead of the real one. test/test_security.py stubs the TEST-NET datagram probe with an inert socket and asserts the peers it would have used; test/test_browser_launcher.py uses the run-owned short_sock_dir as an absolute root instead of chdir plus .. docs/system-specs/common/testing-conventions.md gets the tenth-pass section, docs/system-specs/modules/knowledge.md and docs/system-specs/modules/mcp-gateway-daemon-lifecycle.md record the two new contracts, and src/kiro_crew/builtin_skills/kirocrew-dev/writing-tests/SKILL.md gains nine checklist items. .github/black-baseline.txt drops two files that are now black-clean.

Backwards compatibility

Compatible: KnowledgeStore.close() keeps its per-thread contract (pinned by test_knowledge_cross_thread.py) and _close_all_for_tests() is an additive test-only seam that raises unless the test conftest's flag is set, and production connections keep check_same_thread=True; BackendPool.spawn_shutdown is the renamed _spawn_shutdown with both callers updated; _run in browser_cli/install.py takes a keyword-only cwd that defaults to the old behaviour; _run_job still returns the same exception object, only without __traceback__, and its callers only isinstance it.

Tests

  • test_knowledge.py::TestKnowledgeStore::test_close_all_releases_every_threads_connection_and_reopens_lazily — every thread's connection closed, lazy reopen after.
  • test_knowledge.py::TestKnowledgeStore::test_production_keeps_the_thread_guard_and_the_seam_refuses_without_the_flag — with the flag off, a connection used from another thread is refused and the seam raises.
  • test_knowledge.py::TestKnowledgeStore::test_the_test_mode_connection_still_refuses_another_threads_use — under the flag, use from another thread is still refused; only close() is cross-thread.
  • test_vector_memory.py::TestSchemaInit::test_a_repeated_init_closes_the_handle_it_replaces — no orphaned handle on re-init.
  • test_mcp_gateway_pool_integ.py::test_real_stubs_without_poolable_get_their_own_backend — no private backend pid exists after run_gatewayd returns (the inline reap leaves all three alive).
  • test_dev_fleet_app.py::test_trusted_bin_refuses_target_under_home — the refusal the symlink test now pins around still fires.
  • test_browser_cli_install.py — the staged launcher's cwd is the managed leaf.
  • test_session_cleanup.py::TestTheSuiteNeverRunsTheBootSandboxSweep — the conftest pin on the boot sandbox sweep is in force and reaches the name the session module calls.
  • test_argv_floor_inline_and_brace_scope.py — work-counted linearity and a structural alternation-group bound, with the base's two ceilings kept on thread CPU; a planted per-{ suffix rescan reads 15.9x and fails, as does budget removal.
  • Every changed ratchet re-proven against a planted violation; every fd-class file probed per test (0 tests at +5 or more, last-test delta 0).

Manual verification

Five-run baseline: 126,905 backend tests per run with the reds above; vitest 36,811 and electron 1,999 green in all five. After the fixes, the 57 touched test files together at -n 4: 6,391 passed, 52 skipped. mypy --platform linux src/kiro_crew, isort, flake8, black baseline, docs lint, subprocess-encoding, sync-io-in-async and lockdown gates pass.

Related Issues

no linked issue: findings from a scheduled test-hygiene pass, none tracked as an issue.

Pattern harvest

Rule candidate: review-prompt
Pattern: "a caught exception returned (not raised) with its traceback attached — the traceback → frames → local → exception cycle pins everything those frames hold until cyclic GC"

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@bolichen97
bolichen97 requested review from a team and dwu96 September 22, 2026 13:44
@github-actions github-actions Bot added the fork Pull request from a fork (external contributor) label Sep 22, 2026
@bolichen97 bolichen97 changed the title test: ninth five-run hygiene pass -- 4 reds, 2 prod defects, 11 clusters test: ninth five-run hygiene pass -- 4 reds, 2 prod defects Sep 22, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author

Intent: Close every test-hygiene finding a five-run Linux baseline surfaced -- the one deterministic red, the three single-round flakes, and the descriptor, spawn, kill, network and memory classes behind them -- at the seam production already reads, fixing the production defects those flakes exposed (crew-log lease pinned by a returned traceback; a private MCP backend whose reap teardown cancelled; a knowledge store with no cross-thread close) rather than the assertions that went red, and record each class in the testing spec so the next pass does not re-derive it.
Not a goal: Any user-visible behaviour change; closing follow-ups this pass only named (the unclosed CrewLog handles in test_crew_log_core.py, push_guard.py's patch-id bypass of its injection seam, the history_search index with no close path, class-15 app[...] writes after start, the gateway-lock orphan test that flakes under 1,680-file load); re-deriving the _GIB_PER_WORKER reservation.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 22, 2026
@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5, fork) — ✅ PASS

Design-level review of 6e62dd6da55dc1cbf0621daa9474c8df8723d56e via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Verify the session-wide conftest flag still leaves the flag-off production connection path exercised (the two production-guard tests must build stores outside the flipped flag).

Suggestions

  • The gatewayd orphan-reap fix is a standalone production bug fix buried in an 8k-line hygiene sweep; landing it as its own commit (the two-commit budget allows it) keeps it cleanly revertable and cherry-pickable independent of the test churn.

[DESIGN-REVIEWED] 6e62dd6

@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5, fork) — ✅ PASS

Premise-level review of 6e62dd6da55dc1cbf0621daa9474c8df8723d56e via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Verify the suite-wide test connection factory: _ThreadAffineTestConnection leaves with conn:, backup and cursor methods un-guarded, so all 126k tests lose SQLite's native check on those paths.

What this change ships

Inventory (10 items) — 10 justified

Intent: FIX — make five-run-red test flakes deterministic and reap the production orphan they exposed.

  1. A private MCP backend can no longer outlive the daemon; shutdown_all joins its reap — justified
  2. BackendPool.spawn_shutdown becomes public (renamed _spawn_shutdown, both callers updated) — justified
  3. KnowledgeStore gains _close_all_for_tests() behind a flag production never sets; connections registry + generation bump — justified
  4. A repeated vector_memory.init() now closes the handle it replaces — justified
  5. Staged node probe runs from its managed leaf (_run gains keyword-only cwd) — justified
  6. Suite-wide test plumbing: session flag flip, opened / close_skills_loaders fixtures, autouse no-op pin on the boot sandbox sweep — justified
  7. Four flaky tests rewritten at the seam production reads (pinned Path.home + mirror refusal test, work-counted linearity keeping both CPU ceilings, logger-filtered warning counts, identity-pinned pid reaps, inert datagram stub with stronger peer assertions) — justified
  8. source_corpus streams file text, caching only the path list (drops ~160 MB per-worker retention) — justified
  9. Docs and skill updated in the same commit: two module contracts, tenth-pass section, nine checklist items — justified
  10. .github/black-baseline.txt drops two now-clean files — justified

[FIRST-PRINCIPLES-REVIEWED] 6e62dd6

@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review (fork) — ✅ no blocking findings

Reviewed 6e62dd6da55dc1cbf0621daa9474c8df8723d56e via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 6e62dd6

@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review (fork) — ✅ no blocking findings

Reviewed 6e62dd6da55dc1cbf0621daa9474c8df8723d56e via the fork AI-review pipeline; updated in place on each push.

Review details

Nothing blocks; two advisory findings.

FINDING — test/test_source_corpus.py:399 — assert candidate_sources() == everything materialises the corpus a second time (candidate_sources() → iter_candidate_sources() → iter_source_texts() re-reads every file into fresh str) while everything = source_texts() is still bound, so two independent copies of the ~130 MB text are resident at the comparison — the doubling the sibling hunk two functions above was just changed to avoid ("two calls in one expression would hold the corpus twice"), in the module that pins the streaming rewrite → Fix: compare streamed, e.g. assert all(a == b for a, b in zip(iter_candidate_sources(), iter_source_texts(), strict=True)), or compare paths only, so one file is resident at a time.

FINDING — src/kiro_crew/knowledge/store.py:26 — the flag comment, the RuntimeError text at store.py:2851 ("store is built (the rootdir test conftest does)"), docs/system-specs/modules/knowledge.md:624 and docs/system-specs/common/testing-conventions.md (the rootdir conftest's autouse _no_boot_sandbox_sweep ... for every test, the rootdir conftest's opened register-and-close fixture) all name the rootdir conftest, but all three fixtures landed in test/conftest.py, which the repo-root conftest.py docstring states "only applies to test/" while testpaths also collects src/kiro_crew/apps/builtins — so a developer whose _close_all_for_tests() raises there is sent by the error message itself to a file that contains no such flip, and the "for every test" claim is false for ~108 test modules → Fix: either move the three fixtures to the repo-root conftest.py (what the prose promises, and what makes the pins cover both testpaths) or correct the four sites to say test/conftest.py and state that the app test trees get neither pin.

[OPUS-REVIEWED] 6e62dd6

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 22, 2026
@bolichen97
bolichen97 force-pushed the test/hygiene-sweep-2026-09-22 branch from c99786a to 1eae174 Compare September 22, 2026 14:02
@bolichen97

bolichen97 commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: the ratchet's file pre-filter now carries only the identifier the AST check needs (build_message); the coroutine is found by ast.AsyncFunctionDef, never by a text needle

  • span=5834cf773e6d fixed in 1eae174

parsed_candidates(require_all=("build_message", "async def")) let a coroutine spelled async def (two spaces, valid Python) skip the scan: a gate that can skip a valid offender fails open.
The filter is now require_all=("build_message",): an ast.Attribute with attr == "build_message" cannot exist in a file whose text lacks that identifier, so the narrowing is sound, and coroutines are found by AST alone.
Proven by planting async def _probe(b): return b.build_message("x") under src/kiro_crew/ -- the gate fails on it -- then removing the plant.

@bolichen97

bolichen97 commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: the worker thread now stays alive across close_all() and takes store.db again on the SAME thread

  • span=1d4caa80a15c fixed in 1eae174

Joining the worker before close_all() meant the "reopens lazily" claim was tested on a fresh thread with empty thread-local state, so the generation check could be deleted and the test still passed.
The worker now parks on an Event, close_all() runs, and the same thread asserts its cached handle raises ProgrammingError, that store.db hands it a different connection, and that a query on it answers.
Proven: with the generation check replaced by if conn is None: the test fails with "the worker reused the handle close_all closed"; restored, it passes. The same amend switches both new pins from the stdlib sqlite3 to kiro_crew._sqlite_compat.sqlite3, which is why they were red on the x86_64 shards (pysqlite3's ProgrammingError is an unrelated class).

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 22, 2026
@bolichen97
bolichen97 force-pushed the test/hygiene-sweep-2026-09-22 branch from 1eae174 to 7d27b67 Compare September 22, 2026 14:22
@bolichen97

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: the every-thread close on KnowledgeStore is now the test-private _close_all_for_tests(), the shape crew_log.lease._release_all_for_tests() already uses

  • span=a70b1738ce51 fixed in 7d27b67

Correct: close_all() had zero production consumers, and adding one would be unsound -- production has no moment at which every thread is provably idle short of process exit, and closing a handle from under a worker mid-query is undefined; process exit already closes them.
Taken the second clearing condition: the seam is test-private (_close_all_for_tests), its docstring says nothing in production may call it, and docs/system-specs/modules/knowledge.md records that production closes the other threads' handles by exiting.
The registry, the generation counter and check_same_thread=False stay because the test teardown needs them (an unclosed sqlite3.Connection is a self-cycle on 3.11+); per-thread ownership is still pinned by test_knowledge_cross_thread.py.

@bolichen97

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none -- scripts/check_lockdown_before_publish.py is reverted to origin/main and out of this diff

  • span=8d049b65707f fixed in 7d27b67

Correct: the checker's four-walks-to-one change fixed no red and no leak; it was a speed-up a fixer found on the way, and the intent comment lists no such goal.
Removed from this PR. The ratchet in test_lockdown_before_publish.py keeps its own streaming change, which is the memory finding this pass is about; the checker speed-up is a follow-up on its own.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Sep 22, 2026
@bolichen97
bolichen97 force-pushed the test/hygiene-sweep-2026-09-22 branch from 7d27b67 to 0afc43c Compare September 22, 2026 14:59
@bolichen97

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: a counting str subclass handed to _brace_pairs / _glob_to_regex through the delegating stand-ins counts every character each pass reads (iteration, indexing, slices, find/index/count scans)

  • span=957bd9b588b3 fixed in 0afc43c

Correct: the invocation trace pins the NUMBER of passes, and the thread-CPU backstop ran at the small size only, so a quadratic INSIDE one pass at 12k braces was measured by neither.
The test now also counts the characters the primitives read and asserts 4x the braces reads at most 6x the characters. Measured linear: 6,016 -> 24,016 (4.0x). Planted a semantics-preserving pattern.find("}", j) per { inside _brace_pairs: 67,972,740 -> 1,081,890,240 (15.9x), fails; plant removed.
Deterministic on any host: it counts work, not time.

@bolichen97

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: the mirror test places the candidate directly in the trusted dir (no symlink) and names it gh.exe on Windows, so the skipif is gone

  • span=1b8f4555de66 fixed in 0afc43c

Correct: the mirror test inherited the sibling's skipif(IS_WINDOWS, "POSIX symlink layout"), but the refusal it pins is decided on the resolved path BEFORE any POSIX mode check and needs no symlink, so skipping Windows left the guard unproven there.
Now the fake lives at <home>/Cellar/gh/1.0/bin/gh[.exe] with that dir as the trusted dir; _trusted_bin("gh") must return None on every platform.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=3d1abb1a998d rebutted

Considered a second fix: commit for crew_log/emit.py, crew_log/lease.py, mcp_gateway/gatewayd.py / pool.py: neither ordering yields two independently green commits.
The test pass depends on the production seams (lease._release_all_for_tests, pool.spawn_shutdown, KnowledgeStore._close_all_for_tests) and the production fixes are pinned only by tests in the same files as the hygiene changes, so a bisect landing between the two commits is red either way.
A revert of the production fixes alone is still one git revert of the src hunks; the one-commit shape is what keeps every commit on main green.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@bolichen97 marked the gpt AI finding as false positive, not applicable, or explicitly accepted for 24e370457450de77f328061312621af1d28e43cf.

the Path.home monkeypatch auto-reverts at teardown and pins the one seam _trusted_bin reads; the HOME-refusal branch is asserted by the sibling test that plants inside the pinned home (adjudicator flagged F1 as not a defect, fenced only by class).

This decision applies only to this commit. A new push requires a new judgment.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded, but the following reviewer lane(s) could not be re-run automatically: GPT 5.6. Re-run the lane's latest workflow run for 24e370457450de77f328061312621af1d28e43cf manually from the Actions tab.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Sep 22, 2026
@bolichen97
bolichen97 enabled auto-merge (squash) September 22, 2026 22:43
@bolichen97 bolichen97 changed the title fix: crew-log lease pin, gatewayd orphan reap; ninth hygiene pass fix: gatewayd orphan reap, knowledge close seam; tenth hygiene pass Sep 22, 2026
@bolichen97
bolichen97 force-pushed the test/hygiene-sweep-2026-09-22 branch from 24e3704 to 854d4e0 Compare September 22, 2026 23:00
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running labels Sep 22, 2026
@buluoray

Copy link
Copy Markdown
Contributor

Reviewed 854d4e06f730a564f260d04fc26b0212423b04ff against a worktree pinned to refs/pull/12820/merge. Not approving: GPT 5.6's blocking finding is real, and I confirmed its mechanism against source. Everything else I checked is clean, and the detail below is meant to save you re-deriving it.

The blocking finding is real, but it is pre-existing on main, not introduced here

test/test_platform_compat.py::test_kill_pid_nonexistent_is_safe calls pc.kill_pid(nonexistent, pc.SIGKILL) — the unpinned variant. On Windows that reaches platform_compat.py:6350-6365 and runs a real taskkill /F /PID 2000000000 with no range check and no identity pin (kill_pid_pinned is the variant that holds the query handle across the terminate; this call site does not use it). If a live Windows process held that PID it would be force-terminated, with no recovery path. Windows exposes no readable PID ceiling to bound it the way /proc/sys/kernel/pid_max bounds Linux.

What GPT's write-up and the adjudication both leave out: main already has this exact call with no platform guard at all. Your diff only adds the Linux assertion and the comment, so the Windows exposure is unchanged and the Linux half is strictly safer than before.

The part that is genuinely yours is a claim-vs-code gap in the comment you added. It states the standard —

This is a REAL SIGKILL, so "nonexistent" has to hold by construction, not by luck

— and then establishes it only for Linux (pid_max) and macOS (PID_MAX 99998). On Windows the claim is unbacked and the comment does not say so, which is what makes a reader believe the case is covered. Since the PR already owns these lines, the fix is in scope and small: guard the real call (if not pc.IS_WINDOWS:, or a skipif) and rely on the mocked taskkill mapping tests for the Windows exception shape, then amend the comment to state which platforms the by-construction argument actually covers. If you would rather keep Windows exercising the real path, the comment needs to say that Windows rests on rarity rather than construction — but I would take the guard, because the comment's own standard is the right one and Windows cannot meet it.

Verified clean, for the record

Five production behaviour changes, each pinned by a guard test I mutation-verified — all five mutations caught, each one changing real behaviour rather than breaking compilation or collection, with the tree restored clean between them:

Production change Mutation Result
gatewayd.py pool.spawn_shutdown(orphan) reverted to inline await orphan.shutdown(timeout=2.0) caught — test_mcp_gateway_pool_integ::test_real_stubs_without_poolable_get_their_own_backend
vector_memory.py init() closes the handle it replaces removed the close block caught — test_vector_memory::TestSchemaInit::test_a_repeated_init_closes_the_handle_it_replaces
knowledge/store.py _close_all_for_tests conn.close() → pass caught — test_knowledge::TestKnowledgeStore::test_close_all_releases_every_threads_connection_and_reopens_lazily
browser_cli/install.py cwd=candidate.parent removed the cwd= argument caught — test_browser_cli_install::TestWindowsGatewayCommand::test_install_stages_node_inside_the_managed_root
crew_log/emit.py with_traceback(None) (before the rebase dropped it) reverted to return exc caught — 6 teardown lease-residue ERRORS on exactly the disk-error tests

knowledge/store.py (+129/-5), the largest change and the one the title does not name, is production-inert. With _ALLOW_CROSS_THREAD_CLOSE_FOR_TESTS false the connect kwargs are byte-identical to before, no factory is installed, no _owner_ident is set and nothing is registered in _connections; the db property's new generation check cannot fire, because getattr(self._thread_local, "gen", self._generation) defaults to the current generation and _generation never leaves 0; _close_all_for_tests raises unless the flag is set. Only test/conftest.py sets it, and there is no production caller. No query, index, dedup or delete path changed, so there is no data-loss surface. The one production-visible delta is close() rebuilding an always-empty _connections list under a lock.

The orphan reap introduces no new PID or PGID reaper. Backend.shutdown is unchanged (0 lines) and still acts on its own spawned self.process handle, escalating through platform_compat.kill_process_tree_async. The change is purely scheduling. _safe_shutdown uses timeout=2.0, identical to the timeout=2.0 it replaces, so the reap budget is unchanged. shutdown_all snapshots _shutdown_tasks under self._lock and gathers it, so the task is joined. The _spawn_shutdown → spawn_shutdown rename is complete: 0 references to the old name remain and both production call sites are updated in-diff. The new path also has no race — release_exclusive is awaited, then spawn_shutdown runs synchronously with no suspension point between them, so teardown cannot slip between the map removal and the task creation. That window was previously the whole failure: the inline await was cancelled by the same teardown while release_exclusive had already dropped the backend from the exclusive map, so nothing reaped it.

vector_memory.py: the close sits inside init() under _db_lock, immediately followed by _init_database() reopening the handle. browser_cli/install.py: cwd is keyword-only and defaults to None, preserving the old behaviour for every caller but _staged_node_runs.

No weakened assertion across the test and doc files. Net +30 asserts, no added skip/xfail/skipif, no numeric tolerance introduced. All 21 removed asserts are accounted for: verbatim moves (test_mcp_gateway_daemon_lifecycle.py's three warming asserts move out of a manual try/finally now handled by the autouse _no_git_fingerprint fixture), black re-wraps (test_context_marker_neutralization.py), fixture-parameter rewrites with identical assert bodies (test_skill_search_index.py, test_skill_mapping_and_lazy_default.py), an absolute-path hardening (test_browser_launcher.py), and one genuine strengthening — test_argv_floor_inline_and_brace_scope.py trades a flaky wall-clock ratio for a deterministic visit-count bound plus an anti-vacuous guard, keeping the CPU-time ceilings on time.thread_time().

Both shared-infrastructure files were read in full. conftest.py's two new autouse fixtures are isolation, not weakening: _knowledge_store_cross_thread_close relaxes only SQLite's native guard while _ThreadAffineTestConnection re-applies it in Python for everything but close(), and test_knowledge.py flips the flag off to keep the production path covered; _no_boot_sandbox_sweep rebinds only kiro_crew.session.cleanup_stale_sandbox_profiles while the sweep's own tests call kiro_crew.sandbox.cleanup_stale_sandbox_profiles directly, with test_session_cleanup.py::TestTheSuiteNeverRunsTheBootSandboxSweep pinning the no-op in force. source_corpus.py's streaming rewrite is semantically equivalent — same sorted(rglob("*.py")) walk, same readable filter, and _nfkc's isascii() shortcut is sound because NFKC is identity on ASCII, so homoglyph folding is preserved. No ratchet narrows; the removed _read_tree/_normalized_texts have 0 references.

.github/black-baseline.txt removes exactly the files that became black-clean, verified with black --check, and scripts/check_black_formatting.py reports no pending prune. Docs (knowledge.md, mcp-gateway-daemon-lifecycle.md, testing-conventions.md) describe the seams consistently with the code.

Two description inaccuracies

Neither is a defect, but the review lanes read the description:

  • It says .github/black-baseline.txt "drops three files that are now black-clean"; the diff removes two (test/test_context_marker_neutralization.py, test/test_context_ui_language.py).
  • The Tests section names test_vector_memory.py::TestPersistence::test_a_repeated_init_closes_the_handle_it_replaces. The actual class is TestSchemaInit — it spans lines 898-1204 and the test is at 1182. The test exists and passes.

Test runs

  • test_crew_log_edge_exhaustion, test_mcp_gateway_daemon_lifecycle, test_mcp_gateway_pool_integ, test_vector_memory, test_browser_cli_install, test_eventlog_hooks — 441 passed, 14 skipped in 39.20s.
  • test_knowledge, test_knowledge_store_onloop_db, test_security — 1409 passed, 1 skipped, 3 failed. All three failures are TestFileReaderPdf::* on pdfplumber import failed, an optional dependency missing from my local venv. That class is not in the diff.

Scope note and what I did not verify

The bulk of the verification above ran at 24e370457450de77f328061312621af1d28e43cf, before the rebase. I re-checked the carry-over rather than assuming it: the rebase dropped exactly four files (crew_log/emit.py, crew_log/lease.py, test_crew_log_edge_exhaustion.py, test_eventlog_hooks.py — now on main via #12812) and added none, every surviving production patch is byte-identical line-for-line, and the only conftest.py change is "ninth" → "tenth" in two comments. So the findings transfer, except the emit.py mutation row above, which is no longer in this PR's scope and is listed only as the evidence it was.

Not verified: the three TestFileReaderPdf tests did not execute locally. The Windows behaviour of both kill_pid and platform_compat.kill_process_tree_async is reasoned from source only — I have no Windows host, which is also why I am relaying the blocking finding rather than claiming to have reproduced the kill. The full suite was not run, only the targeted module suites, so a cross-file ordering interaction outside them is unverified. testing-conventions.md's added lines were spot-checked against the seams they describe rather than read line by line.

@bolichen97
bolichen97 force-pushed the test/hygiene-sweep-2026-09-22 branch from 854d4e0 to 8028200 Compare September 25, 2026 16:09
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 25, 2026
@bolichen97

bolichen97 commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: @pytest.mark.skipif(pc.IS_WINDOWS, ...) on test_kill_pid_nonexistent_is_safe; the comment now states which platforms the by-construction argument covers

  • span=42863e1f43fa fixed in 8028200

Correct: the real kill_pid(2_000_000_000, SIGKILL) reached taskkill /F on Windows, where no readable pid ceiling can prove the number unused, and the comment's "by construction" claim was established for Linux and macOS only.
The real-kill branch is now skipped on Windows; the rc=128 -> ProcessLookupError shape it was also pinning stays covered by the mocked TestTaskkillErrorMapping::test_taskkill_rc128_maps_to_process_lookup, and the comment says Windows does not run the real signal and why.
Also in this push: rebased onto main 6b4afd7 (413 commits; two opened-fixture hunks in test_memory_smoke.py / test_perf_memory_quickwins.py reconciled with main's renamed episodic tests and get_activity_context stub) and the two description inaccuracies from the human review corrected (two baseline files, TestSchemaInit). The prior shard reds (test_crew_log_exc_info_sites.py, test_diag_procs.py) were inherited from the stale merge-ref and pass on the current base.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 25, 2026
@bolichen97
bolichen97 force-pushed the test/hygiene-sweep-2026-09-22 branch from 8028200 to 78109bc Compare September 25, 2026 17:16
@bolichen97

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: test_kill_pid_nonexistent_is_safe runs on every platform again; on Windows pc.subprocess.run is stubbed to taskkill's rc=128 and the argv is asserted to name the probe pid

  • span=42863e1f43fa fixed in 78109bc

Taken the lane's own fix: the skip is gone, the contract (ProcessLookupError on a missing pid) is asserted on the Windows shard through the stubbed taskkill, and the real taskkill /F -- the one call no readable pid ceiling can prove harmless, which the previous round of this lane blocked -- is the only thing not run.
The comment states which half is real (POSIX, pinned to pid_max on Linux) and why the Windows arm is stubbed.
Same push: the Opus discovery candidates that survive a read of the code are addressed -- test_source_corpus.py materialises the corpus once per test instead of twice in one expression; the _ThreadAffineTestConnection docstring, _connect comment and both docs name the six guarded entry points instead of claiming "everything but close()"; test_security.py's own-host fixture filters the enumeration to the stub's addresses so /proc/net/if_inet6 and the adapter sweeps cannot seed real host addresses, and its docstrings say so.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 25, 2026
@bolichen97

bolichen97 commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: _pin refuses to store an unreadable identity -- it kills and reaps the just-spawned batch and raises -- and _reap retries an unreadable re-read, then fails the test by name instead of sparing a live sleeper

  • span=a990eeac557c fixed in 6e62dd6

Correct: at fc947e7b65 the macOS ps leg could hand _pin a None identity, which _reap then read as "unconfirmable" and skipped, sparing a live 300-second sleeper. Fixed as recorded in full on the 6e62dd6da5 record below (pin-time kill of the batch that is ours by construction, reap-time retry then a named failure).
Same push: the Opus discovery candidates that survive a read of the code are addressed -- test_source_corpus.py materialises the corpus once per test instead of twice in one expression; the _ThreadAffineTestConnection docstring, _connect comment and both docs name the six guarded entry points instead of claiming "everything but close()"; test_security.py's own-host fixture filters the enumeration to the stub's addresses so /proc/net/if_inet6 and the adapter sweeps cannot seed real host addresses, and its docstrings say so.
Rebase-only push onto main 4926e50 (content-neutral: same 64 files, +2632/-1192; hunk offsets only) to mint a fresh CI wave carrying #13698 (bounded black) after three exit-143 runner deaths on the previous head (#13386).

@bolichen97

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: _pin reads every identity first; if any is unreadable it SIGKILLs and reaps the whole just-spawned batch and raises, so no None pin is ever stored; _reap retries an unreadable re-read three times and, if the pid is still present and still unreadable, fails the test naming it instead of silently sparing it

  • span=a990eeac557c fixed in 6e62dd6

Correct: on the macOS ps leg process_start_time returns None on a timeout or missing binary, _pin stored that, and _reap's token is None -> continue then spared a live 300-second sleeper -- and the _reap([witness_pid, unrecorded_pid]) site has no following wait to surface it.
Pin time is the one moment the pids are ours by construction (just spawned, or just reported by a root that is still ours), so that is where the kill goes; probed by forcing process_start_time to None around _pin on a real sleeper: it raises and the sleeper exits -9. At reap time the teardown neither guesses nor stays silent: a still-present pid whose identity never reads fails the test by name.
Same push: test/test_knowledge.py reformatted with black --target-version py310 (the first black verdict this PR has had -- every earlier lint job died in the runner before black finished; #13698 on the new base fixed that).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants