Skip to content

test(acp): keep the resume lifecycle off the sensitive-path resolver pool - #20564

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
fix/flake-runtime-cleanup-load-wait
Oct 15, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
fix/flake-runtime-cleanup-load-wait

Conversation

@CrysisDeu

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The two [load] cases in test/test_runtime_cleanup_regression.py stop timing out on the Windows backend shard, on every pull request, for good.

Backend Tests (Windows) (2) keeps going red on pull requests that touch nothing near this file:

  • test_session_start_persists_new_descendants[load]
  • test_cancelled_descendant_scan_terminates_only_failed_session[load]

Both expire the 5 s asyncio.wait_for around the resume (rt.load_session) and fail as TimeoutError (GitHub's annotation quotes the inner CancelledError frame from asyncio/locks.py:212, which is the same expiry). The [new] cases of the same tests pass in the same job. A seven-day mine over CI found the pair on 26 and 24 distinct PR branches, Windows only.

Why it matters

Every author whose PR lands on that shard pays a rerun for a test about a runtime they did not touch, and the Windows job is the slow one to rerun. The red also hides real Windows regressions in this file behind "known flaky".

Not a goal

  • Changing the production resume path, the sensitive-path fence or the resolver pool.
  • Widening the 5 s bound: the bound is not what is wrong.
  • The other three tests in the file, which never read an agent spec and never flaked.

What changed (motivation → approach → change)

Symptom: only the [load] cases expire, and only on a loaded Windows shard.

Root cause: load_session resolves its MCP array from the agent spec, where create_session is handed an explicit mcp_servers=[] by this test and never reads one. #19093 sealed one half of that read, the managed-spec self-heal (ensure_agent_materialized). The other half was still live: the spec read's project-checkout scan (session_mcp._project_spec_path_for → agent_discovery.project_agent_files → security.is_sensitive_path(work_dir)) runs the test's work dir through the sensitive-path fence, whose symlink resolution is served by the process-wide resolver child pool (executors.path_resolve_executor()), spawned on first use. The resume path reaches it twice per start (the _ref_spec_snapshot passenger and kiro_control_plane_servers). Profiled locally that first resolution is 36 ms of a 0.5 s start; on a Windows runner with four xdist workers, two fresh python children plus the ~130-path anchor rebuild is seconds, and the product's own budgets for it (_PATH_RESOLVE_TIMEOUT_SECS = 2.0, _PATH_RESOLVE_REBUILD_TIMEOUT_SECS = 8.0) are already wider than the test's 5 s. Whether a given [load] case pays it depends on whether an earlier test in the same worker already warmed the pool, which is why it flakes rather than fails.

Change, all in the lifecycle fixture:

  1. Seal the spec read at its one entry, session_mcp._agent_spec_for → None, in place of the narrower ensure_agent_materialized seal. The resume path is then answered "no spec" before either cost (self-heal or fence) is reached, the same shape the create path already takes with its explicit array. The unresolved-ref guard goes silent (its documented None case) and the control-plane mount yields nothing, which is what the test's empty pooled array already asserted.
  2. Extend the fixture's closed interface to the resolver: security.paths._run_resolution_bounded raises AssertionError naming the path, so a future change that reintroduces a fence hop on this lifecycle fails by name on every host, instead of only on the shard slow enough to expose it. All five tests pass under it, which is the proof that nothing else in the file reaches the pool.

The bounds, the simulated kernel and every assertion are unchanged.

Backwards compatibility

Compatible: test-only change; no production code is touched.

Removes nothing: the fixture replaces one monkeypatch seal with a wider one on the same read path and adds one assertion seal; no production path, key or function changes.

Tests

Changed: test/test_runtime_cleanup_regression.py::lifecycle (fixture). Locks in: the resume path under this lifecycle never reads an agent spec and never reaches the sensitive-path resolver pool (the AssertionError seal), so the [load] and [new] cases do the same amount of work inside their 5 s lost-run bounds.

Determinism proof (testing-conventions § Proving a determinism fix), Linux worktree, Python 3.12.14, uv venv:

Repeats, 20 × -n0:

for i in $(seq 20); do python -m pytest -n0 -p no:cacheprovider -q test/test_runtime_cleanup_regression.py || break; done
n0 passes: 20/20

Repeats, 10 × -n 4 --dist loadgroup --max-worker-restart=2 with two sibling files as co-tenants (test/test_acp_runtime_session_start.py, test/test_runtime_cleanup_windows.py):

n4 passes: 10/10
================= 50 passed, 21 skipped, 5 warnings in 16.45s ==================

Shuffled order (ids of the same three files, @file):

seed=19623
71 shuffled.txt
python -m pytest -n0 -p no:cacheprovider -q @shuffled.txt
================== 50 passed, 21 skipped, 1 warning in 28.05s ==================

Forced condition: a throwaway -p plugin (not committed) holds the first security.paths._resolve_on_calling_thread call for 6 s, which is the cold resolver pool on a loaded shard.

Parent (10e53cc1, origin/main):

[force] first resolution of '.../test_session_start_persists_ne0': sleeping 6.0s
[force] first resolution of '.../test_cancelled_descendant_scan0': sleeping 6.0s
FAILED test/test_runtime_cleanup_regression.py::test_session_start_persists_new_descendants[load] - TimeoutError
FAILED test/test_runtime_cleanup_regression.py::test_cancelled_descendant_scan_terminates_only_failed_session[load] - TimeoutError
============ 2 failed, 2 passed, 1 deselected, 1 warning in 14.90s =============

Same shape as CI: TimeoutError out of the 5 s wait_for, CancelledError in the inner frame.

Fix:

========================= 5 passed, 1 warning in 3.33s =========================

The [force] line never prints: the resolver is not reached.

Where the time went before the fix (cProfile on the two off-loop hops only the resume path takes, cold worker): _pooled_session_servers_and_ref_spec 36 ms, of which 35 ms is is_sensitive_path → _run_resolution_bounded → subprocess_pool.executor.call_op (select.select on the child pipe); the second hop, kiro_control_plane_servers, 2 ms once the pool is warm.

Gates: python -m pytest -n0 test/test_flake_pattern_ratchet.py 53 passed; test/test_wait_discipline_ratchet.py 39 passed; python3 scripts/check_comment_history.py clean; check_brand_name.py, check_black_formatting.py, isort, flake8 clean; python3 scripts/local-gate.py all selected gates passed.

Manual verification

N/A — unit coverage sufficient: the fixture seal is exercised by the five tests in the file, and the forced-condition run above is the manual reproduction.

Related Issues

Part of #19498

Pattern harvest

Rule candidate: review-prompt
Pattern: a test's lost-run bound wraps a product path whose first call lazily spawns a process-wide pool (the sensitive-path resolver children); the cost lands on whichever test in the worker goes first, so it flakes by test order and host load rather than failing. Seal the pool-reaching seam in the fixture, and make the seal assert, so the next hop that reaches it fails by name.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

…pool

The [load] cases of test_runtime_cleanup_regression.py expired their 5 s
wait on Windows shard 2 while the [new] cases passed beside them.
load_session reads the agent spec where create_session, handed an explicit
array, does not; the spec read's project-checkout scan runs the work dir
through the sensitive-path fence, whose symlink resolution spawns the
process-wide resolver child pool on first use. On a loaded Windows worker
that cold spawn alone outlasts the bound, and whether a case pays it
depends on test order, so it flaked instead of failing.

The lifecycle fixture now seals the spec read at its one entry
(session_mcp._agent_spec_for -> None), replacing the narrower
ensure_agent_materialized seal, and asserts that nothing on this lifecycle
reaches security.paths._run_resolution_bounded, so a reintroduced fence
hop fails by name on every host. Bounds and assertions are unchanged.

Part of #19498
@CrysisDeu
CrysisDeu requested review from a team and KiroCrewCaptain as code owners October 15, 2026 01:07
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 15, 2026
@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision f59b94606ac1dcb6dd2aa7a9c6e672c4a82eb967 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Advisory — does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision f59b94606ac1dcb6dd2aa7a9c6e672c4a82eb967 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of f59b94606ac1dcb6dd2aa7a9c6e672c4a82eb967 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] f59b946

False positive or not applicable? A repository writer can comment:
/ai-review override gpt f59b94606ac1dcb6dd2aa7a9c6e672c4a82eb967: <one-sentence reason>

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of f59b94606ac1dcb6dd2aa7a9c6e672c4a82eb967 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Confirm the AssertionError seal surfaces as a failure. If a broad except in is_sensitive_path swallows it, the "fails by name" guard proves nothing.

[DESIGN-REVIEWED] f59b946

Description read

[DESCRIPTION-READ] 5875a5ab095879306543e53765dfeaa3631baabcf9c101c7fa0b845ca8564b85

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed f59b94606ac1dcb6dd2aa7a9c6e672c4a82eb967 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] f59b946

Verdict parsed from the review's SHA-scoped output markers for commit f59b94606ac1dcb6dd2aa7a9c6e672c4a82eb967.

False positive or not applicable? A repository writer can comment:
/ai-review override fable f59b94606ac1dcb6dd2aa7a9c6e672c4a82eb967: <one-sentence reason>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 15, 2026
@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) October 15, 2026 02:05

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (1 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: test-only change to one regression test, which stubs the agent-spec read and makes any reach into the path resolver pool fail by name.

@KiroCrewCaptain
KiroCrewCaptain merged commit 9163daf into main Oct 15, 2026
100 of 101 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the fix/flake-runtime-cleanup-load-wait branch October 15, 2026 02:07
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants