Repository navigation
test(acp): seal the agent-spec self-heal in the runtime cleanup lifecycle fixture - #19093
Conversation
|
Determinism proof for head a3dab0e (Linux, Where the 5 s went (hop trace, parent Same hop on this branch: Repeats Shuffled order ( Forced condition — throwaway # slow_rebuild.py
import time
from kiro_crew import agent as agent_mod
_real = agent_mod.rebuild_agent_config
def _slow_rebuild(*args, **kwargs):
time.sleep(6.0) # a loaded Windows shard at the seam the diagnosis names
return _real(*args, **kwargs)
agent_mod.rebuild_agent_config = _slow_rebuildBackend gate floor from the resolved profile: all green locally (scoped backend tests, black, isort, flake8, mypy, every |
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that the Windows [DESIGN-REVIEWED] a3dab0e Description read[DESCRIPTION-READ] 3e1f8461c9c014e62347995d2cee16d0e74b4a15be953c02f5dc3dcb9770a13a That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Review detailsI found no defects in this PR's changes, which edit one test fixture. The test patches No findings. [OPUS-REVIEWED] a3dab0e Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
|
iamwhatever
left a comment
There was a problem hiding this comment.
This PR fixes the Windows-only TimeoutError flake in the two [load] tests of test/test_runtime_cleanup_regression.py. The lifecycle fixture now stubs session_mcp.ensure_agent_materialized, so load_session's MCP-array hop no longer runs a full rebuild_agent_config() inside the 5 s budget. That rebuild is the root cause: the [new] twins pass mcp_servers=[] and never enter the hop. I confirmed that session_mcp imports the function by name and calls it at line 679, so the patch takes effect. This is the same seam the fixture already seals for pooled_session_servers and that other ACP tests stub, and no kiro-cli in this test reads the spec. The forced-condition proof makes rebuild take 6 s: the parent goes red on exactly the two [load] tests and this head stays green. All 8 Windows shards passed on this head. The Design-lane note has a posted disposition. This is a test-only change with no production code, base is main, checks are green, and there are no change requests. — crew-pr-reviewer (deep re-check)
Problem / Motivation
Goal: The two
[load]tests intest/test_runtime_cleanup_regression.pypass on every shard, every time, without a wider timeout.test_session_start_persists_new_descendants[load]andtest_cancelled_descendant_scan_terminates_only_failed_session[load]fail withTimeoutError(the 5 sasyncio.wait_foraroundrt.load_session) onBackend Tests (Windows) (2). Seen on PRs that touch nothing near this code (#17684, #17784, #18708, #18964, #18894) while main stays green. The[new]twins of both tests never fail.Why it matters
Every open PR can turn red on a Windows shard for a test it did not touch, and each red costs a rerun and a flake-ledger entry. The test is a lifecycle test against a fake kernel; a timeout there says nothing about the code under test.
Not a goal
load_session,ensure_agent_materializedor the agent-spec self-heal in production code.What changed (motivation → approach → change)
The two
[load]tests callrt.load_session. Its MCP-array hop reads the agent spec throughsession_mcp._agent_spec_and_snapshot_for, which callsensure_agent_materialized("kirocrew")first. The per-test agents dir is always empty, so that call runs the wholerebuild_agent_config()inside the hop: a dozen managed agent spec files, a config migration with a backup, and the sensitive-path anchor caches. That is ~300 ms on Linux and open-ended on a loaded Windows shard; the CI traceback shows the task parked in exactly thatto_threadwhen the 5 s budget expires. The[new]tests passmcp_servers=[], which skips the hop entirely, so only[load]ever paid for the rebuild.The
lifecyclefixture already seals the other half of that same hop (pooled_session_serversreturns[]). It now seals this half too:session_mcp.ensure_agent_materializedis stubbed, the same seamtest_acp_session_mcp.py,test_acp_client.pyandtest_acp_mcp_ref_guard.pystub. No kiro-cli runs in this test, so nothing reads the spec that would have been written.With the stub the
[load]hop is ~10 ms and the[load]tests take the same time as their[new]twins.Backwards compatibility
Compatible: test-only change; no production code is touched.
Removes nothing: one fixture gains one monkeypatch line.
Tests
test/test_runtime_cleanup_regression.py: thelifecyclefixture stubs the agent-spec self-heal, so both[load]tests exercise the lifecycle code with the same inputs the[new]tests already use.Determinism proof (testing-conventions § Proving a determinism fix), Linux,
kc-shared-venv,PYTHONPATH=src:-n0, whole file-n 4 --dist loadgroup --max-worker-restart=2withtest_acp_session_mcp.pyandtest_acp_client.pyas co-tenants@file, seed in PR comment)-pplugin that makeskiro_crew.agent.rebuild_agent_configtake 6 sf07162523a: exactly the two[load]tests red withTimeoutError, 3 passed; this branch: 5 passedThe forced-condition plugin and the commands are in a PR comment; the plugin is not committed.
Manual verification
N/A — the fixture change is covered by the five tests in the file plus the forced-condition run above. The Windows CI shard is the real-platform check.
Related Issues
Closes #18874
Closes #18676
Pattern harvest
Rule candidate: review-prompt
Pattern: a lifecycle/fake-kernel test fixture that stubs one half of a two-read
to_threadhop (the pooled array) but leaves the other half (the agent-spec read with its self-heal) real, so one parametrization pays for a full on-disk rebuild under a per-call budget.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license.