Repository navigation
test(context): bound the import-order subprocess by the lost-run bound - #20795
cruisercohen merged 1 commit into
Conversation
First Principles Review (Opus 5.5, fork) — ⏭️ skippedRevision |
UX Review (Opus 5.5, fork) — ⏭️ skippedRevision |
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing to check. Suggestions
[DESIGN-REVIEWED] 1026ca3 Description read[DESCRIPTION-READ] f142ee5d5aad931e49d200d9758890b5dcd37acfddcab64ad33bd43f4da259e7 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
48849b6 to
61829bc
Compare
61829bc to
c592f01
Compare
c592f01 to
c6cd5b1
Compare
cruisercohen
left a comment
There was a problem hiding this comment.
Reviewed head c6cd5b1.
What I checked.
- The change is test-only:
test_agent_sdk_context_bridge.pyandtest_context_composition_contract.py. Both cold-subprocess import probes now take their bound fromdefault_timeout()(half the running test's timeout) instead of fixed 30 s / 60 s values, so a slow cold interpreter on a Windows shard fails inside the runner's budget rather than at a hard-coded wall. - The
faulthandler.dump_traceback_latercall fires a few seconds before that bound withexit=False, so a real hang leaves a stack in stderr and the assertion names where the import stopped. - The
TimeoutExpiredhandlers decodebytesstderr (POSIX re-raises it undecoded even withtext=True) and raise anAssertionErrorthat names the import order and the bound, withfrom Noneso the message is not buried under the subprocess traceback. - Locally on this head: the 9 tests in the two files pass. A probe that shrinks the bound below the cold-import time confirms the timeout path names the order, carries decoded stderr (no
b'...') and, with a slightly longer bound, includes the faulthandler stack. - The new head is a rebase of c592f01: same patch, same added and removed lines.
CI on this head.
The run has finished red, and none of the failures is in the two files this PR touches:
- Backend Tests 3.12 shards 1-8 and Windows shard 6 fail only on
test_the_lazy_imports_stay_inside_the_functions_that_need_them(assert 84 == 83). That count is wrong on main too; #20930 fixed it on main at 13:35 PDT (20:35 UTC), after this run's merge ref was taken, so a re-run will not pick it up. Updating the branch onto current main should clear it. - Windows shard 2 fails on
test_spawn_delete_queued.py::test_an_internal_parent_deletes_its_own_queued_spawn, which this diff does not touch and which passes locally. Worth watching on the next run.
Looks right to me. I'll approve once the required checks are green on a head that has the main fix for the lazy-import count.
The context import-order test gave its fresh interpreter a fixed 30 s, which a cold import on a loaded Windows runner exceeded. Pass default_timeout() and turn TimeoutExpired into an assertion naming the import order. The facade fresh-import test in the context composition contract gets the same bound and a named failure. A hang inside an import writes nothing to stderr, so each child arms faulthandler.dump_traceback_later shortly before the bound; the assertion then shows the stack the import stopped in. The partial stderr is decoded first, because on POSIX the re-raised TimeoutExpired carries bytes even with text=True. Fixes kirodotdev#20782
c6cd5b1 to
1026ca3
Compare
|
Design Review on
|
|
Re the review on CI on
Timeout path, checked
First-pass check: R1-R8 OK
|
cruisercohen
left a comment
There was a problem hiding this comment.
Approving 1026ca3, a rebase of c6cd5b1 onto main: git range-diff shows the same single commit, with the same patch-id and the same changed lines in both test files.
On this head, both PR test files pass locally (129 tests), and the lazy-import count test that was red on c6cd5b1 passes now that #20930 is in the base. The two subprocess probes take their bound from default_timeout(), dump a stack through faulthandler before it, and fail by name with decoded stderr.
The Backend Tests (3.12, 4) red on this head was test_instances.py::TestOrphanForwarderReclaim::test_recorded_argv_fingerprint_survives_command_line_drift[False], a file this PR does not touch. It passed on rerun and is tracked in #21261.
All required checks are green on this head.
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: test (2 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: the two import-order subprocess tests use the default lost-run bound and dump a stack trace before it, so a hang names the import it stopped in; test-only.
Problem / Motivation
Goal: the context import-order test waits as long as the suite's other bounded waits for its fresh interpreter, and a child that never finishes fails with a message naming the import order.
test/test_agent_sdk_context_bridge.py::test_context_import_orders_do_not_form_a_cyclestarts a fresh interpreter per import order and gave ittimeout=30. On a loaded Windows runner a cold import of those modules took longer than that, so two parametrizations failed with a baresubprocess.TimeoutExpiredon a PR that touched nothing near them. The test checks import order, not import speed.Why it matters
Every PR's Windows shard can go red on this test for reasons unrelated to its diff, and the bare
TimeoutExpireddoes not say which import order was slow, so each one costs a log read and a rerun.Not a goal
What changed (motivation → approach → change)
Symptom:
TimeoutExpiredafter 30 s on Windows (5). Cause: a fixed 30 s literal, shorter than the bound the suite gives every other wait, on a step whose cost is a cold interpreter start. Change:--timeout=120)--timeout=180)test_context_import_orders_do_not_form_a_cycletest_context_composition_contract.py::test_every_owner_is_loaded_by_a_fresh_import_of_the_facadedefault_timeout()(fromkiro_crew.testing.wait: half the running test's timeout, which the rootdir conftest publishes at each test's setup) tosubprocess.run.TimeoutExpiredbecomes anAssertionErrorthat names what hung, e.g.importing kiro_crew.essential_delivery -> kiro_crew.providers.base -> kiro_crew.context in a fresh interpreter did not finish within 60 s.faulthandler.dump_traceback_latera few seconds before the bound (55 s of 60 s, 85 s of 90 s). A hang then puts its import stack in the assertion message. The partial stderr is decoded first, because on POSIX the re-raisedTimeoutExpiredcarries bytes even withtext=True.A child that never finishes fails at half the test timeout, so pytest-timeout never kills the worker first. Measured here: a cold run of one order takes about 4.5 s.
Checked and left alone: the other fresh-interpreter import tests (
test_agent_lifecycle_cycle.py,test_agent_import_hoist.py,test_perf_boot_path.pyand the composition-contract probes) already wait 120 s or more.Why not import every order in one interpreter: the test needs a clean
sys.modulesper order to prove there is no cycle, so each order needs its own interpreter.Backwards compatibility
Compatible: test-only. No production code changes.
Removes nothing: both waits take the lost-run bound in place of a literal, and a timeout now names the import that hung.
Tests
Run on Linux (aarch64, Python 3.12) on this head, basetemp outside the repo:
-pplugin printingdefault_timeout()inside each of the 4 cases): 90.0 s under--timeout=180(the Windows shard's flag), 60.0 s under--timeout=120. The value comes from the published runner timeout, not the 60 s fallback.kiro_crew/context.py, child only, writespartial-éto stderr and then sleeps in a named function),--timeout=20so the bound is 10 s: all 4 cases fail by name in 42 s, and each message shows the decodedpartial-é, the faulthandler dump with the hung function's frame incontext.py, and nob'...'.-pplugin prepends a 35 s sleep to the child's-ccode, as slow as the reported Windows runner),--timeout=120: the parent commit's import-order test fails 3/3 with a baresubprocess.TimeoutExpired ... timed out after 30 seconds; this head passes 3/3.-n0; 10/10 green at-n 4 --dist loadgroup --max-worker-restart=2with both files as co-tenants (129 tests); both files in a shuffled order, seed 27909: 129 passed.test_dashboard_server_composition_contract.py::test_the_lazy_imports_stay_inside_the_functions_that_need_them: 130 passed.test/test_*ratchet*.pyplustest_agent_refactor_facade.py,test_spawn_audit.pyandtest_file_line_cap.py: 394 passed.test_flake_pattern_ratchet.pyat-n0: 58 passed.Manual verification
N/A — test-only change; the bound probe and the hang runs above are the verification.
Related Issues
Fixes #20782
Pattern harvest
Rule candidate: semgrep
Pattern: a test passes a fixed
timeout=literal shorter thandefault_timeout()tosubprocess.runon a cold interpreter start, and letsTimeoutExpiredescape without naming what hung.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)