Repository navigation
test(singles-4): replace fixed waits with signal waits - #22884
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Test-only migration. Each removed assertion is still enforced, because [DESIGN-REVIEWED] c712596 Description read[DESCRIPTION-READ] 49e1d31479b20d2c147c8db2790c637a2ec73e451d6a62930e5ecfaf44862e11 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
|
HOLD -- reviewed 3ba032f. The migration is mechanical and reads correctly: each poll or fixed What blocks approval: the Design Review's two notes have no reply.
To clear: fix or rebut each note on the PR. — crew-pr-reviewer (r118) |
3ba032f to
8bbcb8e
Compare
8bbcb8e to
6eccc8d
Compare
6eccc8d to
2aa4e44
Compare
Fifty test files each held one short fixed wait: a wait_for or Future.result ceiling of a few seconds, or a sleep inside a poll loop. Each site now waits on the signal the code under test emits, through kiro_crew.testing.wait (await_until, finished, async_wait_until, wait_until, until_parked), under the lost-run bound only. The waits that ARE the assertion keep a tight bound on the helper: the stalled resource probe (2 s), the hung usage refresh (10 s) and the frame ceiling of the subagent boundary tests. The shutdown-event timeout test asserts its timeout through pytest.raises. One loop ticker that measures free loop time keeps its sleep with a flake-ok reason. The transfer stall waits on a never-set event instead of a 1 s sleep, since only the lapsed deadline can end it. Each file's line leaves test/wait_discipline_baseline.txt.
2aa4e44 to
c712596
Compare
iamwhatever
left a comment
There was a problem hiding this comment.
APPROVE -- reviewed c712596.
This head is a rebase of 3ba032f (range-diff shows only baseline context lines). Every migrated site keeps its predicate, the bounds that are themselves assertions are kept, and the sandbox/kill-path test files change only how the test waits.
The two r118 notes, re-checked: the stalled-peer await asyncio.Event().wait() (test_session_transfer_outbound.py#L419-L421) still fails by name under the suite's --timeout=120, so a regression stays a red test. The txn_lock._waiters read (test_session_autocompact_override.py#L610-L614) is on an asyncio.Lock that the first request holds, and until_parked cannot express the 'or the second committed' half. Neither is a defect.
Design PASS, Opus and GPT no findings on this head; all checks green.
— crew-pr-reviewer (r119)
Problem / Motivation
Goal: Replace the 50 short fixed waits in 50 single-site test files with waits on the signal each test is actually about, so a slow CI runner cannot red them.
Why it matters
A
wait_for(..., timeout=5), a.result(timeout=5)or a 200-turnsleep(0.01)poll bets that the runner is as fast as the machine the test was written on. That bet is the single commonest cause of a flaky backend test here, and every lost bet reds an unrelated pull request.Not a goal
.github/black-baseline.txtstill lists (they stay listed; only the migrated lines change).# flake-okreason).What changed (motivation → approach → change)
Every counted site in these 50 files (one per file) is replaced with the matching
kiro_crew.testing.waithelper:asyncio.wait_for(event.wait(), N)/wait_for(task, N)/wait_for(gather(...), N)→await_until(event | future, what=...).asyncio.wait_for(coro, N)→finished(coro, what).Future.result(timeout=5)on a thread-pool orrun_coroutine_threadsafefuture →.result(timeout=default_timeout()).for _ in range(N): if cond: break; sleep(x)/while time.monotonic() < deadlinepolls →wait_until(cond, describe=...)orasync_wait_until(cond, describe=...). The poll helpers_await_pass_state,_settle(x3),_queued_on,_await_detached,_await_untiland_awaitkeep their signatures and delegate (_await_detachedand the not-found_settledrop their unusedtries/rounds; the side-steer_settledrops its unusedtimeout).test_session_control_reload.py:_queued_onis exactlyuntil_parked(lock).test_session_autocompact_override.py: the first request's persist is held until the second request is parked on the slot's transaction lock (or, absent serialization, has committed), instead of for a fixed 0.5 s.test_session_start_queued_behind.py: the terminate responder wakes on each frame the runtime writes (aside_effecton the mockedstdin.writesets an event) instead of polling every 10 ms.test_session_transfer_outbound.py: the stalled peer awaits a never-set event; only the lapsed no-progress deadline can end it, which is what the test asserts.test_memory_admission_queue.py:_settlefirst waits for the preparation task's done-callbacks to have fired (the drain is armed by one), then for the slot to go idle with no background task left.test_session_not_found_surfaces.py:_settlewaits on the state between one task ending and its respawn registering, instead of sleeping across that gap.test_shutdown_event.py: the "wait_for times out on the never-set event" test asserts its timeout withpytest.raises(asyncio.TimeoutError).test_opencode_session_mcp.py,test_pod_e2e_video_guard.py,test_push_guard_squash.py: the wait ends on the process / pid / mark state, and the original failure branches run after it.# flake-ok:comment on the line BEFORE the call, where the ratchet does not read it; those sites are migrated rather than re-marked.Kept on purpose:
test_resource_pressure_notify.py:finished(..., timeout=2.0)-- the bound IS the assertion (the sample must be forfeited atPROBE_TIMEOUT_SECS, before the probe's own 5 s release).test_session_usage_scrape.py:finished(..., timeout=10)-- the patched 0.2 s deadline must end the refresh before the hung API's 30 s release.test_native_subagent_boundary.py:finished(..., timeout=_FRAME_CEILING_SECS)keeps the module's documented 20 s ceiling under itstimeout(30)mark.test_messaging_attachments.py: a tickersleep(0.005)measures free loop time during a 150 ms blocking read; it waits on no signal, so it keeps a# flake-okreason on its line.test/wait_discipline_baseline.txt: the 50 lines go to zero and are deleted.Files:
test_mcp_gateway_recaller.py, test_mcp_gateway_session_inject.py, test_mcp_launch_approval.py, test_mcp_preflight.py, test_meetings_audio_import.py, test_members_eventlog_wiring_append_ordering.py, test_memory_admission_queue.py, test_messaging_approval.py, test_messaging_attachments.py, test_messaging_auto_title.py, test_mochi_activity.py, test_mochi_autonomous_unfreeze.py, test_model_access_fallback_interactive.py, test_native_subagent_boundary.py, test_opencode_session_mcp.py, test_papyrus_latex.py, test_pid_lifecycle_process_sweeps.py, test_pod_e2e_video_guard.py, test_pptx_maker_routes.py, test_proc_non_utf8_kill_paths.py, test_provider_spawn_off_loop_18172.py, test_push_guard_squash.py, test_queue_boundary_finalize.py, test_registry_store_art_prewarm.py, test_registry_store_art_prewarm_rewarm.py, test_remote_crew_chaining.py, test_resource_pressure_notify.py, test_runtime_start_gate_headroom.py, test_sandbox_argv.py, test_scenarios_v2_logic.py, test_session_autocompact_override.py, test_session_card_source_links_knob.py, test_session_control_reload.py, test_session_control_revive.py, test_session_map_mirror.py, test_session_more_coverage.py, test_session_not_found_surfaces.py, test_session_pulse_counter.py, test_session_scope_reap_seams.py, test_session_start_queued_behind.py, test_session_storage_api.py, test_session_terminated_reap.py, test_session_transfer_outbound.py, test_session_tree_push.py, test_session_usage_refresh_routes.py, test_session_usage_scrape.py, test_session_work_dir.py, test_shutdown_event.py, test_side_steer_queue.py, test_skill_catalog_snapshot.py
Part of wave-2 fixed-wait migration.
Backwards compatibility
Removes nothing: test-only changes; no production code, API or schema touched.
Tests
test/test_wait_discipline_ratchet.pyandtest/test_flake_pattern_ratchet.py: 113 passed (the 50 files now scan at zero sites; baseline canonical; no changed file gains a flaky shape).-n 8).scripts/check_black_formatting.py) clean on the touched files.Pre-existing failures on main (not from this PR)
Real Adapter Contract Tests:test/test_pi_real_adapter.pysetup raisesSandboxUnavailableErroron the CodeBuild runner. Repo-wide since test(pi): drive pinned pi-acp + pi in the real-adapter lane #22807; tracked in ci: Real Adapter Contract Tests red on every PR since #22807 -- test_pi_real_adapter setup raises SandboxUnavailableError (unshare CLONE_NEWUSER EPERM on CodeBuild runner) #22959, fix in fix(pi): clear main's real-adapter lane and set-gate ratchet reds #22951.test/test_backend_cards_advertised.py::test_no_new_runtime_site_gates_a_field_backed_line_on_a_set(backend shard 7): flagssrc/kiro_crew/acp/client_mcp_array.py/ACP_BACKENDS_SESSION_MCP_ARRAY, introduced by fix(pi): honour an agent spec's zero-tool ban on pi sessions #22808. Fails identically onorigin/maintip (01327a3b3c); this PR touches nosrc/file.Manual verification
N/A — unit coverage sufficient; the two ratchets gate regression.
Related Issues
Part of wave-2 fixed-wait migration.