Repository navigation
fix(dashboard): list the outbox off the event loop - #23967
Conversation
UX Review (Opus 5.5, fork) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing to check. What this change shipsInventory (4 items) — 4 justifiedIntent: keep the gateway responsive while the dashboard lists the outbox. FIX.
[FIRST-PRINCIPLES-REVIEWED] 45a0a59 Description read[DESCRIPTION-READ] 1bd585f90a2991b46532fc3a6c0e85e2847b6be7eabf662377d32df2f74f780f That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS No design concerns. This is a proportionate fix aimed at the cause: the scan of the gateway-owned outbox moves to the default executor, and the route's output and filters stay the same. Notes
[DESIGN-REVIEWED] 45a0a59 Description read[DESCRIPTION-READ] 3c42199ff73bd4f4c80d2efdc664f688e06efefce63500ab8ee7a73e5a3a3157 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
|
CI on
Coverage Gate and PR Readiness fail closed only because |
GET /api/outbox ran one iterdir() plus a stat() per entry inside the coroutine. The outbox grows with every file an agent sends, and on a data home whose mount stops answering each of those syscalls blocks in the kernel, stalling every session on the gateway's single loop until the watchdog restarts it. check_sync_io_in_async listed the three calls in its baseline. The scan now lives in _list_outbox_entries and runs on asyncio.to_thread; the directory is the gateway's own, so the default executor is the lane, not the bounded probe pool request-chosen paths use. Filtering, ordering and the 50-entry cap are unchanged. The baseline row for file_api/outbox.py is pruned with --update-baseline.
2acd6ce to
45a0a59
Compare
|
Rebased onto
Coverage Gate and PR Readiness fail closed only because of those shards. A maintainer re-run of the failed jobs should clear them; this fork account can't trigger a re-run. |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: fix (4 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: GET /api/outbox moves its directory scan into asyncio.to_thread with unchanged filtering, plus a test that the scan leaves the event loop thread.
Problem / Motivation
Goal: Listing the outbox (
GET /api/outbox) no longer blocks the gateway's event loop.api_outbox_listranis_dir(),iterdir(), and astat()plusis_file()for every entry, all inside the coroutine. The outbox grows with every file an agent sends. On a data home whose mount has stopped answering, each of those calls blocks in the kernel, so all of the gateway's work stops while the listing waits.check_sync_io_in_asyncalready listed the 3 calls in its baseline.Why it matters
The gateway runs one event loop. A stalled scan freezes every session, channel, cron and MCP call at once. If the stall lasts longer than
dashboard.loop_stall_exit_after_secs, the watchdog restarts the gateway and every in-flight turn is lost (#1572, #3057).Not a goal
file_api/desktop.py,uploads.py,workspaces.py).What changed (motivation → approach → change)
Symptom: a slow or wedged outbox mount stalls every session while the dashboard lists the outbox. Root cause: the directory scan ran on the event-loop thread.
The scan moves into a sync helper,
_list_outbox_entriesinfile_api/outbox.py, andapi_outbox_listawaits it withasyncio.to_thread. The outbox is the gateway's own directory, never a path the request picks. So the default executor is the right lane, not the bounded path-probe pool that request-chosen paths use (the same call as #22222). The helper keeps the same filters: it skips directories, names thatredact()would rewrite, and entries deleted mid-scan, and it sorts newest first.The facade
dashboard/handlers/files.pyre-exports the new helper, the way it re-exports every owner name. Thefile_api/outbox.pyrow is pruned from.github/sync-io-in-async-fs-baseline.txtwith--update-baseline(143 to 140 calls).Backwards compatibility
Compatible: the route, response shape, filters, ordering and the 50-entry cap are unchanged; only the thread the scan runs on moves.
Removes nothing: the baseline row is dropped because the calls it allowed no longer run on the loop. No reader, route or key goes away.
Tests
test/test_dashboard_files_coverage.py::TestOutboxList::test_scan_runs_off_the_event_loop_thread: makes a real request with a file, a subdirectory and a credential-shaped name in the outbox. It asserts that only the file is listed (same size, same filters) and that the outbox'siterdirandstatcalls ran on a thread other than the event loop's. The four existingTestOutboxListtests (absent outbox, newest-first and skipping dirs, credential-shaped name hidden, entry removed mid-scan) still pass unchanged.TestOutboxListclass also passes in reversed order. The test only observesthreading.get_ident()and uses no timing.Proof: PROVEN test/test_dashboard_files_coverage.py::TestOutboxList::test_scan_runs_off_the_event_loop_thread (head 45a0a59)
Gates run locally:
check_sync_io_in_async.py(140 calls / 59 files),check_black_formatting.py,check_subprocess_encoding.py, comment-history, brand-name, isort, flake8, mypy,docs-lint.sh,test_flake_pattern_ratchet.py.test_dashboard_files_composition_contract.pyandtest_dashboard_files_onloop_fs.pypass (239 passed together with the coverage file).local-gate.py's related set fails 70 tests in 5 unrelated files (test_ledger_sync_git.py,test_design_tweak_backend.py,test_acp_spawn_offload.py,test_apps_registry_identity_admission.py,test_cron_run_scratch.py). The same 70 node ids fail on a cleanorigin/mainworktree on this host, so they come from the environment, not this diff. semgrep is not installed here, so the SAST lane covers it.Manual verification
N/A: the unit test drives the real aiohttp route through
TestClientand checks the thread the syscalls run on, which is the property this change exists for.Related Issues
Closes #23942
Sibling sweep
Swept:
git grep -nE "outbox_dir\(\)" -- src/and thefile_api/rows of.github/sync-io-in-async-fs-baseline.txtSibling: src/kiro_crew/dashboard/file_api/channel_upload.py:100 -- not affected --
_gate_upload_fileis sync and its callers already run it off the loopSibling: src/kiro_crew/mcp_tools/messaging.py:1060 -- not affected --
file_sendis a sync MCP tool, not a coroutine on the gateway loopSibling: src/kiro_crew/dashboard/file_api/workspaces.py -- not affected -- a different handler family still on the baseline, out of this PR's goal
Pattern harvest
Not generalizable: the class (a blocking fs call inside
async def) is already enforced repo-wide byscripts/check_sync_io_in_async.py; this PR only pays down one baselined row of it.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributed under the project's Apache-2.0 license.