Repository navigation
fix: hold the remaining dropped asyncio tasks until they finish - #26463
Conversation
The event loop keeps only a weak reference to a task, so a task nothing else holds can be garbage-collected mid-run and its work silently never finishes. Ten sites still dropped theirs after the four area PRs: - sprites_session/terminal.py: the stdin and resize sends - subagent_manager/admission/gate.py: the spawn_batch_started event - shared_crew/admission_entry.py: the child and run cancel hooks - remote_session/worker.py: the SIGTERM retire (a signal-handler lambda) - dashboard/handlers/crew_log.py: the flush pass and the session send - knowledge/artifact_ingest.py: the ingest/remove task - subagent_manager/terminal.py: the subagent_injection_failed event Each now keeps its task in a set until done with add_done_callback(set.discard), or joins the manager's existing _report_tasks / track_store_task set where the module already has one. A collected crew-log flush also left _flushing set, which stopped the publisher for good. The dropped-task guard in test_dashboard_task_refs.py now covers these files and two more drop shapes: a lambda that returns the task, and a sync function's local that only gets callbacks. Closes #25953
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: each of the ten sites now keeps its task in a set the owner already has or a module set, and the guard over Notes
[DESIGN-REVIEWED] 66f1c31 Description read[DESCRIPTION-READ] fc5961754e44c3e399be11c006514fb18f458ec8d72ebecb94040bcca35359a4 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Every item is a direct hold of a task that was being dropped. The provenance is #25953 plus gc tests that fail on base, and no shared helper exists on base that the PR skips. Notes
What this change shipsInventory (8 items) — 8 justifiedIntent: background work Kiro Crew starts runs to the end instead of being garbage-collected part way. FIX.
[FIRST-PRINCIPLES-REVIEWED] 66f1c31 Description read[DESCRIPTION-READ] df0b4c8f4dd45b1f0715adb93cd977c25b2ebf8c2b6ce3871c157a01d01013fe That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: fix (11 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: bug fix; background asyncio tasks in the crew-log publisher, artifact ingest, remote worker, shared-crew cancel hooks, sprites terminal and subagent manager are now held in a set until they finish, so the loop's weak reference cannot drop them mid-run, with retention tests.
Backport to release/0.11.0, one commit: - fix(install): download the pinned kas engine in cli installers (#26384) (cherry picked from commit 0a49fdb); docs/system-specs/modules/cli.md took only this PR's sentence on the `kas install` row. - fix(electron): never spawn a gateway once the app is quitting (#26445) (cherry picked from commit 6a4d4e0), clean. - fix: hold the remaining dropped asyncio tasks until they finish (#26463) (cherry picked from commit 8b3c698); adapted: the five sites in modules 0.11.0 has (crew_log, artifact_ingest, remote_session/worker, subagent_manager admission gate and terminal). shared_crew/admission_entry.py, sprites_session/terminal.py and their tests are not on 0.11.0, so neither is the dropped-task guard test that lists them. The crew-log publisher test gains the held-task assertions, with `_flush` stubbed as on main. - fix(kas): refuse a file write a fallbackAction deny spec does not cover (#26597) (cherry picked from commit 84c2aaa); doc and comment conflicts only: kept 0.11.0's wording around the sub-agent rules (#25361 is not on 0.11.0) and dropped main's "v2 parity" acp-client block for the same reason. Not picked: #26456 needs main-only `security/argv_rules.py` and the s3 download narrowing of #24562 in `deny_gate.py`. The v0.11.1-rc.1 Release run's test failures are fixed by #26729. All three version files move from 0.11.1-rc.1 to 0.11.1-rc.2, and the in-flight [0.11.1] CHANGELOG section gains the four fixes and the filelock bump of #26536 and the cron label fix of #26729 (#24718), both already on the branch. #26149 and #25703 are test-only and have no reader-visible line.
Backport to release/0.11.0, one commit: - fix(install): download the pinned kas engine in cli installers (#26384) (cherry picked from commit 0a49fdb); docs/system-specs/modules/cli.md took only this PR's sentence on the `kas install` row. - fix(electron): never spawn a gateway once the app is quitting (#26445) (cherry picked from commit 6a4d4e0), clean. - fix: hold the remaining dropped asyncio tasks until they finish (#26463) (cherry picked from commit 8b3c698); adapted: the five sites in modules 0.11.0 has (crew_log, artifact_ingest, remote_session/worker, subagent_manager admission gate and terminal). shared_crew/admission_entry.py, sprites_session/terminal.py and their tests are not on 0.11.0, so neither is the dropped-task guard test that lists them. The crew-log publisher test gains the held-task assertions, with `_flush` stubbed as on main. - fix(kas): refuse a file write a fallbackAction deny spec does not cover (#26597) (cherry picked from commit 84c2aaa); doc and comment conflicts only: kept 0.11.0's wording around the sub-agent rules (#25361 is not on 0.11.0) and dropped main's "v2 parity" acp-client block for the same reason. Not picked: #26456 needs main-only `security/argv_rules.py` and the s3 download narrowing of #24562 in `deny_gate.py`. The v0.11.1-rc.1 Release run's test failures are fixed by #26729. All three version files move from 0.11.1-rc.1 to 0.11.1-rc.2, and the in-flight [0.11.1] CHANGELOG section gains the four fixes and the filelock bump of #26536 and the cron label fix of #26729 (#24718), both already on the branch. #26149 and #25703 are test-only and have no reader-visible line. Co-authored-by: w <w@k.local>
Problem / Motivation
Goal: Every background task Kiro Crew starts and does not await runs to its end instead of being garbage-collected part way.
After the four area PRs (#25968, #25971, #25984, #25986), ten sites on main still start a task with
asyncio.create_task,loop.create_taskorasyncio.ensure_futureand keep no strong reference to it. The event loop holds only a weak reference to a running task, so a task parked on an await that nothing else reaches can be collected mid-run, and Python logsTask was destroyed but it is pending!.Why it matters
The work silently never finishes. A collected crew-log flush never runs its done callback, so
_flushingstays set and the crew log publisher stops broadcasting until restart. A collected shared-crew cancel hook leaves the child or run it was cancelling still running. A collected SIGTERM retire leaves a remote worker running after it was told to stop. A collected sprites terminal send drops a keystroke or a resize.Not a goal
add_done_callback(set.discard), and this PR follows the same pattern or joins a set the module already has.What changed (motivation → approach → change)
Symptom: an unawaited task can vanish mid-run. Cause: nothing holds the returned task (a bare expression statement, a lambda that returns it, or a local that only gets callbacks). Change: each site keeps its task in a set until it finishes, using the set the module already has where there is one.
sprites_session/terminal.py(2)_send_in_backgroundsubagent_manager/admission/gate.pyspawn_batch_startedeventtrack_store_task, aswaits.pydoes for the same event familyshared_crew/admission_entry.py(2)_holdremote_session/worker.pyWorker._term_tasks, through a named_on_sigtermhandlerdashboard/handlers/crew_log.py(2)CrewLogPublisher._tasksknowledge/artifact_ingest.pyArtifactKnowledgeSync._taskssubagent_manager/terminal.pysubagent_injection_failedevent_report_tasksThe dropped-task guard in
test/test_dashboard_task_refs.pynow covers these seven files and two more drop shapes: a lambda whose body is the task call, and a sync function's local task that is used only foradd_done_callback. Run over all ofsrc/, the extended guard reports no remaining site.Backwards compatibility
Compatible: each task does the same work at the same time; it is only held until it finishes.
Removes nothing: no key, row, path or capability is removed; the diff only adds references to tasks.
Tests
test/test_dashboard_task_refs.py::test_no_dropped_task_statement[...]: one case per newly guarded file fails if that file drops a task again.test/test_dashboard_task_refs.py::test_the_guard_sees_a_task_dropped_by_a_lambdaand::test_the_guard_sees_a_local_task_given_only_a_callback: the guard catches the two new drop shapes.test/test_runtime_background_task_retention.py: each test parks the task on an await, runsgc.collect(), and checks the owner still holds the task and lets it go once it finishes:test_the_shared_crew_child_cancel_task_outlives_a_collection,test_the_shared_crew_run_cancel_task_outlives_a_collection,test_a_sprites_terminal_send_outlives_a_collection,test_the_remote_worker_sigterm_retire_outlives_a_collection,test_an_artifact_knowledge_sync_task_outlives_a_collection,test_a_crew_log_flush_outlives_a_collection,test_a_crew_log_session_send_outlives_a_collection.test/test_crew_log_routes.pyandtest/test_shared_crew_admission_failures.py: updated fakes that pinned the old callback count or returnedNonefrom a fakecreate_task.Determinism: the new gc tests pass 10 of 10 runs at
-n 0. The related suites pass locally (5378 tests).Proof: PROVEN test/test_dashboard_task_refs.py::test_no_dropped_task_statement[src/kiro_crew/dashboard/handlers/crew_log.py], test/test_dashboard_task_refs.py::test_no_dropped_task_statement[src/kiro_crew/knowledge/artifact_ingest.py], test/test_dashboard_task_refs.py::test_no_dropped_task_statement[src/kiro_crew/remote_session/worker.py], test/test_dashboard_task_refs.py::test_no_dropped_task_statement[src/kiro_crew/shared_crew/admission_entry.py], test/test_dashboard_task_refs.py::test_no_dropped_task_statement[src/kiro_crew/sprites_session/terminal.py] (head 66f1c31)
A manual revert of all seven production files fails 14 tests: the five guard cases above, the guard cases for
subagent_manager/admission/gate.pyandsubagent_manager/terminal.py, and all seven gc tests.Manual verification
N/A — unit coverage sufficient: each site is a two-line hold whose effect the gc tests observe directly.
Related Issues
Closes #25953
Sibling sweep
Swept: AST walk of every src/kiro_crew module with the extended guard in test/test_dashboard_task_refs.py (bare create_task/ensure_future statements, lambda bodies, callback-only locals) on origin/main c21f5a0
Sibling: src/kiro_crew/sprites_session/terminal.py:138 -- changed here
Sibling: src/kiro_crew/subagent_manager/admission/gate.py:1561 -- changed here
Sibling: src/kiro_crew/shared_crew/admission_entry.py:365 -- changed here
Sibling: src/kiro_crew/remote_session/worker.py:262 -- changed here
Sibling: src/kiro_crew/dashboard/handlers/crew_log.py:1406 -- changed here
Sibling: src/kiro_crew/knowledge/artifact_ingest.py:943 -- changed here
Sibling: src/kiro_crew/subagent_manager/terminal.py:1587 -- changed here
Sibling: src/kiro_crew -- not affected -- every other create_task/ensure_future result is awaited, returned, stored on an attribute, or added to a held set; the guard over all of src reports none after this change
Pattern harvest
Rule candidate: lint
Pattern: a create_task/ensure_future result that is dropped (bare statement, lambda body, or a local used only for add_done_callback); now enforced for these files by the AST guard in test/test_dashboard_task_refs.py.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.