Repository navigation
feat(issue-radar): wake an idle crew only when claimable work or capacity grew - #25435
oliviergaumond wants to merge 1 commit into
Conversation
16e3f15 to
c79586e
Compare
|
Hi @oliviergaumond, a user asked in Slack today to prioritize #20552 (T1). This PR is the claimed fix. It is still a draft, so it can't be reviewed or merged yet. When it's ready, please mark it "Ready for review", rebased on the latest main with 1–2 commits, and I'll review it on that head. #22123 by @migh000 targets the same issue. Please say here if one of the two should be closed so reviews don't split. AI-assisted, sent by CommunityCrew |
c79586e to
05edee9
Compare
|
Thanks @bolichen97. Neither should close: they're complementary. #22123 is the wake-signal groundwork, as your review there says, and #25435 is the idle-tick gate for #20552 defect 1, stacked on it. #25435 is now marked ready and rebased on the latest
When #22123 merges, I'll drop its commit so #25435 is a single commit. |
bolichen97
left a comment
There was a problem hiding this comment.
Requesting changes on 05edee9. Thanks for marking it ready and rebasing, @oliviergaumond. I reviewed only the last commit, as the body asks. The first commit is identical to #22123's 180f836. The idle-tick gate is the right fix for #20552 defect 1. The watchdog's precondition re-check under the AutoNudge lock looks correct, and operator-set intervals are left alone. The 3 touched test files pass on this head (94 passed). One thing blocks it:
A signalled item that a started turn does not handle can now wait up to 24 h.
sweep_repoincrew_runtime.pywrites each signalled item's new fingerprint before the wake (around line 1661). The new restore puts the old fingerprint back only when the wake started no turn (not results.get(cid)).- "Started" means the turn was dispatched, not that the item was handled. The brief's per-turn protocol advances only the single most advanceable item per turn.
- Example: in one sweep, item A becomes mergeable and item B's requester replies. One wake carries both reasons. The turn handles A and stops. B's signal is already marked as handled. If no item is in an acting phase, the loop stays at
DEFAULT_IDLE_SECS = 86400, so B waits for the daily backstop. Before this commit, the 300 s tick picked it up within minutes. - A started turn that fails partway (usage limit, ACP error, turn ceiling, restart) loses its signals the same way.
- Fix, either of:
- Keep the pre-wake mark for each signalled item with a
signalled_attime, and commit the new fingerprint only once that item's ledger record is newer thansignalled_at. Otherwise raise the signal again. - Smaller: have
crew_loop_gapreturnACTIVE_IDLE_SECSwhile any open item has a signal newer than its last ledger step.
- Keep the pre-wake mark for each signalled item with a
- Tests: two items signal in one sweep and the turn records progress on only one, so the other is raised again next sweep; and a started turn that records nothing gets its signal raised again.
Non-blocking:
- If
_wake_and_settleraises or is cancelled at shutdown, the restore never runs. Writing once after the wake, intry/finally, closes that window. - The daily backstop still fires one full turn every 86400 s, so "Part of #20552" is right and the issue should stay open. Defect 2 (usage-limit brake and stand-down) is still open too: watcher-dispatched turns don't count toward stand-down, and there is no usage-limit pause or notice. #21919 only stops a stood-down loop from being revived.
crew_runtime_pickup.py:119counts free slots asmax_open - len(set(issue numbers)), not theopen_slot_countits docstring names.- Two branches have no test:
crew_loop_gap's failed-ledger-read fallback (acting = True), and_retime_loopskipping a stood-down loop.
AI-assisted, sent by CommunityCrew
UX Review (Opus 5.5, fork) — ✅ PASS (a BLOCK with no Trigger and no Outcome does not block)UX-level review of UX-Verdict: PASS cannot evaluate: missing the "Wake crews when new work appears" hint. The PR rewrites that hint, but neither committed screenshot shows the settings toggle it sits under. Evidence gaps
Notes
[UX-REVIEWED] fb3162b Description read[DESCRIPTION-READ] 8582198ee61305799c97c2e4bda4a032183beb0323086769ea53c8191c189eb9 That sha256 names everything this verdict read: the PR description, plus the 5 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. The day-long default sits inside the shape the base RFC Notes
What this change shipsInventory (10 items) — 9 justifiedIntent: stop an idle Issue Radar crew paying a model turn every 300 s when nothing it could act on has changed. This is an ADDITION that changes a default.
[FIRST-PRINCIPLES-REVIEWED] fb3162b Description read[DESCRIPTION-READ] b6089602b7a5d7f70a5f829170f03356d9b8e52932716a9cc60fc9d70eb4237d That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
|
@oliviergaumond, this PR has been ready for over 24 hours with a merge conflict and an open change request, so per the team SOP CommunityCrew is taking it over. We will open an in-repo PR from latest main that keeps your commit and authorship, adds a fix for the change request (signalled items a started turn does not handle), and links back here. If you push a fix here first, tell us and we will use yours instead. AI-assisted, sent by CommunityCrew |
05edee9 to
89f66fe
Compare
89f66fe to
eb7193a
Compare
|
Thanks @bolichen97. I've pushed the fix here myself, so CommunityCrew doesn't need to take this over. Head Blocking finding: fixed (option 1). A signalled item keeps its old fingerprint, with the new one held as pending. The new one is committed only after a turn carrying the signal has started and the crew has recorded a step on that item. Until then the signal is raised again at the phase's recheck cadence. Bound: at most two started turns without a step; a dropped wake doesn't count, and only a new signal restarts it. Your two cases are Non-blocking:
From the rebase: #22123 now gates the open-issues read behind CI: the backend and frontend failures were ours and are fixed. Coverage failed only because of those two. GPT 6.1 shows "verdict not published"; could a maintainer re-run it? Ready for another look. |
f3d0628 to
d7af5d7
Compare
e7ab6f7 to
24f0e6e
Compare
|
Rebased onto
The other red jobs on
|
bolichen97
left a comment
There was a problem hiding this comment.
Approving 24f0e6ea48: the line-cap red is fixed. crew_runtime.py is 1970 lines now, with the signal hold moved to crew_runtime_signal_hold.py, and the behaviour I approved earlier is unchanged. The 6 touched radar test files pass locally: 392 tests.
Heads-up so CI doesn't bounce this: main gained the Semgrep rule kirocrew.test-radar-store-root-self-removed (#37622). It rejects tempfile.TemporaryDirectory()/mkdtemp() in test/test_issue_radar*.py, because the radar store re-creates its root on read after cleanup. This PR touches two such files:
test/test_issue_radar_crew_pickup.py:85,:573,:849(newsetUproots)test/test_issue_radar_crew_runtime.py:230,:244,:264,:281,:419,:576,:612
The fix is mechanical and matches test_issue_radar_crew_runtime_wake.py and #37888: take the root from tmp_path, e.g. an autouse fixture method def _root(self, tmp_path): self.root = tmp_path in the unittest classes, and the tmp_path fixture for the with blocks. I approved the held fork CI for this head. Auto merge is on and lands it once SAST is green.
AI-assisted, sent by Bolin Review bot
Head branch was pushed to by a user without write access
24f0e6e to
c335eb8
Compare
|
Fixed the upheld GPT 6.1 blocker (unconsumed dispatches exhaust the held-signal budget), now
Red first, against Backend Issue Radar set: 2,250 passed. mypy (whole package), black, isort, flake8, comment-history, brand, line-cap and flake-ratchet gates pass. |
bolichen97
left a comment
There was a problem hiding this comment.
Approving c335eb83b5.
- The GPT 6.1 blocker is fixed the right way: item-signal reasons now ride
crew_wake_inflightbeside asks and pickups. A turn that hands its reason back (permit timeout, cancel, restart) gets its charge refunded at the next sweep (return_turns→return_turn), and while a turn still carries the reason,resolve_mark(..., waiting=True)neither re-raises nor commits it. Both failure paths are safe: an unreadable in-flight file refunds nothing and holds nothing, and a failedsettle_returnedonly costs one extra wake, never a lost one. crew_runtime.pyis 1990 lines, still under the cap. The 6 touched radar test files pass locally: 394 tests.
Still open from my last review: test/test_issue_radar_crew_pickup.py (3 roots) and test/test_issue_radar_crew_runtime.py (9) still use tempfile.TemporaryDirectory(), which the Semgrep rule kirocrew.test-radar-store-root-self-removed rejects. SAST will stay red until they take their root from tmp_path (see test_issue_radar_crew_runtime_wake.py for the pattern). I approved the held fork CI for this head. Auto merge is on.
AI-assisted, sent by Bolin Review bot
|
@oliviergaumond thanks for the held-signal fix in c335eb8; my approval stands. Three checks still block the merge:
Backend Lint was cancelled, not failed; it will rerun on your next push. AI-assisted, sent by Bolin Review bot |
Head branch was pushed to by a user without write access
c335eb8 to
dfd0d41
Compare
|
Now
Backend Issue Radar set: 2,253 passed. mypy (whole package), black, isort, flake8, line cap ( |
bolichen97
left a comment
There was a problem hiding this comment.
Re-approving dfd0d41. Thanks @oliviergaumond, two of the three blockers are fixed:
- GPT F1:
watchdog_cyclenow marks a crewresumedwhen it relaunches the slot or re-activates a paused loop, and wakes it once withRESUMED_REASONwhen_has_open_itemssays it holds open work. An unreadable ledger answers yes, which is the right side to fail on. The three new tests cover resume with items, resume without items, and relaunch. - PR Hygiene: the body now names
KiroCrewCaptain/kirocrew-app-issue-radar#4. test_issue_radar_crew_pickup.pynow uses thetmp_pathautouse fixture.
Locally on the head, test_issue_radar_crew_pickup.py and test_issue_radar_crew_runtime.py pass (133 passed).
Still open: test/test_issue_radar_crew_runtime.py has 9 tempfile.TemporaryDirectory uses left, which the Semgrep rule kirocrew.test-radar-store-root-self-removed covers (test/test_issue_radar*.py). The same autouse fixture fixes it. I approved the 33 held fork CI runs; auto merge is on.
AI-assisted, sent by Bolin Review bot
Port of kirodotdev/KiroCrew#25435 (core head dfd0d4181a), merged three-way onto this tree, so only the imports differ. An idle crew's AutoNudge loop is now a day-long backstop instead of a 300 s model turn; a crew with an acting item keeps 300 s. The watcher wakes a crew with room when a slot frees or a claimable issue appears (crew_runtime_pickup.py). An item signal stays raised until the crew records a step on that item, bounded at two turns that reached the model (crew_runtime_signal_hold.py); item reasons ride the in-flight carry so a turn that never reached the model hands its signal back. A resumed crew with open items gets one wake. The crew editor's two hint strings change in every catalog, and ui/dist is rebuilt.
Head branch was pushed to by a user without write access
dfd0d41 to
a720bd1
Compare
|
Now
Backend Issue Radar set: 2,257 passed. mypy (whole package), black, isort, flake8, line cap ( |
bolichen97
left a comment
There was a problem hiding this comment.
Re-approving a720bd1. Both new fixes look right:
crew_loop_gaps.pyrecords the interval the app last wrote per crew and loop id, and_retime_loopmoves a loop only while its live interval still equals that record (or, with no record, sits on one of the two gaps). When the precondition refuses, the new value is still recorded, so the next pass sees the operator's interval differ from the record and keeps it._resume_wakeswakes a resumed crew only when its loop is active after the pass and no STOP sentinel exists, so an operator's STOP file still wins.
Still open from my last note: test/test_issue_radar_crew_runtime.py keeps 9 tempfile.TemporaryDirectory uses, which the Semgrep rule kirocrew.test-radar-store-root-self-removed flags. Auto merge is on.
AI-assisted, sent by Bolin Review bot
bolichen97
left a comment
There was a problem hiding this comment.
Re-approving 588fae3. The only change since a720bd1 moves the STOP-sentinel exists() check in _resume_wakes onto a thread, so the watchdog no longer touches the disk on the event loop. Held fork CI runs approved; auto merge is on.
Reminder: test/test_issue_radar_crew_runtime.py still has 9 tempfile.TemporaryDirectory uses for the Semgrep rule kirocrew.test-radar-store-root-self-removed.
AI-assisted, sent by Bolin Review bot
…city grew
The new-work signals fire once, on first sight, so an issue that arrived
while a crew was at max_open was consumed, and nothing woke the crew when
a slot freed. Only the 300 s idle turn covered that, one model turn every
five minutes for a crew with nothing to do.
The sweep now raises pickup-due per live crew from the open-issues page.
It stores the last observed state per crew (sorted claimable numbers,
free slots, the numbers of its open items) in crew-wake-checks.json and
wakes when the crew has something new to decide: a claimable number not
in that state, a different number of free slots, or a different set of
open items. A turn takes one issue, so with room left the next pick is
due. Candidates only leaving is written at once without a wake, and a
turn that takes nothing commits what it saw, so the same choice never
wakes the crew twice. The state commits only after wake_crew reports a
started turn, and the pickup reason rides that turn in
crew-wake-inflight.json like a mention ask: a turn that ends before the
model reads its prompt (permit timeout, cancel, gateway restart) hands
it back, which clears the crew's state so the same sweep wakes it again.
A failed open-issues read wakes nobody and writes nothing.
First observation counts as growth, so a freshly launched or upgraded
crew gets its selection turn.
Pickup-due runs whatever crew_wake_on_new_work says: that setting gates
only the scope-label and new-issue wakes, while pickup is what wakes a
crew with room once its idle clock is the day-long backstop. So the
open-issues page is read for every repo with a live crew. Candidates
exclude only the crew's open items, so an issue it resolved and someone
reopened counts. A crew with an acting item is not read: the brief has it
advance that item first. When another signal wakes the crew in the same
sweep, pickup-due commits only with a selection turn (scope-label,
new-issue or human-answered), whose reasons then name it too.
The idle clock now depends on the crew's items. A crew with an item in an
acting phase keeps the 300 s gap; any other crew gets a day-long
backstop. launch_crew arms the right interval, and the watchdog moves a
loop between those two values when the phase changes, with a
precondition the service re-checks under its lock. A loop on any other
interval was set through PATCH /api/autonudge/{loop_id} and is left
alone. The crew editor's read-only idle wake shows the backstop, and its
line names both gaps. The crew_wake_on_new_work hint says a crew still
wakes for a free slot and something new to pick up when it is off.
With the idle turn a day away, it can no longer be what picks up an item
signal (CI, review, reply, conflict, merge, unblocked) that a started
turn did not handle: the brief advances one item per turn, and a turn can
fail partway. So a signalled item's mark keeps its old fingerprint, with
the new one beside it as pending, written before the wake, and the new
one is committed only once a turn carrying the signal has started and the
item's ledger shows a step newer than that start (its last_progress_at, or
any event on it), so a step by a turn that never saw the signal does not
count. Until then the signal is raised again at the item's phase recheck
cadence. A pending signal is carried to at most two started turns that
record no step on its item; then it is committed, so a crew that looked
and did nothing is not paid for again. A wake that started no turn does
not count, and only a new signal on the item restarts the count: a field
no signal reads moving under it does not.
bolichen97
left a comment
There was a problem hiding this comment.
Re-approving fb3162b. Both AI findings from 588fae3 are handled:
_retime_loopnow records the gap only whensvc.updatereturned a loop with that interval. ANonefrom an operator's precondition, a pause quiescing the loop, or a shutdown leaves the record as it was, so an operator's value is kept and anything else retries next pass. A legacy loop already on the right gap is recorded once.resolve_markcompares a first look at a waiting item withfirst_look_basis, so an outcome that settled before the watcher first read the item (CI finished, a verdict, a conflict, a merge) still reaches the crew instead of waiting for the day-long backstop. Running CI, comment counts and blockers still seed silently.
Held fork CI runs approved; auto merge is on. Still open: the 9 tempfile.TemporaryDirectory uses in test/test_issue_radar_crew_runtime.py for the Semgrep radar rule.
AI-assisted, sent by Bolin Review bot
Problem / Motivation
Goal: an idle Issue Radar crew spends no model turn unless something it could act on, claimable work or free capacity, has grown.
An Issue Radar crew arms an AutoNudge loop that fires a full model turn every 300 s whether or not anything changed. In #20552 a crew with no actionable work burned about 200 credits over 3.2 h on 38 no-op turns. #22123 moves most wakes onto the watcher (new in-scope issues, @mentions) and keeps the idle timer as a fallback. One case still needs that fallback: #22123 consumes a scope-label or new-issue signal on first sight, so an issue that appears while the crew is at
max_opennever wakes it once a slot frees. Only the next idle turn finds it, so the paid 300 s turn stays.Why it matters
Every crew pays roughly 288 turns a day for nothing while its repo is quiet, and per-turn cost grows with the conversation. That makes crews too expensive to leave running unattended, which is what they are for.
Not a goal
What changed (motivation → approach → change)
The base design already makes the watcher the crew's clock and AutoNudge the backstop "only when the watcher has nothing and the crew stalled" (
docs/request-for-change/rfc-issue-radar-crews.md, statuspartial, Mental model table and §4). This change makes the code match that.crew_runtime_pickup.py, fromdetect_new_work). Per live crew it keeps the claimable numbers, free slots and open item numbers increw-wake-checks.json, and wakes a crew with room when a candidate is new or its slots or open items changed (a turn takes one issue). A turn that takes nothing commits what it saw. The pickup reason rides its turn increw-wake-inflight.jsonlike a mention ask, so a turn that ends before the model reads it (permit timeout, cancel, restart) hands it back and the crew is woken again; an arrival turn that commits pickup names it too. It runs whatevercrew_wake_on_new_worksays: that setting gates feat(issue-radar): the watcher wakes crews on new work and @mentions #22123's arrival wakes, while pickup is what wakes a crew once its idle clock is a day, so the open-issues page is read for every repo with a live crew.crew_runtime_session.py: 300 s (ACTIVE_IDLE_SECS) while an item is acting, 86400 s (DEFAULT_IDLE_SECS) otherwise. The watchdog retimes a loop only while its interval is the one the app last wrote (crew_loop_gaps.py), with apreconditionre-checked under the service lock, so an interval set throughPATCH /api/autonudge/{loop_id}is kept, even 300 or 86400. The crew editor's read-only idle wake shows 86400 and its line names both gaps; thecrew_wake_on_new_workhint says pickup still runs with the switch off (all 12 catalogs inwebsite/src/i18n/locales/plus the pseudolocale).crew_runtime_signal_hold.py, fromsweep_repo). A started turn may handle another item or fail partway, so a signalled item's mark keeps its old fingerprint with the new one aspending, written before the wake. It commits only once a turn carrying it started and the item's ledger shows a step after that start; until then the signal is raised again at the phase's recheck cadence. Bound: at mostSIGNAL_TURNS(2) started turns without a step, and only a new signal restarts the count. A turn that stepped on another item it carried is refunded (credit_turn), and the bound waits for a charged turn still running. Each turn is stamped right before its dispatch (dispatched_at), so a step the previous turn wrote meanwhile does not count. Item-signal reasons ridecrew-wake-inflight.json: a turn handing one back unread is refunded (return_turns), and one still carried is not raised again.watchdog_cyclewakes a resumed crew with open items once, unless its STOP file halted the loop.crew_brief.mdexplains thepickup-duewake reason.Backwards compatibility
Compatible: the
pickupsection ofcrew-wake-checks.jsonand thependingfield of acrew-signals.jsonmark are additive, and a missing or malformed one reads as absent; loops armed at 300 s are moved by the existing watchdog pass, and a loop on any other interval keeps it.Removes nothing: no setting, field, key, record or capability is taken away; the idle default changes for crews with nothing acting.
Tests
test/test_issue_radar_crew_pickup.py(new, 60 tests):crew_wake_on_new_workoff; no wake atmax_open, for candidates leaving the page, after a failed forge read, over a quiet day, for a crew with an acting item, or on an item-signal turn. A turn that never reached the model (a queued turn whose permit wait times out in the real_capped_run_chat, a restart, an arrival turn) gets its pickup back, and a same-count pickup cannot take over one still queued.Updated:
test/test_issue_radar_deps.py's fires-once case now records the crew's step before asserting no re-fire; #22123's opt-in test intest/test_issue_radar_crew_runtime_wake.pyasserts no arrival wake while pickup still runs;test/test_issue_radar_crew_close_race.pypinscrew_loop_gap; arrival-turn expectations intest_issue_radar_crew_runtime_wake.pyandtest_issue_radar_crew_runtime_sweep.pynow include the pickup reason, and the ask tests' setup consumes its first-look pickup turn;IssueRadarCrewEditor.test.tsxnow reads bothDEFAULT_IDLE_SECSandACTIVE_IDLE_SECSfrom the backend source, so either side drifting fails it..github/black-baseline.txtdropstest/test_issue_radar_crew_runtime.py, which is now black-clean.Red before: the reviewer's two cases (
test_a_turn_that_advances_one_of_two_items_gets_the_other_raised_again,test_a_started_turn_that_records_nothing_gets_its_signal_raised_again) fail on05edee9c22. Each later fix's test fails on the head before it, including the three never-reached-the-model cases on945efce0f0. Determinism (testing-conventions D12): 20 runs at-n0, 10 at-n 4, one shuffled order (187 tests), all green.Gates on the head: every
test/test_issue_radar_*.py,test/test_slot_close_nudge_race.pyandtest/test_facade_mirror_guards.pyandtest/test_gateway_notice_sites.py(2261 passed);src/apps/issue-radar/andsrc/i18n/vitest (2902 passed),tsc,i18n:checkand eslint; black, isort, flake8, mypy, comment-history, brand, file-line-cap, harness parity and the flake ratchet exit 0. Semgrep was not run locally (not installed); the threetempfile.TemporaryDirectory()roots CI'stest-radar-store-root-self-removedflagged are nowtmp_path.scripts/local-gate.py's first backend batch had 47 failures (file explorer, sandbox, real codex), and the same node IDs fail on cleanorigin/mainon this host.Manual verification
N/A — unit coverage sufficient for the backend: the tests drive the real sweep (
detect_new_work,sweep_repo),launch_crewandwatchdog_cyclewith a fake forge client and AutoNudge service.Screenshots / video
The crew editor's read-only "Idle wake" field now shows the backstop and names both gaps (
CrewEditor.tsx,backstop_fixed):Captured with
website/scripts/capture-crew-editor-idle-wake.mjsagainst thewebsite/capture/crew-editor-idle-wakeharness (the real dialog and provider, network stubbed); the script asserts the field reads 86400 and the line names 300 and 86400 before shooting. The settings hint forcrew_wake_on_new_work(wake_new_work_hint) now says that with the switch off a crew still wakes when it has a free slot and something new to pick up; that is a text-only change to an existing line.Related Issues
Part of #20552
App repo: KiroCrewCaptain/kirocrew-app-issue-radar#4
Builds on #22123 (merged).
Checklist
crew_brief.mdContribution License Agreement
Contributed under the project's Apache-2.0 license.