Skip to content

fix(autonudge): enforce runtime budget and cycle cap without waiting a full interval - #26767

Closed
rohnkum wants to merge 2 commits into
kirodotdev:mainfrom
rohnkum:fix/autonudge-budget-cycle-arming
Closed

rohnkum wants to merge 2 commits into
kirodotdev:mainfrom
rohnkum:fix/autonudge-budget-cycle-arming

Conversation

@rohnkum

@rohnkum rohnkum commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: a prompt loop stops when its runtime budget or cycle cap is spent, not up to a full interval later.

Both bounds are enforced only when the loop's timer wakes (_timer in src/kiro_crew/autonudge_service/firing.py), but _arm_from_deadline always arms toward the interval deadline, so:

  • Runtime budget shorter than the interval: monitor_start(interval_secs=86400, max_runtime_secs=600) arms an 86400 s timer; the budget runs out at 600 s but nothing wakes the loop until the next day.
  • Cycle cap spent on the last cycle: when the final turn ends, notify_turn_complete re-arms a fresh full interval, and the cap is applied only when that timer next wakes.

In both cases the loop stays active with its bound spent for up to a day, emits no expired event, and refuses a create-only monitor_start on its slot.

Why it matters

A long-interval loop that has hit its budget or cycle cap looks like it is still running on the dashboard, sends the user no stop notification, and holds its session slot — so a new monitor_start on that slot is refused — for up to the 86400 s maximum interval.

Not a goal

  • Structured monitors: their tick re-arms itself from its own finally, so arming them at delay 0 could spin. They are explicitly left out.
  • The bound branches in _timer themselves, which already deactivate and emit expired correctly once the timer wakes.

What changed (motivation → approach → change)

Symptom: a spent runtime budget or cycle cap leaves the loop armed for its full interval. Root cause: _arm_from_deadline picks the arm delay from the interval alone and never from the bounds.

Change: in _arm_from_deadline (src/kiro_crew/autonudge_service/timers.py), for a prompt loop (not a structured monitor): arm with delay 0 when cap_reached(loop), otherwise cap the delay at the runtime budget left (max_runtime_secs - runtime_elapsed(loop, created_ts), floored at 0). The existing bound branches in _timer then deactivate the loop with stopped_reason runtime_budget or cycle_cap and emit expired on time. The bounds are read through the model's defensive readers (cap_reached, _positive_number), so a malformed stored bound means no bound rather than a raise. A loop held for approval or a person still returns from its tick without arming, and a work-ledger extension re-arms through this same function with the raised bound, so neither path spins.

Backwards compatibility

Compatible: a loop with no cycle cap and no runtime budget, or whose budget is longer than its interval, arms exactly as before.

Tests

  • test/test_autonudge_budget.py: test_a_budget_shorter_than_the_interval_stops_the_loop_when_it_is_spent (F7) and test_a_spent_cycle_cap_stops_the_loop_without_waiting_an_interval (F8). Each drives the loop with asyncio.sleep recorded and asserts the loop ends inactive with stopped_reason runtime_budget / cycle_cap and that expired was emitted.
  • With the change reverted both fail: armed ~86400 s against a 600 s budget; re-armed 86400 s after the cap was spent.
  • Proof: PROVEN test/test_autonudge_budget.py::test_a_budget_shorter_than_the_interval_stops_the_loop_when_it_is_spent, test/test_autonudge_budget.py::test_a_spent_cycle_cap_stops_the_loop_without_waiting_an_interval (head e9dfc21)
  • Related backend tests: the 58 test files mentioning _arm_from_deadline, notify_turn_complete, _finish_fire_cycle, timers.py or autonudge gave 2415 passed. The change touches no frontend file.

Manual verification

On main the two tests reproduce both bugs (armed ~86400 s past a 600 s budget; re-armed a full interval after the cap). On this branch both stop on time.

Related Issues

Closes #26764

Sibling sweep

Swept: grep for the arming and bound-check sites in autonudge_service/ (_arm_from_deadline, _timer, notify_turn_complete, _finish_fire_cycle).
Sibling: src/kiro_crew/autonudge_service/firing.py -- not affected -- the cap and budget branches (lines 203 and 251) that run when the timer wakes; this PR only makes the timer wake on time.
Related open PRs (not duplicates): #21377 adds a held-subject-read branch at the top of _arm_from_deadline, and #22855 adds an overdue backstop; both touch this area, so whichever lands second needs a small rebase.

Pattern harvest

Rule candidate: review-prompt
Pattern: a bound (cap or budget) enforced only at wake time, while the thing that schedules the wake ignores the bound and arms for the full interval.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license, as the Licensing section of CONTRIBUTING.md describes.

@rohnkum
rohnkum requested review from a team and KiroCrewCaptain as code owners October 31, 2026 15:46
@rohnkum
rohnkum requested a review from CrysisDeu October 31, 2026 15:46
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Oct 31, 2026
@iamwhatever
iamwhatever force-pushed the fix/autonudge-budget-cycle-arming branch from e9dfc21 to 04941ae Compare November 6, 2026 08:14
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 8101e7bea5e4a75d66a7cd03eeca3691a23e840b via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 8101e7b

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — 🔴 changes requested (blocking)

Reviewed 8101e7bea5e4a75d66a7cd03eeca3691a23e840b via the fork AI-review pipeline; updated in place on each push.

BLOCKING: a loop's final nudge turn can be expired while it is still running. A cycle cap or runtime budget that is already spent now wakes the timer immediately, even when the re-arm comes from a mid-turn path that previously waited a full interval.

BLOCKING — src/kiro_crew/autonudge_service/timers.py:1112

        if cap_reached(loop):
            delay = 0.0

Trigger: a dashboard loop with max_cycles=N delivers cycle N. _settle_fire_cycle clears next_due_ts, nothing re-arms the timer, and the final turn runs past two reconciler passes (_RECONCILE_INTERVAL_SECS = 60). _reconcile_once then calls _arm_from_deadline on a loop with no live timer. A person answering an approval mid-turn reaches the same arm through release_approval_hold(arm=True). Either way cap_reached sets the delay to 0. _run_timer_callback checks the stop sentinel (the turn has not written it yet), then reaches the cap branch before any busy-slot refusal.
Outcome: the loop is deactivated as cycle_cap and expired ("stopped short of its goal") is emitted while the final turn is still running. If that turn then writes the stop sentinel, the loop is still recorded as cycle_cap, never stop_sentinel, and the user was told it fell short. notify_turn_complete then returns early because the loop is inactive, so the judge label for that turn is dropped too. Before this change, the cleared deadline led to a full-interval countdown, and the sentinel was checked after the turn ended. The budget branch shortens mid-turn arms the same way.
Anchor: defect
Fix: shorten the delay only on arms that happen after the turn has ended (e.g. the notify_turn_complete/_arm_and_announce_deadline path, or when the slot has no turn in flight). Leave the reconciler and approval-release arms on the deadline delay.

[BLOCK-MERGE] 8101e7b
[OPUS-REVIEWED] 8101e7b

@iamwhatever
iamwhatever force-pushed the fix/autonudge-budget-cycle-arming branch from ffd360f to 09e646e Compare November 6, 2026 10:28
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision 8101e7bea5e4a75d66a7cd03eeca3691a23e840b touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 8101e7bea5e4a75d66a7cd03eeca3691a23e840b via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

No design-level concerns: _arm_from_deadline now arms toward the bounds that _timer already enforces, which fixes the cause rather than the symptom. Each zero-delay path that does not stop the loop ends in a hold that arms nothing, or in a re-arm that the person-wait guard excludes, so none of them can spin.

[DESIGN-REVIEWED] 8101e7b

Description read

[DESCRIPTION-READ] 7cbced9f2f9406bc30c38c4373a9c6d3ef4abdab33dfbb45eb6765be24c5a14b

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 6, 2026
@iamwhatever
iamwhatever force-pushed the fix/autonudge-budget-cycle-arming branch from 09e646e to e853317 Compare November 6, 2026 11:04
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 6, 2026
@iamwhatever
iamwhatever force-pushed the fix/autonudge-budget-cycle-arming branch from e853317 to 25e49fd Compare November 6, 2026 14:11
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ✅ PASS

Premise-level review of 8101e7bea5e4a75d66a7cd03eeca3691a23e840b via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing to check.

What this change ships

Inventory (5 items) — 4 justified

Intent: stop a prompt loop as soon as its runtime budget or cycle cap runs out, instead of up to a full interval later. FIX.

  1. A loop whose runtime budget is shorter than its interval now stops and sends expired when the budget runs out — justified
  2. A loop that has used its last allowed cycle now stops right after that turn instead of one interval later — justified
  3. A loop held for a person keeps its full interval even when a bound is spent, so the hold does not spin — justified
  4. The wake after a budget runs out is set one monotonic clock tick (about 15.6 ms on Windows) past the budget, so it never wakes early and fires one more turn — undeclared
  5. A stored bound that is not a number counts as no bound when arming, reusing cap_reached and _positive_number — justified

[FIRST-PRINCIPLES-REVIEWED] 8101e7b

Description read

[DESCRIPTION-READ] 29abeeaf44703059978de9f434e8c3b9efc3f6314b97c9fa4ad29c613fc4f0d4

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 6, 2026
rohnkum and others added 2 commits November 6, 2026 17:20
…a full interval

The cycle cap and the runtime budget are checked only when a loop's
timer wakes, and _arm_from_deadline always armed toward the interval
deadline. A loop whose interval is longer than its remaining budget
slept past the budget, and the turn that ended the last allowed cycle
re-armed a full fresh interval. Either way the loop sat active with a
spent bound for up to a day, emitted no expired event, and refused a
create-only arm on its slot.

Cap the armed delay at the runtime budget left, and arm at once when
the cycle cap is reached, so the timer's existing bound branches stop
the loop on time. Structured monitors keep their own deadline because
their tick re-arms itself.

Closes kirodotdev#26764
A loop held for a person stops at the hold before either bound is checked, so the early wake for a spent cap or budget re-armed it at zero on every tick. The bound shortening now skips a held loop. A budget-bound wake is padded by one monotonic clock tick, because the event loop can run a timer up to one tick early and the budget check would then find the budget unspent. Also marks the two delay-recording sleep patches in the budget tests flake-ok.
@iamwhatever
iamwhatever force-pushed the fix/autonudge-budget-cycle-arming branch from 25e49fd to 8101e7b Compare November 6, 2026 17:20
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Nov 6, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 6, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Status at 8101e7bea: CI and GPT 6.1 pass. Opus 5.5 raised a new blocking finding in the same lines after our fix rounds, so the crew stops here for a maintainer decision.

Finding (autonudge_service/timers.py, _arm_from_deadline): the early wake for a spent cycle cap or runtime budget also applies to arms made while the last turn is still running: the reconciler (_reconcile_once, after two 60 s passes) and release_approval_hold(arm=True). With the cap spent the delay becomes 0, and the loop is stopped as cycle_cap mid-turn. If that turn then writes the stop sentinel, the loop is still recorded as cycle_cap and the user is told it stopped short. notify_turn_complete returns early, so the judge label for that turn is lost.

Suggested design: shorten the delay only on arms made after the turn has ended (the notify_turn_complete / _arm_and_announce_deadline path, or when the slot has no turn in flight). The reconciler and approval-release arms keep the deadline delay. That needs _arm_from_deadline to know its caller, for example a keyword flag set only by the post-turn path. Please confirm the approach, or take it over.

AI-assisted, posted on behalf of iamwhatever

@bolichen97

Copy link
Copy Markdown
Contributor

[operator: bolichen97] Continuing this in #35437; thanks for the start.

bolichen97 added a commit that referenced this pull request Nov 12, 2026
…ter (#35437)

_timer stops a loop on a spent runtime budget only when it wakes, but
_arm_from_deadline armed toward the interval deadline alone. A loop
whose budget was shorter than its interval stayed active with the
budget spent for up to a full interval (a day at most): no expired
event, and a create-only arm on its slot was refused.

The timer now wakes one second after a budget that runs out before the
next deadline; the second keeps a timer asyncio runs a clock tick early
from firing a turn. A budget already spent arms as before, since app
reconcilers revive such a loop every few seconds.

Part of #26764. Builds on #26767.
@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 12, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Thanks for starting this. A maintainer continued this work in #35437, which has since merged on main, so I'm closing this PR as superseded. Please reopen if anything here is still missing from main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: action required A blocking check or review needs attention

Projects

None yet

Development

Successfully merging this pull request may close these issues.

autonudge: a spent runtime budget or cycle cap waits a full interval before the loop stops

3 participants