Repository navigation
agent-runtime: expire turn-scoped nudges instead of carrying them forever - #1587
Merged
obadakhalili merged 2 commits intoOct 30, 2026
Merged
Conversation
…ever The four mid-turn nudges (FOLLOWUP_TODO_NUDGE, SUGGEST_PROMPTS_BEFORE_ANSWER, GRAVITY_REPORT_NUDGE, TODO_PROGRESS_REMINDER) and the completion-check user message were pushed with no expiry of any kind, so every dead reminder was re-sent in every later step's prompt: per-step re-billing of its tokens, and each mid-history survivor shifted the shared suffix across two steps, adding a prompt-cache-write span each time. The existing timeToLive could not carry them: 'agentStep' dies at the very next expireMessages call — including the step the nudge steers, which is why the step prompt must be re-pushed (rather than re-stamped) every step — and 'userPrompt' buys nothing, since every endOf:'userPrompt' call site runs after the turn's messages have been consumed. expiresAt is the declarative replacement on AuxiliaryMessageData, with the AgentExpiresAt contract in packages/agent-runtime/src/cache-expiry.ts. Semantics, implemented in expireMessages: - 'userPrompt' (what the nudges use): lives for its whole turn in every step prompt — the nudge decisions rate-limit themselves by counting their own earlier nudges in history, so expiring them mid-turn would re-arm the limiter and loop — and dies at the first step carrying a newer USER_PROMPT (or at a turn-end endOf pass), so the next turn's first step never re-sends it. - 'agentStep': drops at the first step boundary strictly newer than the message, pinned by a snitch marker expireMessages appends behind surviving expiresAt'd messages. The mid-step cleanup pass sits between the push sites, so a marker timestamp strictly before the push is what proves one real boundary elapsed. The marker is a user-role message, and endsOnGravityReport now skips it when finding the last user message — otherwise it shadows the nudge and the turn never ends after the report step. - 'turn'/'conversation'/'notImplemented' are type-level placeholders that never expire on their own. Test pinning: nudge present in the immediately following step and through the turn, gone at the next USER_PROMPT; final turn state carries no expired nudge tags; dropUnansweredToolCalls repair semantics unchanged in the presence of live and expired nudge boundaries. Existing loop tests that counted nudges in the returned (post-turn) state now count them in the streamed prompts.
keepLastTags shipped as a type-only field (AuxiliaryMessageData) with exactly one producer — the per-turn instructionsPrompt carries keepLastTags:['INSTRUCTIONS_PROMPT'] — and zero consumers, so one instructions copy piled up per user turn, re-sent verbatim in every later step. expireMessages now applies it as a content-supersede rule on every assembly pass: the latest keepLastTags declaration wins, and earlier messages sharing any of the listed tags (the declaration's own tags overlap counts) drop out of the assembled history. It does not wait for the model to observe the newest copy — the declaration restates what the superseded copies said, so they inform nothing and only shift the cached suffix. Declarations never reach forward: an earlier message cannot delete a later one. The scaffolded agent-types duplicate of AuxiliaryMessageData (common/templates/initial-agents-dir) documents the behavior alongside the field; its expiresAt is kept as an inline union because the scaffolded types cannot import across packages.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The four mid-turn nudges (
FOLLOWUP_TODO_NUDGE,SUGGEST_PROMPTS_BEFORE_ANSWER,GRAVITY_REPORT_NUDGE,TODO_PROGRESS_REMINDER) and the completion-check message were pushed with no expiry, so every dead reminder was re-sent in every later step's prompt: per-step re-billing of its tokens, plus each survivor shifted the shared suffix, adding a prompt-cache-write span per step. Separately,keepLastTagsonINSTRUCTIONS_PROMPThad zero consumers — old instructions notes piled up one per user turn.Change (2 commits)
1. New
expiresAtfield onAuxiliaryMessageData('userPrompt'semantics: lives for its whole turn in every step prompt, dies at the first step carrying a newerUSER_PROMPT), implemented inexpireMessages. The existingtimeToLivecan't express this:'agentStep'dies at the very next pass (including the step the nudge steers), and everyendOf:'userPrompt'call site runs after the turn's messages are consumed.Why turn-scoped, not single-step: the nudge decision functions rate-limit themselves by counting their own prior nudges in history (
followup-todo-nudge.tscountsFOLLOWUP_TODO_NUDGEtags againstMAX_FOLLOWUP_TODO_NUDGES = 3). Expiring mid-turn would re-arm the limiter and loop the turn — empirically verified. Turn-scoped lifetime keeps current-turn, model-visible behavior bit-identical while stopping cross-turn re-sends.2. Wire
keepLastTags: latest declaration wins; earlier messages with an overlapping tag drop. OldINSTRUCTIONS_PROMPTcopies are pruned when a new one lands.Also covered: the
agentStepgrace step is pinned by a snitch marker (endsOnGravityReportskips it when finding the last user message — otherwise turns never end after a report step); expiring nudges count as ephemeral for conversation summaries.Tests
New tests pin: nudge present on the next step, survives its turn, absent after the next
USER_PROMPT; marker lifecycle; old instructions pruned on new declaration, non-overlapping tags untouched;dropUnansweredToolCallsrepair parity at expiry boundaries. agent-runtime: 980 pass / 1 pre-existing base failure; sdk: 1083 pass / 0 fail; typecheck clean on both.