Skip to content

fix(chat): say when a promised follow-up has nothing to wake it - #23248

Merged
iamwhatever merged 1 commit into
mainfrom
fix/23243-deferred-followup-notice
Oct 29, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
fix/23243-deferred-followup-notice

Conversation

@JLargent42

@JLargent42 JLargent42 commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: When the agent ends a turn saying it will report back later and nothing is set up to wake it, the person sees a notice that no follow-up is scheduled.

The agent ends a turn with "I'll report back when the build finishes" or "I'll watch the pipeline and act when it is green". The turn arms no monitor, scheduled job or background run, and none is running. Nothing wakes the session again, so the update never arrives, and the person learns this only by writing again (#23243).

Why it matters

People take the promise at face value and walk away. Two users reported the silence in two days. It looks like the agent broke, when in fact nobody asked it to come back.

Not a goal

What changed (motivation → approach → change)

Symptom: a promise of a later update, with no wake behind it, ends the turn silently. Cause: the runner's two existing checks cover only "I'll do it now" and "I'm still working", not a promise deferred to a future event.

Approach: at turn end the runner now checks four things. If all four hold, it appends one info notice and changes nothing else.

Check Where Passes when
The closing two sentences promise a later follow-up has_deferred_followup_promise in chat_utils_turn_text.py first-person future commitment + a follow-up verb (report back, let you know, keep an eye on, watch…); quoted code and [OPTIONS:] stripped; skips user-gated ("let me know when you're ready and I'll…") and negated forms
This turn called no wake-arming tool turn_armed_a_wake no monitor_*, cron_add/update/resume/trigger, register_hook, spawn_run/continue, workflow_* or task_run call; wait is excluded because it blocks inside the turn
Nothing else owns a later turn background_wake_absent in chat_utils_completion.py no active monitor loop, attached child run, sub-agent result being delivered, any entry waiting in the slot queue (cron notification, hook continuation), pending follow-up watcher, pending workflow, pending synthesis, or enabled cron job this session created or whose runs land here; a probe failure counts as "maybe scheduled"
No Stop hook could continue the turn _stop_hook_configured in chat_turn/turn_ending.py no enabled Stop hook, stored or from the agent spec; Stop hooks fire after the ending chain and may queue a continuation, and an unreadable hook store counts as one

The first four probes moved out of chat_done_payload into background_wake_pending, so the completion sound and the new notice share one answer. The arm in chat_turn/turn_ending.py is an elif after the "still in progress" notice, so a turn gets at most one of the two. Slack-born turns run through the same runner. The session spec (docs/system-specs/modules/session.md) and the chat_utils module map document the notice.

Backwards compatibility

Compatible: one new info notice on turns that previously ended with no row. chat_done_payload returns the same verdict from the same probes.

Removes nothing: the probes moved into a shared helper, and every probe is still read.

Tests

test/test_deferred_followup_notice.py:

  • the detector fires on nine promise phrasings and stays quiet on finished answers, user-gated and negated forms, early plans, quoted text, option labels and third-person monitors;
  • the tool check counts each wake-arming tool, including server-qualified names, and excludes wait/spawn_steer; a malformed identity, or a call whose frame names no tool (claude-agent-acp, opencode), counts as armed;
  • the probe sees a monitor, an owned cron job, children, workflows and synthesis, and a failed probe never reports "absent";
  • the runner appends the notice on an unbacked promise and re-queues nothing, and stays quiet when monitor_start ran, when background work exists, and on a finished answer.

Proof: with only the runner arm disabled, test_unbacked_promise_gets_the_notice fails and the other runner tests still pass. Determinism: 5 serial repeats, plus -n 4 alongside test_facade_mirror_guards.py and test_campaign_facades_base_public_names.py, all green.

Manual verification

N/A — the runner tests drive the real _run_chat with a scripted stream, which is the path a live turn takes.

Related Issues

Closes #23243
Part of #9928
Follow-up: #24916 counts a Claude run_in_background Bash call as a wake once agent-started turns are shown (#17566). Today the notice is accurate for that call, because #18285 shipped the transport only.

Sibling sweep

Swept: git grep -nE "has_unfinished_progress_claim|is_promise_only_terminal" -- src/
Sibling: src/kiro_crew/dashboard/chat_turn/turn_ending.py -- changed here
Sibling: src/kiro_crew/dashboard/chat_utils_turn_recovery.py -- not affected -- it decides recoveries for immediate promises only, and this arm never recovers

Pattern harvest

Not generalizable: the defect is one missing turn-end text claim (deferred promise), not a class a static rule can catch.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@JLargent42

Copy link
Copy Markdown
Contributor Author

Coupling with #18285, noted so neither PR merges blind. This PR decides "nothing will wake the session" from the wake-arming tool list plus the runtime probes. On the claude backend, a Bash call with run_in_background is also a wake once #18285 lands: Claude Code starts its own turn when the task finishes, and #18285 reads that turn. On main today that turn is unread until the next prompt (#17566), so the notice is accurate for now. Whichever PR merges second should make that call count as a wake. The turn_armed_a_wake check would need the call's input, not just its name.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 23, 2026
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision cab4b2d99c4bb27d6399af203d9045962803ea91 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of cab4b2d99c4bb27d6399af203d9045962803ea91 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

The change only adds a notice and probes nothing new, and every probe that fails stays quiet. A wrong regex match costs one extra info row, and a revert removes it cleanly.

Notes

  • The notice text tells the person to "ask the agent to set up a monitor." That steers people toward an always-on monitor for what is often a one-shot wait. "Ask the agent to check back" covers both and keeps the "not a goal" line about cost and authority intact.

[DESIGN-REVIEWED] cab4b2d

Description read

[DESCRIPTION-READ] bbd40967181b801c8878dd5a60d88d8967caa9f35293be9346e9bbf53b5fbf0f

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of cab4b2d99c4bb27d6399af203d9045962803ea91 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing to check.

What this change ships

Inventory (7 items) — 7 justified

Intent: when the agent promises to report back later and nothing will wake the session, the person learns that no follow-up is scheduled (FIX, #23243).

  1. A new info notice appears after a turn that ends on an unbacked "I'll report back later" promise — justified
  2. The closing two sentences are read for a first-person deferred promise; user-gated, negated, quoted and option-label text is skipped — justified
  3. Calling a monitor, cron, hook, spawn, workflow or task tool this turn counts as having set up a wake (WAKE_ARMING_TOOLS) — justified
  4. The runtime check for a later turn (monitor, children, workflows, synthesis, slot queue, pending watcher, owned cron job) treats a failed probe as "maybe scheduled" — justified
  5. Any enabled Stop hook keeps the notice quiet — justified
  6. The completion sound and the new notice now get their answer from one shared helper, background_wake_pending, with the same verdict as before — justified
  7. Updates to the session spec and the chat_utils module map — justified

[FIRST-PRINCIPLES-REVIEWED] cab4b2d

Description read

[DESCRIPTION-READ] 7b15917bc703cad8f8584fb7a285bbf7b4d64edc7e1cbbb57ac16cce4e4e3985

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed cab4b2d99c4bb27d6399af203d9045962803ea91 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] cab4b2d

Verdict parsed from the review's SHA-scoped output markers for commit cab4b2d99c4bb27d6399af203d9045962803ea91.

False positive or not applicable? A repository writer can comment:
/ai-review override fable cab4b2d99c4bb27d6399af203d9045962803ea91 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of cab4b2d99c4bb27d6399af203d9045962803ea91 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] cab4b2d

False positive or not applicable? A repository writer can comment:
/ai-review override gpt cab4b2d99c4bb27d6399af203d9045962803ea91 false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 23, 2026
@JLargent42
JLargent42 force-pushed the fix/23243-deferred-followup-notice branch from d84e021 to cffbd9f Compare October 23, 2026 01:04
@JLargent42

Copy link
Copy Markdown
Contributor Author
  • fixed: offers and noun uses of watch/monitor/poll read as a deferred promise (chat_utils_turn_text.py:222)

Legitimate. The person-gated exemption now covers "if you'd / if you would (want|like|need|prefer)".
Bare watch/monitor/poll now count only directly after the subject, behind at most one adverb (then/also/just/now). So "I'll set up a monitor" and "I'll explain how to poll" no longer match.
Regression cases added to test_deferred_followup_notice.py; they fail before the fix and pass after. Fixed in cffbd9f.

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Oct 23, 2026
@JLargent42
JLargent42 force-pushed the fix/23243-deferred-followup-notice branch from cffbd9f to 1e6037a Compare October 23, 2026 01:30
@JLargent42

Copy link
Copy Markdown
Contributor Author

Rebased: resolved a conflict with main (#22682 moved the turn-end chain into chat_turn/turn_ending.py, so the notice now sits there); CI re-running.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 23, 2026
@JLargent42
JLargent42 force-pushed the fix/23243-deferred-followup-notice branch from 1e6037a to d6ddd1b Compare October 23, 2026 01:32
@JLargent42

JLargent42 commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor Author
  • fixed: span=24aeb56d5864 Pending follow-up watchers are omitted from the wake predicate

Legitimate. background_wake_absent now also reports a wake unless subagents.has_in_memory_pending_work_for(session_key) is a definite False, the same helper synthesis_fire_verdict uses.
Regression test test_probe_sees_a_pending_follow_up_watcher fails without the change. Fixed in d6ddd1b.

@JLargent42

JLargent42 commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor Author
  • rebutted: span=f8ab89034b60 Sentence-local matching flags fulfilled and user-gated replies

Disproportionate; the reply only produces an info line. The detector already handles the common fulfilled forms: a colon lead-in, and a promise followed by more content in a sentence beyond the closing two.
Judging whether a later sentence fulfils or gates an earlier promise is open-ended language inference. A wrong guess either way only shows or hides an advisory row and never changes the turn. Opus adjudication rated it LOW.

@JLargent42

JLargent42 commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor Author
  • rebutted: span=db37e66c9e96 The generated warning never reaches linked messaging users

Disproportionate here and not a regression. A linked thread got no warning before this PR either, and the sibling progress-claim notice in the same chain is dashboard-only for the same reason.
Mirroring notice rows through the audience-fenced channel paths is a separate cross-module change. The Not-a-goal scope keeps this PR to the runner notice. Opus adjudication rated it LOW.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: action required A blocking check or review needs attention labels Oct 23, 2026
@JLargent42
JLargent42 force-pushed the fix/23243-deferred-followup-notice branch from 7704f0e to 9285426 Compare October 28, 2026 02:45
@JLargent42

Copy link
Copy Markdown
Contributor Author

Follow-up to the rebase onto #25040, pushed as 9285426e01d2462b13ca7b1092da8ccdf3d59dd9 (no rebase, base unchanged).

#25040 counts a sub-agent result waiting in the slot queue (_slot_holds_delivery) as pending work for synthesis. background_wake_absent did not, so a turn that said "I'll report back" while a finished child's completion event sat in the slot queue got the "no follow-up scheduled" notice, even though that queued event starts the next turn. The new test test_probe_sees_a_sub_agent_result_waiting_in_the_slot_queue failed on 7704f0e2 and passes now.

Fix: background_wake_absent calls the existing _slot_holds_delivery(slot) right after background_wake_pending and returns False when it holds a delivery. A queued ordinary entry still does not count. The docstring, the session spec line and the PR description table are updated to match.

Local checks: 1149 passed at pytest -n 2 across the 28 targeted files; black (py310 and the bounded gate), isort, flake8, mypy, docs-lint and the comment-history gate pass.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 28, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Holding on 9285426 for one unanswered AI finding.

Both items from my r128 review are done: #24916 records the run_in_background wake (open, assigned, due 2026-11-24), and the cron-cache rebuttal holds because _synced_snapshot returns the in-memory cache on CronStoreBusy.

Unanswered current-head GPT finding. The GPT 6.1 lane on this head says the notice is decided and persisted in _turn_ending_chain before _turn_stop_hook can queue an accepted continuation (chat_runner.py#L1038-L1041, arm at turn_ending.py#L557). I confirmed the ordering. Adjudication downgraded it and the harm is one wrong info row, but it has no disposition. A short target=gpt head=9285426e... rebuttal, or a check that skips the notice when a Stop hook is configured, would clear this.

The rest reads sound: every unknown probe fails toward no notice, unnamed tool calls count as armed, and checks on this head are green.

— crew-pr-reviewer (r131)

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes at 9285426e01d2462b13ca7b1092da8ccdf3d59dd9 for one untracked follow-up. The r128 items are done (#24916, cron-cache rebuttal), every probe fails toward "something is scheduled", and CI is green.

To clear:

  • GPT 6.1 F1 is correct: the notice is appended in _turn_ending_chain and persisted before _turn_stop_hook can queue a continuation (chat_runner.py#L1038-L1041, turn_ending.py#L546-L559). With a blocking Stop hook the person is told nothing will wake the session while a continuation is about to run. It is rare and the next turn contradicts it, so deferring is fine, but please fix it now or open an issue for it and link it in the description.

AI findings judged non-blocking:

  • Opus note (filler slot matches "I'll help you follow up with the vendor"): a heuristic false positive; the notice's claim that nothing is scheduled is still true there (chat_utils_turn_text.py#L209-L218).
  • Design note (English-only detector): a doc nit; other languages fail safe to the old silence.
  • Design note (notice suggests a monitor even if denied): wording nit; it also says to send a message (chat_utils_turn_text.py#L321-L326).
  • First Principles item 5 (_bare_tool_name duplicates a Slack helper): style; the Slack helper also strips a Running: prefix (chat_utils_turn_text.py#L258-L260).

— crew-pr-reviewer (r132)

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision labels Oct 28, 2026
@JLargent42
JLargent42 force-pushed the fix/23243-deferred-followup-notice branch from 9285426 to c531c00 Compare October 29, 2026 00:02
@JLargent42

Copy link
Copy Markdown
Contributor Author
  • span=255690624a30: Notice precedes Stop hooks that queue a continuation

Fixed in c531c0085bbefacf0f673d822772e8d856a0a731. The notice arm in chat_turn/turn_ending.py now also requires _stop_hook_configured(turn) to be False. Any enabled Stop hook, stored or from the agent spec, keeps the notice quiet, because _turn_stop_hook runs after the ending chain and may queue a continuation the chain cannot see yet. An unreadable hook store counts as a hook (fails toward no notice, like the other probes). No hook store means _fire runs no hook, so the notice still applies there.
The ordering is unchanged; the notice simply does not fire when a Stop hook could continue the turn. New test test_promise_with_a_stop_hook_configured_is_quiet fails with the check disabled and passes with it. The session spec says the same.

@JLargent42

Copy link
Copy Markdown
Contributor Author

@iamwhatever Re the r131 HOLD (comment): I took the second option and pushed it as c531c0085bbefacf0f673d822772e8d856a0a731.

  • The notice now stays quiet when any enabled Stop hook is configured for the turn (stored or from the agent spec), since _turn_stop_hook runs after the ending chain and may queue a continuation. An unreadable hook store also keeps it quiet. The new check is _stop_hook_configured in chat_turn/turn_ending.py; the ordering of the chain is unchanged.
  • New test test_promise_with_a_stop_hook_configured_is_quiet fails with the check disabled and passes with it, plus two unit tests for the check itself.
  • The GPT finding has its disposition: comment (span=255690624a30, scoped to 9285426e). The session spec and the PR description table now list the fourth check.
  • Local: 1947 passed, 3 skipped across 52 explicit test files at pytest -n 2; black (py310), isort, flake8, mypy, docs-lint, comment-history, the file-length cap and the async-I/O check pass. No rebase.

Could you re-review and clear the CHANGES_REQUESTED when you have a moment?

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Oct 29, 2026
@JLargent42
JLargent42 force-pushed the fix/23243-deferred-followup-notice branch from c531c00 to bafc69d Compare October 29, 2026 01:10
@JLargent42

Copy link
Copy Markdown
Contributor Author

@iamwhatever Following up on the r131 HOLD (comment): #23248 conflicted with main, so I rebased it once. The new head is bafc69d2704.

  • The only conflict was one paragraph in docs/system-specs/modules/learn-cron-dashboard.md. Main changed the slot_retention.py entry and this PR changed the chat_utils.py entry next to it, so I kept both changes. No code conflicted, and the scope is the same 8 files.
  • The Stop-hook fix from c531c008 still works: test_promise_with_a_stop_hook_configured_is_quiet fails when _stop_hook_configured is switched off and passes when it is on.
  • Local checks: 1950 passed and 2 skipped across 52 explicit test files at pytest -n 2. black, isort, flake8, mypy, docs-lint, comment-history, the file-length cap and the async-I/O check all pass.

The GPT disposition for the Stop-hook finding is here. Could you re-review and clear the CHANGES_REQUESTED?

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Oct 29, 2026
@JLargent42
JLargent42 force-pushed the fix/23243-deferred-followup-notice branch from bafc69d to cab4b2d Compare October 29, 2026 01:32
@JLargent42

Copy link
Copy Markdown
Contributor Author
  • fixed span=35aee9aadff9: a queued cron notification was not counted as a wake (chat_utils_completion.py:266)

Fixed in cab4b2d99cdb. background_wake_absent now returns False when the slot queue holds an entry of any kind, not only a sub-agent result. A cron notification, a hook continuation or any other queued entry starts the next turn once this one ends, and chat_done_payload already counts queue_depth the same way for the completion sound. _slot_holds_delivery still covers a sub-agent result being delivered.
New test test_probe_sees_a_cron_notification_waiting_in_the_slot_queue queues a cron_notification entry with an empty job list. It fails without the fix and passes with it. The session spec lists the queued-entry case.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 29, 2026

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve at cab4b2d. This clears my r132 change request.

The r132 item is fixed: the notice now stays quiet when any enabled Stop hook (stored or from the agent spec) is configured, and an unreadable hook store also counts as one (turn_ending.py#L49-L64, used at L579). The later Opus finding is fixed too: any entry waiting in the slot queue now counts as a wake (chat_utils_completion.py#L262-L270). Both come with tests that pin them. The run_in_background follow-up is tracked in #24916. Every probe still fails toward no notice, all AI lanes are clean on this head, CI is green and the branch merges cleanly with main.

AI findings judged non-blocking:

  • Design Review (PASS note): the notice suggests a monitor rather than a check-back. Wording nit; the notice also tells the person they can send a message (chat_utils_turn_text.py#L321).

— crew-pr-reviewer (r134)

@iamwhatever
iamwhatever enabled auto-merge (squash) October 29, 2026 06:56
@iamwhatever
iamwhatever merged commit 7b0ab42 into main Oct 29, 2026
110 of 117 checks passed
@iamwhatever
iamwhatever deleted the fix/23243-deferred-followup-notice branch October 29, 2026 06:56
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agent promises to report back later, but nothing is scheduled to wake it

2 participants