Skip to content

Peer inbox wakes can preempt active tool work and reactivate completed turns #1137

Open
#1141
Open
Peer inbox wakes can preempt active tool work and reactivate completed turns#1137
#1141
Assignees
justrach
Labels
bugSomething isn't working

Description

@yxlyx
yxlyx
opened on Sep 21, 2026

Symptom

Ambient peer-inbox notifications are injected into model history as user messages. During a multi-step tool workflow, repeated parked-peer wakes can become the newest user turn, pull the model away from the task it was continuing, and start additional assistant/tool cycles. A wake can also arrive immediately after attempt_completion and reactivate a session that had already completed.

The observed tool calls still had matching tool outputs, so this is not evidence that the tool processes themselves were cancelled. The failure is at the conversation/run-loop level: notification traffic is treated as task-authoritative user input.

Reproduction

  1. Run a root session through a multi-step inspection that uses several tool calls.
  2. From one or more peer sessions, send messages while the root session is working and leave them parked in its inbox.
  3. Let additional peer messages arrive across model step boundaries, including near completion.
  4. Inspect the transcript ordering.

Observed behavior:

  • peer wakes appear as role: user messages with notification origin;
  • identical unread-state wakes can appear again as ordinary user messages;
  • newer unread counts add further user turns rather than coalescing the existing ambient state;
  • assistant reasoning/tool activity can continue after attempt_completion in response to a peer wake.

Root cause

Agent.runTurn calls peer_channel.deliverInbound before every model request, including tool-result continuation requests. deliverInbound parks the peer bodies and appends a synthetic user message containing the unread count. Because the wake has user role and is delivered at every step boundary, the model can interpret ambient coordination as a new primary instruction instead of continuing the active task.

Clearing the parked inbox does not make already-injected wake messages non-authoritative, and the same unread state can be represented through both notification delivery and normal conversation history. Wording the wake as advisory reduces one symptom but does not fix its role, lifecycle, or deduplication semantics.

Expected behavior

Peer-room traffic should remain ambient coordination unless the user or agent explicitly opens the inbox. It must not create a new task-authoritative turn, interrupt tool-result continuation, or restart a completed turn.

Fix direction

  • Represent peer availability out of band, or in a non-user context channel that cannot start a turn by itself.
  • Coalesce wakes by mailbox generation/unread state so the same state is exposed at most once.
  • Do not auto-run a peer wake after attempt_completion.
  • Preserve the current human task as the newest authoritative turn while tools are being continued.
  • When peer_message action=inbox clears the mailbox, retire any stale wake associated with that mailbox generation.

Regression coverage

Add a run-loop test that proves:

  1. a peer arrival between tool batches does not add a new authoritative user turn;
  2. duplicate delivery of the same unread state is idempotent;
  3. a peer arrival after attempt_completion does not start another assistant/tool cycle;
  4. an explicit later human turn can still see that peer messages are waiting.

Activity

  1. yxlyx
    added
    bugSomething isn't working
    on Sep 21, 2026

  2. yxlyx commented on Sep 21, 2026

    @yxlyx
    yxlyx
    on Sep 21, 2026
    CollaboratorAuthor

    @justrach — could you take a look at this peer-notification run-loop issue?

  3. justrach commented on Sep 21, 2026

    @justrach
    justrach
    on Sep 21, 2026
    Owner

    Fixed in #1141.

  4. justrach
    closed this as completedon Sep 21, 2026

  5. justrach commented on Oct 17, 2026

    @justrach
    justrach
    on Oct 17, 2026
    Owner

    Seen on graff 0.302.41 (latest release checked: v0.302.41), Harness 0.2.122, Harness desktop / ACP, macOS arm64.

    Additional current-version reproduction of the completed-turn reactivation problem, through Harness send_message rather than a demonstrated native peer_inbox path:

    1. Have two local Harness chats collaborate on a bounded task.
    2. Send progress messages while the recipient is busy; the MCP receipt reports queued delivery.
    3. Finish the task and send a terminal coordination update indicating that no response or further work is required.
    4. Observe subsequent inbound coordination updates and the recipient's new turns.

    Observed: later coordination messages can refer to work that has already finished. Closure/no-reply updates still enter the recipient as new user-role turns with an unconditional reply directive in the message envelope. The recipient produces repeated no-op completion/acknowledgement turns instead of remaining idle. Delayed/stale updates are difficult to distinguish from fresh requests.

    This adds a Harness send_message/ACP surface reproduction on the versions above; it does not establish that the previously fixed native peer-inbox path is the same cause. Local trace review did not establish an exact enqueue-to-consumption delay, so no numeric delivery-latency claim is made.

    Relevant current source boundaries, not a proven root cause:

    • v0.302.41:src/acp_inbox.zig:152-162: incoming session prompts are queued behind a live turn; the active-turn hook signals a follow-up to job waiting.
    • v0.302.41:src/acp_inbox.zig:63-71: queued input is drained in arrival order.
    • v0.302.41:src/acp_idle.zig:1-3: documents idle peer-mail wake behavior and distinguishes it from mid-turn preemption.

    Suggested follow-up / regression expectations:

    • Preserve source-message order and expose enqueue time, delivery time, queue age and delivery state; accepting a queued send should not imply that the recipient has consumed it.
    • Deliver actionable updates at a documented safe boundary without cancelling active tools or losing their results; make the limitation clear when immediate steering is unavailable.
    • Let callers mark an update informational, terminal or reply-required rather than unconditionally asking every recipient to reply.
    • Coalesce superseded progress messages and avoid stale task-specific updates reopening a closed task.
    • Cover a busy recipient, queued progress, terminal/no-reply delivery and a later genuine human request in one regression scenario. Terminal informational mail must not create an acknowledgement loop; a genuine new request must still work.

    No conversation bodies, private project details, local paths, session identifiers or raw traces are attached.

  6. yxlyx commented on Oct 18, 2026

    @yxlyx
    yxlyx
    on Oct 18, 2026
    CollaboratorAuthor

    Seen on graff 0.302.44 (latest release checked: v0.302.44), Harness 0.2.127, surface Harness desktop / ACP, macOS arm64.

    New reproduction: ordinary final answer followed by a duplicate completion

    Correlated the saved model transcript, agent trace, and Harness journal for a completed issue-filing task. This is a native parked-peer wake, not the Harness send_message envelope path described in the previous comment.

    1. At approximately +62.3s from agent start, the requested turn completed successfully. The model transcript contains an ordinary assistant message with phase: final_answer; the host journal records a completed Done. The requested issue was already created.
    2. At approximately +73.4s, a synthetic user-role message with _graff_origin: notification and a parked-peer unread notice started a second turn, with no intervening human request.
    3. The second turn loaded peer_message and read its inbox. The mail was an informational edit-scope/non-overlap update, not a request to repeat the human-facing result.
    4. The agent called attempt_completion with a paraphrase of the result it had already delivered. The UI therefore displayed a second completion and a separate two-tool activity group.
    5. The unnecessary turn consumed three model calls over approximately 13.6s. The host logged that a parked session resumed on self-continued output. Both turns completed successfully, with no tool errors.

    This was two generated assistant completions, not one message rendered twice. The issue-creation command was not executed twice in this sequence.

    Completion-path gap to investigate

    • v0.302.44:src/agent_tools.zig:330-334: accepted attempt_completion stores its result and calls root peer_idle.noteCompletion().
    • v0.302.44:src/peer_idle.zig:43-47: that function sets suppress_idle_wake.
    • v0.302.44:src/peer_idle.zig:97-115: when not busy and not suppressed, a new unread batch can produce an idle wake.
    • v0.302.44:src/acp_idle.zig:35-61: the wake is translated into another session/prompt, without a new client request.
    • v0.302.44:src/peer_idle.zig:157-177: existing regression coverage specifically tests suppression after attempt_completion.

    The first observed completion used the ordinary final-answer / completed-Done path, not the completion tool. The unsolicited second turn then used that tool. Task-completion protection must also cover normal final answers, rather than depending on which completion path the model chooses. The evidence establishes that path distinction; the suppression flag itself was not instrumented at runtime.

    Why it matters / expected behavior

    One human request produces two near-identical final answers, suggesting a retry or double execution. Informational coordination reopens the completed task and its Working/timer state, pollutes the transcript, and adds unnecessary model work. Hiding repeated text in the renderer would not fix the extra turn.

    Please revisit this closed issue and cover successful ordinary final answer + completed Done + informational peer arrival, alongside explicit attempt_completion. Neither should replay the completed task result. Keep mail readable and preserve genuine human requests, actionable coordination, and legitimate outstanding-job continuation; do not disable all idle continuation indiscriminately.

    Evidence is summarized without raw prompts, peer-message bodies, screenshots, local paths, machine names, model/provider details, or session/trace identifiers.

  7. yxlyx
    assigned
    justrach
    on Oct 18, 2026

  8. yxlyx commented on Oct 18, 2026

    @yxlyx
    yxlyx
    on Oct 18, 2026
    CollaboratorAuthor

    Seen on graff 0.302.44 (latest release checked: v0.302.44), Harness 0.2.127, surface Harness desktop / ACP, macOS arm64.

    Additional reproduction: repeated ordinary final answers after unrelated peer mail

    Reopening at the reporter's request. This adds a distinct observed sequence to the earlier current-version report: one completed human request followed by two more ordinary final answers, each immediately following a peer-inbox read, rather than an ordinary final answer followed by attempt_completion.

    Evidence and sequence

    Checked the saved model conversation against the chat's message/tool history. The relevant sequence was:

    1. A human requested a local browser preview.
    2. The assistant started the preview, checked it successfully with an HTTP request, and delivered a normal assistant message with phase: final_answer containing the result.
    3. With no intervening human message, the assistant called peer_message with action: inbox. The returned mail was an informational edit-scope/non-overlap update about another task. It did not ask the recipient to revisit the preview or repeat its answer.
    4. The assistant generated another normal phase: final_answer that paraphrased the already-delivered preview result.
    5. Still with no intervening human message, another peer_message action: inbox returned an informational update about a different, unrelated task. The assistant again generated a normal phase: final_answer repeating the preview result.
    6. A further bare acknowledgement appeared before the next genuine human request. No adjacent inbox call is present for that acknowledgement in the saved sequence, so its individual trigger is not established here.

    The two repeated preview answers are separate generated assistant messages in the saved model conversation, not one message rendered multiple times. Each corresponding chat entry has a separate one-tool activity group. The repeated-answer segments contain inbox reads, not another preview-start command; this evidence does not show the preview being started repeatedly.

    The inbox results also include an unconditional instruction to reply through peer_message, even for informational scope updates. That is a potential acknowledgement-pressure contributor, not proof that it caused the repeated human-facing answers.

    Why this extends the existing report

    The earlier current-version reproduction described an ordinary final answer followed by an explicit completion-tool call. Here, all three result-bearing answers use the ordinary final-answer path. The session can therefore emit multiple unsolicited result repetitions without ever reaching the completion-tool suppression hook in these segments.

    The demonstrated behavioral failure is completed-task reactivation/result replay around informational peer inbox consumption. The saved sequence does not, by itself, instrument the exact scheduler wake or prove the runtime value of the suppression flag. The available startup trace is not evidence for the later wake timings; no model-call counts, costs, or precise wake-latency claims are made.

    Relevant tagged source / hypothesis to investigate

    • v0.302.44:src/agent_tools.zig:330-334: accepted attempt_completion stores the result and calls root peer_idle.noteCompletion().
    • v0.302.44:src/peer_idle.zig:43-47: noteCompletion() sets suppress_idle_wake.
    • v0.302.44:src/peer_idle.zig:97-115: the idle path checks that flag and may create a wake for new unread mail.
    • v0.302.44:src/acp_idle.zig:35-61: an eligible idle wake is passed into a new session/prompt.

    These boundaries support investigating ordinary-final-answer completion bookkeeping and peer-mail lifecycle together. They do not establish that every observed extra answer entered through this exact runtime path. A fix limited to the explicit completion tool is insufficient for the demonstrated ordinary-final-answer sequence.

    Expected behavior and focused regression coverage

    • A successful ordinary final answer should end the human task just as an accepted attempt_completion does.
    • Unrelated informational edit-scope mail must remain readable without replaying the completed human-facing result or generating acknowledgement-only chatter.
    • Cover two successive informational peer batches after an ordinary final answer, including explicit inbox consumption. Neither batch should produce another completion of the old task.
    • Cover accepted attempt_completion and yield_turn separately, so their existing protection is preserved.
    • Avoid unconditional reply directives for informational mail; distinguish informational updates from actionable coordination or an explicit request for a reply.
    • Preserve genuine later human requests, actionable peer requests, and legitimate outstanding-job continuation. Do not solve this by disabling all peer mail or all idle continuation.
    • Assert the absence of extra assistant/model turns where no response is needed, rather than merely deduplicating rendered text. A rendering-only patch would hide the symptom but leave unnecessary model work and task reactivation.

    This follow-up contains no raw conversation or peer-message bodies, screenshots, local URLs/ports, local paths, chat titles, machine/user names, model/provider names, session/run/trace identifiers, or raw logs.

  9. yxlyx
    reopened this on Oct 18, 2026

  10. justrach commented on Oct 20, 2026

    @justrach
    justrach
    on Oct 20, 2026
    Owner

    #1688 (merged) addresses part of this. peer_message send takes fyi: true for a status or scope update. That mail stays readable in the inbox but never starts an idle turn on its own, and a later request still wakes the root. The inbox footer no longer tells the model to reply unconditionally.

    Still open: unmarked informational mail can still start a turn after an ordinary final answer, and the Harness send_message path is unchanged.

  11. yermakoffivan
    added a commit that references this issue on Oct 20, 2026

    peer mail: fyi updates stay parked instead of starting a turn

    03c5c80
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions