Skip to content

[Story 4.2b] Core parallel dispatch in presentAssistantMessage #375

Open

Description

@edelauna

Part of #359 (Epic 4: Parallel Tool Execution).

Depends on: Story 4.2a (DispatchState), #362 (Story 1.2 — TaskSemaphore), #363 (Story 1.3 — experiment flag).

Fixes #190 (missing tool_result blocks — root cause is the missing Promise.allSettled guarantee this story adds).
Relates to #131 (parallel read-only tools — our plan's parallel dispatch for read-only tools directly enables this; execute_command stays sequential per the plan).

Context

With DispatchState in place (Story 4.2a), the parallel dispatch path can be added safely. When PARALLEL_TOOL_EXECUTION is enabled and the stream is complete, tools are dispatched via Promise.allSettled rather than the sequential switch. Promise.allSettled (not Promise.all) is critical — a missing tool_result for any tool_use block causes provider API errors on the next call (all major providers require a result for every tool call), so sibling results must always be collected even when one tool rejects.

This story covers core dispatch logic: tool partitioning, parallel execution, rejection strategy, and DispatchState interlock. File-level write serialization and observability are in Story 4.2c.

Developer Notes

In src/core/assistant-message/presentAssistantMessage.ts:

  • Add import pLimit from 'p-limit' — dependency already at ^6.2.0 in src/package.json.
  • Add presentAssistantMessageParallel(cline: Task): Promise<void>:
    • Gate: if !cline.assistantMessageSavedToHistory or cline.dispatchState !== IDLE → set pendingWork = { mode: 'parallel' } and return.
    • Set dispatchState = PARALLEL.
    • Wait for didCompleteReadingStream = true.
    • Partition all tool_use blocks into three categories:
      • Sequential (always serial even with flag on): new_task, attempt_completion, ask_followup_question, execute_command, switch_mode. execute_command is kept sequential as a deliberate safety margin — same stance as OpenAI Codex (shell_command defaults supports_parallel_tool_calls() = false).
      • MCP tools: Default to sequential — unknown side effects. Future opt-in via server metadata.
      • Parallelizable: read_file, list_files, search_files, list_code_definition_names, write_to_file, apply_diff, insert_code_block. Write tools gain per-file serialization in Story 4.2c; treat as fully parallelizable here (safe since this story ships behind the flag and 4.2c lands before flag promotion to default-on).
      • Default: unclassified tools are sequential. Add a compile-time or test-time exhaustiveness check: every tool type in the switch must be in exactly one classification bucket.
    • Dispatch parallelizable tools with Promise.allSettled via pLimit(PARALLEL_TOOL_CONCURRENCY). Define PARALLEL_TOOL_CONCURRENCY = 8.
    • Dispatch sequential tools one at a time after the parallel batch.
    • Rejection handling (two-tier): When askApproval returns false → cancel all not-yet-dispatched tools (they receive tool_result with is_error: true and cancellation message). Already-executing tools finish normally.
    • pLimit cancellation caveat: pLimit has no API for cancelling queued work. Check a shared cancelled flag inside each wrapped function before executing tool logic.
    • All supported providers require a tool_result for every tool_use. Denied tools produce { is_error: true, content: "Tool use was denied by user." }. Cancelled tools produce { is_error: true, content: "Tool execution cancelled — a sibling tool was denied." }.
    • In finally: reset dispatchState = IDLE, drain pendingWork.

In src/core/task/Task.ts:

  • After didCompleteReadingStream = true, check experiments.isEnabled(stateExperiments, "parallelToolExecution") and dispatch to presentAssistantMessageParallel or presentAssistantMessage accordingly.

Files: src/core/assistant-message/presentAssistantMessage.ts, src/core/task/Task.ts

Tests (src/core/assistant-message/__tests__/presentAssistantMessageParallel.spec.ts):

Dispatch behavior:

  • Controlled promises for each tool — resolve in arbitrary order; assert all results appear in userMessageContent.
  • Result ordering: tool_result entries match tool_use block ordering (required by all providers).
  • new_task in same response → dispatched sequentially after parallel batch.
  • MCP tool in same response → dispatched sequentially.
  • Flag off → presentAssistantMessageParallel never called.

Rejection strategy:

  • Tool A rejected, Tool B already executing → B finishes; A has denial result, B has normal result.
  • Tool A rejected → Tool C (not yet started) gets cancellation result, never dispatched.
  • All tool_use blocks have a corresponding tool_result regardless of outcome.
  • pLimit(8) with 10 tools — 8 dispatched, 2 queued. Rejection cancels the 2 queued tools. Assert all 10 have results.

Tool classification:

  • Exhaustiveness check: Every tool type in the switch is present in exactly one bucket. Adding a new tool without updating classification must fail this test.

All mocked; completes in < 100ms.

Acceptance Criteria

  • Flag off: zero observable change — all existing presentAssistantMessage tests pass.
  • Flag on: multiple read_file / list_files calls in one response execute concurrently.
  • pushToolResultToUserContent's duplicate guard (lines 371–384) prevents double-writes.
  • No duplicate dispatch possible via the dirty-flag path during parallel execution.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions