Skip to content

fix(cron): keep a script cron's MCP server after a tool error - #16905

Merged
iamwhatever merged 1 commit into
kirodotdev:mainfrom
unstablebrainiac:fix/cron-tool-error-keeps-server
Oct 5, 2026
Merged

iamwhatever merged 1 commit into
kirodotdev:mainfrom
unstablebrainiac:fix/cron-tool-error-keeps-server

Conversation

@unstablebrainiac

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A script cron's call that the MCP server answers with a tool error keeps the run's server, so a rate-limited call no longer makes the run's next call start a new server that signs in again.

Since #16661, a script cron run keeps one MCP server per name for its later calls. A call the server answered with a tool error still stopped that server, the same as a call that got no answer. A Slack server answers a rate-limited call with a tool error, so the run's next Slack call started a new server, and that server signed in to Slack again. Sign-ins that come close together are what Slack refuses with HTTP 429 (#16659).

The approval on #16661 also asked for one sentence in the cron spec on what happens to kept servers when a run is killed at its timeout.

Why it matters

Every tool error from a server that signs in when it starts costs the run another server start, handshake and sign-in. For Slack, that extra sign-in comes seconds after the last one, and sign-ins close together are what Slack refuses. Without the spec sentence, a reader cannot tell whether a timeout leaves kept servers running.

Not a goal

  • Retrying a call that got a tool error, or any rate limiting.
  • Changing what happens after a call that gets no answer: its server still stops.
  • Changing how a timed-out run is killed. The spec sentence describes what the kill already does.

What changed (motivation → approach → change)

A call the server answers with a tool error now keeps the run's server, the same as a call it answers with a result. The script still gets the error. Only a call that gets no answer stops its server.

The call Before After
answered with a result server kept server kept
answered with a tool error (a JSON-RPC error reply or an isError result) server stopped; the next call starts a new one, which signs in again server kept
gets no answer (the server exited, stopped answering, or wrote no answer) server stopped server stopped

McpToolClient.call_tool in src/kiro_crew/cron_script.py raised a plain RuntimeError for both kinds of failure, so KeptMcpServers could not tell a server that answered from one that did not. It now raises McpToolError, a new RuntimeError subclass with the same message, for the two error answers. KeptMcpServers.call_tool keeps the client when it catches McpToolError, then re-raises it. Both cron contexts, the run's and kirocrew cron preview's, go through KeptMcpServers, so both behave this way.

The server-lifetime bullet in docs/system-specs/modules/learn-cron-dashboard.md says this. It also says what a timeout does: the kill skips the launcher's finally, but kept servers still stop, because each runs inside the script child's process group (SIGKILL on POSIX) or process tree (taskkill /T on Windows). Only a server that starts a session of its own outlives the kill, until it reads end-of-file on its stdin.

Backwards compatibility

Compatible: a script gets the same results and the same error messages from the same calls. McpToolError subclasses RuntimeError, so code that catches RuntimeError still catches it. Only how long a server stays up after a tool error changes.

Removes nothing: no key, function or record is removed or renamed. cron_script.py gains McpToolError.

Tests

  • test_a_tool_error_keeps_its_server_for_the_next_call in test/test_cron_script.py: a call that raises McpToolError leaves its server running, the next call uses the same server, and close() stops it once.
  • test_a_call_without_an_answer_stops_its_server_and_the_next_call_starts_another (renamed from test_a_failed_call_stops_its_server_and_the_next_call_starts_another): a disconnect still stops the server, and the next call starts another.
  • TestMcpToolClientProtocol: an error reply and an isError result raise McpToolError; a disconnect raises a RuntimeError that is not one.

Each part reverted on its own turns a named test red: stopping the server after a tool error, keeping it after a call without an answer, and either error answer raising a plain RuntimeError.

Manual verification

On Linux, two script crons ran through run_script_sandboxed with a 10-second timeout. Each made one ctx.call_tool and then slept. Both servers ignore end-of-file on stdin, so only a signal could stop them.

  • An ordinary server ran in the script child's process group and was gone after the timeout kill.
  • A server that called os.setsid() first was still running 5 seconds after the kill. The check then stopped it.

This host has no systemd user session, so systemd-run --scope was not in the spawn path. cgroup_scope_argv documents that --scope execs into the target, so it leaves the process group as it is. The Windows path (taskkill /T) was traced in code, not run.

Related Issues

Refs #16659

Follow-up to #16661: its Design review's tool-error point and the timeout note in its approval.

Pattern harvest

Not generalizable: KeptMcpServers.call_tool is the one place that builds an McpToolClient, so it is the one place that decides whether a server outlives a failed call, and both cron contexts go through it.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

A call the server answers with a tool error (a JSON-RPC error reply or an
isError result) now keeps its server for the run's next call, so a
rate-limited call no longer makes the next call start a new server and
sign in again. McpToolClient raises the new McpToolError for those
answers; a call that fails any other way still stops its server.

The cron spec also says what happens to kept servers when a run is killed
at its timeout.
@unstablebrainiac
unstablebrainiac requested a review from a team as a code owner October 5, 2026 05:25
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Oct 5, 2026
@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of a50334e3f19f594011e3837e3e640305ed805001 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check one case before merge: a server whose startup sign-in failed and that reports it as an isError result will now stay broken for the whole run instead of being restarted on the next call.

[DESIGN-REVIEWED] a50334e

Description read

[DESCRIPTION-READ] 95ff96fb6d81312738eaaecef0bbc954e598ecc15070d3b635d484681716d824

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ✅ PASS

Premise-level review of a50334e3f19f594011e3837e3e640305ed805001 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Before merging, check that the Slack MCP server answers a rate-limited call with an isError result or a JSON-RPC error, and doesn't exit. Item 1 rests on that.

What this change ships

Inventory (3 items) — 3 justified

Intent: let a script cron's Slack calls keep going after a rate-limited answer without signing in to Slack again and drawing another HTTP 429. This is a FIX.

  1. A call the server answers with a tool error now keeps that server for the run's next call. The script still gets the error. — justified
  2. A call that raises McpToolError (a new RuntimeError subclass) is now told apart from a call that got no answer. Its one consumer is KeptMcpServers.call_tool. — justified
  3. The cron spec now says that kept servers stop when a run is killed at its timeout, except a server that called setsid. — justified

[FIRST-PRINCIPLES-REVIEWED] a50334e

Description read

[DESCRIPTION-READ] 6717a2b8a865d7b854405612a1b0646b123dd7d441cbbb3c90cc51a03d33ccc8

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed a50334e3f19f594011e3837e3e640305ed805001 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] a50334e

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed a50334e3f19f594011e3837e3e640305ed805001 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] a50334e

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 5, 2026

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error split is correct. Only a reply whose id matched becomes McpToolError, so a reused server cannot be out of sync with the client. Disconnects, the 1000-line cap and malformed replies still stop the server. The tests pin both branches, and every check is green. Please answer the Design note: a server whose startup sign-in failed and that reports it through isError now fails for the rest of the run instead of restarting. That only lasts one run, but say in the spec whether you accept it. Approving.

@iamwhatever
iamwhatever enabled auto-merge (squash) October 5, 2026 16:16
@iamwhatever
iamwhatever merged commit 82869d9 into kirodotdev:main Oct 5, 2026
100 checks passed
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 5, 2026
@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

@iamwhatever Accepted. A server that reports a failed sign-in as a tool error is still answering, so it is kept, and its own retry rules decide when it signs in again. For the Slack server behind this change that beats a restart: after a failed sign-in it fails calls fast until Slack's Retry-After has passed, then signs in again on the next call. A restarted process would sign in again at once, and sign-ins close together are what drew the 429s in #16659. A server that never retries a failed startup sign-in stays failed until the run ends, and the next run starts it fresh. The spec does not say this yet.

unstablebrainiac added a commit to unstablebrainiac/KiroCrew that referenced this pull request Oct 5, 2026
…accepted

A server whose start-up sign-in failed and that reports it as a tool error is kept for the rest of the run since kirodotdev#16905. The spec now says that this is accepted and why: the server's own retry rules decide when it signs in again, the next run starts fresh servers, and restarting it after each error would sign in again at once.

Refs kirodotdev#16659
unstablebrainiac added a commit to unstablebrainiac/KiroCrew that referenced this pull request Oct 6, 2026
…accepted

A server whose start-up sign-in failed and that reports it as a tool error is kept for the rest of the run since kirodotdev#16905. The spec now says that this is accepted and why: the server's own retry rules decide when it signs in again, the next run starts fresh servers, and restarting it after each error would sign in again at once.

Refs kirodotdev#16659
unstablebrainiac added a commit to unstablebrainiac/KiroCrew that referenced this pull request Oct 6, 2026
…accepted

A server whose start-up sign-in failed and that reports it as a tool error is kept for the rest of the run since kirodotdev#16905. The spec now says that this is accepted and why: the server's own retry rules decide when it signs in again, the next run starts fresh servers, and restarting it after each error would sign in again at once.

Refs kirodotdev#16659
iamwhatever pushed a commit that referenced this pull request Oct 6, 2026
…accepted (#17121)

A server whose start-up sign-in failed and that reports it as a tool error is kept for the rest of the run since #16905. The spec now says that this is accepted and why: the server's own retry rules decide when it signs in again, the next run starts fresh servers, and restarting it after each error would sign in again at once.

Refs #16659
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants