Skip to content

perf: relay own MCP servers' tool lists from the gateway - #35833

Merged
bolichen97 merged 1 commit into
mainfrom
perf/mcp-early-entry
Nov 13, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
perf/mcp-early-entry

Conversation

@bolichen97

@bolichen97 bolichen97 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A new chat or sub-agent gets to its first prompt sooner, because Kiro Crew's own four tool servers list their tools without waiting for their own start-up.

An agent engine starts every MCP server of a session when the session starts and holds the first prompt until each has listed its tools. Kiro Crew's own servers (kirocrew-core, kirocrew-cron, kirocrew-guide, kirocrew-mate) each loaded their configuration and the platform before they could list, and kirocrew-core also read the agents folder and the configuration for its descriptions. On Linux the four needed about 0.9 s from spawn to the last list.

Why it matters

Every new chat, every sub-agent and every scheduled run pays this before its first prompt. It is a fixed wait the person sees as a slow start, and it grows when several sessions start at once.

Not a goal

  • No change to which tools a session may list or call. tools/call still reads the policy in the server and refuses there.
  • No change to the text of any tool description.
  • No cached tool list, in the gateway or on disk.
  • No change to the sandbox, or to how a runtime's process tree is read, recorded or killed.
  • Reading the macOS process tree without a pgrep and a ps per process is a follow-up, not part of this change.
  • No relay on a governed install, on a platform overlay, for a private member session, for a pooled backend or under an enforced harness: there the server builds its own list as before.

What changed (motivation → approach → change)

Goal → approach. The gateway is one warm process that already has the configuration, the platform and the agent specs loaded, and every own server already asks it for its tool policy before it lists. So the server asks once, earlier, and the gateway answers with the two answers the server would have built.

One commit: own servers relay their first two answers from the gateway.

  • mcp_listing_relay (server side): after the root-run refusal and before the configuration loader, the platform or the audit log is imported, one GET /api/session-tool-policy with a listing parameter, carrying the same internal credential, session key and session token as the policy read. On a list, the server answers initialize and tools/list and goes on starting; the handlers load where they always did. On anything else (no session identity or port in the environment, a refusal, a timeout, the gateway's "build locally", a malformed answer) it starts in the order it always did and builds both answers itself.
  • mcp_listing_gateway (gateway side): the server's declared list_tools behind the one filter within_policy, and initialize_result, run in a worker thread under that session as the request-local caller, restored on failure, nothing kept between requests. A description that reads a value only the gateway process holds (the sub-agent cap in force) is built as a tool server builds it, so the sentence is unchanged.
  • The gateway builds a list only on an install the sandbox seals. Building one imports listing modules into the gateway, which runs outside the sandbox, on a request a session can cause. So it is done only where the sandbox seals the package tree against a session's agent, by the sandbox's own classification (sandbox_tiers.install_root_targets_for, the one source; no second rule and no seal added): a packaged install, on every system. An editable or from-source checkout stays writable to the agent, so there the gateway answers "build locally" before any listing code runs and the server starts as on main. The classification is the install kind and does not read the sandbox setting: with agent.sandbox off, a packaged install is still answered a list and a source checkout is not. An answer that cannot be had counts as not sealed.
  • The gateway answers a list only where it is the server's own: the server finds on disk the same listing code as the gateway found when it started (the modules that build the lists and every module of the package those import by name; a test derives that set from their import statements, so a builder that starts to import a module fails until the stamp covers it; 86 files, under a millisecond to stat, once in the gateway as it starts and once in each server; the gateway reads it once, in a worker thread a startup hook starts, so nothing is read when the route module is imported or before the listener accepts requests, and a server that asks before it is in builds its own list; it is read at start and not at first use because a later stamp would vouch for what is on disk by then and not for the code the gateway loaded), reads the same data home and agents directory (device and inode), the platform is the standalone composition, no governance policy is installed and no profile is bound to the session, and the session is not a private member's. Otherwise it answers a reason and the server builds its own.
  • A session that is still starting: its servers ask before the start has registered the session, when the policy read answers 404 agent not resolved and a server lists everything. The session allocation records the agent of a start in flight (starting_agents, written only by that start, removed at registration and at every exit). Only a listing request reads it, and the gateway answers a list only when that agent's policy withholds none of the server's tools, which is the same unfiltered list. A plain policy read keeps its 404 in that window.
  • The discovery probe (kirocrew doctor, the dashboard's server list) no longer stops at tools/list for these servers: it asks for one more answer, so a server whose handlers cannot load probes as an error with its reason. A failed probe of any stdio server now shows the END of its stderr (its last whole lines, at most 500 characters). It reads at most 32 KiB, from the first byte, and redacts all of it before it takes the end, because the redactors know a private key by its first line: a server prints its start-up warnings first and the reason it stopped last, and the first 500 characters could be all warnings.

Measured on Linux, on packaged installs (aarch64, 32 cores; this branch and its base each installed non-editable into site-packages, the kind of install that is answered a list; one isolated gateway at a time, default backend). Taken before the last two changes to the gateway side (the install check and the stamp read at startup), which add no work to a relayed request:

What Base This branch n
Four own servers of a live session, started together: spawn to all four listed 913 ms 187 ms 5
New chat, time the gateway spends creating the session 2191 ms 1617 ms 5
Five new chats at once, the same, median per round 2620 to 3387 ms 2054 to 2800 ms 6 rounds of 5
The server's one request, quiet gateway: kirocrew-core / the other three 33 ms (p90 38) / 14 ms (p90 21) 20 each
The same during five chats at once median about 60 ms, p90 about 390 ms; 2 of 120 ran into the 1.0 s bound 120
Gateway's cheapest route during 20 server starts at once median 1.4 ms, max 30 ms median 1.4 ms, max 61 ms 1200

The caveat. With five chats starting at once, 2 of 120 requests (both kirocrew-core's) did not answer within the bound. Those two sessions took 3230 and 3484 ms, at the slow end of the base's range (2264 to 3535 ms): for them the relay cost time instead of saving it. The gateway's own answer times also have a longer tail while lists are built (max 61 ms against 30 ms).

The bound is 1.0 s (RELAY_FETCH_TIMEOUT_SECS), and these numbers are why: a request that answers inside it is sooner than building the list in the server (the list is out about 0.15 s after the answer, against about 0.9 s for the server's own start), and one that does not costs the bound on top of the old start. A shorter bound would turn the 0.4 to 0.9 s answers seen under load into fallbacks that are slower than waiting. Each timed-out request logs a warning; every other fallback logs one informational line with its reason.

A governed install builds locally. With a governance policy installed, a profile bound to the session, or a platform that is not the standalone composition, the gateway answers "build locally" and the server starts as before. No test claims identical bytes there.

macOS and Windows. The relay ran on both, against its base, on an earlier head: all four servers relayed in every chat, and the relayed bytes are the server's own; the figures are in the results comment on this PR. Those runs were on source checkouts. From this head on a source checkout builds its own lists, so those figures show what the relay does where it engages, not what a source checkout gets; runs on packaged installs of this head are to follow in a comment. That earlier head also carried a second commit that read the macOS process tree in process. It is taken out of this PR: with it, two tests of test/test_acp_client_shutdown_eof_grace.py failed on macOS, because a new child's name was recorded before the child had finished starting and the kill path, which needs the recorded name to read back, then left that child alive. The fix belongs in a change of its own. Nothing of that commit is in this head.

Backwards compatibility

Compatible: a policy read without the new parameter is answered exactly as before, and a server that gets no list starts and answers exactly as before.

One thing is no longer written. For a list the gateway relays to a session that is still starting, the audit log no longer gets tool_policy.agent_not_resolved and tool_policy.unfiltered_listing from the server, nor the gateway's session_tool_policy "agent not resolved" row for that request: the gateway read the policy, and it answers a list only when the policy withholds nothing. A server that builds its own list writes all of them as before.

Reader: src/kiro_crew/mcp_shared_policy.py:_resolve_tool_policy -- chat -- writer only; no code reads the row back (test_mcp_listing_relay.py::test_a_server_that_gets_no_list_has_loaded_nothing_and_answers_as_it_always_did)
Reader: src/kiro_crew/mcp_shared_listing.py:listable_tools -- chat -- writer only; no code reads the row back (test_mcp_listing_relay.py::test_a_client_reads_the_same_bytes_from_a_relayed_and_a_server_built_list)
Reader: docs/architecture/mcp.md -- release -- the description of both rows is updated in this PR

Tests

  • test/test_mcp_listing_relay.py: the bytes a client reads are the same from a relayed and a server-built list, for all four servers, with no policy, with excluded tools, with an include list and for a session that is not registered yet (two real processes); a server that gets no list has loaded no configuration loader, platform or audit module when the relay step returns, and answers what it always answered; kirocrew-core's gateway-built list equals its server-built list across backend, Tool Search, a pinned and an automatic sub-agent ceiling, and a parent with and without availableAgents; the gateway builds off the event loop, under the asking session, restores the caller on failure and keeps nothing; another session's key gets the refusal and no list; a plain policy read of a starting session keeps its 404; the route tells the builder a private member's session from that session's own execution record; the code stamp's set of imported modules equals the one read off the builders' import statements; the two-process rig waits the test's own bound, not the production one, for the gateway's answer and for each frame; a table over the install (packaged, source checkout, editable, the sandbox off, a classification that is unknown or raises) says which is answered a list and with which reason, and that no listing code runs for the others; the rig's gateway on the real source checkout answers "build locally"; no list is built before the stamp is read, loading the builder the route module imports reads no stamp, and the startup hook returns before the read is done.
  • test/test_session_starting_agent.py: the starting agent is readable until the session is registered, and the row is gone after a failed and after a cancelled start.
  • test/test_mcp_probe_past_listing.py: a server whose handler import fails turns the probe red, with the reason, also when it printed more start-up text than an error carries before it; a private key longer than the shown text is redacted whole; a stderr that never ends is read to a bound.
  • test/test_mcp_slim_entry.py: the in-process entry tests no longer read the test process's own stdin or keep its stdout.

Manual verification

  • Linux, packaged installs: the table above; a real sub-agent start relayed all four of its servers; kirocrew doctor on a scratch home shows a broken handler import as "the server listed its tools and then stopped before it could run one" with the import error.
  • macOS and Windows: the relay, the byte comparison and the doctor ran on two other machines against the base; see the results comment on this PR. The doctor's end-of-stderr text was run again on Windows on 0ed974e6e504 and shows the import error.

Related Issues

Part of #35785

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license.

@bolichen97
bolichen97 requested review from a team and KiroCrewCaptain as code owners November 12, 2026 12:32
@bolichen97

bolichen97 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: 4396d78
  • What rule changed: GET /api/session-tool-policy (dashboard/handlers/sessions_parts/tool_policy.py) takes one optional query parameter, listing. With it, and only for a request the route authenticated internally as the session it names, the answer also carries that session's initialize result and filtered tool list for one of Kiro Crew's own four servers, or the reason the server must build them itself. No deny rule, sandbox rule, mask or redaction changed. No new route, listener or credential.
  • Before -> after (what was allowed/blocked, now):
    • A policy read without the parameter is answered exactly as before, every refusal included.
    • Another session's key still gets 409 member_identity_unavailable and no list. A caller without internal authentication gets no list.
    • tools/call is unchanged: the server still reads the policy itself and refuses there.
    • A session whose start has not registered it yet: before, its policy read answered 404 agent not resolved, so its server listed every tool and wrote two audit rows. After, a listing request alone reads the agent that start recorded, and gets a list only when that agent's policy withholds none of the server's tools, which is the same unfiltered list. A plain policy read in that window still answers 404.
    • The audit log: for a list relayed to a session that is still starting, tool_policy.agent_not_resolved and tool_policy.unfiltered_listing are no longer written by the server, nor the gateway's session_tool_policy "agent not resolved" row for that request. The gateway read the policy and nothing went out unfiltered by default. A server that builds its own list writes all of them as before.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): a session's engine would cache a tool list that differs from the one its own server would have built: a tool shown that the policy hides, or hidden that it allows. A shown tool still refuses when called, because the call-time check did not move. No secret is in a tool list. No command breaks: any relay failure falls back to the server's own start.
  • Evidence checked (which sandbox result proves it): N/A for a sandbox result, no sandbox rule is touched. Evidence is the tests: test/test_mcp_listing_relay.py compares, in two real processes, the bytes a client reads from a relayed list and from a server-built one, for all four servers, with no policy, with excluded tools, with an include list and for an unregistered session; it also holds the refusal for another session's key, the unchanged 404 of a plain read in the start window, and the two audit rows when the server builds its own list. test/test_session_starting_agent.py holds that the starting-agent row is gone after registration, a failed start and a cancelled start.
  • No regression:
    • Tools still work (which tools/commands were run, result): on Linux, on both backends and with Tool Search on and off, initialize, tools/list and one tools/call of each own server, plus one model turn that calls a tool of each, were compared between the base and this branch: no difference in 88 rows. A sub-agent start relayed all four of its servers.
    • Backward compatible (old config / old data / old callers still work, how checked): no config or data shape changes. A server that does not send the parameter, and a gateway that does not know it, behave as before; the server treats an answer without a listing as "build locally".
    • Existing tests (which suites passed on this head): locally, 2501 tests in the files around every changed module passed three rebases before this head, and 1388 of them again two rebases before it; the rebases since brought, of the files this change touches, only docs/system-specs/modules/security.md. mypy src/kiro_crew, flake8, isort and the diff-scoped gates exit 0 on the first of those trees. CI on this exact head is running.
  • How to undo: revert this PR. The relay can also be left unused without a revert: a server whose environment carries no session token never asks.
  • Not run when this comment was written: macOS and Windows. Commit 2 (the macOS process tree read in process) had not run on a Mac.

@bolichen97
bolichen97 enabled auto-merge (squash) November 12, 2026 12:33
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 12, 2026
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision add3644d8d9e2c68d69724d08afcf42ca4a70424 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of add3644d8d9e2c68d69724d08afcf42ca4a70424 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: every case where the gateway can't answer falls back to the server's own start, and the route answers exactly as before when listing is absent.

Notes

  • Equivalence rests on a convention. as_a_tool_server() covers one value that only the gateway process holds (the sub-agent cap). A future description that reads other gateway-only state would be relayed with different text, and no test would fail. A test that compares the gateway-built list with the server-built list for all four servers would pin this.
  • The 1.0 s bound costs time under concurrent starts: 2 of 120 requests hit it, and those sessions got slower. The bound is a single constant, so it is easy to tune or remove if the tail grows.

[DESIGN-REVIEWED] add3644

Description read

[DESCRIPTION-READ] e8aa22451629b402e94b5fa7f42c12e335a48fa4d3716c760918a9187783e760

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of add3644d8d9e2c68d69724d08afcf42ca4a70424 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. The relay addresses a measured cost (913 ms → 187 ms spawn-to-listed). The only finding is a probe-diagnostics rider outside the Goal.

Notes

  • Item 6 — rides along: nothing in the Goal ("first prompt sooner") needs the probe to show "the END of its stderr… reads at most 32 KiB", and no defect is linked for it. Subtraction: delete _stderr_text, _last_lines and the _STDERR_* constants in mcp_discovery_probe.py, and keep the base proc.stderr.read(4096) / clean[:500].

What this change ships

Inventory (8 items) — 7 justified

Intent: an ADDITION (perf). New chats, sub-agents and cron runs reach their first prompt sooner because Kiro Crew's own four servers relay a tool list the gateway built.

  1. The four own servers answer initialize/tools/list with lists the gateway built, before they load config or the platform — justified
  2. The policy route takes a listing parameter and answers either a filtered list or a "build locally" reason — justified
  3. Lists are relayed only on a packaged install the sandbox seals; source checkouts build their own as on main — justified
  4. Lists are relayed only when the code and data-home stamps match and the session is standalone, has no governance and is not a private member's — justified
  5. A session still starting gets a relayed list when its agent's policy withholds nothing (starting_agents) — justified
  6. A failed stdio probe shows the end of a redacted stderr of up to 32 KiB instead of its first 500 characters — rides along
  7. doctor and the dashboard ping own servers after tools/list, so one whose handlers fail to load probes as an error — justified
  8. The sub-agent cap sentence is left out of relayed lists, so the description text stays the same — justified

[FIRST-PRINCIPLES-REVIEWED] add3644

Description read

[DESCRIPTION-READ] 86d74aa55012d198141c200d29f131ac2f0ad06c7676d92e4cfa6afef197550d

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of add3644d8d9e2c68d69724d08afcf42ca4a70424 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] add3644

False positive or not applicable? A repository writer can comment:
/ai-review override gpt add3644d8d9e2c68d69724d08afcf42ca4a70424 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed add3644d8d9e2c68d69724d08afcf42ca4a70424 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] add3644

Verdict parsed from the review's SHA-scoped output markers for commit add3644d8d9e2c68d69724d08afcf42ca4a70424.

False positive or not applicable? A repository writer can comment:
/ai-review override fable add3644d8d9e2c68d69724d08afcf42ca4a70424 false-positive: <reason, 20 to 500 characters>

@bolichen97

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: 34c8686
  • Since the note for 4396d78fdc19: no rule moved. Two fixes for that head's red checks: the listing code stamp joins its source paths with os.path.normpath (the stamp is the same bytes), and mcp_listing_relay.py is now in the list of modules test/test_mcp_call_site_auth_coverage.py scans, since it sends the internal credential; its one call site resolves to /api/session-tool-policy, a route already granted to internal callers.
  • What rule changed: GET /api/session-tool-policy (dashboard/handlers/sessions_parts/tool_policy.py) takes one optional query parameter, listing. With it, and only for a request the route authenticated internally as the session it names, the answer also carries that session's initialize result and filtered tool list for one of Kiro Crew's own four servers, or the reason the server must build them itself. No deny rule, sandbox rule, mask or redaction changed. No new route, listener or credential.
  • Before -> after (what was allowed/blocked, now):
    • A policy read without the parameter is answered exactly as before, every refusal included.
    • Another session's key still gets 409 member_identity_unavailable and no list. A caller without internal authentication gets no list.
    • tools/call is unchanged: the server still reads the policy itself and refuses there.
    • A session whose start has not registered it yet: before, its policy read answered 404 agent not resolved, so its server listed every tool and wrote two audit rows. After, a listing request alone reads the agent that start recorded, and gets a list only when that agent's policy withholds none of the server's tools, which is the same unfiltered list. A plain policy read in that window still answers 404.
    • The audit log: for a list relayed to a session that is still starting, tool_policy.agent_not_resolved and tool_policy.unfiltered_listing are no longer written by the server, nor the gateway's session_tool_policy "agent not resolved" row for that request. The gateway read the policy and nothing went out unfiltered by default. A server that builds its own list writes all of them as before.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): a session's engine would cache a tool list that differs from the one its own server would have built: a tool shown that the policy hides, or hidden that it allows. A shown tool still refuses when called, because the call-time check did not move. No secret is in a tool list. No command breaks: any relay failure falls back to the server's own start.
  • Evidence checked (which sandbox result proves it): N/A for a sandbox result, no sandbox rule is touched. Evidence is the tests: test/test_mcp_listing_relay.py compares, in two real processes, the bytes a client reads from a relayed list and from a server-built one, for all four servers, with no policy, with excluded tools, with an include list and for an unregistered session; it also holds the refusal for another session's key, the unchanged 404 of a plain read in the start window, and the two audit rows when the server builds its own list. test/test_session_starting_agent.py holds that the starting-agent row is gone after registration, a failed start and a cancelled start.
  • No regression:
    • Tools still work (which tools/commands were run, result): on Linux, on both backends and with Tool Search on and off, initialize, tools/list and one tools/call of each own server, plus one model turn that calls a tool of each, were compared between the base and this branch: no difference in 88 rows. A sub-agent start relayed all four of its servers.
    • Backward compatible (old config / old data / old callers still work, how checked): no config or data shape changes. A server that does not send the parameter, and a gateway that does not know it, behave as before; the server treats an answer without a listing as "build locally".
    • Existing tests (which suites passed on this head): CI on the previous head ran the whole backend suite on Linux and Windows with one failing test, the call-site guard above, now fixed. Locally on this head's tree: 3459 tests in the changed test files and the files around the changed modules pass, and mypy src/kiro_crew, flake8, isort, black, the portability scan and the diff-scoped gates exit 0. CI on this exact head is running.
  • How to undo: revert this PR. The relay can also be left unused without a revert: a server whose environment carries no session token never asks.
  • Not run when this comment was written: a real session start on macOS or on Windows. CI's Windows shards ran the backend suite on the previous head; the macOS suite did not run there. Commit 2 (the macOS process tree read in process) had not run on a Mac.

@bolichen97 bolichen97 added the ci:macos Run the on-demand macOS suite against this PR head label Nov 12, 2026
@KiroCrewCaptain

KiroCrewCaptain commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider raised 1 possible problem and estimates a 60% chance that this change introduces a defect; a person should check it.

The relay is carefully built: any answer other than a list falls back to the old start, and the session-still-starting window is handled with care. But it now has the unsandboxed gateway import and run the listing builder modules on every session start, on every host. The repo's own code (mcp_discovery_probe.py:1021-1028, mcp_discovery_managed.py:71-79) says this import must not happen on hosts where the sandbox works, because editable installs leave the package agent-writable. The code stamp only checks .py metadata one import deep, so a swapped .pyc or a deeper edit passes. The PR moves this trust boundary without acknowledging it. The boot-path finding is covered by the rule's own exception.

Things to check (possible problems the decider raised; at this confidence many turn out to be fine):

  • The relay makes the unsandboxed gateway import and run mcp_*_tool_list and every mcp_tools/catalog module. It does so on an agent-triggerable request path (any new session or sub-agent), on every host, including hosts where the sandbox works. On editable and source installs, an agent can swap a .pyc (src/kiro_crew/mcp_listing_gateway.py, high)

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit cb336b037b · 2758 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 12, 2026
@bolichen97
bolichen97 disabled auto-merge November 12, 2026 15:05

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. One test can fail under load, and it needs a fix before this can merge.

  1. test/test_mcp_listing_relay.py:1061 (with the audit count at line 1069). The child in _SERVER runs the real relay_before_boot, which calls fetch_listing(command) at src/kiro_crew/mcp_listing_relay.py:316 with no timeout. So the call uses the production RELAY_FETCH_TIMEOUT_SECS (1.0 s), and that default is bound when the function is defined. The rig gateway builds the whole body before it sends headers, and the first mcp-core request also pays a cold import of the listing module. On a busy CI runner with -n auto, that can take more than 1.0 s. Then fetch_listing returns the timeout result, the child builds its own list, assert asked["listed"] fails, and for the unregistered case the tool_policy.unfiltered_listing count is 2 instead of 1. The PR body's own load table already shows 2 of 120 requests hitting the 1.0 s bound. Smallest fix: in _SERVER, wrap mcp_listing_relay.fetch_listing so it passes timeout=FALLBACK_TIMEOUT_SECS. Patching the constant does not work, because the default is already bound. The timeout fallback keeps its own stubbed test, and the production bound stays as it is.

Optional:

  • The probe fix for #35785 sits inside the perf: commit. A separate fix: commit would follow one logical change per commit. Also, another contributor claimed #35785 shortly before this PR opened, so please settle that before it closes the issue.
  • No route test reaches the builder with a real member store, so a change to private_member=bool(member_store) at tool_policy.py:723 would not turn any test red. Also, test_a_refused_root_run_falls_back_before_the_data_home_is_touched could assert startup["relay"] == [].
  • code_stamp does not cover mcp_instructions.py or the cron constants module, so an edited dev checkout can relay stale description text until the gateway restarts. The body could say "a fixed set of listing sources" instead of "the same listing code".

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision labels Nov 12, 2026
@bolichen97

bolichen97 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor Author

Runs on macOS and Windows

Two other machines ran this PR against its base. Both ran head 34c868698e98; Windows took its timing rows on the head before it, 4396d78fdc19, whose source differs from it in one line of the code stamp. Model and kiro-cli were the same on both builds of a machine. Numbers are medians.

These runs were on source checkouts (edited after head add3644d8d9e). From that head on the gateway builds a list only on a packaged install, where the sandbox seals the package tree against a session's agent; a source checkout builds its own lists, as before this PR. So the relay rows below show what the relay does where it engages, not what a source checkout gets from that head on. Runs on packaged installs are to follow. The macOS tree-read rows no longer apply either; see the comment on why the goal changed.

Windows (Windows Server 2025, 8 vCPU, kiro-cli 3.2.0)

Base This PR
Own servers whose list was relayed no relay 4 of 4 in every chat: zero config on both backends, and 64 servers (120 relayed lists, none built locally)
The server's one request 50 to 110 ms; in the first chat of a gateway 120 to 600 ms
Relayed bytes against the server's own the same bytes, all four servers, two protocol versions
The gateway's session/new phase, zero config / 64 servers 2078 / 7890 ms 953 / 6547 ms
First token, zero config / 64 servers 7.42 / 13.84 s 5.97 / 12.93 s
Five chats at once, first token, zero config / 64 servers (15 chats each) 14.57 / 26.84 s 12.90 / 23.96 s
Smoke: first prompt with a fast server, 64 servers, the own tools pass pass, 3 of 3
kirocrew doctor with one own server's handlers broken exit 0, "12 tools" exit 1, names the server, shows the import error

macOS (Apple silicon, macOS 26.2, kiro-cli 3.2.1)

Base This PR
The two tree records of a new chat, no servers 279 ms, 16 pgrep and 6 ps 9 ms, no process started
The two tree records of a new chat, 64 servers 1389 ms, 66 pgrep and 31 ps 19 to 26 ms, no process started
Time the gateway spends creating the session, 0 / 64 servers 1.05 / 7.56 s 0.50 to 0.55 / 5.82 to 5.84 s
First token, 0 / 64 servers 5.38 / 13.12 s 5.0 / 11.1 to 11.8 s
Five chats at once, first text, 3 rounds 8.4 to 9.0 s 7.3 to 7.8 s
Own servers whose list was relayed no relay 4 of 4 in 12 of 12 chats; the request took 17 to 40 ms, the slowest 159 ms
Relayed bytes against the server's own the same bytes, all four servers, two protocol versions
Process names, in-process reader against ps 0 of 317 differ
Smoke: first prompt with a fast server, 64 servers, the own tools pass pass, 3 of 3 on both backends
kirocrew doctor with one own server's handlers broken says "12 tools" names the server as failed

What the macOS run found, fixed in head 0ed974e6e504; the first is moot now that the tree-read commit is out

  1. The in-process reader listed a child that had ended and was not collected yet. pgrep never listed one (3 of 3 in a direct test, and once in a real 64-server record). Such a child is now left out in the reader itself, with no process started for it, so a record is the set it was before this PR.
  2. With a broken handler the doctor named the server but not the reason. It showed the first 500 characters of the server's stderr, which on that Mac were two permission warnings; the import error came after them. A failed probe now shows the end of stderr.

Also seen, not a fault of this PR

  • The comparison script exits 1 on macOS: the child lists of the script's own ancestors differ, because pgrep leaves its own ancestors out. With pgrep -a the lists are equal, 8 of 8. The gateway's walk starts at the engine it started, which is never its own ancestor.
  • At 64 servers the PR's record holds 30 processes where the base's holds 29, and in 10 of 10 chats one or two recorded pids are gone a few seconds later (base: none, in 4 chats). The PR reads about a second earlier, so it sees processes that live only for the start. A kill path skips a pid that is gone, and signals a live one only when its start id and name read back as recorded.
  • The smoke case "a server that is still loading at the first prompt" fails on both builds, on both systems. It is main's; perf: send the first prompt without waiting for user MCP servers #35722 addresses it and is in neither build.
  • A chat that calls a tool of the server with broken handlers gets "MCP tool call failed: Not connected" on both builds: the list is out before the handlers load on both.

Not run

  • The two fixes of the current head on a Mac or on Windows. The tree comparison, one 64-server record and the doctor line are to be run again on it.
  • On Windows the timing, five-at-once and smoke rows are from 4396d78fdc19; on 34c868698e98 only the relay, the bytes, the doctor and two first-prompt rows ran.
  • A fallback inside a real chat (a gateway that answers late, a governed install, a private member's session), an enforced harness and a pooled backend. No fallback line was logged in any chat; that path ran only in controls with the gateway port taken out.
  • Five chats at once with 64 servers on the Mac; a broken handler in a server other than the guide server; smoke cases other than the four named.

@bolichen97

bolichen97 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor Author

Reply to the review of 15:17Z, on head add3644d8d9e (edited to match this head: the PR is now one commit, see the comment on why its goal changed).

  1. Fixed. The child server of the two-process rig now waits the test's own bound for the gateway's answer, and for each frame on its stdin; the two in-process fetch tests do the same. The production bound and its stubbed test are unchanged. With a rig gateway that takes 1.5 s the relayed case passes, and fails again when the child is put back on the production bound. The other tests of this PR that start a real child (test_mcp_probe_past_listing.py, test_mcp_listing_import_budget.py, test_mcp_slim_entry.py) give it no session token, so that child never asks the gateway and the fetch bound is not in play.
  2. The probe fix stays in the perf: commit. It is the relay's own precondition: a server may list before its handlers load only if the probe goes past the list. The body says Part of #35785, so this PR does not close the issue, and a note for its assignee is on the issue.
  3. Both test points taken. A route test where the session's own execution record names a member's store and the route's real member scope reads it; private_member=False on that line turns it red. And startup["relay"] == [] in the refused-root test.
  4. code_stamp. It now covers the modules that build the lists and every module of the package those import by name, 52 of them, mcp_instructions.py and cron_service/model.py among them, and a test derives that set from the builders' import statements. It reaches one import deep; the body and the spec say so.

Also in this head: a failed probe shows the end of the server's stderr, redacted whole before it is cut.

And, from the review lanes on cb336b037bb6: the gateway reads its listing code stamp in a startup hook, off the loop, and nothing at import; and it builds a list only on an install whose package tree the sandbox seals against a session's agent (the sandbox's own classification), so an editable or from-source checkout builds its own lists as before.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 12, 2026
@bolichen97

bolichen97 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: add3644
  • Since cb336b037bb6, two changes, both narrowing:
    • Where a list is built. Building a list imports listing modules into the gateway, which runs outside the sandbox, on a request a session can cause. Before: on any install. After: only where the sandbox's own classification (sandbox_tiers.install_root_targets_for) says the package tree is sealed against a session's agent, which is a packaged install; checked first, before any listing code is imported or run; an answer that cannot be had counts as not sealed. An editable or from-source checkout, which the sandbox leaves writable, is answered "build locally" and its servers start as before this PR. No seal, mask or refusal was added to the sandbox and its scope is untouched; the code stamp is not extended to bytecode. With agent.sandbox off the classification is unchanged (it is the install kind): a packaged install is answered a list, a source checkout is not.
    • When the code stamp is read. Before: when the builder module was imported, as the gateway starts. After: by a startup hook, in a worker thread; until it is in, every server is answered "build locally".
  • Since 7fe1e0427905: the second commit (the macOS process tree read in process) is out of this PR, so nothing here touches how a process tree is read, recorded or killed.
  • Since the note for 34c868698e98, in the relay commit: no deny rule, sandbox rule, mask or route moved. Two things changed that touch what a person is shown or what the gateway vouches for:
    • The text a failed server probe shows from that server's stderr (doctor, the dashboard's server list). Before: the first 500 characters of the first 4 KiB, redacted. After: the probe reads at most 32 KiB from the first byte, runs the same two redactors over ALL of it, and only then shows the last whole lines, at most 500 characters. The text is never cut at its start before redaction, because the redactors know a private key by its first line; test_a_long_private_key_in_stderr_is_redacted_whole_before_its_end_is_shown holds that.
    • The listing code stamp covers more files (the modules that build the lists and the 52 modules those import; one more than on cb336b037bb6, because a list builder on the newer base imports agent_sdk/mcp_problems.py), so the gateway declines to vouch for a list in more cases, never in fewer.
  • What rule changed: GET /api/session-tool-policy (dashboard/handlers/sessions_parts/tool_policy.py) takes one optional query parameter, listing. With it, and only for a request the route authenticated internally as the session it names, the answer also carries that session's initialize result and filtered tool list for one of Kiro Crew's own four servers, or the reason the server must build them itself. No deny rule, sandbox rule, mask or redaction changed. No new route, listener or credential.
  • Before -> after (what was allowed/blocked, now):
    • A policy read without the parameter is answered exactly as before, every refusal included.
    • Another session's key still gets 409 member_identity_unavailable and no list. A caller without internal authentication gets no list.
    • tools/call is unchanged: the server still reads the policy itself and refuses there.
    • A session whose start has not registered it yet: before, its policy read answered 404 agent not resolved, so its server listed every tool and wrote two audit rows. After, a listing request alone reads the agent that start recorded, and gets a list only when that agent's policy withholds none of the server's tools, which is the same unfiltered list. A plain policy read in that window still answers 404.
    • The audit log: for a list relayed to a session that is still starting, tool_policy.agent_not_resolved and tool_policy.unfiltered_listing are no longer written by the server, nor the gateway's session_tool_policy "agent not resolved" row for that request. The gateway read the policy and nothing went out unfiltered by default. A server that builds its own list writes all of them as before.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): a session's engine would cache a tool list that differs from the one its own server would have built: a tool shown that the policy hides, or hidden that it allows. A shown tool still refuses when called, because the call-time check did not move. No secret is in a tool list. No command breaks: any relay failure falls back to the server's own start.
  • Evidence checked (which sandbox result proves it): N/A for a sandbox result, no sandbox rule is touched. Evidence is the tests: test/test_mcp_listing_relay.py compares, in two real processes, the bytes a client reads from a relayed list and from a server-built one, for all four servers, with no policy, with excluded tools, with an include list and for an unregistered session; it also holds the refusal for another session's key, the unchanged 404 of a plain read in the start window, and the two audit rows when the server builds its own list. test/test_session_starting_agent.py holds that the starting-agent row is gone after registration, a failed start and a cancelled start.
  • No regression:
    • Tools still work (which tools/commands were run, result): on Linux, on both backends and with Tool Search on and off, initialize, tools/list and one tools/call of each own server, plus one model turn that calls a tool of each, were compared between the base and this branch: no difference in 88 rows. A sub-agent start relayed all four of its servers.
    • Backward compatible (old config / old data / old callers still work, how checked): no config or data shape changes. A server that does not send the parameter, and a gateway that does not know it, behave as before; the server treats an answer without a listing as "build locally".
    • Existing tests (which suites passed on this head): locally on fbf839fedd59, of which this head is a rebase with the same added and removed lines: 1303 tests in the changed test files, the probe's, the startup hooks', the shutdown tests and the base's new server-wait tests pass; mypy src/kiro_crew, flake8, isort, black, the portability scan, the determinism scan and the diff-scoped gates exit 0. A table test covers the install kinds (packaged, source checkout, editable, the sandbox off, a classification that is unknown or raises); it and the two-process rig's source-checkout case fail when the install check is removed. On Linux a packaged install of this change, built before its last test lines and the one added stamp row, relayed all four servers of a real session (four lists in 182 ms against 909 ms built by the servers). CI on this exact head is running.
  • How to undo: revert this PR. The relay can also be left unused without a revert: a server whose environment carries no session token never asks.
  • Not run when this comment was written: this head on macOS or Windows, on a packaged install or a source checkout; a real gateway on a source checkout on Linux (the rig's source-checkout case stands for it).

@github-actions github-actions Bot removed the readiness: checking Automated validation is still running label Nov 12, 2026
@github-actions github-actions Bot added the readiness: passed Eligible automated validation passed for the current revision label Nov 12, 2026
@bolichen97

bolichen97 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor Author

Why the goal of this PR changed (this comment first asked not to merge head 7fe1e0427905; the push it announced has landed).

At the maintainer's request this PR now carries the relay only. The head is one commit, add3644d8d9e (the first relay-only head was cb336b037bb6; the head since answers its two review-lane findings). The second commit, which read the macOS process tree in process, is taken out.

With that commit, two tests of test/test_acp_client_shutdown_eof_grace.py failed on macOS. The cause is a race in how a child process is recorded: the in-process reader recorded a new child's name before the child had finished starting (a launcher that execs the real program changes the name a moment later), and the kill path, which needs the recorded name to read back, then left that child alive. The fix for that belongs in a change of its own. The title, the Goal line and "Not a goal" are edited to match.

In the results comment above, these macOS rows were measured with that commit and no longer apply to this PR:

  • "The two tree records of a new chat", both rows.
  • "Process names, in-process reader against ps".
  • Of "Time the gateway spends creating the session" and "First token": the part that came from the tree read, about 0.27 s with no servers and about 1.4 s with 64. The relay's share of those rows stands.
  • Under "What the macOS run found": point 1, and the two tree-record notes under "Also seen".

The relay rows, the byte comparison, the smoke rows, the doctor rows and every Windows row stand.

@bolichen97 bolichen97 changed the title perf: relay own MCP tool lists and read the macOS tree in process perf: relay own MCP servers' tool lists from the gateway Nov 12, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running labels Nov 12, 2026
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Follow-up on the review of 15:17Z, checked on head cb336b0.

  1. Rig child bound to the 1.0 s production timeout: fixed. _SERVER now wraps fetch_listing and relay_before_boot with the test's own wait.
  • Optional notes from that review: the member-store route test and startup["relay"] == [] were added. The code stamp now covers mcp_instructions.py and cron_service/model.py. The note about the probe fix in the perf: commit still stands (see below).

I found no code bug on this head. It cannot be approved yet, though, and two things need a person.

For the security review: the security lane's blocking finding at src/kiro_crew/mcp_listing_gateway.py:147 has no disposition. The gateway now imports listing modules in its own process. On an editable install, code_stamp checks the .py files but not the .pyc. As far as I can see, the same class is already open on main: slack/gateway.py imports mcp_cron at module scope, and a dashboard route lazily imports mcp_core. So this PR newly runs only the guide and mate listing modules in the gateway. A security owner should decide between two options. One is to record this as the existing editable-checkout residual. The other is to relay only where the install-root seal applies. Either way, the comment in the SandboxUnavailableError branch of mcp_discovery_probe.py, which says importing these modules in the gateway beats the spawn's isolation, is now out of date.

The no-new-work-on-gateway-boot-path BLOCK at src/kiro_crew/mcp_listing_gateway.py:41 (code_stamp() runs at import) has no override or reply. The PR body says why the stamp cannot be deferred and gives its bound, which is what the rule's exception asks for. So a maintainer override that cites the body would clear it. The other way is to take the stamp at readiness and decline to relay until it is set.

PR Readiness is red on this head because of these two lanes. The other reds look unrelated: Analyze failed on a rate-limit annotation, and a macOS CPU-timing test failed in a file this PR does not touch.

Done well: any answer other than a verified list sends the server back to its own start. The two-process rig shows byte-identical lists for all four servers.

Optional:

  • The doctor: the MCP handshake probe reports ok for a server whose tool handlers fail to import #35785 probe fix and the stderr tail change are still in the perf: commit. Listing before handlers has been on main since perf: list MCP tools before loading tool handlers #35293, so the probe fix does not depend on the relay. A second fix: commit would keep one logical change per commit. Please also settle the claim on the issue with the other contributor.
  • test_the_code_stamp_reaches_every_module_a_list_builder_imports does not tie _LISTING_BUILDERS to mcp_entry.EARLY_ANSWERS. The test stays green if the mcp_guide_tool_list.py row is dropped. Asserting that each EARLY_ANSWERS module is in the builder set would close that gap.
  • The Backwards compatibility section could say that the listing builders now run inside the gateway process. "No change to the sandbox" can read as no change at the trust boundary.

@KiroCrewCaptain
KiroCrewCaptain dismissed their stale review November 12, 2026 23:04

Withdrawn: every item in this review is fixed or answered on the current head (see the follow-up comment). Other holds on this PR are listed there.

An agent engine starts every MCP server of a session at session start
and holds the first prompt until each has listed its tools. Kiro Crew's
four own servers each loaded their configuration and the platform, and
kirocrew-core its live descriptions, before they could list.

A server now asks its gateway once, before it loads any of that, on the
policy route it already reads, with the same credential, session key
and session token. The gateway answers the server's initialize result
and filtered tool list, built by the functions the server would have
called, in a worker thread under that session, with nothing kept. Any
other outcome leaves the server on the order it always started in.

The gateway builds a list only on an install whose package tree the
sandbox seals against a session's agent, by the sandbox's own
classification: building one imports listing modules into the gateway,
outside the sandbox, on a request a session can cause. A packaged
install is answered a list; an editable or from-source checkout builds
its own, as before.

The gateway answers a list only where it is the server's own: same
listing code on disk (the modules that build the lists and every module
those import, read once by a startup hook in a worker thread and not at
import), same data home and agents directory, standalone
platform, no governance policy or bound profile, not a private member
session. For a session whose start has not registered it yet, the
allocation records the starting agent for that one request, and a list
is answered only when the policy withholds none of the server's tools,
which is the unfiltered list a server builds in that window. A plain
policy read keeps its 404 there.

The discovery probe no longer stops at the list for these servers: it
asks for one more answer, so a server whose handlers cannot load probes
as an error with its reason. That reason is the last thing a server
writes, so a failed probe now shows the end of its stderr, not the
start.
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 12, 2026
@bolichen97
bolichen97 merged commit 9d6325f into main Nov 13, 2026
119 of 134 checks passed
@bolichen97
bolichen97 deleted the perf/mcp-early-entry branch November 13, 2026 01:01
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:macos Run the on-demand macOS suite against this PR head

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants