Skip to content

perf(mcp-gateway): let a Bot's worker sessions share Crew's own MCP servers - #37695

Merged
buluoray merged 1 commit into
mainfrom
crew/kiro-crew-rust/worker-base-stubs
Nov 16, 2026
Merged

buluoray merged 1 commit into
mainfrom
crew/kiro-crew-rust/worker-base-stubs

Conversation

@buluoray

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A Bot's worker sessions use the shared copies of Kiro Crew's own MCP servers instead of each starting five of their own.

Every worker session starts its own kirocrew mcp-core, mcp-cron, mcp-guide, mcp-mate and mcp-work, even with the MCP gateway on. The overlay pass never rewrote the stock worker spec (kirocrew-mate-worker.json), and worker_overlay_spec skipped every server the base spec declares, so only servers a Bot's own file adds could be stubbed. A Bot's conductor chat already shares these servers.

Why it matters

A conductor hands work to several worker sessions at once, and each one pays for five Python servers: about 309 MB PSS per worker session and about 0.96 s before the five servers answer tools/list (measured below). Sharing cuts both for every worker after the first.

Not a goal

  • Changing what a worker may call. Its limits stay the governed merged spec's (excludedTools on KAS).
  • Sharing for workers on hosts other than KAS. They cannot apply a worker's merged limits per session, so they keep starting their own servers.
  • Making a cold start (no shared backend running yet) faster. See the measurements: that case is slower.
  • The fingerprint mismatch that turned sharing off after website-only commits; that is PR #37686. Without it, a gateway that outlived a commit refuses these stubs too, and they fall back to starting the server themselves.

What changed (motivation → approach → change)

Cause: the worker's base servers never got stubs. Change, reusing the conductor's path rather than a new one:

  1. mate_overlays.with_conductor_source now adds the stock worker spec next to the conductor spec, so the agents half writes <overlay_dir>/kirocrew-mate-worker.json with stubs under --agent kirocrew-mate-worker. Their launch argv equals the conductor stubs' apart from --agent, which is not a pool dimension, so a worker joins the conductor's backends (checked on a pod's real overlays).
  2. worker_overlay_spec returns those base-server stubs, shaped by the governed merged worker spec the same way it already shaped a Bot's added stubs: a server the merged spec drops, mutes or marks type: registry is left out, disabledTools comes from the merged spec, autoApprove is dropped. A worker whose Bot owns no file gets them too. A Bot file's added stubs are unchanged.
  3. session_servers._load_overlay_for_agent never answers kirocrew-mate-worker by name. Before, nothing was written under that name, so the by-name path was empty by accident. Now a file is there, and a host that passes no KAS session key would otherwise get the base stubs with the file's autoApprove and none of the session's limits.

docs/architecture/mcp.md ("Bot overlays") and the module docstring describe the new behaviour.

Session isolation

The pool already keys on launch inputs, not agent, and each forwarded call carries its own caller block. The new integration test runs two workers and a conductor on one real gatewayd and one shared backend, interleaves their calls in reverse order, and checks that the backend saw each caller's own session key. A run of the real kirocrew-work server under sharing was not done; that test uses the repo's fake backend.

Measurements

Rig: a pod (pod_up, its own home, port and gatewayd; the live gateway was not touched) running this branch with mcp_gateway.enabled and the five Crew servers stubbed. Each "session" starts the five servers at once exactly as the worker's spec (arm D, main today) or this branch's worker overlay (arm S) declares them, sends initialize and tools/list, and is ready when all five answered. kiro-cli/KAS was not in the loop: its own time before it starts the servers is the same in both arms and is not in these numbers. Both arms returned the same tool counts (81/9/12/13/7). 6 rounds per arm in ABBA order, 4 sessions per round, host load 7.5-10.4.

D: own servers (main) S: shared (this PR)
ready, median (24 sessions) 963 ms 461 ms
ready, p95 1067 ms 515 ms
A/A: odd vs even rounds, median 951 / 976 ms 461 / 461 ms
PSS of the session's own processes 309 MB 133 MB
processes per session 5 5 (stubs)
shared backends, counted once per host none 5 processes, 331-366 MB
errors 0 0

The ready-time gap (about 500 ms) is about 20 times the A/A spread (25 ms). Each extra worker session saves about 176 MB. The 5 shared backends are the same processes a running conductor already uses, so a worker adds no backends when a conductor is up.

Cold start, the first session after the gatewayd restarted (no backend running): S 1445, 1494 and 1678 ms against D 936, 977 and 1003 ms (3 samples each). So a worker that starts when no shared backend is warm is about 0.5-0.7 s slower than today. How often a worker meets a cold backend on a real host was not measured; a conductor keeps them warm while it runs, and the gatewayd evicts an idle backend after idle_timeout_secs.

Backwards compatibility

Compatible: workers on KAS get stubs where they had none, and every stub falls back to starting its server directly when the gatewayd refuses it, as stubs already do.

Reader: src/kiro_crew/mcp_gateway/session_servers.py:_load_overlay_for_agent -- chat|subagent -- non-KAS hosts read kirocrew-mate-worker by name; it now answers nothing, which is what they got before because no such overlay existed (test_without_a_kas_session_key_no_worker_stub_is_injected_by_name)
Reader: src/kiro_crew/acp/harness/kas.py:injection_server_names -- chat -- the KAS projection's fallback when it has no worker stub names; it now gets an empty set and projects every declared server, as before

Tests

  • test_the_worker_gets_the_base_servers_stubs_from_the_worker_overlay: the real rewrite writes a wrapped base stub under --agent kirocrew-mate-worker; the worker gets it with the merged disabledTools and no autoApprove, also without a Bot file; a muted or dropped base server is not injected.
  • test_a_worker_overlay_with_another_identity_gives_no_base_stub.
  • test_without_a_kas_session_key_no_worker_stub_is_injected_by_name: with the worker overlay on disk, a by-name lookup injects nothing.
  • test_worker_and_conductor_stubs_on_one_backend_keep_their_own_identity (integration, real gatewayd and stubs).
  • Updated: four refusal tests now assert "no stubs" instead of None, since a refused Bot file no longer stops the base stubs.

Proof: not provable -- other: prove.py reverting every production hunk makes these tests error at collection. Reverted by hand instead: without the source changes the three overlay tests fail (no worker overlay is written); with only the by-name guard reverted, the by-name test fails because the lookup injects the base stub carrying autoApprove: ["*"].

Ran locally: 128 test files that import the touched modules: 5329 passed. 7 failures in test_doctor_authored_agent_mcp_refs.py fail the same way on the unchanged tree.

Manual verification

On the pod, the rewriter wrote kirocrew-mate-worker.json with wrapped stubs for all five servers, and their argv matched the conductor overlay's apart from --agent. A real KAS worker session receiving the stubs end to end was not run.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

@buluoray
buluoray requested review from a team and KiroCrewCaptain as code owners November 15, 2026 20:58
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 15, 2026
@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 35f741b4174840481f2471f50b94b64e96f0737c touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 35f741b4174840481f2471f50b94b64e96f0737c — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

No design-level concerns: the change reuses the conductor's overlay path, still applies the worker's own merged limits (disabledTools, no autoApprove), and closes the new lookup by name in _load_overlay_for_agent.

Notes

  • A cold start is 0.5–0.7 s slower (S 1445–1678 ms vs D 936–1003 ms), and the PR doesn't measure how often a worker meets a cold backend. If workers often run with no conductor up, check that hit rate before relying on the speed-up.
  • No real KAS worker session was run end to end; the integration test uses the repo's fake backend. One run on a pod with the real kirocrew-work server would show that each caller keeps its own session key under sharing.

[DESIGN-REVIEWED] 35f741b

Description read

[DESCRIPTION-READ] 39d76dd93df7a648f6928dc5d4e419b2e99472d2971fa2e49eb60e973fc34563

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 35f741b4174840481f2471f50b94b64e96f0737c — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. Each item has a measured or named cause: per-worker server cost, and the autoApprove leak a by-name lookup would open. The one gap is a deleted pin that the description glosses over.

Notes

  • Item 4 — undeclared: the deleted pin said "A leftover ~/.kiro/agents/kirocrew-mate-worker.json overlay still answers the by-name lookup, as on main". The Reader line says non-KAS hosts got nothing before "because no such overlay existed". That holds for the default case but not for this leftover case, which now loses its stubs without being named.

What this change ships

Inventory (6 items) — 4 justified

Intent: let a Bot's worker sessions on KAS share Kiro Crew's five MCP servers through the gateway instead of each spawning its own. ADDITION (perf).

  1. Bot workers on KAS now share Crew's five MCP servers when the gateway is on — justified
  2. Workers whose Bot owns no file get the shared stubs too — justified
  3. A refused or mismatched Bot file no longer stops the base stubs (four refusal pins changed from None to empty) — justified
  4. Non-KAS hosts no longer get worker stubs by name, including from a leftover ~/.kiro/agents overlay — undeclared (leftover case)
  5. The first worker after a gatewayd restart now starts about 0.5–0.7 s slower — justified
  6. The stock worker spec now replaces any same-stem ~/.kiro/agents source in the rewrite and joins the fingerprint — undeclared

[FIRST-PRINCIPLES-REVIEWED] 35f741b

Description read

[DESCRIPTION-READ] 11944991e4e5fca009dbbc4854f101b5cd601fcc6f32b4768bca417139ca15f5

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 35f741b4174840481f2471f50b94b64e96f0737c and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 35f741b

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 35f741b4174840481f2471f50b94b64e96f0737c false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 35f741b4174840481f2471f50b94b64e96f0737c — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 35f741b

Verdict parsed from the review's SHA-scoped output markers for commit 35f741b4174840481f2471f50b94b64e96f0737c.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 35f741b4174840481f2471f50b94b64e96f0737c false-positive: <reason, 20 to 500 characters>

…ervers

A worker session started its own kirocrew-core/cron/guide/mate/work, because
the overlay pass never rewrote the stock worker spec and worker_overlay_spec
skipped every base server. The pass now rewrites kirocrew-mate-worker.json
next to the conductor's, and worker_overlay_spec returns its base-server
stubs (shaped by the governed merged spec: muted, dropped and registry
servers left out, disabledTools from the merged spec, autoApprove dropped)
plus the stubs the Bot's own file adds.

The worker overlay is never served by agent name: without the KAS session
key the lookup answers nothing, so a host that cannot apply a worker's
merged limits keeps declaring the servers itself.
@buluoray
buluoray force-pushed the crew/kiro-crew-rust/worker-base-stubs branch from 1fdf0fd to 35f741b Compare November 15, 2026 21:16
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 15, 2026
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider thinks the change is probably fine but estimates a 14% chance that this change introduces a defect, above the 12% limit for approving on its own.

Worker sessions on KAS now get the shared stubs for Crew's own MCP servers. The stock worker spec is rewritten through with_conductor_source: the file's name is ignored, its stubs carry --agent kirocrew-mate-worker, and the new file joins the fingerprint, so the first start after upgrading does a full rewrite and the overlay is pruned if the file goes away. worker_overlay_spec builds the base stubs only through _governed_stub, which leaves out servers the merged spec drops, mutes or registry-governs, takes disabledTools from the merged spec and removes autoApprove. Any error before the worker is verified still returns nothing. On hosts other than KAS, a lookup of kirocrew-mate-worker by name is now closed, so the new on-disk overlay (and its autoApprove) can't reach a session that lacks its merged limits. On KAS, the servers held back from the projection and the stubs injected come from one lookup. Calls on the shared backend are told apart by each call's caller block, which the new integration test checks. Each change has a test that fails without it, and CI is green. Remaining issues are stale docstrings and an unmentioned edge case with a leftover user-level sp

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 35f741b417 · 333 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

The review found no code defect on this head, but one design point needs a maintainer before merge.

The new guard in src/kiro_crew/mcp_gateway/session_servers.py:376 makes every non-KAS host return nothing for kirocrew-mate-worker by name. The maintainer decision recorded on #36370 asks for the opposite: a worker on Codex or Claude finds its stubs by name, so it can call work_report. Open PR #37352 implements that decision on the same lines of mate_overlays.py and mcp.md, and neither PR mentions the other. A maintainer should settle the order and the design (keep the guard, or leave the non-KAS branch to #37352) and post the decision here.

What the review checked on this head:

  • Base stubs and Bot-added stubs both go through _governed_stub, which drops autoApprove and takes mutes and disabledTools from the merged spec.
  • KAS subtracts exactly the stub names it injects.
  • Each worker keeps its own session token on the shared backend.
  • The rewriter signs and writes the stock worker overlay at both with_conductor_source call sites.
  • The related tests pass on the head tree, and removing the guard, the identity check or the base-wins check makes a test fail.

Done well: the ABBA measurements with an A/A spread, and the real-gatewayd test that proves two Bots' workers and a conductor share one backend while each call keeps its own caller key.

Optional:

@bolichen97

Copy link
Copy Markdown
Contributor

Review of 35f741b417: no code defect found. One design point needs a maintainer before this can merge, so I'm commenting rather than approving.

Design conflict (needs a maintainer call). The new guard in src/kiro_crew/mcp_gateway/session_servers.py:376 makes every non-KAS host get nothing for kirocrew-mate-worker by name. The maintainer decision on #36370 (option 2) says a Codex or Claude worker finds its stubs by that name so it can call work_report, and #37352 implements exactly that on the same lines of mate_overlays.py and mcp.md. Merging this one first turns #37352 into a no-op; merging #37352 first gets reverted by this guard. Neither PR mentions the other.

My read, for whoever decides: the guard is the safer default. A by-name lookup hands the stock worker overlay's stubs to any session running that spec, without that session's merged limits (the Bot's mutes, drops and disabledTools). So if the Codex worker fix is still wanted, it would be safer as a session-keyed lookup for the member-capable mirrored hosts (acp.session_mcp.session_mcp_projection already takes a session_key) than as the by-name path. That is a design choice, not mine to make.

Isolation checks (all hold on this head):

  • Identity: stubs carry no per-Bot identity; each call's caller block is built from the stub process's own env (mcp_caller / CallerContext.from_env), and the new integ test shows a conductor and two workers on one backend each keep their own session key.
  • Limits: base and Bot-added stubs both go through _governed_stub, which drops autoApprove, takes disabledTools from the merged spec, and skips muted, dropped or registry-governed servers. A Bot file cannot override a base server (base wins).
  • Fail closed: an unreadable execution record or worker kit returns None before any overlay is read; a missing, symlinked or wrong-name base overlay gives no base stubs, so the session starts its own private servers.

Tests. 61 test files that import the touched modules: 3312 passed, 15 skipped, 5 xfailed. I also wrote five throwaway checks (not pushed), all passing: a user ~/.kiro/agents/kirocrew-mate-worker.json with a planted server and autoApprove: ["*"] cannot reach the worker's stubs; a symlinked base overlay gives no stub; an unreadable record or kit returns None with the base overlay present; an owner-chat session key gets nothing; a worker of a Bot with no file does not see another Bot's added server. CI on this head: 79 pass, 18 skipped, none pending.

Optional (non-blocking): the stale docstrings and the missing worker twin of test_the_conductor_spec_gets_its_own_overlay_by_name that KiroCrewCaptain listed are still worth doing; in my check the stem replacement was not load-bearing (the stock file still won when I narrowed it), so that test would pin the outcome rather than the line.

AI-assisted, sent by Bolin Review bot

@bolichen97

Copy link
Copy Markdown
Contributor

Maintainer call on the conflict with #37352 / #36370: keep this PR's guard at session_servers.py:376 (a non-KAS host gets no kirocrew-mate-worker stub by name). A by-name lookup can hand the stub to a session that does not carry this session's restrictions; the session-key projection (session_mcp_projection) cannot. #37352 will move the Codex/Claude worker path onto that session-key lookup instead, so work_report keeps working there.

With that settled, nothing else blocks here from my side; I'll approve once #37352 is updated to match or the two are confirmed not to collide on the same hunk.

AI-assisted, sent by Bolin Review bot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants