Skip to content

perf: send the same small request on every install and in a long chat - #35756

Merged
bolichen97 merged 2 commits into
mainfrom
perf/small-request-every-install
Nov 13, 2026
Merged

bolichen97 merged 2 commits into
mainfrom
perf/small-request-every-install

Conversation

@bolichen97

@bolichen97 bolichen97 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Kiro Crew sends the same small request on every install and keeps it small in a long chat.

Two things make a request bigger than it needs to be, and both ride on every request.

  1. When kiro-cli is found through PATH only, or is named by KIROCREW_KIRO_BIN, all of Kiro Crew's own tools stay in every request: 125 tools and 243,046 characters against 15 tools and 93,174 on a standard install.
  2. The [REPLY FORMAT RULES] block is appended to every message, 1,347 characters each time, although the engine replays its own history and already holds it.

Why it matters

The first costs about 54k tokens on every request of every session on such an install: a request 2.6 times as big, paid in model time and in credits. KIROCREW_KIRO_BIN is what Kiro Crew's own note tells a PATH-only install to set, so following the advice does not help today.

The second makes every chat grow twice as fast as it has to. A short turn adds about 2,550 characters of history, of which more than half is the repeated block.

Not a goal

  • Changing the pin, the known install directories, or which file a session runs.
  • Running any file outside the sandbox that is not run outside it today.
  • A new seal, mask or sandbox rule. No sandbox code changes.
  • Lowering the 2.27.0 floor.
  • Changing the scrub that stops other content from minting the [REPLY FORMAT RULES] marker, the [RUNTIME] and [PROJECT] blocks, or the agent's system prompt.
  • A rebuild-and-retry on the surfaces that have none today (channels, crons, tasks), or for a refusal that arrives after a turn already ran a tool.
  • Shorter tool schemas.

What changed (motivation → approach → change)

1. Every install gets the small request. Kiro Crew's own tools now stay out of the request (17 tools instead of 128) when kiro-cli is named by KIROCREW_KIRO_BIN or found only on PATH, not just on the pinned install. The decision is made on the file the session is about to run (src/kiro_crew/kiro_cli.py, mandatory_mcps_drop_allowed).

The rule exists because an engine below 2.27.0 can refuse a request whose tools list changed under a signed thinking block. The version of the file the session runs answers that for every install:

The spawn runs Version read Before After
the pinned install or its kiro-cli-chat inside the sandbox at the configured tier, as on main since #35731 defers at >= 2.27.0 unchanged
the operator's KIROCREW_KIRO_BIN the same read: an absolute override is the pin always resident defers when new enough
a kiro-cli found only through PATH new: inside the strict sandbox, in the background always resident defers once the answer is in
anything else (a pod's bundle, a sibling nobody probed) none resident resident

Neither new shape derives a sibling, so a POSIX kiro-cli below 3.0.0, a launcher, stays resident there.

A PATH file can sit in a directory an agent writes to, so one rule governs its read (src/kiro_crew/kiro_cli_path_probe.py): the probe never has more reach than the session's own launch of the same file. It runs through the readiness probe's own runner for an unverified candidate (kiro_prerequisite._run_process): the existing chokepoint, sandboxed_spawn_argv(mode="strict"), behind the system env entry, with that probe's credential-free environment, its output captured up to the runner's 64 KB bound and its process tree ended on a timeout. One opt-in keyword on that runner, require_crew_sandbox (default off; no existing caller passes it), refuses the spawn with no child started when the wrap did not put it in Kiro Crew's own sandbox. It is not run on Windows, which has no sandbox backend. It is never waited for: it starts at gateway boot (src/kiro_crew/slack/gateway_runtime/process_entry.py) and on a spawn's first miss. Only the file's own answer is cached, by path and mtime: a read that did not run or was cut off (a timeout, a sandbox that could not be built, more output than the bound) is not kept, so the next spawn asks again. Every failure and every missing answer keeps the full list.

The floor is measured, so a refusal above it is handled. The first request refused for a changed tools list on a kiro-cli backend is remembered for the life of the process. From then on every spawn keeps Kiro Crew's servers resident, on the pinned install too (the one way the table's first row changes), a warm-pool process started earlier is discarded at claim (src/kiro_crew/session_pool.py), and the dashboard's existing one-time rebuild of the refused conversation starts on the full list. The refusal's wording is read in acp/thinking_binding_errors.py, moved whole out of transport_errors.py, which was at the 2,000-line cap.

2. A chat grows half as fast, on the harnesses whose compaction Kiro Crew sees. There the engine is sent the reply-format rules once per conversation instead of with every message. The built message does not change: it still carries the block, as on main. On a turn with no member essentials, build_message binds the block as the provider's delivery envelope (EssentialDelivery, the send-once writer the member essentials already use), and the provider takes the block out of the prompt at the wire while this conversation has acknowledged it (context_assembly/turn_message.py: 42 added lines, no changed line; essential_delivery.py and every provider stream are untouched).

Turn Reply-format rules on the wire
kiro, kas, codex, claude, with no member essentials once per engine conversation, and again as below
a chat with a crew member (a turn carrying member essentials) every message, as today
opencode, goose, pi, copilot, strands, deepseek every message, as today
not known (no provider on the turn, one that does not say, one with no delivery) every message, as today

There is no second record of what was sent. The delivery is the one place that knows when a conversation may have lost it, and it decides at the send:

What happens Why the next prompt carries the block again
the turn that carried it did not complete (cancelled, failed, no answer) no acknowledgment; the delivery voids the attempt
a start, a reload, a new provider the provider's start invalidates; a new provider has a new delivery
a new engine session, a fresh conversation, a replaced client or process the acknowledgment names another conversation
/clear, /compact, a compaction Kiro Crew starts, a cancel invalidated before the send
a native clear, a compaction or an agent switch seen on a turn's stream the delivery's own invalidating events
the rules' text changed another digest
a compaction in a turn the agent started by itself the bind is forced

The bind is forced for that last event only, the one the delivery cannot see: such a turn is read without sending a prompt, so no delivery stream sees its compaction; it reaches only the session's re-injection flag, which the builder is handed. A new session is not forced, because the delivery sees it.

The first row of the first table is main's own set for the same question, ACP_BACKENDS_SENT_IMAGE_RECORD, under the name of what it means, ACP_BACKENDS_COMPACTION_REPORTED (one list, defined from it): the harnesses whose compaction, started by Kiro Crew or by the harness itself, sends a frame this client reads. The builder asks the turn's provider (LLMProvider.reports_compaction, False unless the harness is in the set). opencode, goose and pi compact on their own and send nothing Kiro Crew reads, so they keep the block on every message.

Limit. A chat with a crew member is unchanged by this PR: the delivery holds one envelope and the member's essentials take it, so every prompt there still carries the 1,347-character block. A second envelope slot in the delivery is the follow-up that would cover it. The sizes below are for a chat with no member.

Specs, operator help and the packaged guides say the same: docs/architecture/mcp.md and context-management.md, six files under docs/system-specs/modules/ (harness-parity.md and providers.md among them), config/sections_agent.py, config-baseline.json, and configuration.md, coming-from-kiro-cli.md and followup-suggestions.md under src/kiro_crew/docs/.

Sensitive change review

This diff touches none of the paths the Sensitive Change Review gate watches, and it changes a rule that sits beside the install-trust boundary, so the template is filled in here and the change waits for a sign-off from a maintainer who is not the author.

The choice, in two lines: a PATH-only kiro-cli is version-checked only inside the strict sandbox, never outside it, and never waited for; an override takes the answer the pin's own read already gives, which on main runs inside the sandbox at the configured tier. Anything that cannot be answered that way keeps today's full tool list.

  • What rule changed: which kiro-cli installs let Kiro Crew's own MCP servers defer under Tool Search; one new sandboxed --version read of a PATH-only kiro-cli; and one opt-in keyword on the readiness runner (_run_process(require_crew_sandbox=True)), which only refuses a spawn and changes nothing the sandbox seals, masks or wraps.
  • Before -> after: only the pinned install could defer -> the override and a PATH-only install can too, each judged by its own version; a refusal turns deferral off for the process.
  • Worst case if wrong: an engine that should have kept the tools resident defers them and refuses one request; that refusal is remembered, the conversation is rebuilt once on the full list, and no later session defers.
  • Evidence checked: the read's argv, mode, environment and the keyword are pinned by test_kiro_cli_path_probe.py; no credential or proxy variable reaches it; a wrap outside Kiro Crew's sandbox is refused before any spawn, on POSIX and on Windows, and the default path still spawns (test_kiro_prerequisite_process.py, red with the check removed); a planted file that prints four times the bound keeps 64 KB in the gateway and one that prints without end is cut off and its process group ended, both "unknown" and not cached; a PATH file is never read on the pin's route (test_tool_deferral_install_shapes.py). No other read of, or wait on, a child is in this diff.
  • No regression:
    • Tools still work: smoke cases 1, 10 and 12 pass before and after on both backends, for the pinned and the override shape (table below).
    • Backward compatible: the pinned install, the pod bundle and an operator ASBX_KIRO_MANDATORY_MCPS behave as before; nothing main accepts is refused.
    • Existing tests: on a0e61957c7, this branch before its last rebase onto main (the two commits are patch-identical to the pushed ones), 11,761 tests pass across 163 named test files: this PR's own, their neighbours, every test file of the readiness runner and of its other callers, the harness-parity, backend-card and provider tests, and every test file that keeps a frozen inventory or contract of a module this PR touches. Five more could not pass on the test host: main's own in test_kas_cli_floor.py needs os.pidfd_open, which that host's Python lacks, and four (test_harness_parity.py::test_is_kiro_cli_is_positive and three in test_acp_copilot_backend.py) are refused with the same message because that host's home holds a .mcp.json; the first of those four was run on main's tree there and fails the same way. Two tests that pinned "an override never defers" pin the new rule.
  • How to undo: revert the first commit, or set ASBX_KIRO_MANDATORY_MCPS in the gateway's environment, which every spawn honours before any version check.

Environment handed to the read on Linux, by name: DBUS_SESSION_BUS_ADDRESS, HOME, LANG, LC_ALL, LOGNAME, NO_COLOR, PATH, TEMP, TERM, TMP, TMPDIR, USER, XDG_RUNTIME_DIR.

OS What mode="strict" does for this read
Linux the namespace launcher; a host with no backend, or an unsandboxed opt-in, is not run and keeps the full list
macOS the seatbelt profile; the env entry keeps a file named kiro-cli off the path that delegates to the harness's own sandbox
Windows no sandbox backend exists, so the read never runs and a PATH-only install keeps the full list; KIROCREW_KIRO_BIN is the way to the small request there

Verified here: everything above, on Linux. Not verified here: the wall time of the sandboxed read on a real gateway (a gateway started from inside an agent session is itself sandboxed, so the read declines there and the full list is kept; the gateway logs the time of each read), the read under the macOS sandbox, a live refusal on an engine above the floor (none could be produced on 3.2.1; the test feeds the provider's own wording), and the chat_runtime_sharing preview, where a rebuilt conversation can still land on a shared process that started before the refusal.

Backwards compatibility

Compatible: no accepted input is refused, renamed or removed. An install that kept Kiro Crew's tools resident may now defer them, which the engine already does for every other server and for the pinned install; an operator's ASBX_KIRO_MANDATORY_MCPS still decides before anything else.

Removes nothing: the reply-format rules are still delivered to every engine conversation, and every built message still carries them. On kiro, kas, codex and claude, in a chat with no member, the engine is sent them once instead of on every message, and again whenever the conversation may not hold them; on every other harness and in a chat with a crew member they are sent on every message, as before.

Tests

  • test/test_tool_deferral_install_shapes.py: each install shape (override new, old, unreadable, a 2.x launcher, another file spawned, an override that is not the pin, a replaced override, read once per file identity through main's sandboxed probe; PATH-only new, old, not answered yet, a 2.x launcher, a file the resolver did not choose, a file changed between read and spawn; no install); the provider's refusal is typed and remembered on both kiro backends and the next spawn names every Kiro Crew server; any other refusal and another harness's refusal are not remembered; a warm-pool process started before the refusal is discarded at claim.
  • test/test_kiro_cli_path_probe.py: a miss answers "unknown" at once and starts one background read; the read goes through the readiness runner with one file, strict mode, require_crew_sandbox and the allowlisted environment; a non-zero exit, an unparsable line and Windows answer "unknown" and that answer is kept; a spawn refused for not being in Kiro Crew's sandbox, a missing wrapper, a timeout, a wrapper ended by a signal, a sandbox refusal and output at the bound leave no answer, so the next caller asks again (on Linux the launcher reports a file a signal killed as exit 1, so that one run is kept as "no version"); through the real runner, a planted file that prints past the bound and one that prints without end; a replaced file is read again; on Windows the log line says nothing was run and names no sandbox.
  • test/test_kiro_prerequisite_process.py: require_crew_sandbox refuses an unconfined wrap before any spawn and removes the wrap's launcher, refuses on Windows where the wrap is skipped, lets a spawn inside Kiro Crew's sandbox through, and the default still runs the spawn it ran before.
  • test/test_acp_tool_search_spawn_env.py: the two tests of the override are rewritten to the new rule; the pinned-install cases are untouched and pass.
  • test/test_reply_format_once.py: every assertion reads the prompt the transport was handed, through the real AcpProvider and AcpSessionProvider. Sent once and not again; the prompt without the block is the built message minus the block; a turn that did not complete (cancelled, no answer, cut off, raised) sends it again; changed rules send it again. A table of every event that can drop what a conversation was sent, 22 rows over both providers (a compaction seen on the stream, a native clear, an agent change, /compact, /clear, a compaction Kiro Crew starts, a cancel, a start, a reload, a fresh conversation, a replaced process, another engine session): the next prompt carries the rules and the one after does not. Acknowledged, then /clear: it comes back. A compaction in a turn the agent started: forced. A new-session flag alone: not sent again. A table over every selectable harness on both providers: in the set once, outside it every message. A provider that does not say, one with no delivery, one whose answer is truthy and not True, no provider, and one that cannot name its conversation: every message, and the build does not fail. A member turn, with essentials and with none built: every message, while its essentials are still sent once. The marker scrub neutralizes a forged marker with and without the block, and the same text typed by the person is left in the prompt.
  • test/test_acp_bindings.py: the frozen inventory of the ACP provider's members lists the new reports_compaction property.
  • test/spawn_audit_helpers.py lists the read where the other two drivers of the readiness runner are listed: it creates no process itself, the runner does; test/conftest.py starts every test with no refusal on record and no probe answer, and with the boot hook's warm-up of the read turned off, so a test that boots a gateway starts no probe thread; one test pins that the boot hook calls that warm-up.

Determinism: the new tests use no sleep, no real thread (the probe's thread is replaced by a recorder that runs it on demand), no network and no real subprocess; the cut-off test allows the read no time at all, so it does not wait on the clock. python -m pytest -n0 test/test_flake_pattern_ratchet.py passes. The five new and changed test files passed 20 runs at -n0, 10 runs at -n 4 --dist loadgroup --max-worker-restart=2 beside two sibling files, and one shuffled order (seed 35741), last run on 8183e978e5, the head before this one; the test files have not changed since.

Manual verification

Linux, kiro-cli 3.2.1 with its v3 engine (3.2.1), model claude-fable-5.2, sizes from the engine's own request dumps. Not timed runs. The three reply-format rows are from a default install, which runs kiro-cli's v3 engine (the kas backend), a harness in the set above; a harness outside it sends what it sent before.

Before After
First request with KIROCREW_KIRO_BIN set: tools (both backends) 128 17
First request with KIROCREW_KIRO_BIN set: characters (both backends) 250,146 99,701
A warm message, characters 2,282 935
History growth per short turn, characters (median, 3 chats of 30 turns) 2,553 1,188
Requests carrying the rules block, per 30-turn chat 30 1
Reply format kept at turns 10, 20 and 30 (3 chats each) 9 of 9 9 of 9
Reply format kept, all turns 90 of 90 90 of 90

The format check is the one rule the block carries that a script can judge: a reply that offers choices ends with exactly one [OPTIONS: a | b | c] line, as the last line. It held on every turn with the block sent once, so no periodic re-send was added.

The second commit was rebuilt after those runs (the block now stays in the built message and is taken out at the wire), so the request sizes were taken again, on this branch at c23ab178a8 against the head before it, fd405f3973: 3 chats of 5 turns each, same engine and model. The pushed head differs from c23ab178a8 in comments, docs, tests, one Windows log line, one import moved to the top of its module, and main's own commits.

Turn fd405f3973: message characters / carries the block c23ab178a8: message characters / carries the block fd405f3973: whole request, median c23ab178a8: whole request, median
1 44,848 / yes 44,726 / yes 122,990 122,866
2 937 / no 936 / no 124,170 124,045
5 939 / no 938 / no 127,742 127,620

The three chats of each build agree to the character on the message. The block rode 1 of 5 requests in every chat on both, and the reply format was kept on 15 of 15 turns on both. The 122 characters on turn 1 are session-start context: the two checkouts' own paths, the minute, and the 32 commits of main between the two.

MCP smoke test (each case a real tool call whose result is checked), cases 1 (a fast server's tool on the first prompt), 10 (Tool Search off) and 12 (Kiro Crew's own tools: the same names listed, one call of each server answers):

Install shape Backend OS Case 1 before / after Case 10 before / after Case 12 before / after
pinned default Linux PASS / PASS PASS / PASS PASS / PASS
pinned kiro Linux PASS / PASS PASS / PASS PASS / PASS
KIROCREW_KIRO_BIN default Linux PASS / PASS PASS / PASS PASS / PASS
KIROCREW_KIRO_BIN kiro Linux PASS / PASS PASS / PASS PASS / PASS

Before is main at 610c8616bf; after is this branch at 855b6409b9, its first complete build. The first table and the smoke runs were not taken again after the rebases; since then the branch gained main's sandboxed read of the pinned install (#35731), the rule that a cut-off PATH read is not kept, and the PATH read's move onto the readiness runner. Case 12 lists the same tool names before and after in every row. One sample is not in the table: case 10 on the pinned shape with the default backend failed once after the change (the stand-in server that starts 25 s late never became callable in that run, on a path this change does not touch) and passed when that case was run again; both results are kept with the run. PATH-only is not in the table: the test gateway is started from inside a sandboxed session, where the read declines by design and the full list is kept, as before.

Still required from someone with those machines: the smoke test and one PATH-only start on macOS, and the smoke test on Windows.

Related Issues

Closes #35741

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@KiroCrewCaptain

KiroCrewCaptain commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider thinks the change is probably fine but estimates a 20% chance that this change introduces a defect, above the 12% limit for approving on its own.

Two changes. (1) Kiro Crew's own MCP servers can now defer under Tool Search when kiro-cli comes from KIROCREW_KIRO_BIN or is found only on PATH. The override uses main's existing sandboxed version read for the pin; the PATH install is read in the background under strict mode with require_crew_sandbox and a credential-free environment. Every unknown answer keeps the full list, and a tools-list refusal turns deferral off for the rest of the process, with warm-pool processes started before it discarded at claim. (2) The [REPLY FORMAT RULES] block is bound to the existing EssentialDelivery, so it is taken out at the wire only after this conversation (same identity, epoch and text) acknowledged it on a completed turn. Every existing reset path (compaction or clear frames, /compact, /clear, cancel, start, new incarnation, stale-compaction readers) re-arms it, and a bind forced on needs_reinjection covers compactions in agent-started turns. Harnesses outside the set and providers that do not say keep the block on every message. The edge cases I traced only remove text the same conversation already acknowledged, and the trust boundary holds. Tests are thorough. The main issue is descripti

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 8183e978e5 · 2623 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@bolichen97
bolichen97 force-pushed the perf/small-request-every-install branch from 5cc331e to 049f0fa Compare November 12, 2026 06:18
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 3a5eb311ab617f828f2dfe705768a0dec818d9de and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 3a5eb31

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 3a5eb311ab617f828f2dfe705768a0dec818d9de false-positive: <reason, 20 to 500 characters>

@bolichen97
bolichen97 enabled auto-merge (squash) November 12, 2026 06:19
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 12, 2026
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 3a5eb311ab617f828f2dfe705768a0dec818d9de touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 3a5eb311ab617f828f2dfe705768a0dec818d9de — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: both savings reuse seams that already exist (the EssentialDelivery send-once writer and the strict-sandbox runner), and every unknown or failed case falls back to today's full list or full block.

Notes

  • The PR carries two separate changes: Tool Search deferral for more install shapes, and sending the reply-format rules once. Neither depends on the other, so splitting them into two PRs would let either be reverted on its own.
  • The process-wide note_tools_list_refusal latch is not applied to processes shared under the chat_runtime_sharing preview. The PR records this as a shortcut: comment, so it is a known gap to fence before that preview graduates, not a design flaw now.
  • The ## Backwards compatibility section falls past the truncation point. The only take-away found (the thinking_binding_errors move) is re-imported into transport_errors, so no reader breaks.

[DESIGN-REVIEWED] 3a5eb31

Description read

[DESCRIPTION-READ] 6efb779ca977b155913f3e50c0b52a11fc0b55a122f65cb6949603bd7a1878e2

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 3a5eb311ab617f828f2dfe705768a0dec818d9de — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. One note: ACP_BACKENDS_COMPACTION_REPORTED is a second name for an existing set, and its only reader could use the existing set directly.

Notes

  • Item 6 — duplicate of src/kiro_crew/agent_sdk/backends_turns.py:472: the diff adds ACP_BACKENDS_COMPACTION_REPORTED = ACP_BACKENDS_SENT_IMAGE_RECORD, which is the same object. It is read in only 2 places (the two reports_compaction properties, providers/acp.py and acp/session_provider.py), yet the name has to be registered in 3 more places (check_harness_parity.py X_FEATURE_SETS, backend_cards.py OFF_CARD_SETS, the backends.py doc table) plus re-exports in acp_backends.py/acp/types.py. Subtraction: delete the alias and its 3 registry rows, have both properties test ACP_BACKENDS_SENT_IMAGE_RECORD, and fold the reply-format reader into that set's existing comment.

What this change ships

Inventory (8 items) — 7 justified

Intent: Make every kiro-cli install send the small tool list, and stop resending the reply-format rules every turn. This is an ADDITION (perf).

  1. A KIROCREW_KIRO_BIN install now defers Kiro Crew's tools once the file is ≥2.27.0 — justified
  2. A kiro-cli found only on PATH defers once a background version check inside the sandbox answers — justified
  3. That PATH check now starts at gateway boot — justified
  4. After the first refusal for a changed tool list, every later spawn keeps the full list until restart — justified
  5. Warm-pool processes started before that refusal are thrown away when claimed — justified
  6. New harness set ACP_BACKENDS_COMPACTION_REPORTED / reports_compaction — duplicate of ACP_BACKENDS_SENT_IMAGE_RECORD
  7. The reply-format rules go to the engine once per conversation on kiro/kas/codex/claude — justified
  8. The thinking-refusal parsing moves out of transport_errors.py into its own module — justified

[FIRST-PRINCIPLES-REVIEWED] 3a5eb31

Description read

[DESCRIPTION-READ] aba4a62399b131f61d5cb10258e2746793539a84827bdb41fd611f7d47acac38

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 3a5eb311ab617f828f2dfe705768a0dec818d9de — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 3a5eb31

Verdict parsed from the review's SHA-scoped output markers for commit 3a5eb311ab617f828f2dfe705768a0dec818d9de.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 3a5eb311ab617f828f2dfe705768a0dec818d9de false-positive: <reason, 20 to 500 characters>

@bolichen97
bolichen97 force-pushed the perf/small-request-every-install branch from 049f0fa to 7fa08e4 Compare November 12, 2026 07:18
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 12, 2026
@bolichen97
bolichen97 force-pushed the perf/small-request-every-install branch from 7fa08e4 to 44c9737 Compare November 12, 2026 08:57
@bolichen97

bolichen97 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in config-baseline.json, docs/architecture/mcp.md, docs/system-specs/modules/memory-skills-hooks.md, docs/system-specs/modules/metrics.md and src/kiro_crew/docs/configuration.md since this head went green (169429f -> 9ef0fc4); CI re-running.

The previous head 7fa08e460c was green: every review lane PASS, CodeQL, Fast Gate, all Linux and Windows shards (Linux shard 6 on its one re-run, a ledgered flake recorded on #34376), Coverage Gate, and PR Readiness passed. The rebase had no conflict and changes none of this PR's own lines: the difference between the two heads in the files this PR touches is exactly what main added to those five files. On the rebased tree: 592 named tests pass (the new and changed test files, the config baseline, the spawn audit, the two structure tests), and black, isort, flake8, mypy, comment-history, brand, harness-parity, line cap, subprocess-encoding and docs-lint all exit 0.

macOS Tests (on demand) shard 4 was red on the previous head on test/test_installer_disk_space.py only, a file this PR does not touch. That was main's own red (#35693); its fix #35695 merged as 04cad0664b and is in the base of the rebased head, so the shard is expected green on this run.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running labels Nov 12, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author

Disposition for macOS Tests (on demand) on head 44c97375eadf (not a required check): not this change; ledgered flake #31637, no push.

Every required check is green on this head, every review lane is PASS, PR Readiness is success and the label is readiness: passed. The head is frozen here: no further push for main drift. What remains is a person's: an approval by a maintainer who is not the author (this is a sensitive change).

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. The change itself looks careful, but one test it adds breaks once it is merged with current main.

  1. test/test_tool_deferral_install_shapes.py:225 (TestOperatorOverride::test_a_replaced_override_is_judged_by_its_new_version). The test patches kiro_cli.subprocess.run. #35731 landed on main after this branch's merge-base. It removed import subprocess from kiro_cli.py, and kiro_cli_version_at now reads the version through kiro_cli._sandboxed_probe. The textual merge is clean. On the merged tree this test fails with AttributeError: module 'kiro_crew.kiro_cli' has no attribute 'subprocess', so merging this head as it is would turn main red. Even if that name were still there, the stub would never run, so runs == [[binary, "--version"]] could not hold. The green readiness on this head comes from before #35731. Smallest fix: rebase on main, then stub kiro_cli._sandboxed_probe the way main's own tests do (for example test/test_kas_permissions.py). Make it return an ok ProcessResult with kiro-cli 3.2.1, then 2.20.0, and assert one probe per file identity.

Optional:

  • While you rebase: the new paragraph in docs/system-specs/modules/security.md (around line 4270), the kiro_cli_path_probe.py module docstring and the version-read table in the body all say the pinned and KIROCREW_KIRO_BIN read runs outside the sandbox. Since #35731 that read is sandboxed, so these lines would describe a trust boundary wrongly.
  • kiro_cli_path_probe._probe_and_record caches None on a timeout or a failed wrap. One slow read at boot then keeps the full tool list until the file changes or the gateway restarts. Main's kiro_cli_version_at deliberately does not cache a sandbox failure. You could cache only a parsed version or a clean non-zero exit.
  • Deleting self._reply_format_pending.pop(key, None) in delivery_records.py:347 still leaves every test green. The sequence send (not settled), then a non-interactive send, then land, then an interactive send that must still carry the rules would pin it.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision labels Nov 12, 2026
@bolichen97
bolichen97 force-pushed the perf/small-request-every-install branch from 44c9737 to 0930fd0 Compare November 12, 2026 12:14
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 12, 2026

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Both earlier change requests are resolved on this head, but the new reply-format record has one runtime gap.

Earlier reviews:

  • 44c97375, the kiro_cli.subprocess.run stub: fixed (it now stubs kiro_cli._sandboxed_probe).
  • 00f25eb5, the record kept on harnesses that compact silently: fixed. The record is now kept only on ACP_BACKENDS_COMPACTION_REPORTED, and the table test covers every selectable harness.
  • 00f25eb5 optional items (stub warm, pin the boot hook's warm() call, the two stale comments): all fixed.
  1. context_assembly/delivery_records.py:364 (_reply_format_due, gated at context_assembly/turn_message.py:975). On the kiro backend, /clear is not answered locally: kiro is not in ACP_BACKENDS_LOCAL_CLEAR, so the Kiro CLI runs the clear in the same ACP session. The confirmed frame reaches dashboard/chat_turn/session_events.py:165 (_on_clear_status). That handler clears the slot, the replay lease and the first-turn history debt. It never calls mark_needs_reinjection and never drops the builder's reply-format record. On the next ordinary turn, is_new_session, needs_reinjection and conversation_reset are all False and the digest still matches, so the [REPLY FORMAT RULES] block (the [OPTIONS:] format and the ask_question, suggest_followup and card guidance) is left out. The engine conversation that held the rules was just wiped, so the cleared chat goes without them until a compaction, a new session or a rules change. On main the block rides every message. For an agent with includeCrewContext: false this block is the only source of the [OPTIONS:] rule. The sent-image record already forgets on this same clear frame (_forget_sent_images()), and essential_delivery.py treats a clear as invalidating. test/test_reply_format_once.py has no clear case. Smallest fix: when a clear the session owns is confirmed, drop that session's _reply_format_sent and _reply_format_pending entries, or arm mark_needs_reinjection in _on_clear_status. Add a test: a turn lands, a native clear arrives, and the next warm build must carry the block. The test should fail without the fix.

Optional:

  • test/conftest.py:657 stubs _sandboxed_version, _versions, _in_flight and warm, but not kiro_cli_path_probe.version_if_known. On a host where kiro-cli is found only on PATH, a test that reaches mandatory_mcps_drop_allowed can still start a real probe thread that outlives the test. Stubbing it in the same fixture would close that.
  • The reports_compaction ... is True guard in turn_message.py has no mutation pin. Adding a provider whose attribute is a truthy non-bool (for example a MagicMock) to the parametrize in test_a_harness_that_is_not_known_gets_the_block_every_time would catch a loosening to bool(...).
  • The body's table says the pinned install is "unchanged", but the process-wide tools-list refusal latch also applies to it. One line in the description would say so.

@bolichen97
bolichen97 force-pushed the perf/small-request-every-install branch from fd405f3 to 8183e97 Compare November 12, 2026 20:53
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 12, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author

Round 9, head 8183e978e52a82f82699aa24994f8fd54eb5f9d1: an answer to the review of fd405f3973 (a native /clear leaves the reply-format record stale).

The finding is right. A confirmed native clear empties the engine conversation, and nothing reset the builder's "the rules were sent" record. It is also the third finding of one kind on this PR: a second record of what a conversation holds goes stale whenever the conversation loses it and the builder is not told. So the record is removed instead of being given one more reset.

What changed in the second commit.

  • The built message is main's again: it always carries the [REPLY FORMAT RULES] block. context_assembly/turn_message.py gains 42 lines and changes none.
  • On a turn with no member essentials, on a harness whose compaction Kiro Crew sees, the block is bound as the provider's delivery envelope (EssentialDelivery, the send-once writer the member essentials already use). The provider takes the block out of the prompt at the wire while the conversation has acknowledged it, and sends it again after a start, a reload, a clear, a compaction, an agent switch, a cancel, a failed turn or a change of conversation. essential_delivery.py and the provider streams are not touched.
  • context.py, delivery_records.py and their two pin tests are back to main's content and leave the PR.
  • A chat with a crew member is unchanged from main: its essentials take the delivery's one envelope, so the block still goes out on every message there. A second envelope slot would cover it and is not in this PR.

The test asked for. test/test_reply_format_once.py now reads the prompt the transport was handed, through the real AcpProvider and AcpSessionProvider. It has a table of 22 rows of events that can drop what a conversation was sent, a native clear and /clear among them, and test_acknowledged_then_cleared_comes_back. Each fails with the bind removed.

The optional points.

  • A provider whose reports_compaction is truthy and not True: added to the unknown-provider cases (_AnswersTruthy); a loosening to a truthy test turns it red.
  • The pinned install and the remembered refusal: the description now says the refusal applies to the pinned install too.
  • Stubbing version_if_known in test/conftest.py: not in this push. The fixture already replaces the read itself (_sandboxed_version), so a probe thread a test starts runs the stub and ends; it can still outlive the test by that moment.

Also in this push. On Windows the PATH probe runs nothing, and its log line said "inside the sandbox"; it now says that the version is not read on Windows and that nothing was run.

Measured (Linux, kiro-cli 3.2.1, 3 chats of 5 turns, engine request dumps, on this branch at c23ab178a8): the message is 936 characters on turn 2 and 938 on turn 5, against 937 and 939 on fd405f3973; the block rides 1 of 5 requests on both.

CI on this head so far. Await Fast Gate, Opus 5.5 Review and Analyze (javascript-typescript) stopped on "API rate limit exceeded for installation" and reached no verdict; GPT 6.1 Review reached "no blocking findings" and could not publish it for the same reason; Coverage Gate failed closed behind the Fast Gate. Those jobs are re-run once.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Nov 12, 2026
Kiro Crew's own MCP servers deferred under Tool Search only on the
pinned kiro-cli install. An install found through PATH only, or named
by KIROCREW_KIRO_BIN, kept all of them resident: 125 tools and 243k
characters per request against 15 and 93k.

The decision is now made on the file the session is about to run. An
absolute override is the pin, so it takes the answer the pin's version
read already gives. A PATH file may sit in a directory an agent writes
to, so its version is read inside the strict sandbox, through the
readiness probe's own bounded runner, in the background, and never
waited for; no answer keeps the servers
resident, and only the file's own answer is cached, so a read that did
not run or was cut off is asked again. Neither derives a sibling, so a
POSIX launcher below 3.0.0 stays resident.

The floor is measured, so a refusal above it is remembered: after the
first request refused for a changed tools list, every spawn in the
process sends the full list, and the rebuilt conversation starts on it.
build_message appended the [REPLY FORMAT RULES] block to every message,
1,347 characters each time, although the engine replays its own history
and already holds it.

The built message is unchanged: it still carries the block. On a turn
with no member essentials, build_message now binds the block as the
provider's delivery envelope (EssentialDelivery, the send-once writer
the essentials already use), and the provider takes it out of the
prompt at the wire while this conversation has acknowledged it. There
is no second record: the delivery is the one place that knows when a
conversation may have lost what it was sent (a start, a reload, a
clear, a compaction, an agent switch, a cancel, a failed turn, another
conversation), and it decides at the send. The bind is forced for the
one event the delivery cannot see, a compaction in a turn the agent
started by itself, which reaches only the session's re-injection flag.

That rests on Kiro Crew hearing of every compaction, so the block is
bound only on a harness whose compaction it sees: kiro, kas, codex and
claude, the set the sent-picture record uses, read from the turn's
provider. On every other harness, when the harness is not known, and
on a member turn, whose essentials take the delivery's one envelope,
the block goes out on every message as before.
@bolichen97
bolichen97 force-pushed the perf/small-request-every-install branch from 8183e97 to 3a5eb31 Compare November 12, 2026 23:17
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Nov 12, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author

Rebased: head 3a5eb311ab617f828f2dfe705768a0dec818d9de on main ee18fa5303. The previous head 8183e978e5 had a merge conflict after #35722 landed.

Two things differ from 8183e978e5 (git range-diff shows nothing else in either commit):

Checked on the rebased tree: the reply-format block that is bound as the delivery envelope is still byte-equal to the block in the built message with #35722's new [MCP SERVERS] part in place (that part is added before the block), and test/test_reply_format_once.py passes unchanged.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Nov 12, 2026
@bolichen97
bolichen97 enabled auto-merge (squash) November 13, 2026 00:50
@bolichen97
bolichen97 disabled auto-merge November 13, 2026 01:01
@bolichen97
bolichen97 merged commit 0842788 into main Nov 13, 2026
108 of 113 checks passed
@bolichen97
bolichen97 deleted the perf/small-request-every-install branch November 13, 2026 01:01
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 13, 2026
@LuisBrel LuisBrel mentioned this pull request Nov 13, 2026
5 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Requests stay large when kiro-cli is on PATH only or set by KIROCREW_KIRO_BIN, and grow by the reply-format rules every turn

3 participants