Skip to content

perf(agents): share the agents-dir revision memo with the KAS session start - #13067

Merged
bolichen97 merged 1 commit into
kirodotdev:mainfrom
Pearcekieser:perf/kas-spec-memo
Sep 23, 2026
Merged

bolichen97 merged 1 commit into
kirodotdev:mainfrom
Pearcekieser:perf/kas-spec-memo

Conversation

@Pearcekieser

@Pearcekieser Pearcekieser commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Every KAS session start resolves its agent through agent_discovery.spec_by_declared_name, which hardened-reads every spec in the agents directory to find one declared name. On the profiled host (~2000 installed specs) that is most of a second of GIL-holding path validation and JSON parsing on a worker thread per session start. #12722 pinned the tool-policy read's answer to a stat-only directory revision, but KAS start is the second caller of the same scan and still paid the full cost, so a session start plus its managed MCP servers' policy request cost 2N reads. The revision helper also lived in dashboard/handlers/sessions.py, duplicating the walk agent_discovery._dir_signature already does.

Why it matters

The scan holds the GIL for most of a second, which the gateway's event loop sees as loop_lag; the adaptive concurrency controller reads that as pressure and pins the subagent cap at 1 on an idle 128-core host. Session start is the most frequent path that pays it. This closes the three deferrals recorded on #12721 from PR #12722.

What changed (motivation → approach → change)

Motivation: two callers of one expensive scan, one memoized, one not, with the memo's revision logic living in a handler module.

Approach: move the revision into agent_discovery, next to _dir_signature, and put the store/hit rules in one small primitive both callers use, so the rules that took #12722 fifteen review rounds to settle are written once.

Change:

  • src/kiro_crew/agent_discovery.py: _iter_spec_entries is the one scandir walk; _dir_signature projects (name, mtime_ns) from it and keeps its never-None contract (the catalog caches do not change what they invalidate on). agents_dir_revision is the stricter fingerprint with the same behaviour as the handler's private copy (Windows off, symlink, an entry whose kind cannot be read or a directory that cannot be listed → None (_dir_signature keeps treating the last as empty), entry cap → None plus one warning per directory (its text is now agents-dir memo disabled for <dir>: N spec entries exceed M, replacing the handler's tool-policy memo disabled for ..., since the revision now serves both callers), racy window, spec generation in the tuple). AgentsDirMemo[T].get(agents_dir, key, compute): revision before, hit on same revision, else compute(), revision after, store only when equal and not None; exceptions propagate unstored; _AGENTS_DIR_MEMO_MAX_KEYS (256) keys per directory. There is no registry or clear(): clear_list_agents_cache bumps the spec generation, which is part of every revision, so stored answers stop matching on their own. The stored object is returned as-is, so a caller that hands it to mutating code copies it.
  • sessions.py: one _TOOL_POLICY_MEMO instance and a thin _read_managed_tool_policy_sync; the private revision function, its constants and dicts are deleted. _read_managed_tool_policy_uncached and _refuse_if_any_spec_is_unreadable are unchanged.
  • src/kiro_crew/acp/kas_agents.py load_agent_spec: the scan answer is served from its own _SPEC_SCAN_MEMO (separate instance: the two surfaces carry different SEL operation labels) and returned as a deep copy, so it is still a parse the caller owns. AmbiguousAgentSpecError/OSError still raise every call; the <id>.json fallback read stays uncached.

Backwards compatibility

Compatible: no wire format, config, or public API changes; _dir_signature keeps its exact return type and semantics, and every refusal path is re-derived per call exactly as before.

Tests

  • test/test_agents_dir_memo.py (new): the revision tests moved from test_session_tool_policy_cache.py (content/permission changes, racy window, entry cap with one warning, platform gate, symlink entry and the catalog signature staying on for it (both through a fake DirEntry, no symlink created), an entry whose is_symlink raises, an unlistable directory, stray files ignored) plus test_the_catalog_signature_stays_on_where_the_revision_refuses; memo primitive tests (unchanged directory served / edit seen, exception stores nothing, per-directory cap, clear_list_agents_cache invalidates through the generation); KAS tests test_kas_load_agent_spec_reads_once_per_directory_revision (N reads, then 0, then N after an os.utime-bumped edit; fails on main), test_kas_memoized_spec_is_a_copy, test_kas_refusals_are_re_derived_on_every_call, test_kas_direct_filename_fallback_is_not_memoized.
  • test/test_session_tool_policy_cache.py: keeps the policy-semantics tests on the shared memo and adds test_the_policy_memo_is_separate_from_the_projection_memo.
  • Existing test_session_tool_policy_off_loop.py, test_agent_discovery.py, test_kas_agents.py, test_parsed_agents_cache.py, test_pathres_loop_starvation.py pass unchanged.

Manual verification

N/A — unit coverage sufficient: the read-count tests observe the hardened reader directly, which is the cost being removed.

Related Issues

Fixes #12721

Pattern harvest

Not generalizable: this is the second caller of a scan whose first caller was already memoized in #12722; the primitive introduced here is what stops the pattern from recurring for a third caller.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Fixes #12721

@Pearcekieser
Pearcekieser requested review from a team and cixuuz September 23, 2026 11:43
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Intent: one directory-revision memo in agent_discovery serves both spec_by_declared_name callers (the tool-policy read and the KAS session-start projection); sessions.py loses its private copy of the revision machinery.

Not a goal: changing what the catalog caches (list_agents, parsed_agent_specs) invalidate on, or touching the hardened readers.

@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 23, 2026
@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5, fork) — ✅ PASS

Design-level review of a9afa41bc1da926657353403cfc03a3302c8336b via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Verify SEL consumers tolerate a memo-hit KAS session start emitting zero kas_agent_projection read events — same trade #12722 already made for tool-policy.

[DESIGN-REVIEWED] a9afa41

@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5, fork) — ✅ PASS

Premise-level review of a9afa41bc1da926657353403cfc03a3302c8336b via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Verify the memo hands back the stored tool-policy dict itself — confirm no policy caller mutates it, since only the KAS side deep-copies.

What this change ships

Inventory (9 items) — 9 justified

Intent: stop a KAS session start from re-parsing every installed agent spec when the agents directory is unchanged — a FIX (perf defect, linked issue #12721, deferrals from #12722; the measured cost is already recorded in the base comment at sessions.py:3610).

  1. KAS session start on an unchanged directory answers from a revision-pinned memo, no spec re-read — justified
  2. The spec handed to a KAS session is now a deep copy the session owns — justified
  3. Revision fingerprint and store/hit rules moved from the sessions handler into agent_discovery as one shared primitive — justified
  4. An unlistable directory or unreadable entry kind now disables memoization instead of fingerprinting as empty — rides along, declared; closes a staleness hole in the moved mechanism
  5. Overflow warning relabeled "agents-dir memo disabled…" — justified (old text: zero consumers outside the moved test; grepped tool-policy memo disabled, 2 hits, both this mechanism)
  6. All refusal paths stay re-derived per call — justified
  7. The two callers keep separate memo instances per SEL operation label — justified (both spec_by_declared_name prod callers counted: exactly 2)
  8. Revision tests moved to test_agents_dir_memo.py plus new KAS read-count, copy and refusal tests — justified
  9. _dir_signature shares one scandir walk, contract unchanged — justified

[FIRST-PRINCIPLES-REVIEWED] a9afa41

@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review (fork) — ✅ no blocking findings

Reviewed a9afa41bc1da926657353403cfc03a3302c8336b via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] a9afa41

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 23, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • fixed — the copy convention is on the class

AgentsDirMemo's docstring states that get returns the stored object itself and that a caller handing it to mutating code must copy, naming the two callers' choices. No hook or parameter: one sentence keeps a third caller honest at the point they read the API. Head bc003cd.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • fixed — has_answers_for deleted

Tests assert through compute call counts (a stored answer means the second get does not call it), the behaviour the memo exists for. No production caller remained. Head bc003cd.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • fixed — max_keys parameter removed

The cap is the module constant _AGENTS_DIR_MEMO_MAX_KEYS = 256 beside the other _AGENTS_DIR_* constants; the overflow test monkeypatches it. Head bc003cd.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • fixed — is_symlink() inside the per-entry guard

An OSError from is_symlink() (unknown d_type, lstat refused) returns None: the entry's kind cannot be determined, so freshness cannot be proven and nothing is memoized. test_an_entry_whose_kind_cannot_be_read_gives_no_revision reproduced the escaping PermissionError on the prior head. Head bc003cd.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Pushed bc003cd: the review-round fixes above (net smaller: agent_discovery.py loses has_answers_for and the max_keys parameter), plus a re-run of test_diag_procs::test_gil_hint_stays_off_without_futex_waiters on Windows shard 4, a CPU-percentage timing assertion (88.7 vs 90±1) unrelated to this diff.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 23, 2026
@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review (fork) — ✅ no blocking findings

Reviewed a9afa41bc1da926657353403cfc03a3302c8336b via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] a9afa41

@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • rebutted — a copy policy on the constructor is the shape First Principles rejected last round

Round 1 removed max_keys because only one production value existed; a copy= policy would have exactly one non-default caller (the KAS projection) and the same objection. The tool-policy read's answer is serialized and never mutated, so a forced copy there is pure cost on the hot path this PR exists to cut.
The ownership rule is stated once on the class docstring, where a third caller reads the API; test_kas_memoized_spec_is_a_copy pins the one caller that hands the object to mutating code.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 23, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • fixed — docstring matches the implementation

load_agent_spec describes the revision as a stat of the directory and every spec entry, and names what is never memoized: a directory containing a symlinked spec, one written inside the racy window, an entry whose kind cannot be read, or a directory past the entry cap. Head 9bad637.
served model: the pinned gpt-5.6-sol lane failed at turn 0 with a backend service error; the repair ran on claude-fable-5.1 (auto selection) and was verified in the parent.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • fixed — the store-refusal guard is pinned again

test_an_edit_during_the_read_is_not_memoized asserts str(tmp_path) not in _TOOL_POLICY_MEMO._answers after the straddling read; removing the guard in AgentsDirMemo.get goes RED (reproduced). Head 9bad637.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • fixed — no real symlink is created

test_the_catalog_signature_stays_on_where_the_revision_refuses uses the file's fake-DirEntry seam (is_symlink() -> True, real stat) like its sibling, so it runs on an unelevated Windows shell without a skip marker; symlink_to no longer appears in either test file. Head 9bad637.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Pushed 9bad637: the round-2 fixes above; also re-runs the test_terminal_handler pty timeout on Linux shard 6 (Coverage Gate failed closed on it), which fails identically on main in this sandbox.

@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Sep 23, 2026
@github-actions github-actions Bot added the readiness: action required A blocking check or review needs attention label Sep 23, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • fixed — the walk is iterated inside the try, never materialised

The list(_iter_spec_entries(...)) was mine from the previous round (the unlistable-directory fix) and it did put every DirEntry in memory before the entry cap could fire. The loop now runs directly over the generator inside a try/except OSError: return None, so the cap returns on the 4,097th entry with nothing else retained and an unlistable directory still gives None. No new state; the existing cap test (_AGENTS_DIR_REVISION_MAX_ENTRIES patched to 2) and the unlistable-directory test cover both paths. Formatting-only re-indent of the loop body otherwise. Head 26d746c.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 23, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • fixed — the description now names the warning-text change

The body's entry-cap sentence states the new text (agents-dir memo disabled for <dir>: N spec entries exceed M), what it replaces, and why (the revision serves both callers). Description-only; no code change.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • rebutted — a copy-on-hit constructor flag is the shape First Principles struck in round 1

Round 1 removed max_keys because exactly one production value existed; a copy= flag would have exactly one non-default caller (the KAS projection) and the same objection, and the tool-policy read's answer is serialized and never mutated, so a forced copy there is pure cost on the hot path this PR exists to cut. The ownership rule is stated once on the class docstring where a third caller reads the API, and test_kas_memoized_spec_is_a_copy pins the one caller that hands the object to mutating code. Same reasoning as the round-2 disposition on this suggestion (span b3ccdf96bc9d).

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Re-pushed the identical tree (26d746c → d42a497, no code change) to re-run Windows shard 7, whose only failure was test_runtime_cleanup_windows.py::test_repeated_windows_provider_shutdown_reclaims_entire_tree ("owned processes still alive") — a process-teardown timing test unrelated to the agents-dir memo — and to re-trigger the fork GPT lane, whose job sat at zero steps after its parent run completed. Design PASS, First Principles PASS and Opus clean on 26d746c.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 23, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Rebased onto main b013853 (12 upstream commits; kas_agents.py and sessions.py moved upstream in #13014/#13015/#13033, merged clean). Same diff, head d42a497 → a9afa41. Scoped tests for the overlap files (574) plus flake8/mypy/black pass locally.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Sep 23, 2026

@cixuuz cixuuz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the perf claim (KAS session start paying the same full agents-dir scan the tool-policy read already memoized in #12722) and the shared AgentsDirMemo primitive against source: the store/hit rules (revision before+after compute, exception unstored, per-directory key cap) are correctly extracted into one class used by both callers with separate instances (different SEL operation labels), and the KAS side deep-copies its answer so caller mutation can't leak into a later session. All 4 review lanes (Design, First Principles, Opus, GPT) are PASS/no-blocking at this exact head, dispositions across rounds all resolved, and CI is fully green. High confidence.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

session-tool-policy read re-parses every agent spec per request (750-965 ms, GIL stalls)

3 participants