Repository navigation
perf(crew): offer a Bot approval's Remove without rebuilding the file per row - #37773
Conversation
… per row worker_summary checked each added row's Remove by rebuilding the file's additions without that row, so a read made one whole-file rebuild per approval: 1021 rebuilds for a thousand approvals. That kept the scale test near the 120 s pytest timeout on 4-core CI runners. A glob-free entry that no other effective entry reaches surely disappears when removed, so its Remove is offered directly; anything else still takes the rebuild path. Fixes kirodotdev#37743
UX Review (Opus 5.5, fork) — ⏭️ skippedRevision |
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the shortcut runs only when the rebuild must give the same answer, and a differential test compares the two on every row. Notes
[DESIGN-REVIEWED] 4ae29f7 Description read[DESCRIPTION-READ] 81a273733b051349e0fef483bc179a7fc5c99fd21a1b29b1e5903419c6727e83 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS This is a performance fix for issue #37743, aimed at its cause: the read rebuilt the whole file once per row. The rows that still rebuild are listed in the description. Notes
What this change shipsInventory (4 items) — 3 justifiedIntent: open a Bot that has about a thousand approvals without the capabilities read timing out. FIX.
[FIRST-PRINCIPLES-REVIEWED] 4ae29f7 Description read[DESCRIPTION-READ] e47dc94696630ca6265a9785aacfcd794e117f0d43da1cbb404ee0fa4d512e10 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Code change approver recommendation: ApproveWhy: the decider estimates a 10% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes. This change adds a shortcut in worker_summary._remove_ref. A tools or allowedTools row that has no glob, and that no other effective entry reaches, gets its Remove directly instead of a whole-file rebuild. I traced worker_additions, _materialize, _apply_operations and the reach pass in _effective. Removing a stored entry only shrinks the additions, the key's only source is its own file entry, and the reach pass can only grow the set. So the shortcut answers only rows where the rebuild gives the same Remove. Whole-server selectors count as globbed and still rebuild, so the existing tests with a stored '*' are unaffected. OtherReach's lookups by folded pattern and by verb cover every way entry_reaches can match a key with no glob, and every hit is checked again. When the summary falls back on a read failure, every row takes the old rebuild path. The new tests cover the behaviour: the scale test counts rebuild calls, which fails on main, and a comparison test checks every row with the shortcut on and off. An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Reviewed on head 4ae29f7. We read the full diff and checked the shortcut's soundness against worker_additions and the reach pass in _effective. A removed glob-free key that no other entry reaches cannot come back. We ran the three affected test files with -n0 (29 passed). Reverting the shortcut reds the scale test (1021 < 100) and the differential test. Six mutations of OtherReach each red 2 to 8 tests. A 20k-case random comparison against trying every other entry found 0 mismatches. The branch merges clean with current main.
Done well: the differential test runs every row with the shortcut on and with it forced off. It also asserts that the fast path answered more than the bulk rows, so it cannot pass vacuously. The new bound counts rebuild calls instead of timing them, so a loaded runner cannot flake it.
Optional:
grant_foldhas one consumer, andReachIndex.reaches(mcp_refs.py:374) still writes the same fold inline. Making it_grant_foldand calling it from there would leave one copy of the rule.- The
glob_free(key)guard in_remove_ref(mate_capabilities.py:551) is redundant: a globbed key always takes the rebuild anyway. You could drop it, or add a comment saying it only skips a lookup.
Problem / Motivation
Goal: Opening a Bot that has about a thousand tool approvals is fast again, and its scale test stops timing out in CI.
test/test_mate_capabilities_scale.py::test_get_preview_and_put_do_not_match_every_approval_against_every_otherhit the 120 s pytest timeout on a 4-core Linux runner (Backend Tests (3.12, 4)) at line 74, theservice.get("big")read. On a fast local host the same test took 14.4 s.Why it matters
The Abilities and Approvals panes read
GET /api/agents/<bot>/capabilities, which gives up after 20 s. A Bot with hundreds of approvals could not open its panes, and the scale test fails PRs that do not touch this code.Not a goal
What changed (motivation → approach → change)
Symptom: one
getread of a Bot with 1,000 approvals took 14.4 s locally and over 120 s on CI.Root cause:
worker_summarydecides each added row's Remove by building a candidate file without that row, then re-running_materialize,worker_additionsand_effectiveon it. That is one whole-file rebuild per row, so the read is quadratic in approvals: the new probe counts 1,021worker_additionscalls for one read. #37691 indexed the reach matching inside each rebuild but kept the per-row rebuild.Change: a glob-free row that no other effective entry reaches gets its Remove directly, without the rebuild. Removing an entry only shrinks the file's additions, and the reach pass is monotone, so a key that nothing else reaches surely disappears. The new
OtherReachinagent_sdk/mcp_refs.py, besideReachIndex, answers whether any effective entry other than the key reaches it. Globbed entries go throughReachIndex. Glob-free entries are looked up by the key's case-folded pattern and by its verb, then re-checked withentry_reaches. Glob rows, rows something else reaches, and the skills, resources and server sections keep the existing rebuild path.Result: the scale test's call phase went from 14.4 s to 2.9 s locally.
Backwards compatibility
Compatible: the Remove shown for each row is unchanged. The shortcut only answers for rows where the rebuild would give the same answer, and every other row still takes the rebuild path.
Removes nothing: the change adds a shortcut in front of an existing check and drops no rows, keys or functions.
Tests
test/test_mate_capabilities_scale.py: countsmate_worker.worker_additionscalls during onegetread and bounds them at a tenth of the approval count. Main makes 1,021 calls here. This is a call count, not a wall-clock bound.test/test_mate_capabilities_remove_differential.py::test_every_added_row_offers_the_remove_the_rebuild_offers: for a file with globbed entries, case variants,@srv/toolbeside bare tools, a slash-bearing bare entry, a reach-pass-derived entry, a base-server selection and 300 plain rows, every added row's Remove is the same with the shortcut on and with it forced off (every key read as globbed, so every row takes the rebuild). It also asserts that the shortcut answered more than 300 rows, so the comparison is not vacuous.test/test_mate_capabilities_remove_differential.py::test_other_reach_matches_trying_every_other_entry:OtherReach.reached_by_anotherequalsany(entry_reaches(e, key) for e in entries if e != key)for every glob-free key, including a slash-bearing bare key with a bare-verb reacher. This fails if the verb lookup is limited to@keys again.test_a_glob_entry_withholds_remove_on_a_tool_it_still_grantscases intest/test_mate_capabilities_worker_summary.pystill pass, so a row that a glob still grants keeps no Remove.Proof: PROVEN test/test_mate_capabilities_scale.py::test_get_preview_and_put_do_not_match_every_approval_against_every_other (head 4ae29f7)
Manual verification
N/A: unit coverage is sufficient. The change is backend-only and the scale test exercises the same
get,previewandputpath the panes call.Related Issues
Fixes #37743
Sibling sweep
Swept:
git grep -n "copy.deepcopy(file_spec)" -- src/Siblings: none -- worker_summary's _remove_ref is the only per-row whole-file rebuild of a Bot's spec
Pattern harvest
Not generalizable: the per-row rebuild was specific to how worker_summary decides Remove, and the scale test already guards it.
Checklist
Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license.