Repository navigation
fix(security): size the path gate's anchor cache by what the rebuild cost - #12664
Conversation
|
Intent: Stop the sensitive-path gate refusing ordinary project files under CPU contention, by sizing its anchor-cache expiry to what the rebuild actually costs instead of to a constant picked against an idle interpreter. Not a goal: Making a single cold rebuild cheap (that needs the resolution to leave this interpreter), caching the candidate-path side, changing the stall-cooldown key, or raising the load-arm probe allowance. |
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify "Fixes #10255" shouldn't auto-close: stow/chezmoi hosts stay floor-pinned (no-op fix) until #12714 narrows the pin. [DESIGN-REVIEWED] 2d87346 |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS
Not justified as shipped
What this change shipsInventory (10 items) — 8 justifiedIntent: FIX — stop the sensitive-path gate refusing ordinary files under CPU load by sizing the anchor-cache expiry to the rebuild's own measured cost (issue #10255;
[FIRST-PRINCIPLES-REVIEWED] 2d87346 |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsFINDING -- src/kiro_crew/security/paths.py:2176 -- False positive or not applicable? A repository writer can comment: |
edb18fa to
1c3373c
Compare
|
self-added: no
|
|
self-added: yes
|
|
self-added: no
|
|
self-added: no
|
|
self-added: yes
|
|
self-added: yes
|
|
self-added: yes
|
Security Scope Review -- ✅ PASSWhich legitimate operations does Verdict basis: adjudicated, zero confirmed regressions. Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. Scope-Verdict: PASS TIGHTENING: ADJUDICATED: 14 candidates, 0 confirmed newly-refused PLATFORM COVERAGE: Candidates were written in posix and windows spellings; no [SCOPE-REVIEWED] 2d87346 |
1c3373c to
f107159
Compare
|
span=6740b53bc685 -- fixed. Correct: two different sweeps were quoted as though one number sequence were a The spec now labels its row as one sweep of |
|
span=0a81c8095013 -- rejected, with the reason named. The subtraction is well argued: no host is named, the sweep says a window wider I am not deleting it because the maintainer's own acceptance condition for this What I can do without touching it: the exposure the knob governs is now much |
|
span=6ad2bbe979c5 -- rejected, and this is the third round on the theme. The cost is real and I am not disputing it: the script reads private names of the It stays because a committed, runnable measurement is an acceptance condition of Per the same-span rule I am not patching this a third time. The conflict is |
|
span=2872b3c17caa -- fixed, and it was my bug. Exactly right, including the consequence. The cache now calls The wrapper now takes Verified by running it, not by reading it: |
|
span=1a82848489c5 -- fixed. Same defect as the GPT lane found one line up, and the same fix clears both: the Verified by execution on this head rather than by inspection: |
|
span=ac4d18a4e002 -- fixed. The claim was false as written, and for the reason given: the expiry now starts at The comment now says only that an idle interpreter selects the same 0.1s EXPIRY it |
|
span=5cff7c1f0a24 -- fixed. A security pin that skips on a platform is not a pin there, and the two tests that
The end-to-end symlink tests keep their skip, which is correct -- they assert the |
|
span=c2e5c92f85c1 -- fixed. "Byte-for-byte unchanged" was wrong for the same reason as the module comment: the The spec now says an unloaded host selects the same 0.1 s expiry it selects today, |
|
span=7751e3f8029e -- accepted as stated, narrower pin tracked, coverage gap named The observation is correct and worth stating plainly: The direction of the mismatch is the safe one, which is why it ships this way: the That cost is no longer silent, which was the other half of making this acceptable. The narrower crew-home-scoped pin is tracked as #12714, with the shape it needs So the remaining step is the one this item names: a maintainer accepting the |
|
span=a26aa8d4895d -- rejected, with the reason named. Same theme as The grep is right: nothing in-repo sets It stays because the maintainer's acceptance condition for this change requires Two things narrow the exposure meanwhile. The cap applies ONLY where no target |
|
span=2ad466927c5c -- rejected, with the reason named. Third round on this theme The cost is real and not disputed: the script reads private names of the security It stays because a committed, runnable measurement is an acceptance condition of Per the same-span rule I am not patching this a third time. The conflict is between |
|
span=5cff7c1f0a24 -- answered with what changed and what I am forbidden to The first raise was taken and fixed rather than argued: The re-raise is about the remaining
What I have NOT established, and am not claiming: whether the runner actually skips |
26345d9 to
0012a73
Compare
GPT 5.6 Review —
|
GPT 5.6 Review —
|
|
Answer to the Security Scope Review CONCERNS on Taking the three unadjudicated candidates in turn, because they are not equal. (1) The pin is per-build, not per-entry, so one managed dotfile pins every build. (2) A builder returning a plain (3) An operator setting the cap above 300 has the value "silently replaced by the and, for an unparseable value:
On the held verdict. This run classified So the remedy will be the re-measure the lane asks for, not an override. It is |
…cost The sensitive-path gate caches its protected-anchor set for a fixed 0.1s. That value is sized for an idle interpreter, where one rebuild costs about 2ms and the cache spends about 2% of the wall clock rebuilding. The rebuild is ~130 realpath calls and every syscall hands the GIL over, so its cost is set by CPU contention rather than by the disk: measured on a 64-core Linux host with one local xfs mount, 2ms idle and 647/870/2760/ 3906ms with 1/2/4/8 sibling threads running pure Python. A fixed expiry does not move with that, so under load the gate re-pays a multi-second rebuild every 100ms until it misses its budget and refuses ordinary project files as sensitive, with the filesystem healthy throughout. The expiry now tracks the measured cost of the build it expires: clamp(cost * _HOME_TARGETS_TTL_COST_RATIO, _HOME_TARGETS_TTL_SECS, _HOME_TARGETS_TTL_MAX_SECS). The ratio is the reciprocal of the share of the wall clock the gate may spend rebuilding, so that share holds at 2% at every load level instead of only at the one the constant was sized for, and the rebuild's own duration is the load signal. At idle the ratio lands exactly on the floor, so an unloaded host is unchanged. The expiry also starts after the build rather than before it, which a 2ms build hides and a 0.4s build does not. Measured with two contending threads: 14 to 29 rebuilds and up to 86,467 refusals of ordinary project files under the fixed expiry, against 3 rebuilds and zero refusals under the law. _HOME_TARGETS_ADAPTIVE_TTL pins the expiry at the floor for an operator who wants the old shape. Trade, pending maintainer decision: a longer expiry lengthens the window in which a symlink swapped deeper inside the crew home is answered from the stale set, now bounded by the 30s cap rather than by 0.1s. Neither reproduced bypass widens, because both repoint an anchor and every anchor is part of the cache key; the regression test is repeated at the cap. scripts/measure_path_gate_ttl.py reproduces the measurement. Fixes #10255
0012a73 to
2d87346
Compare
…cost (kirodotdev#12664) The sensitive-path gate caches its protected-anchor set for a fixed 0.1s. That value is sized for an idle interpreter, where one rebuild costs about 2ms and the cache spends about 2% of the wall clock rebuilding. The rebuild is ~130 realpath calls and every syscall hands the GIL over, so its cost is set by CPU contention rather than by the disk: measured on a 64-core Linux host with one local xfs mount, 2ms idle and 647/870/2760/ 3906ms with 1/2/4/8 sibling threads running pure Python. A fixed expiry does not move with that, so under load the gate re-pays a multi-second rebuild every 100ms until it misses its budget and refuses ordinary project files as sensitive, with the filesystem healthy throughout. The expiry now tracks the measured cost of the build it expires: clamp(cost * _HOME_TARGETS_TTL_COST_RATIO, _HOME_TARGETS_TTL_SECS, _HOME_TARGETS_TTL_MAX_SECS). The ratio is the reciprocal of the share of the wall clock the gate may spend rebuilding, so that share holds at 2% at every load level instead of only at the one the constant was sized for, and the rebuild's own duration is the load signal. At idle the ratio lands exactly on the floor, so an unloaded host is unchanged. The expiry also starts after the build rather than before it, which a 2ms build hides and a 0.4s build does not. Measured with two contending threads: 14 to 29 rebuilds and up to 86,467 refusals of ordinary project files under the fixed expiry, against 3 rebuilds and zero refusals under the law. _HOME_TARGETS_ADAPTIVE_TTL pins the expiry at the floor for an operator who wants the old shape. Trade, pending maintainer decision: a longer expiry lengthens the window in which a symlink swapped deeper inside the crew home is answered from the stale set, now bounded by the 30s cap rather than by 0.1s. Neither reproduced bypass widens, because both repoint an anchor and every anchor is part of the cache key; the regression test is repeated at the cap. scripts/measure_path_gate_ttl.py reproduces the measurement. Fixes kirodotdev#10255
…cost (kirodotdev#12664) The sensitive-path gate caches its protected-anchor set for a fixed 0.1s. That value is sized for an idle interpreter, where one rebuild costs about 2ms and the cache spends about 2% of the wall clock rebuilding. The rebuild is ~130 realpath calls and every syscall hands the GIL over, so its cost is set by CPU contention rather than by the disk: measured on a 64-core Linux host with one local xfs mount, 2ms idle and 647/870/2760/ 3906ms with 1/2/4/8 sibling threads running pure Python. A fixed expiry does not move with that, so under load the gate re-pays a multi-second rebuild every 100ms until it misses its budget and refuses ordinary project files as sensitive, with the filesystem healthy throughout. The expiry now tracks the measured cost of the build it expires: clamp(cost * _HOME_TARGETS_TTL_COST_RATIO, _HOME_TARGETS_TTL_SECS, _HOME_TARGETS_TTL_MAX_SECS). The ratio is the reciprocal of the share of the wall clock the gate may spend rebuilding, so that share holds at 2% at every load level instead of only at the one the constant was sized for, and the rebuild's own duration is the load signal. At idle the ratio lands exactly on the floor, so an unloaded host is unchanged. The expiry also starts after the build rather than before it, which a 2ms build hides and a 0.4s build does not. Measured with two contending threads: 14 to 29 rebuilds and up to 86,467 refusals of ordinary project files under the fixed expiry, against 3 rebuilds and zero refusals under the law. _HOME_TARGETS_ADAPTIVE_TTL pins the expiry at the floor for an operator who wants the old shape. Trade, pending maintainer decision: a longer expiry lengthens the window in which a symlink swapped deeper inside the crew home is answered from the stale set, now bounded by the 30s cap rather than by 0.1s. Neither reproduced bypass widens, because both repoint an anchor and every anchor is part of the cache key; the regression test is repeated at the cap. scripts/measure_path_gate_ttl.py reproduces the measurement. Fixes kirodotdev#10255
Problem / Motivation
Under CPU load the sensitive-path gate refuses ordinary files. An agent reading
a source file, running
git statusor writing a scratch file getsaccess to sensitive path: <ordinary path>, and then every path under thatprefix is refused for 30s, 60s, 120s as the cooldown doubles.
The disk is fine the whole time. One host logged 57 of these in a day with a
healthy local mount.
Why it matters
The refusal is wrong twice. The file does not hold a credential, and nothing
was locked down. An agent that reads the message concludes one of those anyway
and burns rounds on it. Worse, a worker session can be refused at startup: the
gateway reads its own agent spec through this gate, and a refused read reports
the spec as unreadable, so the worker never starts.
It is also the only pending change that lowers the refusal RATE on a single-user
host. The sibling fix in #12647 (
84a9edfab1, merged and in this PR's base) givesthe stall cooldown key one more path component, which separates one user's tree
from another user's. It does not separate a repo, a workspace, a data home and a
worktree that all sit under the SAME user, so on a one-user host the blast radius
is unchanged by design. Measured on such a host with that fix live: three
consecutive refusals inside one minute on three DIFFERENT cooldown keys, one of
them keyed off the literal text of a shell command, because a command string is
handed to the gate as a path (#11223 is the fix for that half). Different keys
refusing at the same moment means the firing branch is prefix-independent, so
re-scoping the key cannot reach it. This change removes the resolver work rather
than re-scoping its damage, which is why it is not redundant with either sibling.
What changed (motivation → approach → change)
The gate caches the protected anchors it compares paths against, and that cache
expires after 0.1s. That is sized for an idle interpreter, where one rebuild costs
2ms, so re-paying it every 100ms spends 2% of the wall clock. But the rebuild is about 130
os.path.realpathcalls, and every syscall hands the GIL over. Its cost is setby how many threads compete, not by the disk:
A fixed expiry does not move with that. At two busy threads the gate pays an
870ms rebuild ten times a second, misses its 8s budget, and refuses a healthy
file.
So the expiry now tracks what the build cost:
The ratio is 50, the reciprocal of the share of wall clock the gate may spend
rebuilding, so
ttl = cost * 50holds that share at 2% at every load level andthe rebuild's own duration is the load signal. No load metric is read, nothing is
configured per host. At idle
2ms * 50is exactly the floor, so an unloaded hostis unchanged.
Both are operator knobs:
KIROCREW_PATH_GATE_TTL_COST_RATIO(0 to 1000) andKIROCREW_PATH_GATE_TTL_MAX_SECS(0.1 to 300), read once at import, fail-soft tothe reviewed default. A ratio of 0 is the revert: it clamps to the floor for
every cost, so one env var restores a fixed 0.1s expiry. The ceiling's own ceiling
stops an env var widening the window past the bound.
The expiry also starts at the clock read taken after the build. Invisible for
a 2ms build; for an 870ms one, an expiry started before it has already elapsed on
return, so the entry is dead on arrival.
Measured, 15s per cell, one caller gating project files:
The cap is measured too. At ratio 50, a 10s cap pays 7 and 6 rebuilds,
30s pays 6 and 3, 60s pays 6 and 5: the count stops falling at 30s, so a wider
window buys nothing.
src/kiro_crew/executors.pyand the security system spec asserted a fixed cache;both now describe the cost-tracking expiry.
What this does not fix. An expiry controls how often the rebuild is paid,
never what one costs. Past about four busy threads one cold rebuild exceeds its 8s
budget, and every expiry refuses alike. Separate work.
Disclosed limitation, on the Security Scope Review's own point. The floor pin is
per BUILD, not per entry. So on a host whose sensitive dotfiles are symlinks, and a
stow or chezmoi home is the ordinary case rather than an exotic one, a single managed
dotfile anywhere in the target set makes every build report a traversal. Every expiry
is then the floor, this fix does not apply on that host, the refusals #10255 describes
persist, and both knobs read as no-ops to whoever tunes them. Two things bound it: the
direction is the safe one, since the floor is the shorter window and therefore today's
behaviour, and
_report_expiry_pinlogs that state once per transition so it can beattributed from one log line instead of being re-reported as a fresh defect.
Narrowing the pin to traversals at or below the resolved crew home is tracked as
#12714.
The trade, and why the reachable case no longer takes it. A longer expiry
would lengthen the window in which a symlink swapped deeper inside the crew
home, a keystone leaf or a directory on the way to one, is still answered from the
stale set. So the build now reports whether it traversed one: the returned set
carries
resolution_differed, read off the memo the build already filled, truewhen some target's canonical form differed from its lexical one, which is exactly
the entry a repoint can strand. A true flag pins the floor. The install where the
stale-credential path is reachable therefore keeps its 0.1 second window, and
the long expiry applies only where no target resolved differently, where reaching
the case would first require creating a symlink inside the crew home, a write
is_sensitive_write_pathrefuses. The flag defaults to true in both places it isread, so a builder that reports nothing gets the floor. Roots cannot trip it:
_resolve_root_anchorshas already canonicalised them, so a symlinked$HOMEstill earns the adaptive expiry. For the symlink-free install the age a caller can
be served is the build's own duration plus the expiry, so the ceiling plus the 8s
rebuild budget, not the expiry alone. And because an install whose dotfiles are
symlinks (stow, chezmoi) pins the floor on every build, the module logs once per
TRANSITION which expiry is in force, so the availability half cannot self-disable
in silence and leave both knobs reading as no-ops to whoever tunes them.
Neither reproduced bypass widens: both repoint an anchor, and every anchor is
part of the cache key, so a repointed
$HOMEorKIROCREW_HOMEmisses at anyexpiry.
test_repointed_home_symlink_is_not_served_from_cacheis untouched andpasses;
test_repointed_home_symlink_is_not_served_from_cache_at_the_longest_expirypins that at the cap, on the one install shape that can still select it.
Backwards compatibility
Compatible: the gate's verdict for any path is unchanged, only how long a
correct anchor set is reused. An idle host gets the same 0.1s expiry it has
today, arithmetically rather than approximately. No name is removed or renamed;
three constants and one helper are added and exported.
Tests
test/test_security.py::TestAdaptiveHomeTargetsExpiry, seventeen tests. One pernamed input: the floor is what an idle rebuild selects, the expiry scales with
cost between floor and cap, the cap binds at the measured worst case, a zero or
negative cost falls to the floor, and a ratio of 0 pins the floor. Two end to
end: a costly build is not re-paid at the old fixed expiry, and a build costing
more than its own expiry is not expired on arrival. One keeps the committed
measurement script honest, asserting every private name it reads still exists,
because nothing in CI runs a contention sweep.
Eight on the trade and the knobs. Five on the flag: a symlinked keystone leaf
inside the crew home pins the floor and the repoint is live one floor-tick
later, an install with no symlinked leaf keeps the long expiry and is still
expiry-bounded, a builder returning a plain
setgets the floor at both defaults,a build that REPORTS a traversal gets the floor without any symlink being created
(so the pin also runs on a platform that refuses to make one, where its
end-to-end sibling skips), and the pin is reported to the log once per transition
rather than once per rebuild. The other three pin the knobs: the ceiling cannot be
raised without bound, a bad value keeps the reviewed default, and the shipped
defaults make an idle rebuild land exactly on 0.1s.
test_repointed_home_symlink_is_not_served_from_cacheis unmodified andpasses. The two tests here whose contract needs a real symlink carry no skip guard
of their own: their exact node ids are listed in
test/requires-real-symlinks.txt, so the root conftest's capability probe ownsthat decision and an elevated or Developer-Mode Windows runner keeps the coverage
rather than silently losing it.
TestHomeDirTargetsCacheandtest_security_path_resolve_bounded.py(71 tests) keep passing; the frozen-clock jumps now read the effective expiry
through
_home_targets_ttl(0.0)rather than the floor constant, so they stayhonest if the floor is ever reached differently.
Fifteen mutations, each breaking one line of the implementation, each red on its
own pin and green when restored. The eight marked
*were re-run against this head;the rest break lines this round did not touch and were verified when they landed:
*flag never pins the floor (early return disabled)*build stops reporting the symlink*cache read defaults to "no symlink"*class default flips to "no symlink"*cache key drops the resolved home*cost not measured (fed a constant zero)*the pin report stops deduplicating*the cache fill stops reporting the pinminremoved)maxremoved)Manual verification
scripts/measure_path_gate_ttl.pyis the measurement, committed so the numbersabove can be re-derived rather than trusted:
It drives the shipped law, reaching the fixed-expiry baseline through the
module's own switch, and ranks policies on refusals first because a cooldown
answers instantly and would otherwise look fast.
rebuild --hogs 0,1,2,4,8alone shows the mechanism in about a minute.
Also verified by the defect itself: this change was written on a loaded host and
the gate refused the author's own reads and edits of
paths.pyrepeatedly,each time clearing on an identical retry.
Related Issues
Fixes #10255
Pattern harvest
Rule candidate: review-prompt
Pattern: a cache TTL sized against an idle measurement, guarding work whose cost
is set by contention rather than by the resource it touches. The expiry is a
constant while the thing it amortises varies by three orders of magnitude, so
the amortised share silently rises with load until a budget breaks. Worth asking
of any TTL whose comment cites a single measured cost.
Notes for reviewers
The security trade here is approved, not proposed. Raymond ruled on it on
2026-09-21, and it has since NARROWED. The review's finding that a repointed
keystone leaf is reachable on an ordinary contended host was correct, so the build
now reports whether it traversed a symlink and that install is pinned to the
0.1 second floor instead of the cap. What is still traded is the symlink-free
install, where reaching a stale set requires first creating a symlink inside the
crew home. What it buys is the removal of a gate that currently fails closed
machine-wide under ordinary CPU load: escalating 30/60/120-second refusals of
healthy files across a whole path prefix, plus refused worker-session starts. A
gate that refuses healthy files is itself the failure, so the narrower exposure is
the better one. The pins are
test_a_symlinked_keystone_leaf_pins_the_expiry_at_the_floorand
test_a_builder_that_reports_nothing_gets_the_floor, andtest_repointed_home_symlink_is_not_served_from_cacheis untouched on purpose.A separate change will move the resolution out of process, which is what makes
a single cold rebuild cheap under contention. It is deliberately not in this PR.
It does not supersede this one either: this removes the repeated cost, which is a
different quantity from the cost of one rebuild, and the cache is still what keeps
the gate off the filesystem between builds.
Findings from the earlier rounds are fixed here rather than argued: the revert
path is an env var rather than a private constant, the measurement script's
coupling to private names is pinned by a test that derives the name list from the
script, and the reachable stale-credential path is closed by the flag above.
Issue #12688 carries the remaining prose corrections to the window's arithmetic
and a proposal to lower the operator ceiling to what the sweep supports.
Not in this diff: the per-prefix cooldown key (
_stall_prefix), which PR#12647 owns, and a positive cache for the candidate-path side of the check.
One scope question is open with the maintainers, so treat it as undecided
rather than answered here: whether the load-arm probe allowance
(
_PATH_RESOLVE_LOAD_MAX_PROBES, 3 per prefix per 10s window) should be tuned inthis PR. My reading of the measurement is that it is not the lever, because past
four contending threads a single cold rebuild is already over budget and more
uncharged probes would only spend more event-loop time for the same refusal. That
is an argument, not a ruling, and the allowance is untouched while it is pending.
Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)