Repository navigation
fix(security): stop one slow path resolution from fencing a whole drive - #10276
Conversation
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Sound, well-pinned fix, but the new 12 s wait allowance is sized to the watchdog's default, not its configurable floor of 10 s. WatchThe allowance arithmetic hardcodes the watchdog at 25 s ("Allow 12s total … leaving 13s of the 25s watchdog"), but Suggestions
[DESIGN-REVIEWED] 6c4d63e |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of All lenses run and counts verified. Final review: First-Principles-Verdict: PASS Verify on one real slow-disk host that the 12 s allowance refusing healthy paths for 25 s stays invisible during ordinary heavy tool calls. That arm is the only one that refuses more than base, its trade is argued from the in-repo 25 s watchdog ( What this change shipsInventory (7 items) — 7 justifiedIntent: stop one slow path resolution from refusing paths host-wide (and from killing the gateway) — a FIX.
No deleted pins (test hunks are pure additions); no duplicate mechanism (existing bounds are per-prefix, none per-thread); no unfixed siblings of the drive-arithmetic class (5 other [FIRST-PRINCIPLES-REVIEWED] 6c4d63e |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
Security Scope Review -- 🟡 CONCERNSWhich legitimate operations does Verdict basis: the reviewer's own judgement; no confirmed regression. Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. Scope-Verdict: CONCERNS TIGHTENING: A per-thread cumulative wait allowance inside the bounded sensitive-path resolver ( ADJUDICATED: 13 candidates, 0 confirmed newly-refused PLATFORM COVERAGE: Candidates were written in posix (7 rows) and windows (6 rows) spellings; none in UNADJUDICATED: The tightening itself — the cumulative wait allowance — is a temporal, state-dependent guard inside a path resolver, and the classifier adjudicates shell command strings only. Call path an input takes to reach it: any tool-call command carrying a path → the composite's path fence → [SCOPE-REVIEWED] 6c4d63e |
ede524c to
36f8978
Compare
|
The cumulative path-resolution deadline fix is prepared locally. Its validation exposed an independent app dependency-lock creation failure that also reproduced on clean main. That prerequisite is now isolated in #10464, with serial local regression tests passing and the full platform checks running in CI. Keeping the two changes separate; the deadline revision will follow once the prerequisite lands. |
A sensitive-path check that misses its resolve budget charges a COOLDOWN to the path's prefix, and every path under that prefix is then refused -- fail-closed, without touching the filesystem -- until it lapses. Two things made that blanket far wider than intended on Windows, where one transient miss fenced essentially the whole host for 30s (doubling to 30min). `_stall_prefix` counted path components without splitting the drive. A POSIX absolute path carries a leading empty component, a Windows path does not, so "the first two components" of `C:\Users\bob\...` kept `C:\Users` -- a key that contains $HOME, %TEMP%, the workspace and the checkout. It now splits the drive first, keying on `C:\Users\bob`, and returns a UNC share root whole because `\\server\share` IS the mount point. Byte-identical on POSIX. A missed budget was also treated as proof of a wedged mount. The syscall probe that separates "wedged" from "merely slow" reads `/proc/<tid>/syscall` against a per-architecture table, and `platform.machine()` is `AMD64` on every Windows host and `arm64` on Apple silicon -- neither is mapped, so the probe returned True unconditionally and the prefix was charged for a resolution that was only slow. A miss now earns a bounded GRACE to finish, and one that finishes is a success that charges nothing. The grace is a capped fraction of the caller's own budget, so a caller that chooses a tight budget keeps a tight worst case. A miss of both the full budget and the full grace still charges and still refuses, so the fail-closed stance is unchanged. Each budget is now sized to the work its caller submits: the anchor REBUILD is one pool job doing ~130 realpath calls to build ~200 targets, where a candidate resolution does one or two, so sharing one budget left the rebuild running ~130x closer to its ceiling. It carries its own, larger budget. Every bound above is per resolution, and one tool call carries many path tokens. Tokens under DIFFERENT slow prefixes each pay their own budget and grace, which the per-prefix cooldown cannot collapse, so serial waits inside one `on_tool_call` summed past the 25s loop-stall watchdog and exited the gateway -- the crash the whole bound exists to prevent. Each calling thread now shares a 12s allowance across every prefix and anchor job, counting the real time spent in each result wait that is long enough to be filesystem latency rather than pool round-trip overhead. A wait is clamped to what is left of the allowance, and once it is gone the next resolution is refused without being submitted. The allowance frees 25s after that thread's last counted wait, so two spans cannot combine inside one watchdog gap, and a background thread carries its own allowance and never spends the event loop's. A wait clamped short by that allowance charges no prefix: it ended for a reason that says nothing about the mount, so it refuses that one call, the conclusion the saturated-pool and starved-worker arms already reach. Charging requires a resolution to have received its full entitled budget and full entitled grace and still not finish, so no bound in this module can make another one look like a wedged mount. The test floor resets the four process-global structures per test. They are per-worker state that makes `is_sensitive_path()` answer True for a whole prefix, so one test whose resolution was slow could fail every later test on the same xdist worker -- observed as 77 `ArtifactError: refusing to use sensitive path as artifact root` errors across four unrelated files on one Windows shard, from a single charged stall. `_path_resolve_degraded` and `_path_resolve_wedged` are set through the `security` facade so the export-identity contract holds; `_path_resolve_load_probes` and `_path_resolve_thread_waits` are not re-exported and are set on the submodule.
36f8978 to
6c4d63e
Compare
|
self-added: yes
Tests that fail without the cap: an unsubmitted resolution and a clamped wait each |
|
self-added: yes
The grace still charges nothing when it completes, which was the intended trade you |
Summary
Split out of #9585, where the Windows shards surfaced it. That PR does not touch
src/kiro_crew/security/, so the defect is pre-existing on main — #9585 onlyraised Windows-runner load enough to reach it. It also cannot be fixed in #9585:
that PR's diff is over the Security Scope Review lane's 1 MB cap, so any
security-scope change there fails the lane closed. Hence its own PR, where the
lane can actually review it.
A sensitive-path check that misses its resolve budget charges a cooldown to the
path's prefix, and every path under that prefix is then refused — fail-closed,
without touching the filesystem — until it lapses. Three independent bugs made that
blanket far wider than intended, and a fourth left the total wait unbounded.
1. The stall prefix collapsed to
C:\Users_stall_prefixkeeps "the first two components" so a cooldown is scoped to thewedged mount. But it counted components without splitting the drive: a POSIX
absolute path carries a leading empty component, a Windows path does not, so
C:\Users\bob\AppData\...keptC:\Users— a key containing$HOME,%TEMP%,the workspace and the checkout. One stall anywhere in the profile fenced
essentially the whole host.
Now splits the drive first (
C:\Users\bob), and returns a UNC share root wholebecause
\\server\shareis the mount point. Byte-identical on POSIX, pinnedby a test that drives both
ntpathandposixpath.2. A missed budget was treated as proof of a wedged mount
The probe that separates "wedged" from "merely slow" reads
/proc/<tid>/syscallagainst a per-architecture syscall table.
platform.machine()isAMD64on everyWindows host (and
arm64on Apple silicon) — neither is mapped, so the probereturned
Trueunconditionally and the prefix was charged for a resolution thatwas only slow.
A miss now earns a bounded grace to finish; one that finishes is a success that
charges nothing. The grace is a capped fraction of the caller's own budget, so a
caller choosing a tight budget keeps a tight worst case. A resolution that misses
its full budget and its full grace still charges and still refuses — the
fail-closed stance is unchanged.
3. Budgets are sized to the work submitted
The anchor rebuild is one pool job doing ~130
realpathcalls to build ~200targets; a candidate resolution does one or two. Sharing one budget left the
rebuild running ~130× closer to its ceiling, which is what made the miss reachable.
It gets its own, larger budget.
4. One thread's resolver waits are capped together
Every bound above is per resolution. One tool call carries many path tokens, and
tokens under different slow prefixes each pay their own budget and grace, so the
per-prefix cooldown never collapses them: six serial waits inside one
on_tool_calladd up past the 25 s loop-stall watchdog and the gateway exits —the crash this whole bound exists to prevent.
Each calling thread shares a 12 s allowance across every prefix and anchor job,
counting the real time spent in each result wait: successes, timeouts and graces
alike. A wait is clamped to what is left, and once the allowance is gone the next
resolution is refused without being submitted at all. The allowance frees 25 s after
that thread's last wait, so two spans cannot combine inside one watchdog gap. A
background thread carries its own allowance and can never spend the event loop's,
which leaves 13 s of the watchdog budget for heartbeat age and the rest of the call.
A wait cut short by that clamp charges no prefix. It ended for a reason that says
nothing about the mount, so it refuses that one call — the same conclusion the
saturated-pool and starved-worker arms already reach.
🟩 added · 🟨 changed · 🟦 unchanged
A charge now needs the resolution to have been given everything it was entitled to
and still not finish, so no bound in this module can make another one look like a
wedged mount.
Why it showed up as artifact errors
The degradation structures are process-global and per-xdist-worker, so one test
whose resolution was slow failed every later test on that worker whose paths sat
under the same prefix — observed on a #9585 Windows shard as 77
ArtifactError: refusing to use sensitive path as artifact rootfailures across four unrelatedfiles, all from a single charged stall (one
stall #1warning in 23,894 outcomes;the cascade arm logs nothing). The test floor resets that state per test, so the
cascade can neither be inherited nor exported.
_path_resolve_degraded/_path_resolve_wedgedare set through thesecurityfacade so the export-identity contract
test_security_facadepins still holds;_path_resolve_load_probesand_path_resolve_thread_waitsare not re-exported andare set on the submodule.
Tested
test_security_path_resolve_bounded.py,test_security_facade.pyandtest_artifacts.pygreen — 327 passed.failing-test sets are identical, so this change adds no failure. (That host fails
98 tests either way, on its own layout —
/local/homeowned byuid 65534, anaarch64machine whoseunistd_64.hcarries ARM numbers, and no session dbus.)sync-io / docs-lint gates clean.
reverting
_stall_prefixfails with the literal old valueC:\Users; the gracetest asserts the same slow resolution does charge the prefix with the grace
disabled; the allowance tests assert an unsubmitted resolution and a clamped wait
each leave
_path_resolve_degradedempty, and fail against an implementationwithout the cap. So no mechanism here can be removed while the tests pass.
Security posture
The gate never answers "not sensitive" more often than before. Every arm here
raises
PathResolutionStalled, so a path whose canonical form cannot beestablished is still refused rather than matched on its lexical spelling.
Three of the four changes move refusals toward allowed. The thread allowance is the
one that refuses more: a thread that has already spent 12 s waiting has its
healthy paths refused too, until the allowance frees. That is deliberate. The
alternative on that path is the watchdog killing the gateway, and answering from the
lexical spelling instead would build exactly the lever this module refuses to build
— stall one token under a wedged mount and a workspace symlink into a credential
store passes.
Pattern harvest
Rule candidate: semgrep
Pattern: path-component arithmetic on
split(os.sep)withoutos.path.splitdrivefirstThree defects here are instances of a class, not one-offs.
The prefix bug is component-index arithmetic over a raw
split(os.sep). Any codethat slices a path into "the first N components" is silently wrong on Windows, because
a drive-qualified path has no leading empty component and
C:occupies one of the N.A semgrep rule can flag
\.split\(os\.sep\)(andPath(...).parts) followed by indexor slice arithmetic in a function that has not called
splitdrive, which is cheap tomatch and has few legitimate exceptions.
The stall-attribution bug is a second, more interesting class that a linter cannot
see: a capability probe whose "cannot answer" is spelled the same as one of its two
real answers.
_worker_blocked_in_filesystemreturnsTrueboth for "this worker isblocked in the filesystem" and for "this architecture is not in my table", so every
caller on an unmapped platform silently took the wrong branch — and the docstring
described that as failing toward existing behaviour, which hid it. The reviewable rule
is a review-prompt/AGENTS.md line: a predicate that cannot always answer should return
a three-valued result (or expose a separate
supportedflag) rather than folding"unknown" into one of the answers, so callers must handle the unknown case explicitly.
That is the same shape as the
agent_namedmarker discussed on #9779 and thefloorGen === undefinedgate on #9646 — "treat absent as compatible" — so it is worthstating once as a rule rather than fixing a third time.
The unbounded total is a third class, and the one worth a review-prompt line of its
own: a per-operation bound cited as if it were a per-caller bound. The comment
justifying the grace reasoned "8 s + 4 s stays well inside 25 s", which is true of one
resolution and says nothing about a caller that performs six. A bound whose stated
justification names a budget belonging to a different scope than the loop the bound
protects is reviewable by reading the comment alone: if the ceiling is per-turn or
per-request, the arithmetic beneath it has to sum over everything one turn can do, not
over one call.