Skip to content

fix(security): key the resolver stall cooldown on the home directory, not on its container - #12647

Merged
bolichen97 merged 1 commit into
mainfrom
fix/stall-prefix-home-container-12386
Sep 22, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/stall-prefix-home-container-12386

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

On a host whose home directories sit one level deeper than usual -- /local/home/<user>, with /home/<user> a symlink into it (the standard cloud-desktop layout, already named in #8022) -- the resolver-stall cooldown collapses to one key for the whole machine.

_stall_prefix charges a stall to the first two path components. On this layout that is /local/home, which contains every user, every workspace and checkout, the data home and the credential stores. Measured on origin/main (#12386): five distinct paths under different users, directories and stores all key to /local/home; the same function keys /home/alice/... and /home/bob/... to two per-user keys, as designed.

The function's own docstring already describes this exact failure mode -- for Windows: counting components without splitting the drive collapsed every user path to C:\Users, "so one stall anywhere in the profile refused path resolution for essentially the whole host". That case was fixed by the drive split. A POSIX home at depth three reproduces the same collapse and was not covered.

Why it matters

The sensitive-path gate runs before every other check on every tool call (hooks.on_tool_call), so while the one key's cooldown is open every file read, write, grep, shell command and MCP call on the host is refused without touching the filesystem, in every session at once, with a refusal that names an ordinary project file. The user sees an agent that has apparently lost its tools. On a /home/<user> host the same stall costs one user's tree for 30 s; here it costs the whole machine, and repeat stalls under the shared key double that window toward the 1800 s ceiling with a counter that every session on the host feeds.

Two of the issue's three consequences follow from the single key and are what this PR removes:

  1. Blast radius is the whole host instead of one home.
  2. The early-degrade arm (history is not None and wedged >= W - 1) sees a stall history for every path once any stall has been charged anywhere, so on the shipped two-worker pool a single pinned worker refuses everything.

The third consequence as originally filed ("never decays") was withdrawn by the reporter and is confirmed wrong in code: _run_resolution_bounded pops the prefix's entry on any completed resolution under it.

What changed (motivation -> approach -> change)

Symptom: whole-host refusal for the cooldown after one stall. Root cause: the cooldown key is a fixed component count, and on this layout the second component is the container of homes, not a home. /local/home is to this host what C:\Users was to Windows.

Approach. The key must stay lexical: _stall_prefix is consulted before deciding whether to probe at all, and it names the prefix charged when the $HOME anchor itself stalls, so it cannot touch the filesystem. That rules out the obvious mechanical fix -- a mount-derived key. os.path.ismount walked upward would lstat the very mount that may be wedged, and on this host (one local xfs at /, no network, CIFS or FUSE mount under the home) it reaches /: a process-wide key, worse than the collapse. Anchoring on $HOME's parent does not work either: on the affected host $HOME is the /home/<user> spelling, so its parent is /home and every /local/home/... path would still collapse; it would also make the key depend on the environment.

Change. The key never ends on a container of home directories. A component spelled home is such a container -- /home/<user> is the common layout, and the layouts that put homes one level deeper all keep the word: /local/home/<user> (cloud desktop), /usr/home (FreeBSD, where /home is itself a symlink to it), /var/home (Fedora Silverblue, same), /export/home (Solaris, NFS servers). When the second component is that container and the path continues, the key takes one more component, so the boundary is the home directory itself: /local/home/<user>/oss/x -> /local/home/<user>, exactly the key a /home/<user> layout already gets. Users needs no entry: it only ever sits at depth one (/Users/<user>, C:\Users\<user>), where the two-component key already ends on the user.

Every other key is unchanged: /home/<user>, /tmp/<dir>, /Volumes/<share>, /net/<host>, C:\Users\<user>, UNC roots, relative paths, and the container alone (/local/home stays its own key when the path stops there). Two lines in _stall_prefix with the rationale beside them, the function's docstring, and the one sentence in docs/system-specs/modules/security.md that describes the key; nothing in _mark_stalled, the load arm, the TTL cache or the degraded threshold is touched.

What this does not buy, stated plainly. Two spellings of one tree (/home/<user> and its /local/home/<user> target) remain two keys, each paying its own timeout -- inherent to a lexical key, and the resolution that would unify them is the operation the key exists to avoid. When the container is itself one wedged mount (an NFS /export/home), each user's tree pays its own timeout and pins its own worker: that is the cost a /home/<user> autofs layout already carries per user, bounded the same way by the pool size (KIROCREW_PATH_RESOLVE_WORKERS, #12460), and it is the isolation this key exists for; narrowing the rule to the literal /local/home would leave /usr/home and /var/home hosts -- single local filesystems with the identical collapse -- unfixed. A single-user host still has one key for that user's whole tree; that is the designed per-user granularity, the same one /home/<user> hosts get today. The issue's item 1 (floor the early-degrade threshold at two workers) is a separate, independent change already claimed by another operator on the issue; item 2 (pool size) landed as #12460; why the budget expires at all on a healthy local disk is #10255 / #10256.

Backwards compatibility

Compatible: the only inputs whose key changes are absolute or relative paths whose second component is literally home and that continue past it; those now key one component deeper. The container by itself, every other layout and every Windows spelling key exactly as before, pinned by the three pre-existing _stall_prefix tests, which pass unchanged. The key is an in-memory index into the cooldown dictionary and is never persisted or exposed beyond the PathResolutionStalled message.

Tests

test/test_security_path_resolve_bounded.py:

  • test_the_stall_prefix_never_ends_on_a_container_of_homes -- the issue's reproduction inverted: two users' paths under /local/home/<user>/... are two keys, each the user's own home; the deeper layouts (/usr/home, /var/home, /export/home) key on the user; the container alone (/local/home, /home) is its own key; /home/x, /Volumes/share, /tmp/x are untouched. Driven through posixpath so the layout is asserted on every platform.
  • test_a_stall_under_one_home_leaves_a_sibling_home_resolving -- through the real cooldown machinery: a stall charged under one user's home refuses that user's other paths for free (no re-probe), while a second user's path under the same /local/home container is submitted and resolved during the first user's cooldown, and the second user's key carries no history.

Mutation-verified: with the two-line change disabled (keep += 0), both new tests fail -- the key test on its first assertion, the key being /local/home instead of the user's home; the behavioural test with PathResolutionStalled raised for the second user's path, i.e. the reported collapse -- while the three existing prefix tests still pass. With the fix restored the whole file passes: 73 passed under -n0 with an isolated KIROCREW_HOME and the pool at its shipped size (KIROCREW_PATH_RESOLVE_WORKERS unset). black, isort, flake8 clean on both files; mypy clean on security/paths.py.

Manual verification

On the affected host, with this branch's _stall_prefix, the five paths from the issue's measurement table key to /local/home/<user> (the one user's home) instead of /local/home; the /home/<user>/... spelling still keys to /home/<user>; /var/tmp/... still keys to /var/tmp. Everything else is a pure function of the path string and is covered by the tests above.

Any other suggestions on the work

Related Issues

Fixes #12386

Related: #8022 (introduced _stall_prefix), #12460 (pool size knob, item 2), #10255, #10256 (why the budget expires).

Pattern harvest

Rule candidate: review-prompt
Pattern: a key or prefix derived by counting path components stands in for a filesystem boundary (mount, home) and must be checked against the layouts where that boundary sits one level deeper -- C:\Users\<user> and /local/home/<user> were the same defect in the same function, three weeks apart.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable) -- _stall_prefix docstring and the key's description in docs/system-specs/modules/security.md
  • No secrets, credentials, or internal references in the diff

@chenmingwei23
chenmingwei23 requested review from a team and bolichen97 September 21, 2026 23:08
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 21, 2026
@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of aead815f6c2ff05818b96459d47e33ff22f8bf85 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

A home container not spelled home (e.g. /srv/homes/<user>) still collapses to one key — confirm no supported layout uses one.

[DESIGN-REVIEWED] aead815

@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ PASS

Premise-level review of aead815f6c2ff05818b96459d47e33ff22f8bf85 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Confirm issue #12386 matches the description's account (measured collapse, withdrawn third consequence) — this run could not fetch it; the base-failing test is verified by reading.

What this change ships

Inventory (4 items) — 4 justified

Intent: FIX — stop one path-resolution stall under /local/home/<user> from refusing every tool call on the whole host for the cooldown window (#12386).

  1. A stall under one user's home at /local/home/<user> now refuses only that home, not every path on the host — justified
  2. The same per-user keying covers /usr/home, /var/home, /export/home layouts with the identical collapse — justified
  3. The one spec sentence describing the cooldown key updated in the same commit — justified
  4. Two new tests pin the per-home key and the sibling-home blast radius, failing on base — justified

[FIRST-PRINCIPLES-REVIEWED] aead815

@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed aead815f6c2ff05818b96459d47e33ff22f8bf85 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] aead815

Verdict parsed from the review's SHA-scoped output markers for commit aead815f6c2ff05818b96459d47e33ff22f8bf85.

False positive or not applicable? A repository writer can comment:
/ai-review override fable aead815f6c2ff05818b96459d47e33ff22f8bf85: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of aead815f6c2ff05818b96459d47e33ff22f8bf85 and found no blocking issues.

This comment is updated in place on each push.

Review details

FINDING -- docs/system-specs/modules/security.md:468 -- "one wedged mount costs one timeout per window" is false when one mount contains separately keyed homes -> Fix: say one cooldown key costs one timeout per window.
[GPT-REVIEWED] aead815

False positive or not applicable? A repository writer can comment:
/ai-review override gpt aead815f6c2ff05818b96459d47e33ff22f8bf85: <one-sentence reason>

@chenmingwei23
chenmingwei23 force-pushed the fix/stall-prefix-home-container-12386 branch from 1a1b3a1 to b4c9049 Compare September 21, 2026 23:18
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 21, 2026
@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Security Scope Review -- 🟡 CONCERNS

Which legitimate operations does aead815f6c2ff05818b96459d47e33ff22f8bf85 newly refuse? Updated in place on each push. A script-confirmed regression blocks; the model proposes candidates and never decides a refusal.

Verdict basis: the reviewer's own judgement; no confirmed regression.

Security scope review -- adjudicated candidates

  • macos-latest (platform posix): 12 classified, 0 newly refused, 12 still allowed.
  • ubuntu-latest (platform posix): 12 classified, 0 newly refused, 12 still allowed.
  • windows-latest (platform windows): 4 classified, 0 newly refused, 4 still allowed.

No candidate flipped from allowed to refused.

Scope-Verdict: CONCERNS

TIGHTENING: _stall_prefix in src/kiro_crew/security/paths.py — the key that scopes the fail-closed PathResolutionStalled cooldown — now takes one more component when the key would end on a container of homes: if len(parts) > keep and parts[keep - 1] == "home": keep += 1. The one threat the mechanism keeps refusing is a workspace symlink into a credential store passing on its lexical spelling while resolution is stalled — "A lexical-only match here would pass a workspace symlink into a credential store for the length of the stall" — now refused per user home (/local/home/<user>) rather than per home container (/local/home).

ADJUDICATED: 16 candidates submitted, 0 confirmed newly-refused. No confirmed rows were handed to this review; the adjudication legs run after capture and the deterministic fold reports their verdicts per leg.

PLATFORM COVERAGE: posix — rows for all four deep-home layouts the diff's comment enumerates (/local/home/<user>, /usr/home/<user>, /var/home/<user>, /export/home/<user>), the unchanged-key controls the diff assumes (/home/<user>, /Users/<user>, a plain directory), a relative path whose literal home component is repo content rather than a container, and the touched-area test invocation with the resolver's own KIROCREW_PATH_RESOLVE_WORKERS variable. windows — C:\Users\<user> (the layout the diff asserts needs no entry), C:\msys64\home\<user> (the one common Windows spelling where the new exact-match branch fires under ntpath), a UNC home share \\server\home\<user> (the drive-returned-whole arm the new branch sits beside), and the Windows spelling of the touched-area test run. Not covered, deliberately: the exact, case-sensitive match on "home" means spellings like C:\Msys64\Home\bob or non-home-named containers (/data/users/<user>) keep the OLD, broader key — that is pre-existing behavior, not a new refusal, so no candidate probes it. The per-leg verdicts are the fold's to report, not this section's.

UNADJUDICATED: The tightening lives in a path resolver's stall-cooldown machinery, not in a pattern the shell classifier can flip. Call path: shell command → path fence → is_sensitive_path → _candidate_forms → _resolved_forms_bounded → _run_resolution_bounded (src/kiro_crew/security/paths.py:1653 computes the key; :1656–:1672 refuse on an open cooldown or a pinned pool). The changed value is only consulted when a bounded realpath misses its 2 s budget, and the classifier legs run on healthy checkouts where no resolution stalls — so every allowed/allowed verdict my rows earn is evidence about the fence, not about this diff. Inputs I could not get a verdict for:

  • A sibling home resolving while one home's cooldown is open (/local/home/bob/... during a stall charged to /local/home/alice) — the intended improvement; the diff's own test test_a_stall_under_one_home_leaves_a_sibling_home_resolving pins it through the real machinery.
  • The arguable over-refusal, stated but unconfirmable here: when the container is itself ONE wedged mount (an NFS /export/home), the per-user keys give each user's first probe a fresh key with no stall history, so a second user's probe passes the :1659–:1661 gate and pins the second of the default two mc-pathres workers. With both pinned, _run_resolution_bounded refuses EVERY path process-wide without probing — including paths on healthy mounts — until a wedged stat returns, which on a hard-wedged mount is never. The old container-wide key converged to one pinned worker (a re-probe of a known-stalled prefix is refused unless it leaves a worker free), keeping healthy-mount resolution alive. The docstring accepts the trade ("bounded … by the pool size"), but that bound protects the pool from one prefix's re-probes, not from N fresh per-user keys under one dead mount. No classifier leg can observe this; it is argued, not confirmed.

[SCOPE-REVIEWED] aead815

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Sep 21, 2026
… not on its container

A host whose homes sit one level deeper than the usual layout keyed every path on the machine to the container of homes, so one resolver stall refused every tool call on the host for the cooldown. The key now never ends on a component named home: when the second component is a container of homes it takes the user too, so the boundary is the home directory itself, the key the usual layout already gets. Every other key is unchanged.

Fixes #12386
@chenmingwei23
chenmingwei23 force-pushed the fix/stall-prefix-home-container-12386 branch from b4c9049 to aead815 Compare September 21, 2026 23:46
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 21, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the full diff. Pure fix, two lines in _stall_prefix (security/paths.py:1456-1457) plus docstring, one spec sentence and two tests. Verified the new key can never collapse two trees (the old key is always a prefix of the new one) and cannot be unbounded by an attacker: entries are only created by an observed stall in _mark_stalled and both dicts stay capped at 64. The single-wedged-NFS-container trade-off is disclosed in the body and in the docstring. Both tests go red on revert.

@bolichen97
bolichen97 merged commit 84a9edf into main Sep 22, 2026
100 of 106 checks passed
@bolichen97
bolichen97 deleted the fix/stall-prefix-home-container-12386 branch September 22, 2026 02:25
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 22, 2026
amadsalmon pushed a commit to amadsalmon/KiroCrew that referenced this pull request Sep 22, 2026
… not on its container (kirodotdev#12647)

A host whose homes sit one level deeper than the usual layout keyed every path on the machine to the container of homes, so one resolver stall refused every tool call on the host for the cooldown. The key now never ends on a component named home: when the second component is a container of homes it takes the user too, so the boundary is the home directory itself, the key the usual layout already gets. Every other key is unchanged.

Fixes kirodotdev#12386
wknight1 pushed a commit to wknight1/KiroCrew that referenced this pull request Oct 5, 2026
… not on its container (kirodotdev#12647)

A host whose homes sit one level deeper than the usual layout keyed every path on the machine to the container of homes, so one resolver stall refused every tool call on the host for the cooldown. The key now never ends on a component named home: when the second component is a container of homes it takes the user too, so the boundary is the home directory itself, the key the usual layout already gets. Every other key is unchanged.

Fixes kirodotdev#12386
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Resolver stall cooldown collapses to one key when homes sit one level deeper than /home: whole-host refusal that never decays

2 participants