Repository navigation
fix(security): resolve sensitive paths in the subprocess pool's child, from the calling thread - #12879
Conversation
|
👋 Hi! This PR is currently in draft status. Workflow runs won't be auto-approved until it's marked as ready for review. When you're ready, click "Ready for review" and the workflows will be approved on the next cycle automatically. |
2 similar comments
|
👋 Hi! This PR is currently in draft status. Workflow runs won't be auto-approved until it's marked as ready for review. When you're ready, click "Ready for review" and the workflows will be approved on the next cycle automatically. |
|
👋 Hi! This PR is currently in draft status. Workflow runs won't be auto-approved until it's marked as ready for review. When you're ready, click "Ready for review" and the workflows will be approved on the next cycle automatically. |
e2df7e0 to
0082a4c
Compare
|
👋 Hi! This PR is currently in draft status. Workflow runs won't be auto-approved until it's marked as ready for review. When you're ready, click "Ready for review" and the workflows will be approved on the next cycle automatically. |
|
👋 Hi! This PR is currently in draft status. Workflow runs won't be auto-approved until it's marked as ready for review. When you're ready, click "Ready for review" and the workflows will be approved on the next cycle automatically. |
3 similar comments
|
👋 Hi! This PR is currently in draft status. Workflow runs won't be auto-approved until it's marked as ready for review. When you're ready, click "Ready for review" and the workflows will be approved on the next cycle automatically. |
|
👋 Hi! This PR is currently in draft status. Workflow runs won't be auto-approved until it's marked as ready for review. When you're ready, click "Ready for review" and the workflows will be approved on the next cycle automatically. |
|
👋 Hi! This PR is currently in draft status. Workflow runs won't be auto-approved until it's marked as ready for review. When you're ready, click "Ready for review" and the workflows will be approved on the next cycle automatically. |
GPT 5.6 Review (fork) — ✅ no blocking findingsReviewed Review detailsFINDING -- src/kiro_crew/subprocess_pool/executor.py:87 -- |
First Principles Review (Fable 5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Verify the Windows arm on a real Windows host: every Not justified as shipped
What this change shipsInventory (10 items) — 9 justifiedIntent: FIX — stop the sensitive-path gate refusing ordinary paths on a busy gateway by moving its
More than 10 items exist; the omitted ones (doc rewrite, measure-script rename, +200-line package budget for the moved child script, [FIRST-PRINCIPLES-REVIEWED] b7fc1ed |
Design Review (Fable 5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify the reaper — now the pool's only spawn site — survives a poisoned tick; a dead reaper leaves every gated path refused until restart. Suggestions
[DESIGN-REVIEWED] b7fc1ed |
0082a4c to
fdf0834
Compare
|
Review round on Design Review — BLOCK (Windows loses the caller-side bound). Fixed in Both watch items are also taken:
The two-pass anchor invariant suggestion is left as is: pass 2 asks only for what First Principles — CONCERNS ( GPT — advisory after adjudication (IPC wait blocks the event loop). No change: as the adjudication records, the base already waits inline at the same ≤5 s ceiling, the new wait is bounded absolutely with the child killed at expiry, and offloading it re-introduces the two-handoff shape this PR measures at 2681 ms vs 81 ms. Windows shards 1 and 6 on Local gates on |
Security Scope Review (fork) — 🟡 CONCERNSWhich legitimate operations does the reviewer's own judgement; no confirmed regression Scope-Verdict: CONCERNS TIGHTENING: the sensitive-path resolver behind the tool gate's path fence moved from the ADJUDICATED: 14 candidates, 0 confirmed newly-refused PLATFORM COVERAGE: 10 posix rows and 4 windows rows; no UNADJUDICATED: the tightening itself. The guard is the path fence's resolver:
The candidates in [SCOPE-REVIEWED] b7fc1edSecurity scope review -- adjudicated candidates
No candidate flipped from allowed to refused. |
406f3e4 to
b5757b0
Compare
|
b5757b0 to
dba9e9e
Compare
|
|
|
|
|
|
|
Head |
…, from the calling thread Wire the sensitive-path resolver to kiro_crew.subprocess_pool: the candidate, the root anchors and the target rebuild each travel as one request to the child interpreter, made by the thread that wants the answer, so the budget is spent waiting for the disk rather than for the GIL. The stall classifier is adapted to child-shaped signals: a lease timeout means no child ran (not charged), and a SubprocessPoolTimeout carries the child's syscall sampled before it was killed, read against the existing stat/readlink table. Transport faults refuse fail-closed with no cooldown; only a child that cannot start falls back in-process. The child script moves beside the resolver it serves. Refs kirodotdev#10255, kirodotdev#12709, kirodotdev#10256, kirodotdev#12739
|
Base commit:
e46967b5d502e323728f272933c910bb81767983This is the wiring PR that #12709's scope note (section 5) asks for: the sensitive-path resolver now runs its
realpathwork inkiro_crew.subprocess_pool's child interpreter, reached from the calling thread. It answers the classifier question that note raises rather than guessing at it (see "What changed"). Credit: the pool and its child are @chenmingwei23's (#12709); the child-pid syscall sampling, batched anchor request and fail-closed transport posture come from #10256 (@bolichen97) and #12739, whose own helper/client this supersedes. Refs #10255 -- closed by triage, but the executor swap it describes is what lands here, so please reopen it or read this as the follow-up.Problem / Motivation
executors.path_resolve_executor()on main is still aThreadPoolExecutor, so the gate still refuses legitimate paths when the interpreter is busy:os.path.realpathreacquires the GIL twice per path component, and beside CPU-bound sibling threads that wait, not the disk, spends the budget. Measured here on a 128-core host (medians of 9 uncached anchor rebuilds,scripts/measure_path_gate_ttl.py rebuild):The rebuild budget is 8 s; main is over it from four hogs, and its first refusal there pins both pool workers so every later path under every prefix is refused without probing ("2 of 2 resolver worker(s) pinned").
Why it matters
The user's own file reads, edits and shell calls are refused with the unverifiable-path message on ordinary project paths, more often the busier the gateway is, and the stall cooldown widens one spike into a prefix-wide refusal. #12709 landed the mechanism and deliberately left
security/paths.pyuntouched because wiring it needed a decision; until that decision is made the fix is not in effect.What changed (motivation → approach → change)
The decision the scope note asked for.
_run_resolution_boundedclassified a missed budget with thread-pool signals (future.cancel(), astarted/abandonedhandshake, a worker ceiling) that do not exist once the work is a process. It now reads child signals: a plainTimeoutErrorfromcall_opmeans no child ever took the request (the old never-started arm);SubprocessPoolTimeout, new inexecutor.py, means a child took it and missed the deadline, carryingchild_syscall(/proc/<pid>/syscallsampled just before the kill)._child_blocked_in_filesystemreads it against the existing per-architecture stat/readlink table; on-CPU orfutexis the load arm; unsampleable (Windows, macOS, unmapped arch) fails toward charging. The worker ceiling and_path_resolve_wedgedare deleted: a timed-out child is destroyed and respawned.On the calling thread, one request per unit of work.
_resolve_on_calling_threadruns the worker on the thread that wants the answer, under a thread-local deadline its child requests share (#12709 item 3). The candidate is oneOP_REALPATH_SPELLINGSrequest; the root anchors and the target rebuild are one request each through a newOP_REALPATH_MANYop (_realpaths_or_none). The grace folds into the child's single deadline; a child that misses it is destroyed.Fail-closed on every fault (item 4).
SubprocessPoolUnavailableraisesPathResolutionStalled, uncharged: an empty set would read as "resolved". Three consecutive faults warn once and degrade to the in-process fallback until a child answers. Only the reaper thread spawns children andwaits on killed ones; a caller leases a ready slot within its budget, soPopennever runs on the loop. A killed slot respawns at once until2 * workerskilled children are unexited (callers then refused at once); children run source captured at import, never re-read. A stall-history prefix is not re-probed while one serviceable child remains. A host that cannot spawn raises theOSErrorat once; the resolver falls back in-process. Whereselectcannot arm the pipe deadline (Windows),call_opreads on a dedicated thread under a timed future; at the deadline it kills the child and retires the slot without waiting on the reader. Relative inputs join the CWD withoutnormpath;..after a symlink resolves in the kernel. No deadline armed means in-process:is_sensitive_resolved_path(worker-thread walks) andsandbox_credential_targets(off-loop preflight, bounded byacp/client.py's 60 swait_for) resolve in this interpreter, as before, neither holding a child to the ceiling nor queuing ahead of the loop;dashboard/handlers/files.pycatches nothing in its exclusion build, so a stall there routes_grep_rgto the fail-closed python engine._child_realpath.pymoves tosecurity/.path_resolve_executor()returns theSubprocessPoolExecutor;KIROCREW_PATH_RESOLVE_WORKERSsizes children (floor 2).security.mdandscripts/measure_path_gate_ttl.pyfollow.What the wiring does not buy. Each round trip still costs several parent-side GIL reacquisitions: a warm gate call measured a median of 51 / 413 / 599 / 862 ms at 1 / 2 / 4 / 8 hogs (maxima 3.1 s) against main's 323 ms at one hog and refusal from two. No prefix was charged; the only refusals came from the 12 s wait allowance after fifteen back-to-back calls at eight hogs. A socket pair with one
recvis the pool follow-up.Backwards compatibility
Compatible: no public API, config key or on-disk format changes.
KIROCREW_PATH_RESOLVE_WORKERSkeeps its name, default and range; it now counts children rather than threads. Two private names tests read are gone (_path_resolve_wedged,_wedged_workers) and one is renamed (_worker_blocked_in_filesystem→_child_blocked_in_filesystem, taking the sampled token);scripts/measure_path_gate_ttl.pyis the one in-tree reader and is updated.Tests
test/test_security_path_resolve_bounded.py(78 tests): the stubs now behave as a child does (wait out the shared deadline, raiseSubprocessPoolTimeout), the cooldown/one-hop tests count child requests (OP_REALPATH_MANYonce for the roots, once for the rebuild), the saturated arm leases both real children, and two pool-only tests are removed (leave-one-free re-probe guard, the never-run handshake). New pins: child answers identical to in-process for both ops; resolution runs on the calling thread; aSubprocessPoolUnavailableon either half refuses without a cooldown; a child that cannot start falls back in-process with one warning; an empty spelling never reaches the resolver; and the GIL-hog reproduction (sensitive_path_refusalon a healthy file beside one hog: not refused, no prefix charged, < 1.5 s). Added after review: the in-process fallback is bounded; parity on a..-after-symlink candidate; and intest_subprocess_pool.pythe caller's budget fires withselectdisabled (the Windows shape), so the three budget tests now run on every platform.test_file_grep.pypins that a stall routes the grep to the python engine;test_subprocess_pool.pypins the killed-child cap, the retired-slot arm and reaper-only spawning; the re-probe guard has its own pin.test_subprocess_pool.py,test_executors.py,test_pathres_loop_starvation.py,test_spawn_audit.pyadjusted for the moved child, the executor type and the batch primitive;test_security_gate_liveness.pywidens the security package's line budget by the relocated child script's size (a move, not growth).Ran locally (
-n0): the 17 files named in the gate set, 2032 passed, 2 skipped. flake8, isort, black clean on changed files;mypy src/kiro_crew/security/ src/kiro_crew/subprocess_pool/ src/kiro_crew/executors.py: no issues in 18 files; comment-history gate against origin/main passes; docs lint passes.Manual verification
The two contention tables above (
scripts/measure_path_gate_ttl.py rebuild --hogs 0,1,2,4,8on this branch and on a detached origin/main worktree, plus a stdlib gate-call probe kept outside the repo). No wedged mount was available to observe the child's syscall on a real dead NFS/FUSE/CIFS mount; that caveat stands exactly as it did for the thread sample and is recorded in the spec.Related Issues
Refs #10255, #12709, #10256, #12739.
Pattern harvest
Rule candidate: review-prompt
Pattern: "a classifier reads side effects of a submitted callable (thread id, handshake events, cancel() result) -- when the executor behind it changes shape, every one of those signals must be re-derived, not carried".
Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)