Skip to content

fix(dashboard): key the path-probe stall gate for callers that do not lead with the path - #22321

Merged
bolichen97 merged 1 commit into
mainfrom
fix/21798-probe-stall-key
Oct 20, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/21798-probe-stall-key

Conversation

@iamwhatever

@iamwhatever iamwhatever commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Every dashboard file route that runs a blocking path call is refused under a mount once two calls there are stuck, including routes that do not pass their path as the first argument.

PR #21782 taught _run_path_probe to refuse new calls under a mount once two probes there are stuck past 2 s. It finds the mount from the call's first argument only. Five callers pass their path another way (a closure, a list, a config first, a relative path), so they get no key. A caller naming a dead mount through one of them still parks one worker per request until the probe pool is full.

Why it matters

When the probe pool is full, every file route answers 503, not just the ones on the dead mount. These five routes (reveal/open, the unscoped file search, /api/file-kinds, the terminal cwd, and every resolve=1 read) were the way around the per-mount refusal.

Not a goal

  • Changing the stall threshold, the 2 s budget, or the key rule itself.
  • Keying calls that touch several mounts at once on any one of them.
  • Callers that already pass an absolute path first: they keep their key unchanged.

What changed (motivation → approach → change)

The stall key came only from the first argument, so callers that pass their path some other way had no key.

_run_path_probe takes a keyword-only stall_path=. When it is given, the key comes from it by the same lexical rule; "" means no key; when it is omitted, the first argument is the key as before.

Caller stall_path
reveal/open (desktop.py) the request path
terminal (handlers/terminal.py) the requested cwd
resolve=1 in editor.py, transfer.py, office_preview.py, office_slides.py _project_relative_stall_path: the project dir for a relative path, else the path
/api/file-kinds (pinned_io.py) _batch_stall_path over the list
unscoped file search (search.py) _batch_stall_path over the project dir and workspace
transfers that open the path inside a closure: file_raw, file_download, office_preview.py, sheet.py, grep.py the request path or root
the file-search walk (search.py) _batch_stall_path over the roots it walks

For the list callers the rule is: key only when every path shares one key. Keying on the first path would let two batches stuck on a dead mount listed third refuse a healthy mount listed first, on every route. A batch across mounts stays bounded by the pool, as today.

Four probe-pool calls stay unkeyed: the drive listing (no path), the outbox read and the slide render (paths inside crew-owned directories, not chosen by the caller), and the knowledge-source add check (handlers/knowledge_sources.py, not a file route). The handlers/files.py facade exports the two helpers, and docs/system-specs/modules/security.md describes the rule.

Backwards compatibility

Compatible: stall_path is a new optional keyword; every caller that omits it gets the first-argument key exactly as before. The only new refusals are the documented 503 path_probe_busy, under a mount with two stuck calls.

Removes nothing: no key, route or answer is taken away; calls that had no key gain one.

Tests

test/test_dashboard_files_onloop_fs.py (TestStallKeyForCallersThatDoNotLeadWithThePath) and test/test_terminal_handler.py park two probes under one key on a three-worker pool, with the stall time set to zero so no clock is involved. Then they show the route is refused with path_probe_busy while the third worker is still free:

  • reveal/open: refused, the launcher never called.
  • unscoped file search: refused.
  • /api/file-kinds with an absolute and a relative path in one project: refused.
  • resolve=1 file read: refused before the resolver runs.
  • terminal: refused before _resolve_cwd runs.
  • file_raw, file_download, file_office_preview, file_sheet: each transfer is refused under its path's key.
  • _batch_stall_path / _project_relative_stall_path: one key for one mount, none across mounts, none with no project dir.

All ten fail on origin/main. test/test_dashboard_files_composition_contract.py gets the recaptured shape digest for the new _run_path_probe signature.

Manual verification

N/A — unit coverage sufficient: the change is which key a call is charged to, and each route is driven end to end through its handler against a parked pool.

Related Issues

Closes #21798

Pattern harvest

Not generalizable: the defect was one chokepoint inferring its key from argument position; the explicit keyword removes that inference for this chokepoint only.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@iamwhatever
iamwhatever requested review from a team and KiroCrewCaptain as code owners October 20, 2026 00:02
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 20, 2026
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision e40dc0e36e9fcd209683784c51d44acdcfb33072 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of e40dc0e36e9fcd209683784c51d44acdcfb33072 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check that the terminal call still gets a key when requested_cwd is missing or None. In that case it falls back to the first argument, which is the config object, not a path.

[DESIGN-REVIEWED] e40dc0e

Description read

[DESCRIPTION-READ] 5db9bd3582de2490eef5f22aaad48298a40e76ee7194fd494ef0ea3137b52178

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of e40dc0e36e9fcd209683784c51d44acdcfb33072 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Check that leaving /api/file-kinds and search batches that span mounts unkeyed is an accepted gap: they can still fill the probe pool.

What this change ships

Inventory (8 items) — 8 justified

Intent: a dead mount reached through any dashboard file route gets the per-mount 503 path_probe_busy and can no longer fill the probe pool for every route. FIX (closes #21798; the ten new tests fail on base).

  1. Reveal/open is refused under a stuck mount before it launches anything — justified
  2. A terminal open with a cwd on a stuck mount is refused before the cwd is resolved — justified
  3. resolve=1 reads (file read, download, office preview, slides) are charged to the project dir when the path is relative — justified
  4. /api/file-kinds is refused when every path in it shares one stuck mount — justified
  5. The unscoped file-search fallback and the search walk are refused when their roots share one stuck mount — justified
  6. The file_raw, file_download, office-preview, sheet and grep transfers are refused under their path's key — justified
  7. _run_path_probe takes a new keyword-only stall_path (17 call sites pass it; all other callers unchanged) — justified
  8. Two exported helpers, _batch_stall_path (3 consumers) and _project_relative_stall_path (4 resolve sites plus the batch helper) — justified

[FIRST-PRINCIPLES-REVIEWED] e40dc0e

Description read

[DESCRIPTION-READ] e2d4c97cc2ef53266a92d9bddf39943f888eea8fd4b6af253f1e92d778169afd

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed e40dc0e36e9fcd209683784c51d44acdcfb33072 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] e40dc0e

Verdict parsed from the review's SHA-scoped output markers for commit e40dc0e36e9fcd209683784c51d44acdcfb33072.

False positive or not applicable? A repository writer can comment:
/ai-review override fable e40dc0e36e9fcd209683784c51d44acdcfb33072: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 20, 2026
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of e40dc0e36e9fcd209683784c51d44acdcfb33072 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] e40dc0e

False positive or not applicable? A repository writer can comment:
/ai-review override gpt e40dc0e36e9fcd209683784c51d44acdcfb33072: <one-sentence reason>

@iamwhatever
iamwhatever force-pushed the fix/21798-probe-stall-key branch from fd01037 to e40dc0e Compare October 20, 2026 00:26
@iamwhatever

iamwhatever commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: explicit stall_path on transfer closures

  • span=9eeda9ba95c6 Item 1: six transfer calls ran unkeyed

Fixed in e40dc0e. The six transfer calls named in the item now pass their key: file_raw and file_download (the _open_checked partials) and the office-preview and sheet closures pass the request path, grep passes its root, and the file-search walk passes _batch_stall_path(safe_roots). A parametrized test drives file_raw, file_download, file_office_preview and file_sheet under two stuck probes and gets 503 path_probe_busy with a worker still free.
The Subtraction (a required stall_path at every call site, dropping the first-argument fallback) is declined: it would touch every caller that already passes its path first, which the issue keeps unchanged. The description now lists the four probe-pool calls left unkeyed and why: the drive listing, the outbox read, the slide render, and the knowledge-source add check.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no

Checked. stall_path=requested_cwd uses the same lexical rule as every other key (_path_probe_stall_key): an absolute cwd and ~ / ~/... (expanded from HOME/USERPROFILE only) get a key. A relative cwd and ~name get none, on purpose: _resolve_cwd resolves a relative cwd against the gateway's working directory, which the lexical rule does not guess, and ~name would need a user-database lookup on the event loop. The dashboard sends the chat's project dir, which is absolute, so the terminal route is keyed in practice; an unkeyed cwd stays bounded by the pool, as before.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no

Checked: that fallback is unreachable. stall_path=requested_cwd is passed only inside if requested_cwd: in api_terminal_ws, so a missing, None or empty cwd never reaches _run_path_probe. That branch calls _resolve_cwd(cfg, None) inline, which picks the configured cwd, home or / without a stat. When the probe does run, stall_path is a non-empty string, so the key never comes from cfg.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no

Accepted gap, stated in the description and in the _batch_stall_path docstring. A batch that spans mounts gets no key because any one key would be wrong: two batches stuck on a dead mount listed third would refuse a healthy mount listed first, on every route. Such a batch is bounded by the probe pool, exactly as on main. The issue allowed either rule for the list caller; keying only single-mount batches covers the common case (chips from one project) without that false refusal.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed head e40dc0e. The change looks good to ship.

What I checked:

  • _run_path_probe keys on stall_path when given and on the first argument otherwise, so callers that omit it keep the old key.
  • _batch_stall_path returns no key for an empty list: the set of keys is empty, so paths[0] is never read. This covers the file-search walk when every root is filtered out.
  • A batch that mixes a keyed path with an unkeyed one (relative with no project dir, or ~name) gets no key, which is the documented rule.
  • file_raw keys on the same query path it opens, and the terminal passes stall_path only inside if requested_cwd:.
  • Linked issue is tier:T1. No file is on the security path list. The design and first principles notes have replies that match the code.

Remaining maintainer check: readiness is not passed on this head yet (coverage combine is still pending). I will approve once readiness passes on this same head.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention labels Oct 20, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Review-ready at e40dc0e36e9fcd209683784c51d44acdcfb33072: all 107 checks pass, PR Readiness is green, and the GPT and Opus lanes are clean for this head. Design and First Principles both PASS; each punchline has a disposition reply above.

What it does: _run_path_probe takes stall_path=, and every file route that hands the probe its path some other way passes it, so two stuck calls on a dead mount refuse the third on that route too. Batches that span mounts stay unkeyed on purpose (see the description).

main has moved since this head went green, but only in other files; nothing this PR changes. This Crew will not merge, approve or enable auto-merge; that is the maintainer's call.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving head e40dc0e. Checked on this head: readiness label passed, PR Readiness green, Coverage Combine green, no required check failing, all review lanes PASS with no BLOCK, linked issue is tier T1, no outstanding change requests. Earlier review found it good to ship.

@bolichen97
bolichen97 merged commit cb756d7 into main Oct 20, 2026
107 checks passed
@bolichen97
bolichen97 deleted the fix/21798-probe-stall-key branch October 20, 2026 02:03
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 20, 2026

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: fix (16 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: adds a keyword-only stall_path to the dashboard path probe so callers that do not pass their path first are keyed by the same stall rule, with tests for each route. Spec files changed as a ride-along (a minority of the diff on both file count and changed lines), not reviewed as a design decision: docs/system-specs/modules/security.md.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Key the dashboard probe-pool callers that do not pass their path first

3 participants