Skip to content

test(website): add a capture harness for file-read failure states - #10845

Closed
rnoack1 wants to merge 1 commit into
kirodotdev:mainfrom
rnoack1:test/file-search-timeout-capture-harness
Closed

rnoack1 wants to merge 1 commit into
kirodotdev:mainfrom
rnoack1:test/file-search-timeout-capture-harness

Conversation

@rnoack1

@rnoack1 rnoack1 commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Problem / Motivation

The dashboard's file-read surfaces — the @-mention picker, the folder panel, the project and workspace pickers, the file rail — have a growing set of failure and recovery states: timed out, failed, no access, root missing, stale rows, and the retry affordances for each. None of them could be reviewed as rendered. A reviewer had to read the copy off the diff and take on trust that it appears where and when it is claimed to.

There was no harness that drives those states, so "does this notice actually show up, with that wording, next to that button" was not an answerable question without hand-wedging a gateway.

Why it matters

Failure copy is exactly the category that is easy to get wrong in ways a unit test does not catch: a notice can assert correctly in jsdom and still be unreadable, mispositioned, or attached to a control the user cannot find. Two of the review lanes on the original change asked for rendered evidence and could not be satisfied, because producing it needed tooling that did not exist.

It also closes a specific trap: the harness serves the prebuilt bundle, so a copy change that has not been rebuilt does not appear. Reviewing a stale frame is worse than reviewing none, and having the driver in-tree is what makes the rebuild step explicit rather than remembered.

What changed (motivation → approach → change)

  • Adds website/scripts/capture-file-search-timeout.mjs, a Playwright driver that serves the built dashboard and stubs the gateway per arm, then photographs each failure and recovery state in turn.
  • It reuses the existing screenshot-harness libraries (lib/serve-dist.mjs, lib/stub-dashboard-api.mjs) rather than standing up its own server and stubs, so it behaves like the other capture scripts in this directory.
  • Each arm stubs one specific outcome — a hung request for the timeout, a 403 with an access-denied code for the refusal, a 404 for a missing root — so the state photographed is the state named, not an approximation of it.

This is tooling only. It adds no product code, imports nothing from src/, and changes no behaviour: nothing in the shipped application calls it.

Tests

None, deliberately: this is a developer capture script with no product surface, in a directory of sibling scripts that are likewise untested. Its correctness is observable directly — it either produces the frames or it does not.

Verified by execution rather than by assertion: node --check on the file, eslint clean, and the script runs to completion producing frames locally.

Manual verification

eslint clean on the added file, node --check passes, and the harness runs end to end against a locally built dist.

One ordering note for a reviewer: several arms exercise states that arrive with the other units split out of the same original change. Landing the harness first is harmless — it is the tool, not the subject — but its full output is only meaningful once those merge. It is independent of them at build time and carries no prerequisite.

@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Sep 14, 2026
@dwu96

dwu96 commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

👋 Hi! This PR is currently in draft status. Workflow runs won't be auto-approved until it's marked as ready for review.

When you're ready, click "Ready for review" and the workflows will be approved on the next cycle automatically.

@rnoack1
rnoack1 marked this pull request as ready for review September 15, 2026 00:49
@rnoack1
rnoack1 requested review from a team and smeyffret September 15, 2026 00:49
@github-actions

github-actions Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ✅ PASS

UX-level review of 1bad4673257827f0eb1b760cb078e20c7d5b76fd via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

UX-Verdict: PASS

Nothing to check.

[UX-REVIEWED] 1bad467

Description read

[DESCRIPTION-READ] 92700fc29eabccb9c59ad471f546be2f9f9b08f320ad876146885083cad0adb8

That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — 🟡 CONCERNS

Design-level review of 1bad4673257827f0eb1b760cb078e20c7d5b76fd via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

The harness asserts copy and a CI test that aren't on main, so it fails on the tree it lands in and points readers to a test that doesn't exist.

Watch

  • It points to a test that doesn't exist. The header says "the CI-enforced half of the invariant is FilePickerMenu.timeout.test.tsx", but no such file is in the base tree. Its framing ("The composer's @ file-mention menu had no deadline… The fix bounds the fetch") also describes a product change that this PR doesn't contain. A reader who trusts the header will think a pin exists that doesn't.
    Clears when: the header names only tests on main, or the PR lands after the sibling unit that adds FilePickerMenu.timeout.test.tsx and the header cites it.
  • It fails on main by construction. main() exits 1 unless every arm finds its string. The base en.json has none of "File search timed out", "File search failed", "No access to the project folder" or "Project folder not found". So on main it prints FAIL: an arm did not render its own copy for a tree that is working correctly. The description's "harmless — it is the tool, not the subject" understates this: it ships a check that is red on the tree it ships into.
    Clears when: the PR merges after (or together with) the units that add those strings, or the arms whose copy isn't on main are left out until those units land.

[DESIGN-REVIEWED] 1bad467

Description read

[DESCRIPTION-READ] e87eef003fe0bc74df49a4248d29c30e96526243cafbf6bdd590c03f30c55207

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 1bad4673257827f0eb1b760cb078e20c7d5b76fd via the fork AI-review pipeline; updated in place on each push.

Review details

Exits 1 on every run: the harness waits for @-menu and picker text that this SPA never renders.

FINDING — website/scripts/capture-file-search-timeout.mjs:426 — In the current tree, FilePickerMenu.tsx:316 shows only components.filePickerMenu.search_failed. No website/src file contains the expected text 'File search timed out — Enter sends the message' or 'Recent projects unavailable'. Each captureArm('hang'|'error'|'denied'|'not-found'), staleRows and pickerRecent/pickerBoth wait finds nothing and returns false, so main() prints "FAIL: an arm did not render its own copy" and calls process.exit(1). → Fix: ship the cause-keyed FilePickerMenu and picker copy (and the FilePickerMenu.timeout.test.tsx the header names) with this script, or change the expected text to the strings the catalog actually has.

[OPUS-REVIEWED] 1bad467

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 15, 2026
@github-actions

github-actions Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 1bad4673257827f0eb1b760cb078e20c7d5b76fd via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 1bad467

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 15, 2026
@rnoack1
rnoack1 force-pushed the test/file-search-timeout-capture-harness branch from 1a04387 to ca842c3 Compare September 19, 2026 01:36
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 19, 2026
@rnoack1
rnoack1 force-pushed the test/file-search-timeout-capture-harness branch from ca842c3 to 6604335 Compare September 19, 2026 05:51
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 19, 2026
@rnoack1
rnoack1 force-pushed the test/file-search-timeout-capture-harness branch from 6604335 to 43660b8 Compare September 19, 2026 07:00
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 19, 2026
@iamwhatever iamwhatever added the drive-to-green PR claimed by drive-to-green pipeline label Sep 25, 2026
@iamwhatever iamwhatever self-assigned this Sep 25, 2026
@iamwhatever iamwhatever removed the drive-to-green PR claimed by drive-to-green pipeline label Sep 25, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

This capture harness photographs file-search failure/recovery UI copy ("File search timed out", "No access to the project folder", the picker/rail/folder-panel failure states) that is not on main yet — it belongs to the still-open stack #10834 (bound the file reads and name why one failed) and #10838 (picker failure UI). Verified against a clean build of current main: only the pre-existing "No matching files" arm renders; the other 20 arms assert strings no code in the repo produces, so the harness exits 1 on main today. It should land together with or after that stack rather than alone. Not closing.

@cruisercohen cruisercohen added the needs-pr-triage PR scanner: awaiting automated triage label Oct 4, 2026
@dwu96 dwu96 added needs-author-decision PR blocked on author input and removed needs-pr-triage PR scanner: awaiting automated triage labels Oct 4, 2026
@dwu96

dwu96 commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

🤖 Kiro Crew [operator: dwu96#8c08bee0]: This PR has been inactive for 7+ days. I reviewed the blockers but they require your input:

  • The harness lands before its subject, and only re-sequencing fixes that. Design Review (CONCERNS) reports the script hardcodes ~20 exact user-facing strings ("File search timed out — Enter sends the message", "Folder listing timed out"…) plus a code contract (access_denied, project_not_found) that exists nowhere in website/src/ or the i18n catalog at this base — and that even FilePickerMenu.timeout.test.tsx, which the description calls "the CI-enforced half of the invariant", does not exist yet. @iamwhatever's comment says the same: the copy belongs to the still-open stack starting at fix(files): bound the file reads and name why one failed #10834. It clears only when those feature units merge and the harness runs green against a dist built from HEAD, or this PR is re-sequenced to land with/after them. Nothing I can push to this branch changes that — it is a merge-order decision, and it is yours.
  • How should this be sequenced? Mark it draft until the stack lands, stack it on fix(files): bound the file reads and name why one failed #10834 explicitly, or land it as-is and accept a harness that fails at HEAD until its subject arrives. I won't pick for you, because the third option is a deliberate tradeoff someone has to own.

One genuinely mechanical item, for when you pick that up: GPT 5.6 Review raised a BLOCKING finding at capture-file-search-timeout.mjs:232 claiming the panel-seed addInitScript races the stub's localStorage.clear(). Adjudication (Opus 4.8) did not uphold it — it found the clear registers at stub-dashboard-api.mjs:211-212 and your seeds strictly after at :238-246, that Playwright evaluates init scripts in registration order, and that your other arms (:112, :179, :295) use the identical stub-then-seed pattern and work. It landed on FLAG, noting a human would plausibly accept it with the suggested localStorageEntries fix noted. So if you want that red gone cheaply, passing the seeds through stubDashboardApi(..., { localStorageEntries: ... }) is a one-line change — but the finding is a false positive, not a defect.

Issue Gate also wants a Closes #N / Part of #N reference, which the description doesn't declare yet.

When you've addressed these, the pipeline will re-assess on its next cycle.

Copy link
Copy Markdown
Contributor

PR triage: OVERLAPPING. Keep open, continue development. Do not merge alone.

I checked this diff (merge base 0e7ba9a6f0, head 43660b817a) against origin/main at cc7e79894e. The branch merges cleanly with git merge-tree. I found no breaking change and no P0 bug: this is a dev-only Playwright script that nothing in CI or the shipped app runs.

What has changed in the stack since the last comments:

Subject PR State Effect on this harness
#10834 (bound the file reads) closed 2026-10-06, superseded by merged #7068 (18243811d6) searchErrorCause / withDeadline and the access_denied / project_not_found codes are on main now.
#10838 (@-mention search failure copy) open Still the only source of "File search timed out / failed — Enter sends the message", "No access to the project folder — …" and "Project folder not found — …". None of these strings is on main.
#10842 (project picker recents failure) open Still the only source of "Recent projects unavailable" (6-picker-…, 8-picker-…).
#10839 (folder read failure, keep the rail mounted) closed, not merged The panel and rail arms that depend on it have no subject in flight.

On main today, only the "No matching files" arm and the folder-listing copy from #7068 ("Folder listing timed out", "No access to this folder", "Unable to list folder", "Folder not found") can render. The harness still exits 1.

Recommendation: re-scope the arms to what #7068 shipped plus #10838 and #10842. Either drop the #10839 arms or say which PR now owns them. Then land this with or after #10838 and #10842. The earlier GPT finding on addInitScript is already handled: this head passes every seed through stubDashboardApi(..., { localStorageEntries }).

Drives the built dashboard against a stubbed gateway to photograph each file-search and listing failure state, so the failure copy can be reviewed as rendered rather than read off the diff.
@rnoack1
rnoack1 force-pushed the test/file-search-timeout-capture-harness branch from 43660b8 to 1bad467 Compare October 9, 2026 13:25
@rnoack1
rnoack1 requested review from a team and KiroCrewCaptain as code owners October 9, 2026 13:25
@rnoack1
rnoack1 requested a review from bolichen97 October 9, 2026 13:25
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 9, 2026
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ⏭️ skipped

Revision 1bad4673257827f0eb1b760cb078e20c7d5b76fd ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Advisory — does not block merge.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 9, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Thank you @rnoack1 for this capture harness.

Closing this because its subject is gone. The harness belongs to #10834, which was closed on 2026-10-06 as superseded by #7068. The copy it photographs ("File search timed out", "No access to the project folder") never landed: it is not in website/src on main (74a249d48).

Your branch is kept. If that copy lands later, please reopen this or open a follow-up.

@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) needs-author-decision PR blocked on author input

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants