Repository navigation
test(e2e): wait for the page's slot read before the drop in spec 8 - #37286
Conversation
Spec 8 fired the drop API as soon as the Done column rendered. The column renders before the page's own GET /api/chat/slots returns, so the drop and that read raced. A reply serialized before the drop but applied after the drop's slots push puts the pre-drop row back, and nothing pushes again, so the slot never reaches the Done column. Wait for that read to finish before the drop, and drop the fixed 500 ms sleep: the drop's push is the only thing left that can move the row.
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
UX Review (Opus 5.5) — ✅ PASSUX-level review of UX-Verdict: PASS Nothing to check. The PR only changes one Playwright spec and touches no user-visible control, so no screenshots are needed. [UX-REVIEWED] a79101a Description read[DESCRIPTION-READ] 20a4274728e6a1ef1dbf7325a83410954b58ceda925148a5956bb038fe06b144 That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS This only changes a test. It replaces a fixed sleep with a wait on the page's real slot read, which removes the race named in the description. No product code or contract changes. Notes
[DESIGN-REVIEWED] a79101a Description read[DESCRIPTION-READ] 6c06c079ebed8cdc7e9dfedca0c4129a144f71b4983e6665d6b38931f634dcfe That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
buluoray
left a comment
There was a problem hiding this comment.
🤖 Kiro Crew Auto-Pipeline [operator: buluoray]:
Approve. This is the E2E red on main: e5f1b5e51f and 18a5366c61 both fail E2E (stub ACP backend, offline) on exactly this spec passing only on a retry (#37208).
The change is test-only and goes the right way: it waits on a signal (the page's own GET /api/chat/slots response finishing) instead of the fixed waitForTimeout(500), so the drop can only land after that read, and the later slots push is the one thing that moves the row. That matches the failure artifact in the body: the drop API answered ok with Done, and the page still drew the slot under its old tag.
Non-blocking follow-up: the race the body describes is also a real page behaviour. A slot-list read serialized before a change but applied after that change's push puts the old row back, and nothing pushes again, so a person can see a stale column until the next refresh. That belongs in the reducer (e.g. ignore a list reply older than the newest push), outside this PR's goal; worth an issue.
Problem / Motivation
Goal: Spec 8 of
website/playwright/session-tags-e2e.spec.ts(drop a session into a single-status column) passes on its first attempt, so theE2E (stub ACP backend, offline)job stops going red on PRs that do not touch it.Spec 8 failed its first attempt and passed on the retry, which
test/test_playwright_e2e.py'sMAX_FLAKY_SPECS=0turns into a red job. Hits: run 43511027535 (11-13), run 43588045096 x2 (#37170, docs-only), run 43599069114 (#36027), and maine5f1b5e51frun 43610407890 job 132497469065. First-attempt failure on main:The drop API had answered
ok: truewithDonein its tags (those asserts passed). The page snapshot in the failure artifact (error-context.md) shows the Done column holding E2E-1, E2E-4 and E2E-5 and no E2E-8 row anywhere, so the page still held the slot as ToDo, a tag no column on the board showed.Why it matters
Any PR can go red on this spec for reasons unrelated to its diff, and it has already turned main red once. Each hit costs a rerun.
Not a goal
What changed (motivation → approach → change)
The race chain, as spec 8 drove it on main:
website/playwright/session-tags-e2e.spec.ts:218(the old line 210) firedPOST /api/chat/slots/{slot}/dropas soon aswaitForSelector(column)returned. The column renders from the tag-columns read, before the slot list has loaded.fetchSlots(website/src/shell/appShell.tsx:976→website/src/store/dashboardSlice.ts:83,GET /api/chat/slots?crew=all). In the passing retry's trace that GET started 26 ms after the drop's POST, so the two race. When the GET is served first, its reply holds the pre-drop row (ToDo).api_chat_slot_dropretags the slot and callsstate.push_slots_update()(src/kiro_crew/dashboard/chat_tags.py:1055). The tab applies that list inuseSlotListSync.onSlots→dispatch(sseSlots(data))(website/src/hooks/websocket/slotList.ts:189, reducerwebsite/src/store/dashboardSlice.ts:942), so for a moment the row is Done.fetchSlots.fulfilled(website/src/store/dashboardSlice.ts:1500) replaces the rows with it (applySlots(state, rows), line 1598).sseSlotsdoes mark the newer complete frame on every in-flight GET (dashboardSlice.ts:928), butfetchSlots.fulfilleduses that mark only for the unread drain, not for the rows. The ToDo row comes back, the board's Done filter (website/src/pages/chat-sidebar/boardLane.tsx:444) drops it, and nothing pushes again in a quiet test gateway, so the 5 s expect runs out. This matches the failure snapshot: the Done column held E2E-1, E2E-4 and E2E-5, and E2E-8 was nowhere on the board.The change: spec 8 waits for that GET's response to finish (
page.waitForResponseon the exact path, thenresponse.finished(), lines 213-217) before it fires the drop. The fixedpage.waitForTimeout(500)before the assertion goes (website/docs/testing.md § Rules every test keeps). The 5 s assertion is unchanged.Left for the owner, not fixed here: in step 4, a GET reply older than a complete frame that arrived while it was in flight still wins the rows. This PR narrows the spec's exposure to that gap but does not close it. A tiny window stays open: the reply's body has finished arriving, but the page could still process the drop's push before it applies that finished reply. Closing it is a reducer design change in
fetchSlots.fulfilled, recorded on #37208.Backwards compatibility
Compatible: test-only change; no product code or contract moves.
Removes nothing: one fixed sleep in a spec is replaced by a wait on a real response.
Tests
website/playwright/session-tags-e2e.spec.tsis the changed test.E2E (stub ACP backend, offline)green on head a79101a0ba, with the job's 0-flaky-spec ceiling, so spec 8 passed on its first attempt.Proof: not provable -- test-only change
Manual verification
N/A — test-only change; CI's E2E job runs the spec.
Screenshots / video
Why no screenshot: the diff is one Playwright spec and nothing renders differently.
Related Issues
Fixes #37208
Sibling sweep
Swept: git grep -nE "request.(post|put|patch)(`/api/chat/slots/\${" -- website/playwright/
Sibling: website/playwright/session-tags-e2e.spec.ts:232 -- not affected -- spec 9 asserts the API answer only and loads no page
Sibling: website/playwright/session-tags-e2e.spec.ts:728 -- not affected -- spec 29 asserts the API answer only and loads no page
Sibling: website/playwright/active-slot-persistence.spec.ts:22 -- not affected -- seeds titles before page.goto, never writes while the page loads
Pattern harvest
Rule candidate: review-prompt
Pattern: a Playwright spec that writes state over the request context right after page.goto must first wait for the page's own initial reads of that state.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributed under the project's Apache-2.0 license.