Skip to content

test(e2e): wait for the page's slot read before the drop in spec 8 - #37286

Merged
buluoray merged 1 commit into
mainfrom
crew/unflaky/37208-drop-column-refresh
Nov 14, 2026
Merged

buluoray merged 1 commit into
mainfrom
crew/unflaky/37208-drop-column-refresh

Conversation

@CrysisDeu

@CrysisDeu CrysisDeu commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Spec 8 of website/playwright/session-tags-e2e.spec.ts (drop a session into a single-status column) passes on its first attempt, so the E2E (stub ACP backend, offline) job stops going red on PRs that do not touch it.

Spec 8 failed its first attempt and passed on the retry, which test/test_playwright_e2e.py's MAX_FLAKY_SPECS=0 turns into a red job. Hits: run 43511027535 (11-13), run 43588045096 x2 (#37170, docs-only), run 43599069114 (#36027), and main e5f1b5e51f run 43610407890 job 132497469065. First-attempt failure on main:

Error: expect(locator).toBeVisible() failed
Locator: locator('[data-testid="column-68cb95425d05"] [data-slot-key="chat-63-1794671746"]')
Timeout: 5000ms  Error: element(s) not found
> 216 |     await expect(page.locator(`[data-testid="column-${col.id}"] [data-slot-key="${slotKey}"]`)).toBeVisible()

The drop API had answered ok: true with Done in its tags (those asserts passed). The page snapshot in the failure artifact (error-context.md) shows the Done column holding E2E-1, E2E-4 and E2E-5 and no E2E-8 row anywhere, so the page still held the slot as ToDo, a tag no column on the board showed.

Why it matters

Any PR can go red on this spec for reasons unrelated to its diff, and it has already turned main red once. Each hit costs a rerun.

Not a goal

  • Changing the drop route, the slots broadcast, or the dashboard's slot-list reducer.
  • Touching the other specs in this file.

What changed (motivation → approach → change)

The race chain, as spec 8 drove it on main:

  1. Spec: website/playwright/session-tags-e2e.spec.ts:218 (the old line 210) fired POST /api/chat/slots/{slot}/drop as soon as waitForSelector(column) returned. The column renders from the tag-columns read, before the slot list has loaded.
  2. Page GET in flight: the shell's mount effect dispatches fetchSlots (website/src/shell/appShell.tsx:976 → website/src/store/dashboardSlice.ts:83, GET /api/chat/slots?crew=all). In the passing retry's trace that GET started 26 ms after the drop's POST, so the two race. When the GET is served first, its reply holds the pre-drop row (ToDo).
  3. Drop push: api_chat_slot_drop retags the slot and calls state.push_slots_update() (src/kiro_crew/dashboard/chat_tags.py:1055). The tab applies that list in useSlotListSync.onSlots → dispatch(sseSlots(data)) (website/src/hooks/websocket/slotList.ts:189, reducer website/src/store/dashboardSlice.ts:942), so for a moment the row is Done.
  4. Stale reply applied: the GET reply lands after that push, and fetchSlots.fulfilled (website/src/store/dashboardSlice.ts:1500) replaces the rows with it (applySlots(state, rows), line 1598). sseSlots does mark the newer complete frame on every in-flight GET (dashboardSlice.ts:928), but fetchSlots.fulfilled uses that mark only for the unread drain, not for the rows. The ToDo row comes back, the board's Done filter (website/src/pages/chat-sidebar/boardLane.tsx:444) drops it, and nothing pushes again in a quiet test gateway, so the 5 s expect runs out. This matches the failure snapshot: the Done column held E2E-1, E2E-4 and E2E-5, and E2E-8 was nowhere on the board.

The change: spec 8 waits for that GET's response to finish (page.waitForResponse on the exact path, then response.finished(), lines 213-217) before it fires the drop. The fixed page.waitForTimeout(500) before the assertion goes (website/docs/testing.md § Rules every test keeps). The 5 s assertion is unchanged.

Left for the owner, not fixed here: in step 4, a GET reply older than a complete frame that arrived while it was in flight still wins the rows. This PR narrows the spec's exposure to that gap but does not close it. A tiny window stays open: the reply's body has finished arriving, but the page could still process the drop's push before it applies that finished reply. Closing it is a reducer design change in fetchSlots.fulfilled, recorded on #37208.

Backwards compatibility

Compatible: test-only change; no product code or contract moves.

Removes nothing: one fixed sleep in a spec is replaced by a wait on a real response.

Tests

Proof: not provable -- test-only change

Manual verification

N/A — test-only change; CI's E2E job runs the spec.

Screenshots / video

Why no screenshot: the diff is one Playwright spec and nothing renders differently.

Related Issues

Fixes #37208

Sibling sweep

Swept: git grep -nE "request.(post|put|patch)(`/api/chat/slots/\${" -- website/playwright/
Sibling: website/playwright/session-tags-e2e.spec.ts:232 -- not affected -- spec 9 asserts the API answer only and loads no page
Sibling: website/playwright/session-tags-e2e.spec.ts:728 -- not affected -- spec 29 asserts the API answer only and loads no page
Sibling: website/playwright/active-slot-persistence.spec.ts:22 -- not affected -- seeds titles before page.goto, never writes while the page loads

Pattern harvest

Rule candidate: review-prompt
Pattern: a Playwright spec that writes state over the request context right after page.goto must first wait for the page's own initial reads of that state.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

Spec 8 fired the drop API as soon as the Done column rendered. The
column renders before the page's own GET /api/chat/slots returns, so
the drop and that read raced. A reply serialized before the drop but
applied after the drop's slots push puts the pre-drop row back, and
nothing pushes again, so the slot never reaches the Done column.

Wait for that read to finish before the drop, and drop the fixed
500 ms sleep: the drop's push is the only thing left that can move
the row.
@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision a79101a0ba27c489fc0a8deaeb99c2753141cd67 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of a79101a0ba27c489fc0a8deaeb99c2753141cd67 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

Nothing to check. The PR only changes one Playwright spec and touches no user-visible control, so no screenshots are needed.

[UX-REVIEWED] a79101a

Description read

[DESCRIPTION-READ] 20a4274728e6a1ef1dbf7325a83410954b58ceda925148a5956bb038fe06b144

That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of a79101a0ba27c489fc0a8deaeb99c2753141cd67 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

This only changes a test. It replaces a fixed sleep with a wait on the page's real slot read, which removes the race named in the description. No product code or contract changes.

Notes

  • waitForResponse only catches the first GET /api/chat/slots. If the page refetches later (on a socket reconnect or a crew switch), a stale reply can still land after the drop. The reducer gap the description points to is the real fix, so file it as a separate issue.

[DESIGN-REVIEWED] a79101a

Description read

[DESCRIPTION-READ] 6c06c079ebed8cdc7e9dfedca0c4129a144f71b4983e6665d6b38931f634dcfe

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of a79101a0ba27c489fc0a8deaeb99c2753141cd67 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] a79101a

False positive or not applicable? A repository writer can comment:
/ai-review override gpt a79101a0ba27c489fc0a8deaeb99c2753141cd67 false-positive: <reason, 20 to 500 characters>

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed a79101a0ba27c489fc0a8deaeb99c2753141cd67 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] a79101a

Verdict parsed from the review's SHA-scoped output markers for commit a79101a0ba27c489fc0a8deaeb99c2753141cd67.

False positive or not applicable? A repository writer can comment:
/ai-review override fable a79101a0ba27c489fc0a8deaeb99c2753141cd67 false-positive: <reason, 20 to 500 characters>

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Kiro Crew Auto-Pipeline [operator: buluoray]:

Approve. This is the E2E red on main: e5f1b5e51f and 18a5366c61 both fail E2E (stub ACP backend, offline) on exactly this spec passing only on a retry (#37208).

The change is test-only and goes the right way: it waits on a signal (the page's own GET /api/chat/slots response finishing) instead of the fixed waitForTimeout(500), so the drop can only land after that read, and the later slots push is the one thing that moves the row. That matches the failure artifact in the body: the drop API answered ok with Done, and the page still drew the slot under its old tag.

Non-blocking follow-up: the race the body describes is also a real page behaviour. A slot-list read serialized before a change but applied after that change's push puts the old row back, and nothing pushes again, so a person can see a stale column until the next refresh. That belongs in the reducer (e.g. ignore a list reply older than the newest push), outside this PR's goal; worth an issue.

@buluoray
buluoray enabled auto-merge (squash) November 14, 2026 17:32
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 14, 2026
@buluoray
buluoray merged commit ba71a27 into main Nov 14, 2026
112 checks passed
@buluoray
buluoray deleted the crew/unflaky/37208-drop-column-refresh branch November 14, 2026 18:34
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky: session-tags-e2e.spec.ts passes only on a Playwright retry (E2E stub ACP backend)

2 participants