Skip to content

test(app-sdk): give the stale-cache issues test its own app id per run - #36287

Merged
bolichen97 merged 1 commit into
mainfrom
fix/app-sdk-issues-test-app-id
Nov 13, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/app-sdk-issues-test-app-id

Conversation

@kyleseaman

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: AppSdkIssues.test.tsx passes when its tests are repeated in one run, so vitest --repeat-style runs and reordered runs stop failing on a cache left by an earlier run.

The test IssueList > reads the list again when an app reopens it after the route cache expires mounted its list under the fixed app id issues-app-stale. An external app's query client is held per app id in a module-level map (website/src/app-sdk/appQueryClient.ts), so that client outlives the test. On a repeat, the first mount finds the previous run's cached list and sends no request, and expect(bodies()).toHaveLength(2) sees 0.

KiroCrewCaptain flagged this in its review of #34732.

Why it matters

A test that depends on running once per module is a flake waiting for any change that reorders, repeats or co-hosts the file. It also blocks proving determinism for later changes to the issues SDK, because every repeated run of this file goes red.

Not a goal

  • No product code change. The per-app-id client in appQueryClient.ts is the intended design.
  • No change to what the test asserts.

What changed (motivation → approach → change)

Symptom: the stale-cache test fails on every repeat after the first. Cause: its fixed app id reuses a query client that outlives the test. Change: the test takes a fresh id from the file's existing appSeq counter (issues-app-stale-<n>), the same counter externalApp() already uses. Both mounts inside one run still share that id, which the test needs so the reopen finds the first mount's client.

Backwards compatibility

Compatible: test-only change; no product code, API or config is touched.

Removes nothing: the diff only changes one test's app id.

Tests

Proof: not provable -- frontend test (prove.py runs Python only)

Vitest 4.1 has no --repeat CLI flag, so the proof runs a throwaway, untracked copy of the file with { repeats: 4 } on each top-level describe (5 runs of every test in one module instance), then deletes the copy:

sed -E "s/^describe\('([^']+)', \(\) => \{/describe('\1', { repeats: 4 }, () => {/" src/test/AppSdkIssues.test.tsx > src/test/AppSdkIssues.repeat5.test.tsx
npx vitest run src/test/AppSdkIssues.repeat5.test.tsx --reporter=verbose
  • Before (main at 52cc80f; the file and website/src/app-sdk/ are unchanged on the current base a02e90c; 5 runs each): Tests 1 failed | 24 passed (25). × IssueList > reads the list again when an app reopens it after the route cache expires (repeat x4), failing at expect(bodies()).toHaveLength(2) with 0 received.
  • After, 5 runs each: Tests 25 passed (25).
  • After, 20 runs each (repeats: 19): Tests 25 passed (25).
  • After, 5 runs each, shuffled (--sequence.shuffle --sequence.seed=13013): Tests 25 passed (25).
  • Plain single run of src/test/AppSdkIssues.test.tsx: 25 passed.

Manual verification

N/A — test-only change; the repeat runs above are the verification.

Related Issues

N/A — follow-up to KiroCrewCaptain's review note on #34732; no tracking issue.

Sibling sweep

Swept: git grep -nE "appName=|appId=|issues-app" -- website/src/test/AppSdkIssues.test.tsx website/src/apps/issue-radar/components/IssueList.refreshButton.test.tsx
Sibling: website/src/test/AppSdkIssues.test.tsx:97 -- not affected -- externalApp() already draws a fresh issues-app- per mount from appSeq
Sibling: website/src/test/AppSdkIssues.test.tsx:151 -- not affected -- builtin issue-radar uses AppIdentityProvider only, under a QueryClient each test creates itself
Sibling: website/src/apps/issue-radar/components/IssueList.refreshButton.test.tsx -- not affected -- mounts no app provider and names no app id

The other tests in the file were checked too: scopedIssuesTransport uses a fake api with no client; the no-identity refusal test and the hook refusal test each build their own QueryClient; the IssueCard tests mount no app provider.

Pattern harvest

Rule candidate: review-prompt
Pattern: a test mounts an AppApiProvider with a literal appName, so its per-app query client outlives the test

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@kyleseaman
kyleseaman requested review from a team and KiroCrewCaptain as code owners November 13, 2026 01:49
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 13, 2026
@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 9e05ffd7bc59e318c8e2c88ce9e6e1f96eeda9a9 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of 9e05ffd7bc59e318c8e2c88ce9e6e1f96eeda9a9 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

Nothing to check. The PR only changes a test file and adds or changes no user-visible control, so no screenshot or blind read is needed.

[UX-REVIEWED] 9e05ffd

Description read

[DESCRIPTION-READ] 7a9ea9a7817dc118c0121923817d470ff36a3b2cbfc94fc0a7dcb8f02f230f80

That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 9e05ffd7bc59e318c8e2c88ce9e6e1f96eeda9a9 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Test-only change that fixes the root cause (an app id reused across runs) with the counter the file already has; nothing blocks.

[DESIGN-REVIEWED] 9e05ffd

Description read

[DESCRIPTION-READ] 80929a34d43053499df454f733277d4c332abab66b8767f28856743cebb506c1

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 9e05ffd7bc59e318c8e2c88ce9e6e1f96eeda9a9 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 9e05ffd

Verdict parsed from the review's SHA-scoped output markers for commit 9e05ffd7bc59e318c8e2c88ce9e6e1f96eeda9a9.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 9e05ffd7bc59e318c8e2c88ce9e6e1f96eeda9a9 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 9e05ffd7bc59e318c8e2c88ce9e6e1f96eeda9a9 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 9e05ffd

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 9e05ffd7bc59e318c8e2c88ce9e6e1f96eeda9a9 false-positive: <reason, 20 to 500 characters>

@kyleseaman

Copy link
Copy Markdown
Contributor Author

Backend Tests (3.12, 8) is red on test/test_reply_format_once.py::TestEveryMessageWhereTheBlockCannotBeTakenOut::test_the_table_names_every_selectable_harness. This is inherited from main, not from this test-only diff: the same test fails on a clean origin/main checkout (a5815c1) because #24732 added the mimocode harness without adding it to that test's table. #36290 adds the missing row. This PR will rebase onto main once that merges.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 13, 2026
@kyleseaman
kyleseaman force-pushed the fix/app-sdk-issues-test-app-id branch from 564e595 to 9e05ffd Compare November 13, 2026 03:31
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 13, 2026
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: Approve

Why: the decider estimates a 4% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes.

The PR changes one test and no product code. The stale-cache test used to mount under the fixed app id issues-app-stale. appQueryClient.ts keeps a module-level map of query clients keyed by app id and session binding, so that client outlived the test. On a repeated run, the first mount found the previous run's cached list and sent no request. The fix gives each run its own id, issues-app-stale-<n>, from the file's existing appSeq counter, the same counter externalApp() uses. Both mounts within one run still share that id, so the test still checks that a reopen after the route cache expires reads the list again. On a single run nothing changes: no other test used the old id, so the first mount always got a fresh client. The assertions are unchanged. The added comment matches the code. I found no other test that reads or writes this id.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 9e05ffd7bc · 6 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 13, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. I read the full diff at 9e05ffd. This is a test-only change: AppSdkIssues.test.tsx gives the stale-cache test a fresh issues-app-stale-<n> app id per run, using the file's existing appSeq. A repeated run then cannot inherit the previous run's route cache. Both mounts inside one run still share the id, which the test relies on. No product code changes.

Not checked: CI is green (97). I ran nothing locally.

@bolichen97
bolichen97 merged commit 282fa12 into main Nov 13, 2026
98 of 100 checks passed
@bolichen97
bolichen97 deleted the fix/app-sdk-issues-test-app-id branch November 13, 2026 06:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: passed Eligible automated validation passed for the current revision

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants