Repository navigation
🤖 tests: phase-3 re-audit residuals with no other tracking issue #7635
Copy link
Copy link
Open
Description
The phase-3 test audit re-graded all five areas at main 0059c29c84 (frontend at 9834c30c4d). Each area graded A- or better, and none is borderline. The auditors also listed weaknesses that no existing issue covers. This issue tracks them. Each item gives the place and the reason it matters.
Items already tracked elsewhere are not repeated: #7582 (orchestration), #7587 (streaming), #7604 (seam and export checks), #7608 (fixed sleeps), #7613 (type-aware cast rule), #7615 (orpc casts), #7626 and #7321 (flakes).
Frontend
settle()is weaker than the 20 ms sleep it replaced (tests: settle reconnect suites on promise chains instead of a 20 ms sleep #7607).settle()(connection.testHarness.tsx:129) drains onesetImmediateturn. Mutant evidence: a reset delivered throughsetTimeout(0)survives the new negative checks atRightSidebar.reconnect.test.tsx:55-58,90-91, but the pre-tests: settle reconnect suites on promise chains instead of a 20 ms sleep #7607 test killed it. Fix: advance a fake clock past the owners' timers insettle(), or assert throughadvanceClock. This regression came from a phase-3 PR, so it comes first.as neverbypassesno-cast-to-api-client. Untyped fakes reach the API client atDraftStore.sendIds.test.ts:197,199,WorkflowRunToolCall.test.tsx:118(APIHarnesstakesclient: unknown, so the whole suite runs on an untyped fake),NestedToolRenderer.test.tsx:166,WorkflowTimeline.test.tsx:289,509. 🤖 tests: make no-cast-to-api-client type-aware (P3-4 follow-up) #7613's fix does not catch this. Fix: type the fakes, then flag any type assertion inside acreateTestApiClient(...)argument or anAPIContextapi:value.no-fixed-sleepgaps not in 🤖 tests: P3-4 fixed-sleep residuals in browser tests #7608:setTimeout as delayfromnode:timers/promises, and wait helpers insrc/browsersupport files (outside the rule's file scope).- The
WorkspaceMenuBarimport-count test still runs in its own process. The reason is written only in the tests: retire the Mermaid isolation entry #7591 body. Record it next to the isolation entry, or make the test pass in the shared process. Mermaid.test.tsx:75-86restores itsLazyFeaturespy inline, not infinally, so a failed assertion at:82leaks the spy.
Orchestration
- 9
.sessions.setwrites remain outside the real-host builder:workspaceService.rename.test.ts:217,330,410,workspaceService.initCancellation.test.ts:582,workspaceService.archive.test.ts:954,workspaceService.test.ts:414,workspaceGoals.repro.test.ts:241,serviceContainer.test.ts:526-527. Move each ontocreateRealHostWorkspaceService, or record why it must plant a session. - Two surviving mutants in the task launch path (seen with
taskLaunchReproonly, not the full taskService suite): dropping the re-check under the fork lock (taskService.ts:7934), and forcingreusedExistingCheckout: true(taskService.ts:7973). First run the full taskService suite against both. If they still survive, add tests. - 🤖 tests: remaining private-state access in orchestration tests (turn seeds, WorkspaceTurnManager, contextController) #7582 says 17 turn seeds remain. Its own list has 14 call sites. Correct the count.
Streaming
streamManager.completion.test.ts:97-130("the retry's partial delete holds the history write lock") only exerciseshistoryService.deletePartialIfMessageIdMatches. It is that method's only lock test. Move it into ahistoryService.*.test.tsfile.historyService.test.ts:1498,2000,2037read the privatesequenceCounters. The tests already check the next append'shistorySequence, which is the public behavior, so drop the private reads.- Kept on purpose (no work planned): the 20 private casts in
memoryService.test.tsguard a memoization step that has no public signal. The 8 private reads inagentSession.continuousCompaction.test.tsare older 🤖 tests: drive continuous-compaction collaborators through the session's real seams #4604 residuals.
Tools
installService.test.tsawaits the privatesettledSignOutPassesat:3795, 3846, 3960, 4046, 4094, 4168, 4196, 4247, 4290(9 of its 29 private casts), and patcheswriteRegistry(:1059, 1126, 1354) andremoveDir(:5760, 7063, 7209). Wait on a public signal instead.workspaceMcpOverridesService.test.tscalls the privatebumpOverridesEpochat:2667, 3455, 3628(the largest group of its 27 casts).- Flaky 100 ms stale-lease window.
WorkflowRunner.attemptDisposition.test.ts:18(STALE_LEASE_MS = 100): "already-active errors name the lease owner and its freshness" failed once under load (914 ms), because the lease went stale. Use a fake clock or a much longer window. - The singular "1 more server" branch (
toolCatalog.ts:440) is untested:toolCatalog.test.ts:344matches/\d+ more servers/. toolCatalog.test.ts:611checks the fixed text "Search deferred tools", not the overview its name claims.- Artifact test overlap:
artifact_list.test.ts:39repeats:46, andartifact.test.ts:102only countsexeccalls. - The dynamic
agent_reportschema (agent_report.ts:93,156) is outside the.nullish()check overTOOL_DEFINITIONS.
Cross-cutting
- The seam check misses "injectable for tests", "
xis for tests" and "test double(s)" wording (SEAM_COMMENT_PATTERNS,scripts/check-test-seam-comments.ts:32). Uncaught seams today: thesendRetryDelayMsoption (DraftStore.ts:532, onlyDraftStore.sendIds.test.ts:233passes it), thepathImplparameter (artifactVersionsOperations.ts:65, only tests passpath.win32), and the "(test doubles)" branch atturnRequestBuilder.ts:2684. 🤖 tests: seam check flags test-double seam claims #7620 covers part of the "test doubles" wording. - Exports used only in their own file:
WorkspaceServiceEvents(workspaceService.ts:2305) andMOCK_STREAM_START_GATE_MARKER(mockAiRouter.ts:64). Dropexport, and havecheck-test-only-exports(🤖 tests: fail on production exports that only tests (or nothing) reference #7561) flag this case, which its own error message already describes. - 67
as anyremain insrctest files (43 with eslint-disable lines). This is not a regression. Reduce it as files are touched.
Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high • Cost: $43.07
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity