Repository navigation
Scope E2E state file bridge fake timers to each test's async context - #21927
Conversation
sinon.useFakeTimers replaces timers, Date, nextTick and queueMicrotask for the whole extension-host process. Host callbacks that run during clock.tickAsync's real setImmediate yields can schedule onto the fake clock, so cliSpawn tests asserting clock.countTimers() === 0 failed intermittently on the Windows unit job. Add useScopedFakeTimers, which installs sinon's clock on a private global and routes the real globals to it only for the test's AsyncLocalStorage context, and use it in cliSpawn.test.ts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…bclasses Replace useScopedFakeTimers (enterWith) with withScopedFakeTimers(body), which runs the body in its own AsyncLocalStorage.run scope so work the test runner queues after invoking the body can't inherit the fake clock. Forward newTarget in the routed Date construct trap so Date subclasses keep their prototype. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 21927Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 21927" |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are identified for the scoped fake-timer changes.
No open issues 🎉
What changed in this PR
This PR addresses flaky timer-leak assertions in the VS Code extension’s E2E state-file bridge tests by isolating fake timers to each test’s async context. The stacked diff also includes the shared helper and CLI test migration from #21922.
Changes:
- Add scoped fake timers that leave unrelated extension-host callbacks on real timers.
- Migrate bridge and CLI process tests while preserving existing assertions.
- Add regression coverage for timer isolation, scoped time, and
Datesubclasses.
| File | Description |
|---|---|
extension/src/test/scopedFakeTimers.test.ts |
Tests async-context isolation and scoped timer and Date behavior. |
extension/src/test/helpers/scopedFakeTimers.ts |
Adds async-context timer routing and automatic restoration. |
extension/src/test/e2eStateFileBridge.test.ts |
Migrates fake clocks and adds a host-timer interference regression test. |
extension/src/test/cliSpawn.test.ts |
Migrates process-termination tests to scoped fake timers. |
Review effort: Balanced
|
Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt. |
|
Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt. |
|
[automated] Validation report: blocked for a fresh comparison; inconclusive for flaky-test efficacy. Reviewed head: Existing GitHub evidence confirms the original failure and a passing fixed-source suite, not a measured reduction in flakiness. These are two retrospectively selected CI jobs, not an independently dispatched before/after experiment.
Excerpt from the historical job log (timestamps removed, indentation trimmed, blank lines and the Per each job's runner-setup and step logs, both jobs ran on Windows Server 2025 (runner image The historical checkout was Concrete blocker: the native reproduction workflow resolves only No new workflows or runners were dispatched, and no source changes were made. A macOS run or standalone Node test would not replace the failing Windows Extension Host workload. The actual owner/creation stack of the extra CI timer remains unverified; the new test models that mechanism. Next step: separately scope a faithful Windows Extension Host comparison lane using the existing full-suite command, explicit before/fixed source pins, one invocation per fresh runner, and retained passing/failing results. Admit its exact cohort, destination/ref, and collection plan before dispatch. This is a tooling limitation, not a finding that the PR's fix is wrong. |
Covers bodies that resolve, reject and throw synchronously, so a regression in restore cannot leave the routers installed for the rest of the run. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
James Newton-King (JamesNK)
left a comment
There was a problem hiding this comment.
The bridge test conversion looks right: the harness uses a fake WebSocket emitter whose events fire from in-scope code, so bridge timers are still created in the scope and a genuine leak would still be caught. All assertions and tick amounts are unchanged.
Stacking: this PR carries #21922's first two commits but not 6dda321 (the restore-globals test in scopedFakeTimers.test.ts). Please merge #21922 first and then update this branch from main (rather than rebasing the duplicated commits, which can hit add/add conflicts on the helper), so the restore test isn't lost if #21922 ends up closed as superseded. The helper comments on #21922 apply here as well since the helper is identical.
|
AspireAgent stopped: the review threads are addressed, JamesNK is re-requested and Final Results passed on the new head. 🤖 Posted by AspireAgent (AI Agent) |
Route the globals through one set of routers shared by every active scope, which look up the caller's scope on each call. Settling a scope only retires that scope, and the real globals come back when the last scope settles and only where our routers are still installed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
79160a2 to
339b095
Compare
This comment has been minimized.
This comment has been minimized.
|
Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt. |
|
AspireAgent made the scoped fake-timer routers support 🤖 Posted by AspireAgent (AI Agent) |
The scoped setTimeout and setImmediate routers did not expose util.promisify.custom, so util.promisify(setTimeout) fell back to the callback form and threw ERR_INVALID_ARG_TYPE while any scope was active. Route the promisified form to the calling scope's fake clock, or to the real promisified timer when no scope owns the caller. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The bridge suite installed sinon's clock on the extension host's globals, so a host callback that scheduled a timer during clock.tickAsync()'s real yields was counted by countTimers() as a timer leaked by the natural-close CDP proof. Use withScopedFakeTimers for every clock in the suite and add a regression test that schedules a host timer during the connection timeout. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
339b095 to
b36b299
Compare
Tests selector0 / 101 PR test projects · 2 PR jobs, from 4 changed files. Selected PR test projects (0 / 101)none — no PR-gated .NET test projects run for this change. Selected PR jobs (2)
How these were chosen — grouped by what changedJob reasons
Selection computed for commit |
|
✅ No documentation update needed. docs_optional → test_only. No signals triggered. All 4 changed files match |
Description
What failed. The Windows VS Code extension unit job intermittently fails
E2E state file bridge > bounds the natural-close connection and releases its socket on timeoutonassert.strictEqual(clock.countTimers(), 0)with1 !== 0: 42338413245, 42289556640, 42137910318. That's 4 of 608 first attempts in 7 days, and every one passed on rerun. The failing runs came from unrelated PRs (a Dependabot actions bump and two dashboard UI changes) that don't touch the extension, so this is a flake, not a PR-caused failure.Failure stats (14d, attempt 1)
CIworkflow on microsoft/aspire, runs created 2026-10-25 06:28 → 2026-11-08 06:28 UTC, attempt 1 only. Denominator: attempt-1 jobs that ran (success or failure). Hits: the failed job's log matches the signature. Jobs that lost their runner, which have no log, are excluded. Main runs have no reruns, so their flake check is the next main run of the job.E2E state file bridge›bounds the natural-close connection and releases its socket on timeout(countTimers()1 !== 0)VS Code extension E2E (Windows, tree-actions)withExpected e2e-worker to expose an endpoint URL, which is the known flake #21796 owned by #21914 (this PR touches only unit-test files). Attempt 3 passed and Final Results is green.Failing attempt-1 jobs, most recent first (all 10 hits, all PR runs):
aspireagent/fix-vscode-folder-picker-colliding-selectionbounds the natural-close connection…: AssertionError strictly equal: 1 !== 0sebros/resources-compact-view-regression-4abee4bounds the natural-close connection…: AssertionError strictly equal: 1 !== 0dependabot/github_actions/github-actions-ebbe822166bounds the natural-close connection…: AssertionError strictly equal: 1 !== 0sebros/dashboard-review-tracking-issue-f03a30bounds the natural-close connection…: AssertionError strictly equal: 2 !== 0ievangelist-go-runtime-export-readinessbounds the natural-close connection…: AssertionError strictly equal: 1 !== 0spike/project-v2-csharpprogram-watchbounds the natural-close connection…: AssertionError strictly equal: 2 !== 0spike/project-v2-csharpprogram-watchbounds the natural-close connection…: AssertionError strictly equal: 2 !== 0ievangelist-dashboard-sidebar-auto-collapsebounds the natural-close connection…: AssertionError strictly equal: 1 !== 0damianedwards-launch-groupsbounds the natural-close connection…: AssertionError strictly equal: 1 !== 0copilot/clarify-scope-of-agent-integrationsbounds the natural-close connection…: AssertionError strictly equal: 1 !== 0Same root cause, other tests in this suite (not counted above; this PR migrates them too): 11-05
natural-close response … 2 !== 0· 11-06CDP proxy request … 2 !== 0· 11-06CDP proxy request … 1 !== 0· 11-06natural-close response … 1 !== 0.Verdict: MODERATE. 10 hits since 2026-11-05 (0.9%), still hitting today, and all 7 that were rerun passed on rerun. It has never hit on main (0/158). The fix is validated on a modelled host callback in a real extension host, and its own CI ran the unit job once (green).
Why. The suite calls
sandbox.useFakeTimers(), which replacessetTimeoutand the other timer globals for the whole extension-host process.clock.tickAsync()yields to the real event loop, and host or other-extension callbacks that run during those yields schedule their timers on the test's fake clock.closeBrowserPageThroughCdpandproveBlazorWasmDebuggingclear every timer they create on this path. The extra timer belongs to someone else. I reproduced it locally in a real VS Code 1.131 extension host by looping the full unit suite: the leftover was a 25 ms timer with a bound native callback, created at fake time 1100 (after the tick ended) and not by the bridge code. This is the same root cause #21922 found forspawnCliProcess.Fix. Every clock in
e2eStateFileBridge.test.ts(8 tests) now useswithScopedFakeTimersfrom #21922. Only the test body's async context sees the fake clock, and host callbacks keep real timers. The assertions are unchanged. A new regression test,does not count extension host timers scheduled while the natural-close connection times out, queues a host callback outside the test. The callback runs during the connection timeout'stickAsyncand schedules a timer that's still pending when the wait ends. It stays in this suite as the bridge-level repro of the original flake, and a comment says so. The helper's own isolation guarantees are covered inscopedFakeTimers.test.ts.Depends on #21922. This branch is stacked on #21922's head (
4f84aa90) because it uses that PR's helper, including its fixes for scopes that settle out of order and forutil.promisifytimers. It targetsmainso CI runs, which means the diff includes #21922's commits until that PR merges. After #21922 merges, I'll rebase so only this PR's two commits remain (8d966f54,b36b2994).Confidence
High that this removes the flake. Evidence, all from a real VS Code 1.131.0 extension host under xvfb:
AssertionError: 1 !== 0atcountTimers().withScopedFakeTimers): theE2E state file bridgesuite passes 4/4 (62 passing).vscodeshim, because VS Code can't be downloaded in that environment. The regression test fails 3/3 with1 !== 0atcountTimers()when it uses the global sinon clock, and passes 3/3 withwithScopedFakeTimers. The bridge, helper andcliSpawnsuites together have the same failure set with Scope VS Code extension spawnCli fake timers to the test's async context #21922's version of this file as with this PR: 3 shim gaps unrelated to timers. This PR adds one passing test.tscandeslintare clean.How we stop this recurring: move the remaining extension suites that still install global
sinon.useFakeTimers(as #21922 proposes) ontowithScopedFakeTimers, and add a lint rule against global fake timers in extension tests.Refs #21600, #21922
Checklist
🤖 Posted by AspireAgent (AI Agent)