Skip to content

Scope VS Code extension spawnCli fake timers to the test's async context - #21922

Merged
Eric Erhardt (eerhardt) merged 5 commits into
mainfrom
aspireagent/vscode-spawncli-cleanup-flake
Nov 11, 2026
Merged

Eric Erhardt (eerhardt) merged 5 commits into
mainfrom
aspireagent/vscode-spawncli-cleanup-flake

Conversation

@davidfowl

@davidfowl David Fowler (davidfowl) commented Nov 8, 2026 •

Copy link
Copy Markdown
Collaborator

Description

What failed. The Windows VS Code extension unit job intermittently fails spawnCliProcess tests on assert.strictEqual(clock.countTimers(), 0) (actual 1–2), e.g. 42116644663, 42147483007, 42153240896, 42280991794 (POSIX cleanup retry after confirmation timeout) and the main failure 42288258477 tracked in #21886 (taskkill deadline). 16 first-attempt hits in 7 days, all on runs whose cliSpawn.test.ts/cliProcess.ts matched main, so these are not PR-caused.

Failure stats (14d, attempt 1)

CI workflow on microsoft/aspire, runs created 2026-10-25 06:28 → 2026-11-08 06:28 UTC, attempt 1 only. Denominator: attempt-1 jobs that ran (success or failure). Hits: the failed job's log matches the signature. Jobs that lost their runner, which have no log, are excluded. Main runs have no reruns, so their flake check is the next main run of the job.

Signature spawnCliProcess tests failing with AssertionError … strictly equal (clock.countTimers())
Job Run VS Code extension unit tests (Windows)
main pushes: hits / runs 1 / 158 (0.63%)
main pushes, 30d: hits / runs 1 / 273 (0.37%). The 16 days before the 14d window add 0 hits.
PR runs: hits / runs 20 / 986 (2.03%); 0 PR-caused excluded
Combined rate 21 / 1144 = 1.84%
Flake check main: next main run of the job passed 1/1; PR: 14/16 rerun hits passed on rerun (4 never rerun)
First / last hit 2026-10-31 / 2026-11-08
Still hitting main? Yes, still active: last hit 2026-11-08; last main hit 2026-11-06
Fix evidence Local: modelled-flake test fails before and passes after (no natural repro). This PR's CI: the Windows unit job passed 1/1 on the head commit (plus 1/1 earlier).

Failing attempt-1 jobs, most recent first (10 of 17 log-verified hits):

Date (UTC) Branch Job Signature excerpt
2026-11-08 06:03 aspireagent/fix-vscode-folder-picker-colliding-selection Run VS Code extension unit tests (Windows) spawnCliProcess tests rejects when a POSIX process group remains alive after forced termination: AssertionError [ERR_ASS…
2026-11-08 05:14 aspireagent/resource-attach-proof-resume-aborted-inspection Run VS Code extension unit tests (Windows) spawnCliProcess tests retrying POSIX cleanup after confirmation timeout retains process-group ownership: AssertionError …
2026-11-08 04:51 aspireagent/java-debug-timeout-preserve-failure Run VS Code extension unit tests (Windows) spawnCliProcess tests retrying POSIX cleanup after confirmation timeout retains process-group ownership: AssertionError …
2026-11-07 00:57 aspireagent/warn-cli-path-overrides-dnx-19713 Run VS Code extension unit tests (Windows) spawnCliProcess tests rejects when a POSIX process group remains alive after forced termination: AssertionError [ERR_ASS…
2026-11-06 22:53 eerhardt-update-appcontainers-dependency Run VS Code extension unit tests (Windows) spawnCliProcess tests force kills surviving POSIX descendants immediately when their leader exits: AssertionError [ERR_A…
2026-11-06 20:25 main Run VS Code extension unit tests (Windows) spawnCliProcess tests rejects when a Windows child does not close after successful taskkill: AssertionError [ERR_ASSERTI…
2026-11-06 19:35 sebros/resources-compact-view-regression-4abee4 Run VS Code extension unit tests (Windows) spawnCliProcess tests retrying POSIX cleanup after confirmation timeout retains process-group ownership: AssertionError …
2026-11-06 18:19 eerhardt-update-appcontainers-dependency Run VS Code extension unit tests (Windows) spawnCliProcess tests rejects when a Windows child does not close after successful taskkill: AssertionError [ERR_ASSERTI…
2026-11-06 16:14 ievangelist-go-starter-frontend Run VS Code extension unit tests (Windows) spawnCliProcess tests rejects when a POSIX process group remains alive after forced termination: AssertionError [ERR_ASS…
2026-11-06 05:50 ievangelist-bound-ci-mirror-fallback Run VS Code extension unit tests (Windows) spawnCliProcess tests rejects when a POSIX process group remains alive after forced termination: AssertionError [ERR_ASS…

A strict re-scan of the job logs confirmed 17 of the 21 hits counted above; the table lists only confirmed hits. The rates above use the original count.

Verdict: MODERATE. A real and frequent flake: 21 hits (1 on main), 1.8%, still hitting today. 14 of the 16 PR hits that were rerun passed on rerun. The fix is validated only on a model of the foreign-timer race, and its own CI is 2 runs; at a 1.8% rate, ~160 clean runs are needed for 95% confidence.

Why. sinon.useFakeTimers() replaces setTimeout/setImmediate/Date/process.nextTick/queueMicrotask for the whole extension-host process. clock.tickAsync() yields through real setImmediate, and host/other-extension callbacks that run during those yields schedule onto the fake clock, so countTimers() sees work the test never started. terminateCliProcess itself is deterministic under fake timers.

Fix. New test helper withScopedFakeTimers(async clock => ...) installs sinon's clock on a private global object, runs the body via AsyncLocalStorage.run, and routes the real globals to the clock only from the body's async context (the body and everything it awaits or schedules). Callbacks from any other context, including the test runner that invoked the body, keep real timers and Date. Clears are routed by handle, so either kind can be cleared anywhere.

Scopes do not have to be nested. When mocha times out a test, its body keeps running while the next test opens its own scope, so the two can settle in either order. One set of routers serves every active scope and looks up the caller's scope on each call. Settling a scope only retires that scope. The real globals come back when the last active scope settles, and only where our routers are still installed.

The helper fakes the same timer set the old sinon.useFakeTimers({ shouldClearNativeTimers: true }) faked for this code: timers, Date, process.nextTick and queueMicrotask. With sinon 21 / fake-timers 13, that call's clock.methods also includes nextTick and queueMicrotask, so countTimers() counted queued jobs before this change too. It does not fake hrtime, performance or Intl, because neither cliProcess.ts nor the tests read them.

The setTimeout and setImmediate routers also carry util.promisify.custom, routed per call like the timers. A scope's own code waits on its fake clock; any other caller gets the real promisified timer. Without it, util.promisify(setTimeout) threw ERR_INVALID_ARG_TYPE while any scope was active.

cliSpawn.test.ts now uses it for all 13 clocks; assertions are unchanged.

Confidence

  • New scopedFakeTimers.test.ts models the flake: a callback queued before the test runs during tickAsync's real yield and schedules a timer, nextTick and microtask. Paired run with the same test bodies on plain sinon.useFakeTimers({ shouldClearNativeTimers: true }): failing (countTimers() non-zero; foreign code sees fake Date); with the scoped helper: passing on every repeat.
  • Two more regressions from review (caller-queued work must stay off the fake clock; Date subclasses keep their prototype) fail on the first-iteration enterWith helper and pass now.
  • Restore coverage from review: a test captures the real timer globals, Date, Date.now and process.nextTick, then asserts they are all back after a body that resolves, rejects or throws synchronously. It fails when restore is regressed to run only on success or to skip process.nextTick, and passes on the helper.
  • Out-of-order restore from review: a test opens two scopes, settles the first before the second, and checks that each body keeps its own fake clock and that every real global is back afterwards. On the previous helper it fails (The second scope lost its fake clock when the first settled.). A standalone probe of the previous helper also shows setTimeout, Date and process.nextTick left as stale routers after that sequence. With this change both pass.
  • util.promisify from the Copilot review: a test calls promisify(setTimeout) and promisify(setImmediate) from outside an active scope and inside it. On 4ddd5b11 it fails (TypeError [ERR_INVALID_ARG_TYPE]: The "callback" argument must be of type function. Received type number (1)). With this change it passes (helper suite 7/7, cliSpawn.test.ts 50 passing, 1 pending; tsc and eslint clean).
  • Full extension unit suite in plain-Node mocha with the vscode shim: identical failure set before and after (all shim limitations, none in withScopedFakeTimers or spawnCliProcess), with one more passing test (the new one). tsc and eslint clean on changed files.
  • CI on the first push: Windows VS Code extension unit job in the real extension host ran 4155 passing / 0 failing, including the previously flaky retrying POSIX cleanup ... and taskkill ... bounded deadline tests; Final Results green.
  • cliSpawn.test.ts together with progressNotifier, appHostStopper, runSessionRegistry and e2eStateFileBridge (which keep global sinon clocks) in one mocha process: identical failure set before/after, +2 passing (the new tests at the time). tsc and eslint clean on changed files.
  • Not run locally: the real VS Code extension host (VS Code download is blocked in my environment), so local runs used plain Node mocha with a vscode module shim; failures in that run are shim limitations and occur identically on main. Real-host coverage comes from CI. Not stress-tested with repeated Windows runs.

How we stop this recurring: follow up by moving the remaining extension suites that use global sinon.useFakeTimers to withScopedFakeTimers and adding a lint rule against global fake timers in extension tests.

Refs #21886, #21600

🤖 Posted by AspireAgent (AI Agent)

sinon.useFakeTimers replaces timers, Date, nextTick and queueMicrotask for
the whole extension-host process. Host callbacks that run during
clock.tickAsync's real setImmediate yields can schedule onto the fake
clock, so cliSpawn tests asserting clock.countTimers() === 0 failed
intermittently on the Windows unit job.

Add useScopedFakeTimers, which installs sinon's clock on a private global
and routes the real globals to it only for the test's AsyncLocalStorage
context, and use it in cliSpawn.test.ts.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@davidfowl David Fowler (davidfowl) added the agent:working An AI agent is actively working on this issue or PR label Nov 8, 2026 — with Copilot SWE Agent
@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 21922

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 21922"

@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The timer helper lets unrelated caller continuations inherit fake-clock scope and breaks Date subclass prototypes by dropping newTarget in its construct trap.

2 open issues
What changed in this PR

This PR addresses intermittent VS Code extension test failures caused by unrelated extension-host callbacks scheduling work on Sinon’s global fake clock.

Changes:

  • Adds an async-context-aware fake-timer helper intended to preserve real timers for unrelated callbacks.
  • Migrates all 13 fake-clock setups in cliSpawn.test.ts without changing assertions.
  • Adds regression tests for timer isolation and scoped time behavior.
File Description
extension/​src/​test/​scopedFakeTimers.test.ts Tests isolation from foreign callbacks and scoped timers, microtasks, and dates.
extension/​src/​test/​helpers/​scopedFakeTimers.ts Adds context-based routing between real and fake timer APIs.
extension/​src/​test/​cliSpawn.test.ts Replaces global fake clocks with the scoped helper.

Review effort: Balanced

…bclasses

Replace useScopedFakeTimers (enterWith) with withScopedFakeTimers(body), which
runs the body in its own AsyncLocalStorage.run scope so work the test runner
queues after invoking the body can't inherit the fake clock. Forward newTarget
in the routed Date construct trap so Date subclasses keep their prototype.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the changes isolating fake timers in the CLI process tests.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

@radical

Copy link
Copy Markdown
Member

[automated] Inconclusive for efficacy: the requested fresh before/fixed comparison is blocked. The native reproduce-flaky-tests.yml resolves .csproj projects and runs dotnet test; it cannot execute the affected VS Code extension-host suite. No new CI jobs or proxy tests were allocated.

I inspected complete original Windows x64 unit-job logs:

Evidence Full suite spawnCliProcess cases
Historical main, attempt 1 4152 passing, 9 pending, 1 failing 43 pass, 1 fail
Historical peer PR, attempt 1 4152 passing, 9 pending, 1 failing 43 pass, 1 fail
Existing fixed CI, attempt 1 4157 passing, 9 pending 44 pass, 0 fail

The historical failures are rejects when a Windows child does not close after successful taskkill and retrying POSIX cleanup after confirmation timeout retains process-group ownership, both with:

AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
1 !== 0

The POSIX-named scenario also ran on Windows; the peer run's later successful rerun does not erase its first-attempt failure. All three jobs ran the full extension/ corepack yarn test workload on Windows Server 2025, image windows-2025-vs2026 / 20261103.298.1, with VS Code 1.131.0 x64. Setup Node was 22.23.4; the embedded extension-host runtime version was not recorded.

Fixed evidence covers head 449b1e6a22c02b50542b9e62b1d942b62dc0a2b9: its actual synthetic merge checkout has the same tree. The same 44 spawn cases execute across these logs; the four new scoped-timer regression tests pass separately. The same nine tests (none in spawnCliProcess) are pending in all three logs; pending is not a pass.

Scoped review: no high-confidence defect found in the three test-only files; cleanup assertions remain intact. The new regressions exercise foreign-callback and caller-context contamination, but the original logs do not prove who owned the leftover fake-clock work. CI-confidence is not applicable because this diff changes no CI/build/workflow/toolchain behavior.

These are two outcome-selected historical failures and one existing fixed-suite run—not a contemporaneous baseline campaign, a measured failure rate, or proof that the flake is eliminated.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the new withScopedFakeTimers helper end to end: the fake-timers install on the private global, the AsyncLocalStorage routers, the Date proxy, clear routing for native and fake handles, and restore. I also reviewed its tests and the 13 migrated spawnCliProcess tests against cliProcess.ts.

I didn't find any correctness issues. I left one nonblocking note about test coverage for restore.

@davidfowl David Fowler (davidfowl) added the agent:working An AI agent is actively working on this issue or PR label Nov 10, 2026 — with Copilot SWE Agent

David Fowler (davidfowl) commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator Author

AspireAgent stopped: the restore-coverage feedback is addressed in 6dda321 and CI is green.

🤖 Posted by AspireAgent (AI Agent)

Covers bodies that resolve, reject and throw synchronously, so a regression
in restore cannot leave the routers installed for the rest of the run.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified in the test-only timer-isolation changes.

No open issues 🎉

Review effort: Balanced

@davidfowl David Fowler (davidfowl) removed the agent:working An AI agent is actively working on this issue or PR label Nov 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scoping the clock to the test's async context is a reasonable way to fix this, and simpler alternatives (sinon toFake subsets, relaxing countTimers) would either not fix it or lose the leak check. All converted cliSpawn assertions and tick amounts are unchanged. A couple of things on the helper inline.

@davidfowl David Fowler (davidfowl) added the agent:working An AI agent is actively working on this issue or PR label Nov 10, 2026 — with Copilot SWE Agent

David Fowler (davidfowl) commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator Author

AspireAgent stopped: the review threads are addressed, JamesNK is re-requested and Final Results passed on the new head.

🤖 Posted by AspireAgent (AI Agent)

Route the globals through one set of routers shared by every active scope,
which look up the caller's scope on each call. Settling a scope only retires
that scope, and the real globals come back when the last scope settles and
only where our routers are still installed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed to evaluate a cross-context compatibility risk where timer routers can break promisified Node timers in unrelated extension-host code while a scope is active.

No open issues 🎉

Review effort: Balanced

@davidfowl David Fowler (davidfowl) removed the agent:working An AI agent is actively working on this issue or PR label Nov 10, 2026
@davidfowl David Fowler (davidfowl) added the agent:working An AI agent is actively working on this issue or PR label Nov 10, 2026 — with Copilot SWE Agent

David Fowler (davidfowl) commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator Author

AspireAgent made the scoped fake-timer routers support util.promisify (head 4f84aa900e5c8c5ec8cf9bb3e17088acc3695c54) and restacked #21927 on it (head b36b2994ce604dd13c820ff83a7c1bb3fc7ab9b4). Final Results passed on both heads. JamesNK is re-requested.

🤖 Posted by AspireAgent (AI Agent)

The scoped setTimeout and setImmediate routers did not expose
util.promisify.custom, so util.promisify(setTimeout) fell back to the
callback form and threw ERR_INVALID_ARG_TYPE while any scope was active.
Route the promisified form to the calling scope's fake clock, or to the
real promisified timer when no scope owns the caller.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

Tests selector

0 / 101 PR test projects · 2 PR jobs, from 3 changed files.

Selected PR test projects (0 / 101)

none — no PR-gated .NET test projects run for this change.

Selected PR jobs (2)

extension-e2e, extension-unit


How these were chosen — grouped by what changed

Job reasons

Job Triggered by
extension-e2e extension/src/test/cliSpawn.test.ts, extension/src/test/helpers/scopedFakeTimers.ts, extension/src/test/scopedFakeTimers.test.ts
extension-unit extension/src/test/cliSpawn.test.ts, extension/src/test/helpers/scopedFakeTimers.ts, extension/src/test/scopedFakeTimers.test.ts

Selection computed for commit 4f84aa9.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the test-infrastructure changes intended to isolate fake timers from unrelated extension-host work.

No open issues 🎉

Review effort: Balanced

@davidfowl David Fowler (davidfowl) removed the agent:working An AI agent is actively working on this issue or PR label Nov 10, 2026
@eerhardt
Eric Erhardt (eerhardt) merged commit 394b247 into main Nov 11, 2026
95 checks passed
@microsoft-github-policy-service microsoft-github-policy-service Bot added this to the 17.1 milestone Nov 11, 2026
Eric Erhardt (eerhardt) pushed a commit that referenced this pull request Nov 11, 2026
…21927)

## Description

**What failed.** The Windows VS Code extension unit job intermittently
fails `E2E state file bridge > bounds the natural-close connection and
releases its socket on timeout` on
`assert.strictEqual(clock.countTimers(), 0)` with `1 !== 0`:
[42338413245](https://github.com/microsoft/aspire/actions/runs/42338413245),
[42289556640](https://github.com/microsoft/aspire/actions/runs/42289556640),
[42137910318](https://github.com/microsoft/aspire/actions/runs/42137910318).
That's 4 of 608 first attempts in 7 days, and every one passed on rerun.
The failing runs came from unrelated PRs (a Dependabot actions bump and
two dashboard UI changes) that don't touch the extension, so this is a
flake, not a PR-caused failure.

## Failure stats (14d, attempt 1)

`CI` workflow on microsoft/aspire, runs created 2026-10-25 06:28 →
2026-11-08 06:28 UTC, **attempt 1 only**. Denominator: attempt-1 jobs
that ran (success or failure). Hits: the failed job's log matches the
signature. Jobs that lost their runner, which have no log, are excluded.
Main runs have no reruns, so their flake check is the next main run of
the job.

| | |
|---|---|
| Signature | `E2E state file bridge` › `bounds the natural-close
connection and releases its socket on timeout` (`countTimers()` `1 !==
0`) |
| Job | Run VS Code extension unit tests (Windows) |
| main pushes: hits / runs | 0 / 158 (0.00%) |
| main pushes, 30d: hits / runs | 0 / 273 (0.00%). The 16 days before
the 14d window add 0 hits. |
| PR runs: hits / runs | 10 / 986 (1.01%); 0 PR-caused excluded |
| Combined rate | 10 / 1144 = **0.87%** |
| Flake check | PR: 7/7 rerun hits passed on rerun (3 never rerun) |
| First / last hit | 2026-11-05 / 2026-11-08 |
| Still hitting main? | Yes, still active: last hit 2026-11-08; last
main hit never |
| Fix evidence | Local (real VS Code 1.131 host): the new regression
test fails 3/3 with the CI signature before; after the fix the suite
passes 4/4. The natural flake reproduced 1 time in ~18 suite runs. This
PR's CI
([run](https://github.com/microsoft/aspire/actions/runs/42453400853)):
the Windows unit job passed on attempt 1 (1/1). Attempts 1 and 2 failed
only in `VS Code extension E2E (Windows, tree-actions)` with `Expected
e2e-worker to expose an endpoint URL`, which is the known flake #21796
owned by #21914 (this PR touches only unit-test files). Attempt 3 passed
and Final Results is green. |

Failing attempt-1 jobs, most recent first (all 10 hits, all PR runs):

| Date (UTC) | Branch | Job | Signature excerpt |
|---|---|---|---|
| [2026-11-08
06:03](https://github.com/microsoft/aspire/actions/runs/42453169210/job/128646825791)
| `aspireagent/fix-vscode-folder-picker-colliding-selection` | Run VS
Code extension unit tests (Windows) | `bounds the natural-close
connection…: AssertionError strictly equal: 1 !== 0` |
| [2026-11-07
04:25](https://github.com/microsoft/aspire/actions/runs/42338413245/job/128301352107)
| `sebros/resources-compact-view-regression-4abee4` | Run VS Code
extension unit tests (Windows) | `bounds the natural-close connection…:
AssertionError strictly equal: 1 !== 0` |
| [2026-11-06
20:34](https://github.com/microsoft/aspire/actions/runs/42289556640/job/128140462040)
| `dependabot/github_actions/github-actions-ebbe822166` | Run VS Code
extension unit tests (Windows) | `bounds the natural-close connection…:
AssertionError strictly equal: 1 !== 0` |
| [2026-11-06
15:59](https://github.com/microsoft/aspire/actions/runs/42248127696/job/127997089873)
| `sebros/dashboard-review-tracking-issue-f03a30` | Run VS Code
extension unit tests (Windows) | `bounds the natural-close connection…:
AssertionError strictly equal: 2 !== 0` |
| [2026-11-06
14:50](https://github.com/microsoft/aspire/actions/runs/42237500020/job/127960139523)
| `ievangelist-go-runtime-export-readiness` | Run VS Code extension unit
tests (Windows) | `bounds the natural-close connection…: AssertionError
strictly equal: 1 !== 0` |
| [2026-11-06
02:15](https://github.com/microsoft/aspire/actions/runs/42149206621/job/127671234198)
| `spike/project-v2-csharpprogram-watch` | Run VS Code extension unit
tests (Windows) | `bounds the natural-close connection…: AssertionError
strictly equal: 2 !== 0` |
| [2026-11-06
01:20](https://github.com/microsoft/aspire/actions/runs/42143166267/job/127650342491)
| `spike/project-v2-csharpprogram-watch` | Run VS Code extension unit
tests (Windows) | `bounds the natural-close connection…: AssertionError
strictly equal: 2 !== 0` |
| [2026-11-06
00:35](https://github.com/microsoft/aspire/actions/runs/42137910318/job/127631683952)
| `ievangelist-dashboard-sidebar-auto-collapse` | Run VS Code extension
unit tests (Windows) | `bounds the natural-close connection…:
AssertionError strictly equal: 1 !== 0` |
| [2026-11-05
19:43](https://github.com/microsoft/aspire/actions/runs/42096580384/job/127491142502)
| `damianedwards-launch-groups` | Run VS Code extension unit tests
(Windows) | `bounds the natural-close connection…: AssertionError
strictly equal: 1 !== 0` |
| [2026-11-05
19:01](https://github.com/microsoft/aspire/actions/runs/42089852361/job/127466229297)
| `copilot/clarify-scope-of-agent-integrations` | Run VS Code extension
unit tests (Windows) | `bounds the natural-close connection…:
AssertionError strictly equal: 1 !== 0` |

Same root cause, other tests in this suite (not counted above; this PR
migrates them too):
[11-05](https://github.com/microsoft/aspire/actions/runs/42093304445/job/127478234883)
`natural-close response … 2 !== 0` ·
[11-06](https://github.com/microsoft/aspire/actions/runs/42146938315/job/127661485652)
`CDP proxy request … 2 !== 0` ·
[11-06](https://github.com/microsoft/aspire/actions/runs/42296965127/job/128166382811)
`CDP proxy request … 1 !== 0` ·
[11-06](https://github.com/microsoft/aspire/actions/runs/42304086594/job/128189892984)
`natural-close response … 1 !== 0`.

**Verdict: MODERATE.** 10 hits since 2026-11-05 (0.9%), still hitting
today, and all 7 that were rerun passed on rerun. It has **never hit on
main** (0/158). The fix is validated on a modelled host callback in a
real extension host, and its own CI ran the unit job once (green).

<!-- /failure-stats -->

**Why.** The suite calls `sandbox.useFakeTimers()`, which replaces
`setTimeout` and the other timer globals for the whole extension-host
process. `clock.tickAsync()` yields to the real event loop, and host or
other-extension callbacks that run during those yields schedule their
timers on the test's fake clock. `closeBrowserPageThroughCdp` and
`proveBlazorWasmDebugging` clear every timer they create on this path.
The extra timer belongs to someone else. I reproduced it locally in a
real VS Code 1.131 extension host by looping the full unit suite: the
leftover was a 25 ms timer with a bound native callback, created at fake
time 1100 (after the tick ended) and not by the bridge code. This is the
same root cause #21922 found for `spawnCliProcess`.

**Fix.** Every clock in `e2eStateFileBridge.test.ts` (8 tests) now uses
`withScopedFakeTimers` from #21922. Only the test body's async context
sees the fake clock, and host callbacks keep real timers. The assertions
are unchanged. A new regression test, `does not count extension host
timers scheduled while the natural-close connection times out`, queues a
host callback outside the test. The callback runs during the connection
timeout's `tickAsync` and schedules a timer that's still pending when
the wait ends. It stays in this suite as the bridge-level repro of the
original flake, and a comment says so. The helper's own isolation
guarantees are covered in `scopedFakeTimers.test.ts`.

**Depends on #21922.** This branch is stacked on #21922's head
(`4f84aa90`) because it uses that PR's helper, including its fixes for
scopes that settle out of order and for `util.promisify` timers. It
targets `main` so CI runs, which means the diff includes #21922's
commits until that PR merges. After #21922 merges, I'll rebase so only
this PR's two commits remain (`8d966f54`, `b36b2994`).

## Confidence

High that this removes the flake. Evidence, all from a real VS Code
1.131.0 extension host under xvfb:
- **Before** (same test file, with the suite's original global sinon
clock): the new regression test fails 3/3 with the CI signature
`AssertionError: 1 !== 0` at `countTimers()`.
- **After** (`withScopedFakeTimers`): the `E2E state file bridge` suite
passes 4/4 (62 passing).
- Re-checked on the current stack in plain-Node mocha with a `vscode`
shim, because VS Code can't be downloaded in that environment. The
regression test **fails** 3/3 with `1 !== 0` at `countTimers()` when it
uses the global sinon clock, and **passes** 3/3 with
`withScopedFakeTimers`. The bridge, helper and `cliSpawn` suites
together have the same failure set with #21922's version of this file as
with this PR: 3 shim gaps unrelated to timers. This PR adds one passing
test. `tsc` and `eslint` are clean.
- Main remaining uncertainty: I didn't capture the creation stack of the
host timer in CI. I reproduced the natural flake locally once in about
18 full-suite runs, so the regression test models it rather than
replaying the exact host callback.

**How we stop this recurring:** move the remaining extension suites that
still install global `sinon.useFakeTimers` (as #21922 proposes) onto
`withScopedFakeTimers`, and add a lint rule against global fake timers
in extension tests.

Refs #21600, #21922

## Checklist

- Is this feature complete?
  - [x] Yes. Ready to ship.
  - [ ] No. Follow-up changes expected.
- Are you including unit tests for the changes and scenario tests if
relevant?
  - [x] Yes
  - [ ] No
- Did you add public API?
  - [ ] Yes
  - [x] No
- Does the change make any security assumptions or guarantees?
  - [ ] Yes
  - [x] No

**🤖 Posted by AspireAgent (AI Agent)**

---------

Co-authored-by: davidfowl <95136+davidfowl@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@aspire-repo-bot

Copy link
Copy Markdown
Contributor

✅ No documentation update needed.

docs_optional → test_only. No signals triggered. All 3 changed files match extension/src/test/** (cliSpawn.test.ts, helpers/scopedFakeTimers.ts, scopedFakeTimers.test.ts); no user-facing change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants