Repository navigation
[release/17.0] Wait for e2e-worker's endpoint URL before capturing it in the tree-actions E2E - #22031
Closed
Mitch Denny (mitchdenny) wants to merge 1 commit into
Conversation
…tions E2E (#21914) ## Description The Linux and Windows `tree-actions` VS Code E2E shard fails with `AssertionError: Expected e2e-worker to expose an endpoint URL.` in *routes view, copy, endpoint, log, and resource commands through tree handlers*. It is the most frequent flake since #21624. Examples: [run 41946839177 / job 126989027692](https://github.com/microsoft/aspire/actions/runs/41946839177/job/126989027692) (Linux), [run 41967348150 / job 127053704182](https://github.com/microsoft/aspire/actions/runs/41967348150/job/127053704182) (Windows, main), [run 41980257054 / job 127096573746](https://github.com/microsoft/aspire/actions/runs/41980257054/job/127096573746) (Windows). #21796 lists more. ## Failure stats (14d, attempt 1) `CI` workflow on microsoft/aspire, runs created 2026-10-25 06:28 → 2026-11-08 06:28 UTC, **attempt 1 only**. Denominator: attempt-1 jobs that ran (success or failure). Hits: the failed job's log matches the signature. Jobs that lost their runner, which have no log, are excluded. Main runs have no reruns, so their flake check is the next main run of the job. | | | |---|---| | Signature | `Expected e2e-worker to expose an endpoint URL.` | | Job | VS Code extension E2E (Linux + Windows, tree-actions) | | main pushes: hits / runs | 8 / 306 (2.61%) | | main pushes, 30d: hits / runs | 8 / 530 (1.51%). The 16 days before the 14d window add 0 hits. | | PR runs: hits / runs | 28 / 1605 (1.74%); 0 PR-caused excluded | | Combined rate | 36 / 1911 = **1.88%** | | Flake check | main: next main run of the job passed 8/8; PR: 24/25 rerun hits passed on rerun (3 never rerun) | | First / last hit | 2026-11-05 / 2026-11-08 | | Still hitting main? | Yes, still active: last hit 2026-11-08; last main hit 2026-11-07 | | Fix evidence | Local real E2E: injected 5s delay 3/3 fail → 3/3 pass; natural run 1/1 pass. This PR's CI: tree-actions Linux and Windows passed 1/1 each. | Failing attempt-1 jobs, most recent first (10 of 34 log-verified hits): | Date (UTC) | Branch | Job | Signature excerpt | |---|---|---|---| | [2026-11-08 06:06](https://github.com/microsoft/aspire/actions/runs/42453400853/job/128650958075) | `aspireagent/vscode-bridge-scoped-fake-timers` | VS Code extension E2E (Windows, tree-actions) | `Expected e2e-worker to expose an endpoint URL.` | | [2026-11-08 05:17](https://github.com/microsoft/aspire/actions/runs/42450180289/job/128641295561) | `darc-main-252ee2c2-deac-4980-a34a-b797f49158de` | VS Code extension E2E (Windows, tree-actions) | `Expected e2e-worker to expose an endpoint URL.` | | [2026-11-08 04:40](https://github.com/microsoft/aspire/actions/runs/42447773687/job/128633232882) | `aspireagent/doctor-cancel-dcp-cleanup` | VS Code extension E2E (Windows, tree-actions) | `Expected e2e-worker to expose an endpoint URL.` | | [2026-11-08 04:32](https://github.com/microsoft/aspire/actions/runs/42447210418/job/128631545572) | `aspireagent/resource-attach-proof-resume-aborted-inspection` | VS Code extension E2E (Windows, tree-actions) | `Expected e2e-worker to expose an endpoint URL.` | | [2026-11-07 21:02](https://github.com/microsoft/aspire/actions/runs/42413947055/job/128538664400) | **main** | VS Code extension E2E (Windows, tree-actions) | `Expected e2e-worker to expose an endpoint URL.` | | [2026-11-07 03:55](https://github.com/microsoft/aspire/actions/runs/42336149573/job/128297445902) | `sebros/resources-compact-view-regression-4abee4` | VS Code extension E2E (Windows, tree-actions) | `Expected e2e-worker to expose an endpoint URL.` | | [2026-11-06 23:20](https://github.com/microsoft/aspire/actions/runs/42311290959/job/128230943448) | **main** | VS Code extension E2E (Windows, tree-actions) | `Expected e2e-worker to expose an endpoint URL.` | | [2026-11-06 22:50](https://github.com/microsoft/aspire/actions/runs/42307718588/job/128210216707) | `ellahathaway-stop-idle-extension-polling` | VS Code extension E2E (Windows, tree-actions) | `Expected e2e-worker to expose an endpoint URL.` | | [2026-11-06 20:25](https://github.com/microsoft/aspire/actions/runs/42288258477/job/128147538335) | **main** | VS Code extension E2E (Linux, tree-actions) | `Expected e2e-worker to expose an endpoint URL.` | | [2026-11-06 20:20](https://github.com/microsoft/aspire/actions/runs/42287581178/job/128143472756) | `ellahathaway-clarify-agent-init-output` | VS Code extension E2E (Windows, tree-actions) | `Expected e2e-worker to expose an endpoint URL.` | A strict re-scan of the job logs confirmed 34 of the 36 hits counted above; the table lists only confirmed hits. The rates above use the original count. **Verdict: STRONG.** The most frequent signature here: 36 hits since 2026-11-05 (8 on main), still hitting today. 24 of the 25 PR hits that were rerun passed on rerun, and all 8 main hits passed on the next main run. The real-E2E before/after repro shows the exact CI assertion. The PR's own CI is 1 run per OS. That is not statistical proof on its own: at a 1.9% rate, ~160 clean runs are needed for 95% confidence. <!-- /failure-stats --> **Why:** the spec captures `e2e-worker` from the first snapshot where it is `Running` and later reads its endpoint URL from that same object. DCP activates the worker's endpoint shortly after the resource reaches `Running`, and the backchannel omits inactive URLs, so the first `Running` snapshot can have `urls: []`. The URL-bearing snapshot arrives a few milliseconds later, but the spec never re-reads it. Since #21624, the worker reaches `Running` after the extension's `describe --follow` attaches, so this URL-less snapshot is now visible to the spec. **Fix:** wait with the existing `waitForRunningResourceWithUrl('e2e-worker', 'http:', 180000)` helper (already used by the Azure Functions spec) instead of `waitForResourceState('e2e-worker', ['Running'])`. This is a test-only change, and the endpoint assertion is unchanged. Fixes #21796 ## Confidence - **Real E2E, paired before/after (Linux, xvfb, real VS Code 1.130.0, CLI/DCP/VSIX/NuGets from main CI run 42413947055):** a local-only CLI wrapper delays the first URL-bearing `Running` `e2e-worker` line from `describe --follow` by 5s. The 5s delay is well above the 200ms state poll interval, so the URL-less snapshot is always observed. - Before (spec on `origin/main`): **3/3 failed** (6 passing, 1 failing), each with the exact CI assertion. - After (this PR): **3/3 passed** (7/7). - **Natural (no injection) run with the fix:** 1/1 passed (7/7). Locally, the URL-bearing `Running` snapshot followed the URL-less one by 2ms. - **Other checks:** eslint on the spec, `tsc` for both `tsconfig.e2e.json` and the unit project, and `e2eResourceReadiness.test.ts` (9/9, run with mocha directly) all pass. - **Not run locally:** the Windows `tree-actions` shard, other E2E shards, and `vscode-test`-hosted unit tests (the VS Code download host is blocked in my environment). CI covers these. **How we stop this recurring:** E2E specs that read resource URLs should wait on `waitForRunningResourceWithUrl` instead of a state-only `Running` gate, because state and URL activation arrive as separate snapshots. Related: #21854 made the same change but was closed without being merged. ## Checklist - Is this feature complete? - [x] Yes. Ready to ship. - [ ] No. Follow-up changes expected. - Are you including unit tests for the changes and scenario tests if relevant? - [x] Yes (the change is to an E2E scenario test; the helper it uses already has unit coverage) - [ ] No - Did you add public API? - [ ] Yes - [x] No - Does the change make any security assumptions or guarantees? - [ ] Yes - [x] No **🤖 Posted by AspireAgent (AI Agent)** Co-authored-by: davidfowl <95136+davidfowl@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 37d2680)
14 tasks
This comment has been minimized.
This comment has been minimized.
Mitch Denny (mitchdenny)
force-pushed
the
mitchdenny-backport-17-0-34-pr21914
branch
2 times, most recently
from
November 10, 2026 04:24
c1657d9 to
6ca54c0
Compare
Contributor
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 22031Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 22031" |
Mitch Denny (mitchdenny)
marked this pull request as ready for review
November 10, 2026 05:10
Mitch Denny (mitchdenny)
requested review from
Adam Ratzman (adamint) and
Ella Hathaway (ellahathaway)
as code owners
November 10, 2026 05:10
Member
Author
|
Approved for inclusion in release/17.0. Do not merge yet. |
Mitch Denny (mitchdenny)
force-pushed
the
mitchdenny-backport-17-0-34-pr21914
branch
from
November 10, 2026 05:59
6ca54c0 to
2364a5a
Compare
This comment has been minimized.
This comment has been minimized.
Contributor
|
Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt. |
Mitch Denny (mitchdenny)
force-pushed
the
mitchdenny-backport-17-0-34-pr21914
branch
from
November 10, 2026 06:37
2364a5a to
6ca54c0
Compare
Mitch Denny (mitchdenny)
force-pushed
the
mitchdenny-backport-17-0-34-pr21914
branch
from
November 10, 2026 07:42
6ca54c0 to
2364a5a
Compare
Mitch Denny (mitchdenny)
force-pushed
the
mitchdenny-backport-17-0-34-pr21914
branch
from
November 10, 2026 08:47
2364a5a to
6ca54c0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Backport of #21914 to
release/17.0(cherry-picked from 37d2680 with no modifications).Layer 34 of 55 in the release/17.0 backport stack; stacked on #22030.
Backport recommendation: No — Extension test flake.
Validation: Stable patch IDs match the original commit. Builds/tests not run for this unchanged cherry-pick.
Backport of #21914
Checklist
<remarks />and<code />elements on your triple slash comments?