Skip to content

[release/17.0] Wait for e2e-worker's endpoint URL before capturing it in the tree-actions E2E - #22031

Closed
Mitch Denny (mitchdenny) wants to merge 1 commit into
mitchdenny-backport-17-0-33-pr21963from
mitchdenny-backport-17-0-34-pr21914
Closed

Mitch Denny (mitchdenny) wants to merge 1 commit into
mitchdenny-backport-17-0-33-pr21963from
mitchdenny-backport-17-0-34-pr21914

Conversation

@mitchdenny

Copy link
Copy Markdown
Member

Description

Backport of #21914 to release/17.0 (cherry-picked from 37d2680 with no modifications).

Layer 34 of 55 in the release/17.0 backport stack; stacked on #22030.

Backport recommendation: No — Extension test flake.

Validation: Stable patch IDs match the original commit. Builds/tests not run for this unchanged cherry-pick.

Backport of #21914

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
      • If yes, did you have an API Review for it?
        • Yes
        • No
      • Did you add <remarks /> and <code /> elements on your triple slash comments?
        • Yes
        • No
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
      • If yes, have you done a threat model and had a security review?
        • Yes
        • No
    • No

…tions E2E (#21914)

## Description

The Linux and Windows `tree-actions` VS Code E2E shard fails with
`AssertionError: Expected e2e-worker to expose an endpoint URL.` in
*routes view, copy, endpoint, log, and resource commands through tree
handlers*. It is the most frequent flake since #21624. Examples: [run
41946839177 / job
126989027692](https://github.com/microsoft/aspire/actions/runs/41946839177/job/126989027692)
(Linux), [run 41967348150 / job
127053704182](https://github.com/microsoft/aspire/actions/runs/41967348150/job/127053704182)
(Windows, main), [run 41980257054 / job
127096573746](https://github.com/microsoft/aspire/actions/runs/41980257054/job/127096573746)
(Windows). #21796 lists more.

## Failure stats (14d, attempt 1)

`CI` workflow on microsoft/aspire, runs created 2026-10-25 06:28 →
2026-11-08 06:28 UTC, **attempt 1 only**. Denominator: attempt-1 jobs
that ran (success or failure). Hits: the failed job's log matches the
signature. Jobs that lost their runner, which have no log, are excluded.
Main runs have no reruns, so their flake check is the next main run of
the job.

| | |
|---|---|
| Signature | `Expected e2e-worker to expose an endpoint URL.` |
| Job | VS Code extension E2E (Linux + Windows, tree-actions) |
| main pushes: hits / runs | 8 / 306 (2.61%) |
| main pushes, 30d: hits / runs | 8 / 530 (1.51%). The 16 days before
the 14d window add 0 hits. |
| PR runs: hits / runs | 28 / 1605 (1.74%); 0 PR-caused excluded |
| Combined rate | 36 / 1911 = **1.88%** |
| Flake check | main: next main run of the job passed 8/8; PR: 24/25
rerun hits passed on rerun (3 never rerun) |
| First / last hit | 2026-11-05 / 2026-11-08 |
| Still hitting main? | Yes, still active: last hit 2026-11-08; last
main hit 2026-11-07 |
| Fix evidence | Local real E2E: injected 5s delay 3/3 fail → 3/3 pass;
natural run 1/1 pass. This PR's CI: tree-actions Linux and Windows
passed 1/1 each. |

Failing attempt-1 jobs, most recent first (10 of 34 log-verified hits):

| Date (UTC) | Branch | Job | Signature excerpt |
|---|---|---|---|
| [2026-11-08
06:06](https://github.com/microsoft/aspire/actions/runs/42453400853/job/128650958075)
| `aspireagent/vscode-bridge-scoped-fake-timers` | VS Code extension E2E
(Windows, tree-actions) | `Expected e2e-worker to expose an endpoint
URL.` |
| [2026-11-08
05:17](https://github.com/microsoft/aspire/actions/runs/42450180289/job/128641295561)
| `darc-main-252ee2c2-deac-4980-a34a-b797f49158de` | VS Code extension
E2E (Windows, tree-actions) | `Expected e2e-worker to expose an endpoint
URL.` |
| [2026-11-08
04:40](https://github.com/microsoft/aspire/actions/runs/42447773687/job/128633232882)
| `aspireagent/doctor-cancel-dcp-cleanup` | VS Code extension E2E
(Windows, tree-actions) | `Expected e2e-worker to expose an endpoint
URL.` |
| [2026-11-08
04:32](https://github.com/microsoft/aspire/actions/runs/42447210418/job/128631545572)
| `aspireagent/resource-attach-proof-resume-aborted-inspection` | VS
Code extension E2E (Windows, tree-actions) | `Expected e2e-worker to
expose an endpoint URL.` |
| [2026-11-07
21:02](https://github.com/microsoft/aspire/actions/runs/42413947055/job/128538664400)
| **main** | VS Code extension E2E (Windows, tree-actions) | `Expected
e2e-worker to expose an endpoint URL.` |
| [2026-11-07
03:55](https://github.com/microsoft/aspire/actions/runs/42336149573/job/128297445902)
| `sebros/resources-compact-view-regression-4abee4` | VS Code extension
E2E (Windows, tree-actions) | `Expected e2e-worker to expose an endpoint
URL.` |
| [2026-11-06
23:20](https://github.com/microsoft/aspire/actions/runs/42311290959/job/128230943448)
| **main** | VS Code extension E2E (Windows, tree-actions) | `Expected
e2e-worker to expose an endpoint URL.` |
| [2026-11-06
22:50](https://github.com/microsoft/aspire/actions/runs/42307718588/job/128210216707)
| `ellahathaway-stop-idle-extension-polling` | VS Code extension E2E
(Windows, tree-actions) | `Expected e2e-worker to expose an endpoint
URL.` |
| [2026-11-06
20:25](https://github.com/microsoft/aspire/actions/runs/42288258477/job/128147538335)
| **main** | VS Code extension E2E (Linux, tree-actions) | `Expected
e2e-worker to expose an endpoint URL.` |
| [2026-11-06
20:20](https://github.com/microsoft/aspire/actions/runs/42287581178/job/128143472756)
| `ellahathaway-clarify-agent-init-output` | VS Code extension E2E
(Windows, tree-actions) | `Expected e2e-worker to expose an endpoint
URL.` |

A strict re-scan of the job logs confirmed 34 of the 36 hits counted
above; the table lists only confirmed hits. The rates above use the
original count.

**Verdict: STRONG.** The most frequent signature here: 36 hits since
2026-11-05 (8 on main), still hitting today. 24 of the 25 PR hits that
were rerun passed on rerun, and all 8 main hits passed on the next main
run. The real-E2E before/after repro shows the exact CI assertion. The
PR's own CI is 1 run per OS. That is not statistical proof on its own:
at a 1.9% rate, ~160 clean runs are needed for 95% confidence.

<!-- /failure-stats -->

**Why:** the spec captures `e2e-worker` from the first snapshot where it
is `Running` and later reads its endpoint URL from that same object. DCP
activates the worker's endpoint shortly after the resource reaches
`Running`, and the backchannel omits inactive URLs, so the first
`Running` snapshot can have `urls: []`. The URL-bearing snapshot arrives
a few milliseconds later, but the spec never re-reads it. Since #21624,
the worker reaches `Running` after the extension's `describe --follow`
attaches, so this URL-less snapshot is now visible to the spec.

**Fix:** wait with the existing
`waitForRunningResourceWithUrl('e2e-worker', 'http:', 180000)` helper
(already used by the Azure Functions spec) instead of
`waitForResourceState('e2e-worker', ['Running'])`. This is a test-only
change, and the endpoint assertion is unchanged.

Fixes #21796

## Confidence

- **Real E2E, paired before/after (Linux, xvfb, real VS Code 1.130.0,
CLI/DCP/VSIX/NuGets from main CI run 42413947055):** a local-only CLI
wrapper delays the first URL-bearing `Running` `e2e-worker` line from
`describe --follow` by 5s. The 5s delay is well above the 200ms state
poll interval, so the URL-less snapshot is always observed.
- Before (spec on `origin/main`): **3/3 failed** (6 passing, 1 failing),
each with the exact CI assertion.
  - After (this PR): **3/3 passed** (7/7).
- **Natural (no injection) run with the fix:** 1/1 passed (7/7).
Locally, the URL-bearing `Running` snapshot followed the URL-less one by
2ms.
- **Other checks:** eslint on the spec, `tsc` for both
`tsconfig.e2e.json` and the unit project, and
`e2eResourceReadiness.test.ts` (9/9, run with mocha directly) all pass.
- **Not run locally:** the Windows `tree-actions` shard, other E2E
shards, and `vscode-test`-hosted unit tests (the VS Code download host
is blocked in my environment). CI covers these.

**How we stop this recurring:** E2E specs that read resource URLs should
wait on `waitForRunningResourceWithUrl` instead of a state-only
`Running` gate, because state and URL activation arrive as separate
snapshots.

Related: #21854 made the same change but was closed without being
merged.

## Checklist

- Is this feature complete?
  - [x] Yes. Ready to ship.
  - [ ] No. Follow-up changes expected.
- Are you including unit tests for the changes and scenario tests if
relevant?
- [x] Yes (the change is to an E2E scenario test; the helper it uses
already has unit coverage)
  - [ ] No
- Did you add public API?
  - [ ] Yes
  - [x] No
- Does the change make any security assumptions or guarantees?
  - [ ] Yes
  - [x] No

**🤖 Posted by AspireAgent (AI Agent)**

Co-authored-by: davidfowl <95136+davidfowl@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
(cherry picked from commit 37d2680)
@github-actions

This comment has been minimized.

@mitchdenny
Mitch Denny (mitchdenny) force-pushed the mitchdenny-backport-17-0-34-pr21914 branch 2 times, most recently from c1657d9 to 6ca54c0 Compare November 10, 2026 04:24
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 22031

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 22031"

@mitchdenny

Copy link
Copy Markdown
Member Author

Approved for inclusion in release/17.0. Do not merge yet.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@mitchdenny
Mitch Denny (mitchdenny) force-pushed the mitchdenny-backport-17-0-34-pr21914 branch from 2364a5a to 6ca54c0 Compare November 10, 2026 06:37
@mitchdenny
Mitch Denny (mitchdenny) force-pushed the mitchdenny-backport-17-0-34-pr21914 branch from 6ca54c0 to 2364a5a Compare November 10, 2026 07:42
@mitchdenny
Mitch Denny (mitchdenny) force-pushed the mitchdenny-backport-17-0-34-pr21914 branch from 2364a5a to 6ca54c0 Compare November 10, 2026 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants