Skip to content

[release/17.0] Dump AppHost resource state and logs when Nuxt validation health wait times out - #22042

Closed
Mitch Denny (mitchdenny) wants to merge 1 commit into
mitchdenny-backport-17-0-43-pr21967from
mitchdenny-backport-17-0-44-pr21642
Closed

Mitch Denny (mitchdenny) wants to merge 1 commit into
mitchdenny-backport-17-0-43-pr21967from
mitchdenny-backport-17-0-44-pr21642

Conversation

@mitchdenny

Copy link
Copy Markdown
Member

Description

Backport of #21642 to release/17.0 (cherry-picked from 30f8bf2 with no modifications).

Layer 44 of 55 in the release/17.0 backport stack; stacked on #22041.

Backport recommendation: No — Diagnostics only.

Validation: Stable patch IDs match the original commit. Builds/tests not run for this unchanged cherry-pick.

Backport of #21642

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
      • If yes, did you have an API Review for it?
        • Yes
        • No
      • Did you add <remarks /> and <code /> elements on your triple slash comments?
        • Yes
        • No
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
      • If yes, have you done a threat model and had a security review?
        • Yes
        • No
    • No

… times out (#21642)

## Description

Makes the **Polyglot SDK Validation → TypeScript SDK Validation** Nuxt
dev-mode health-wait hang diagnosable. This is a diagnostics-only
change. It contributes to #21600 and is related to #21146; it does not
close either one.

### Symptom

`aspire start` succeeds. Then `node verify.mjs apphost` polls `aspire
describe` for the full 240s and fails with:

```
AssertionError [ERR_ASSERTION]: Nuxt and the backend did not become healthy within four minutes.
```

It prints nothing about which resource was stuck or why. The work dir is
deleted afterwards, so CI artifacts don't help either.

| Job | Branch | AppHost started | Failed |
|---|---|---|---|
|
[124635347952](https://github.com/microsoft/aspire/actions/runs/41233366554/job/124635347952)
(Node 22) | mitchdenny-init-config-template-sources | 19:36:12 |
19:40:14 |
|
[124524439865](https://github.com/microsoft/aspire/actions/runs/41195598493/job/124524439865)
(Node 22) | tommasodotnet-linux-tray-support | 11:33:02 | 11:37:03 |

In the same window, the step from "AppHost started successfully" to the
next phase took **32–46s in 40 of 40 sampled passing jobs**. This is a
hard hang, not a slow tail, so raising the timeout would not help.

**7-day frequency** (ci.yml runs created 2026-10-25T07:00Z →
2026-11-01T07:20Z, attempt 1, both "TypeScript SDK Validation (Node
22.x/24.x)" jobs; denominator = success + failure, cancelled excluded):

| Branches | Health-wait hang | Denominator | Rate |
|---|---|---|---|
| main | 1
([122922335039](https://github.com/microsoft/aspire/actions/runs/40716902347/job/122922335039),
Node 24) | 121 | 0.8% |
| PR branches | 2 (the two jobs above, Node 22) | 796 | 0.25% |
| **Total** | **3** | **917** | **0.33%** |

The other 14 attempt-1 failures in the window are unrelated:
- 5 were `aspire` CLI exit code 4 (3 runs).
- 6 were TypeScript tsgo/codegen failures (3 runs × 2 Node versions).
- 2 were `TypeScript.PackageManagers.Bun` install failures.
- 1 was an MCR `typescript-node` image pull failure.

### Why diagnostics first

The leading hypothesis is that `web` (`addNuxtApp`, which `waitFor`s
`api`) is stuck behind its hidden `web-installer` / `api-installer` npm
install, or that `nuxt dev` never answers `/api/health`. Existing output
can't distinguish these: `verify.mjs` only reports the boolean outcome,
and the installer resources are hidden from the default `describe`.

### Change

`playground/NuxtApp/verify.mjs`, apphost mode only:
- When the health wait times out, it now runs `aspire describe
--include-hidden --format Json`. For every resource, including hidden
installers, it prints one JSON line with `name, type, state, waitingFor,
exitCode, health, healthReports, started, stopped`.
- It then prints `aspire logs <resource> --tail 200 --timestamps` for
each resource.
- Environment variables, properties and the dashboard URL are
deliberately omitted because they can contain secrets.
- The original assertion still fails the run.
- The timeout stays 240s. `NUXT_VERIFY_APPHOST_TIMEOUT_MS` exists only
so the unit test can exercise the timeout path quickly.
- The file stays self-contained because it is `docker cp`'d into the
validation container.

### Tests

New tests in
`tests/Aspire.Hosting.CodeGeneration.TypeScript.JsTests/tests/nuxt-verification.test.ts`
run `verify.mjs` against a fake `aspire` CLI on `PATH`:
1. **Timeout:** `web` is stuck Waiting on a hidden `web-installer`. The
test asserts that the failure message, the per-resource state lines
(including the hidden installer) and the installer's logs are printed,
and that environment and dashboard secrets are not.
2. **FailedToStart:** `web` exited with code 1. The test asserts that
its state and logs are printed.

| | New tests | Full JsTests suite |
|---|---|---|
| Before (`verify.mjs` from `main`) | 2 of 2 fail (no state or log
output) | — |
| After | 2 of 2 pass | 205 of 205 pass (8 files) |

**Real E2E on this head (f1a4f06):** both **Polyglot SDK Validation /
TypeScript SDK Validation** jobs ran the real `aspire start` →
`verify.mjs apphost` path and passed. The health wait took 35–37s, and
the timeout-only diagnostics did not print (0 `Last 200 log lines`
headers):

| Job | AppHost started | Next phase (Publish) |
|---|---|---|
|
[124787663525](https://github.com/microsoft/aspire/actions/runs/41285477888/job/124787663525)
| 07:52:55 | 07:53:30 |
|
[124787663535](https://github.com/microsoft/aspire/actions/runs/41285477888/job/124787663535)
| 07:53:14 | 07:53:50 |

The new tests also ran in the CI JsTests job
([124786183460](https://github.com/microsoft/aspire/actions/runs/41285477888/job/124786183460)),
where all 8 test files passed.

## Checklist

- Is this feature complete?
  - [x] Yes. Ready to ship.
  - [ ] No. Follow-up changes expected.
- Are you including unit tests for the changes and scenario tests if
relevant?
  - [x] Yes
  - [ ] No
- Did you add public API?
  - [ ] Yes
  - [x] No
- Does the change make any security assumptions or guarantees?
  - [ ] Yes
  - [x] No

**🤖 Posted by AspireAgent (AI Agent)**

Co-authored-by: davidfowl <95136+davidfowl@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
(cherry picked from commit 30f8bf2)
@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@mitchdenny
Mitch Denny (mitchdenny) marked this pull request as ready for review November 10, 2026 05:13
@mitchdenny

Copy link
Copy Markdown
Member Author

Approved for inclusion in release/17.0. Do not merge yet.

@github-actions

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 22042

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 22042"

@github-actions

This comment has been minimized.

@mitchdenny
Mitch Denny (mitchdenny) force-pushed the mitchdenny-backport-17-0-44-pr21642 branch from 523a3db to cea0d43 Compare November 10, 2026 06:24
@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants