Repository navigation
Bound primary apt mirror builds before CLI E2E fallback - #21849
David Pine (IEvangelist) wants to merge 4 commits into
Conversation
Move shared image orchestration into a testable shell entry point, reserve fallback time inside the existing job budget, and fail closed on unsafe cancellation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 21849Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 21849" |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
🟡 Changes recommended
Derived-image fallbacks can reuse the throttled Azure mirror and exhaust the job, while external cancellation can incorrectly trigger a retry that reports success.
2 open issues
What changed in this PR
This PR bounds primary apt-mirror attempts in shared CLI E2E image builds, addressing the slow downloads that blocked validation in #21828 without depending on that PR.
Changes:
- Extracts image-build orchestration into a shared script with five-minute primary limits and fallback handling.
- Adds behavioral coverage for retries, cancellation, cleanup, and image selection.
- Updates test routing, cache-warmer triggers, and documentation.
| File | Description |
|---|---|
tests/Infrastructure.Tests/TestTriggerMap/TestTriggerMapTests.cs |
Covers test selection for the shared script. |
tests/Infrastructure.Tests/Pipelines/CliE2EImageBuildTests.cs |
Registers nine Linux behavioral scenarios. |
tests/Infrastructure.Tests/Pipelines/cli-e2e-image-build.harness.ts |
Exercises orchestration, retries, and process cleanup. |
eng/scripts/build-cli-e2e-images.sh |
Centralizes builds with bounded primary attempts. |
eng/github-ci/test-trigger-map.yml |
Routes script changes to E2E and infrastructure tests. |
docs/ci/cli-e2e-images.md |
Documents timeout budgets and fallback behavior. |
.github/workflows/warm-cli-e2e-image-cache.yml |
Triggers cache warming when the script changes. |
.github/workflows/build-cli-e2e-image.yml |
Replaces inline orchestration with the shared script. |
Review effort: Balanced
Snapshot original Ubuntu sources before configuring mirrors so derived-image fallbacks restore defaults. Normalize cancellation in a supervising shell even when Docker exits with code 1 after TERM. Add external-signal and real Ubuntu restoration regressions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
🟡 Changes recommended
An external TERM during deadline cleanup can be ignored, allowing fallback to report success after cancellation; cancellation must be tracked independently and covered by a regression test.
1 open issue
2 resolved since last review
Review effort: Balanced
Track primary deadlines and external signals independently; retain process-group TERM and fail-closed kill grace. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Tests selectorSelects the full PR test matrix + all PR-gated jobs (ALL) — a rule matching 'tests/Shared/Docker/configure-ubuntu-apt-mirror.sh' selects ALL Selection computed for commit |
|
Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt. |
## Description Go starters should build from their shipped files without an initial `aspire run` repairing dependencies, and Go preview validation should exercise the same runtime and native-runner paths as other AppHosts. - Ship a tidy API `go.mod` and generated `go.sum`. Add starter scenarios checking gitignore, readonly API build, tidy stability, health and weather responses; remove the Kubernetes manual-tidy workaround. - Honor the existing `--use-redis-cache false` option for Go. Omitted/true retains Redis; false omits the integration and resource. Cover all three scaffold choices with generated-source snapshots and both HTTP lifecycle variants with Docker E2E. - Add `go vet` to Go polyglot fixtures, an EF Core fixture, and Foundry tool/AKS node-pool parity. Exercise configuration-section list getters in the real Go runtime smoke. - Add Go to all six native starter-validation platforms, with official checksum-verified Go installation, timeout budget, and runtime-only selector edges for Go hosting, Go codegen, and Redis. Linux retains API/Redis readiness; containerless Windows/macOS use the cache-free starter with API readiness unchanged. - Add isolated-local-feed Go/Java scaffolding and restore to the stabilization smoke without rewriting template pins. Use a genuine Release bundled driver outside repository mode, cold consumer caches, and exact hosting-package extraction/provenance checks. Enable Java only in its subprocess. - Select languages by name rather than picker position. Restore stale Java/TypeScript SDK baselines after `aspire add` so the following `aspire restore` genuinely tests regeneration. Establish the TypeScript baseline with an actual restore before reading cache metadata. - Wait for the TypeScript Kubernetes application's actual Ready pod; exercise the Go Helm namespace/chart-version callback. Relates to #21202, #21206, #21209, #21210, #21201, and #21108. ### User-facing usage ```bash # Default retains Redis. aspire new aspire-go-starter --name GoReady --non-interactive --suppress-agent-init # Containerless starter: the Go API runs without Redis or a Docker daemon. aspire new aspire-go-starter --name GoReady --use-redis-cache false --non-interactive --suppress-agent-init ``` Both variants emit `api/go.sum` and an already-tidy API module. Its API can build with `go build -mod=readonly` without first running the AppHost's tidy resource. ### Dependencies and remaining acceptance This PR is independent of #21823. Root-module brownfield E2E #21825 merged into the #21823 feature branch, not directly into main; its delivery follows #21823. #21806 independently covers other Go init/build/vet/stable-restore scenarios. The existing stabilization gate remains disabled (`STABILIZATION_ENABLED: 'false'`); enabling it needs a separate CI/policy decision. Current `Aspire.Hosting.Java.csproj` still sets `SuppressFinalPackageVersion=true` while the Java starter pins the requested Aspire version. The corrected smoke reaches actual NuGet restore and exposes that mismatch: `Aspire.Hosting.Java (= 17.0.0)` is unavailable, the nearest local package is `17.0.0-dev`, and nuget.org versions are not considered. No pin substitution or Java release-policy change is included. Native Linux validation built the genuine stabilized Release CLI/payload and proved cold Go/Redis package restore and C# restore. This is **not a clean stabilization pass**: the unsuppressed stable pack first reproduced NU5104 on existing .NET 11 preview dependencies. The diagnostic consumer feed downgraded only NU5104 with `WarningsNotAsErrors=NU5104`; that exception is not committed. The final smoke correctly exits 1 with only Java in its failure list. The bounded shared-image fix is separate in #21849; this PR does not modify image Dockerfiles or the image-build implementation. Its `b8274fc8e0` image job and all four uploads passed, Infrastructure passed 2090/2090 including all 13 new image cases, and all 116 CLI archive E2E jobs passed. Earlier Windows extension fake-timer and DCP Docker-startup failures were investigated with same-source passing main controls; the morning live refresh reports its Final Results passing. It remains independently review-gated. Do not infer actual timeout/fallback execution from a healthy-mirror image pass. No action SHA is added or changed. No official Azure DevOps pipeline is modified. Parent's Go-minimum-version map rule/theory and the other owner's extension-e2e map block are untouched. The cache option uses existing embedded CLI inputs; no additional selector rule is required. The stabilization script remains advisory/disabled rather than being presented as a regular-PR gate. This does not close #21108: live Azure Container Apps deployment and real missing/old-toolchain E2E remain uncovered here. Cross-language lifecycle redesigns are deferred. ### CI failures diagnosed and fixed Original-head run [42116644663](https://github.com/microsoft/aspire/actions/runs/42116644663) exposed two causes: - Windows x64/ARM64 and macOS x64/ARM64 starter validation failed because Redis could not start: the Windows Docker engine pipe was absent and macOS had no Docker executable. The API correctly waited on its unhealthy dependency. Those native runners now use the supported cache-free variant; Linux and Docker E2E retain Redis coverage. No readiness assertion was removed. - `TypeScriptCodegenValidationTests` read `.codegen-hash` immediately after init, which supplies SDK sources but not restore-cache metadata. An explicit baseline restore now precedes the snapshot and stale-cache assertions. **Head `1ab3ad1e30`: all six native starter lanes passed**, including both Redis-enabled Ubuntu lanes and all four formerly failing Windows/macOS lanes, in [run 42151619597](https://github.com/microsoft/aspire/actions/runs/42151619597). Packaged CI subsequently passed [GoStarterReadinessTests](https://github.com/microsoft/aspire/actions/runs/42151619597/job/127721672515) (both cache choices), [GoStarterTemplateTests](https://github.com/microsoft/aspire/actions/runs/42151619597/job/127721672968), [JavaCodegenValidationTests](https://github.com/microsoft/aspire/actions/runs/42151619597/job/127721672216), and [TypeScriptCodegenValidationTests](https://github.com/microsoft/aspire/actions/runs/42151619597/job/127721664876). All originally failing targeted lanes/classes therefore have passing coverage. The shared image job first timed out at 45 minutes twice. Attempt 2 fetched 108 MB from Azure Ubuntu in 41 minutes 29 seconds (43.4 kB/s), versus a following GitHub download at 80.1 MB/s; GHA cache import also returned HTTP 400 service-unavailable responses. After #21849's same-input image control passed with all four uploads, one evidence-based product revalidation was started. [Attempt 3's image job passed](https://github.com/microsoft/aspire/actions/runs/42151619597/job/127718680817) and admitted packaged E2E. No blind retries were made. The successful infrastructure control used a healthy primary mirror; it does not demonstrate timeout/fallback execution in that CI run. **Head `bd30794b9e`: historical failures and final outcome are distinct.** In [run 42167868738](https://github.com/microsoft/aspire/actions/runs/42167868738), all six native starter lanes and Go/Java/TypeScript SDK validation passed. The first [shared image job](https://github.com/microsoft/aspire/actions/runs/42167868738/job/127729647462) was canceled after 45 minutes 22 seconds with Azure Ubuntu apt downloads still running and all four uploads skipped; the initial Linux tree-actions attempt also failed on a stale immutable Running snapshot. The morning live refresh subsequently shows [Final Results passing](https://github.com/microsoft/aspire/actions/runs/42167868738/job/127754717945), including [GoStarterReadinessTests](https://github.com/microsoft/aspire/actions/runs/42167868738/job/127747615122). This session did not initiate retries. A passing later attempt does not repair the tree-actions race: its separate test-only fix #21854 still needs independent approval/merge and incorporation before claiming the race fixed in this checkout. **Latest head `0bb6569d91` requires fresh checks**, now running in [42222384609](https://github.com/microsoft/aspire/actions/runs/42222384609). Both new review findings are corrected: SDK selection accepts only exact numeric MAJOR.MINOR.PATCH package filenames, and Go starter startup is inside its existing cleanup scope. Both threads were replied to and resolved with evidence. Earlier-head CI is not presented as latest-head proof. Per the user's explicit authorization, the coordinator enabled normal native **squash auto-merge** at 05:22:46Z on 2026-11-06. It remains enabled behind required fresh checks and review requirements. No admin bypass is authorized or used; the PR is not merged. ### Stabilization review corrections The smoke now publishes a Release CLI with its real stabilized bundle outside the checkout, propagates stabilization into child payload builds, and clears inherited `ASPIRE_REPO_ROOT` for every consumer invocation. Consumer packages are isolated after the driver build so build-time extraction cannot hide a missing stable package. Successful guest restores must produce metadata for the exact hosting pins, including Redis for Go; provenance is retained in the output. The real run also exposed two stale smoke assumptions: a pre-owned parent feed mapping conflicts with scaffolding's project-local channel policy, and current C# file-based templates use `#:package Aspire.Hosting.AppHost@<version>`, not the obsolete SDK directive. The smoke now lets the CLI establish its own authoritative project-scoped feed mapping and checks the current hosting-package contract. No customer NuGet policy is changed. ### Validation - `.\restore.cmd` - passed. - `dotnet build tests\Aspire.Cli.EndToEnd.Tests\Aspire.Cli.EndToEnd.Tests.csproj --no-restore --nologo -v quiet` - passed, zero warnings/errors; repeated successfully after removing temporary fault injection for `0bb6569d91`. - `dotnet test --project tests\Aspire.Cli.Tests\Aspire.Cli.Tests.csproj --no-launch-profile -- --filter-class '*.NewCommandTemplateConfigPersistenceTests' --filter-not-trait 'quarantined=true' --filter-not-trait 'outerloop=true'` - passed, 59 tests including default/true/false Go package and generated-source assertions. Existing ILCompiler NU1903 advisory was emitted. - With `ASPIRE_E2E_CLI_ARCHIVE_DIR=artifacts\readiness-ci\cli-archive-new` and `ASPIRE_E2E_DOTNET_IMAGE=aspire-e2e-polyglot-base:latest`, `dotnet test --project tests\Aspire.Cli.EndToEnd.Tests\Aspire.Cli.EndToEnd.Tests.csproj --no-launch-profile -- --filter-method '*.RestoreGeneratesSdkFiles_WithConfiguredToolchain' --filter-not-trait 'quarantined=true' --filter-not-trait 'outerloop=true'` - passed, all four npm/bun/yarn/pnpm cases against exact `1ab3ad1e30` Linux CLI/packages. The corresponding packaged CI class also passed above. - With `ASPIRE_E2E_CLI_ARCHIVE_DIR=artifacts\readiness-ci\cli-archive-new` and `ASPIRE_E2E_POLYGLOT_GO_IMAGE=aspire-go-e2e-6972:latest`, `dotnet test --project tests\Aspire.Cli.EndToEnd.Tests\Aspire.Cli.EndToEnd.Tests.csproj --no-launch-profile -- --filter-method '*.StarterShipsTidyApiAndServesRequests' --filter-not-trait 'quarantined=true' --filter-not-trait 'outerloop=true'` - 1 passed/1 failed locally against exact `1ab3ad1e30`. Cache-free passed actual scaffolding, readonly API build, tidy check, API-up, health/weather and stop without a Docker socket. Redis timed out locally. The unchanged `2859e199e2` control reproduced the same timeout on this Docker Desktop host; no workaround or weaker assertion was added. **Both cases subsequently passed packaged CI**, linked above. After the cleanup-scope correction, this local command again passed cache-free and failed Redis on API readiness; the independent CLI stop log confirmed successful shutdown, while the shared terminal helper observed the prior ERR prompt. That local run is not claimed green. - Temporary fault-injection validation used that same E2E command with only the cache-free case and a shell wrapper returning HTTP 503 for the Dashboard probe after a real packaged AppHost launch - intentionally failed. Terminal output showed `dashboard-http-failed`, then cleanup invocation; the CLI log confirmed `apphost.go stopped successfully` and exit 0. The wrapper and temporary test edit were removed before the final clean build/commit. This verifies partial-start cleanup, not a successful readiness scenario. - `docker run --rm --entrypoint bash -v "${PWD}:/workspace:ro" aspire-go-e2e-6972:latest -c "sed 's/\r$//' /workspace/artifacts/readiness-ci/validate-stable-sdk-selection.sh | bash"` - passed actual smoke-script preflight cases for missing SDK, prerelease-only, mixed stable/dev/preview/rc/build-metadata/four-component filenames, multi-digit stable versions, and feed paths with spaces. The instrumented driver received `BundleVersion=17.0.0` from the mixed feed. Invalid-only feeds exited 1 with the explicit missing-stable diagnostic. Temporary harness removed after retaining logs; not a new all-language stabilization pass. - `dotnet test --project tests\Infrastructure.Tests\Infrastructure.Tests.csproj --no-launch-profile -- --filter-class '*.TestTriggerMapTests' --filter-not-trait 'quarantined=true' --filter-not-trait 'outerloop=true'` - passed, 176 tests. - `dotnet run --project tools\SelectTests -- --changed-files artifacts\readiness-changed-files.txt --explain` - passed against the real graph/map; Go hosting/codegen, Redis, starter checksum and EF Core fixture inputs select starter-validation/polyglot with zero unattributed files. Schedule-only deployment remains advisory. - `$env:DOTNET_ROOT = Join-Path $PWD '.dotnet'; $env:ASPIRE_REPO_ROOT = $PWD; .\artifacts\bin\Aspire.Cli\Debug\net11.0\aspire.exe restore --apphost tests\PolyglotAppHosts\<integration>\Go\apphost.go --non-interactive` - passed separately for `Aspire.Hosting.EntityFrameworkCore`, `Aspire.Hosting.Foundry`, and `Aspire.Hosting`, generating actual SDK bindings. - `docker run --rm --entrypoint bash -v "${PWD}:/workspace" aspire-go-e2e-6972:latest -c 'set -e; for fixture in Aspire.Hosting.EntityFrameworkCore Aspire.Hosting.Foundry Aspire.Hosting; do cd "/workspace/tests/PolyglotAppHosts/$fixture/Go"; gofmt -w apphost.go; go build -buildvcs=false -o /tmp/fixture ./...; go vet ./...; done'` - passed for all three fixtures. - `docker run --rm --entrypoint bash -v "${PWD}:/workspace:ro" aspire-go-e2e-6972:latest -c 'set -e; bash -n /workspace/.github/workflows/polyglot-validation/test-go.sh /workspace/.github/workflows/polyglot-validation/test-go-playground.sh /workspace/eng/scripts/stabilization-smoke-init-restore.sh; work=$(mktemp -d); trap "rm -rf $work" EXIT; cp /workspace/src/Aspire.Cli/Templating/Templates/go-starter/api/* "$work/"; cd "$work"; sed -i "s/{{projectNameLower}}/starter/g" go.mod; go build -mod=readonly -o /tmp/go-ready-api .; go mod tidy -diff; go vet ./...'` - passed without a dependency-repair command. - Native Linux `./restore.sh -p:SkipTestProjects=true -p:SkipPlaygroundProjects=true` - passed with the pinned SDK. - Native Linux `./dotnet.sh pack src/Aspire.Hosting.AppHost/Aspire.Hosting.AppHost.csproj -c Release -p:StabilizePackageVersion=true -p:SkipNativeBuild=true --nologo -v quiet` - failed as expected on existing NU5104 preview dependencies; not counted as a stabilization pass. - To isolate consumer behavior, the diagnostic feed used `for project in Aspire.Hosting Aspire.Hosting.AppHost Aspire.Hosting.Dotnet Aspire.TypeSystem Aspire.Hosting.CodeGeneration.Go Aspire.Hosting.CodeGeneration.Java Aspire.Hosting.Go Aspire.Hosting.Redis Aspire.Hosting.Java Aspire.Hosting.JavaScript Aspire.AppHost.Sdk Aspire.ProjectTemplates; do ./dotnet.sh pack "src/$project/$project.csproj" -c Release -p:StabilizePackageVersion=true -p:SkipNativeBuild=true -p:WarningsNotAsErrors=NU5104 --nologo -v quiet; done` - passed. No suppression is committed; Java remains `17.0.0-dev` by existing policy. - Native Linux `STABILIZATION_SMOKE_CONFIGURATION=Release ASPIRE_REPO_ROOT=/work/source bash eng/scripts/stabilization-smoke-init-restore.sh` - built the genuine stabilized Release driver/payload and completed the consumer flow. **Exit 1 is the expected Java policy failure, not an all-language pass.** Go/Redis `17.0.0` metadata names `/work/source/artifacts/packages/Release/Shipping`; C# `17.0.0` restored; Java reached real NuGet restore and rejected its absent stable pin with no nuget.org fallback. Output assertions confirmed the final failure list is exactly `java`. Sources were `1ab3ad1e30` plus the final smoke correction; no mixed PR-stamped CLI was used. - `bash -n eng/scripts/stabilization-smoke-init-restore.sh`, PowerShell parser validation of `eng\scripts\cli-starter-validation.ps1`, and `git diff --check` - passed. - Full `Infrastructure.Tests.TestTriggerMap` namespace was not passed on this Windows host: WSL's `bash.exe` cannot find `/bin/bash`; a narrowed run also encountered readonly `.git` cleanup failures. Not counted as successful validation. ## Checklist - Is this feature complete? - [ ] Yes. Ready to ship. - [x] No. Follow-up changes expected. - Are you including unit tests for the changes and scenario tests if relevant? - [x] Yes - [ ] No - Did you add public API? - [ ] Yes - If yes, did you have an API Review for it? - [ ] Yes - [ ] No - Did you add `<remarks />` and `<code />` elements on your triple slash comments? - [ ] Yes - [ ] No - [x] No - Does the change make any security assumptions or guarantees? - [ ] Yes - If yes, have you done a threat model and had a security review? - [ ] Yes - [ ] No - [x] No --------- Co-authored-by: IEvangelist <7679720+IEvangelist@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
## Description Go starters should build from their shipped files without an initial `aspire run` repairing dependencies, and Go preview validation should exercise the same runtime and native-runner paths as other AppHosts. - Ship a tidy API `go.mod` and generated `go.sum`. Add starter scenarios checking gitignore, readonly API build, tidy stability, health and weather responses; remove the Kubernetes manual-tidy workaround. - Honor the existing `--use-redis-cache false` option for Go. Omitted/true retains Redis; false omits the integration and resource. Cover all three scaffold choices with generated-source snapshots and both HTTP lifecycle variants with Docker E2E. - Add `go vet` to Go polyglot fixtures, an EF Core fixture, and Foundry tool/AKS node-pool parity. Exercise configuration-section list getters in the real Go runtime smoke. - Add Go to all six native starter-validation platforms, with official checksum-verified Go installation, timeout budget, and runtime-only selector edges for Go hosting, Go codegen, and Redis. Linux retains API/Redis readiness; containerless Windows/macOS use the cache-free starter with API readiness unchanged. - Add isolated-local-feed Go/Java scaffolding and restore to the stabilization smoke without rewriting template pins. Use a genuine Release bundled driver outside repository mode, cold consumer caches, and exact hosting-package extraction/provenance checks. Enable Java only in its subprocess. - Select languages by name rather than picker position. Restore stale Java/TypeScript SDK baselines after `aspire add` so the following `aspire restore` genuinely tests regeneration. Establish the TypeScript baseline with an actual restore before reading cache metadata. - Wait for the TypeScript Kubernetes application's actual Ready pod; exercise the Go Helm namespace/chart-version callback. Relates to #21202, #21206, #21209, #21210, #21201, and #21108. ### User-facing usage ```bash # Default retains Redis. aspire new aspire-go-starter --name GoReady --non-interactive --suppress-agent-init # Containerless starter: the Go API runs without Redis or a Docker daemon. aspire new aspire-go-starter --name GoReady --use-redis-cache false --non-interactive --suppress-agent-init ``` Both variants emit `api/go.sum` and an already-tidy API module. Its API can build with `go build -mod=readonly` without first running the AppHost's tidy resource. ### Dependencies and remaining acceptance This PR is independent of #21823. Root-module brownfield E2E #21825 merged into the #21823 feature branch, not directly into main; its delivery follows #21823. #21806 independently covers other Go init/build/vet/stable-restore scenarios. The existing stabilization gate remains disabled (`STABILIZATION_ENABLED: 'false'`); enabling it needs a separate CI/policy decision. Current `Aspire.Hosting.Java.csproj` still sets `SuppressFinalPackageVersion=true` while the Java starter pins the requested Aspire version. The corrected smoke reaches actual NuGet restore and exposes that mismatch: `Aspire.Hosting.Java (= 17.0.0)` is unavailable, the nearest local package is `17.0.0-dev`, and nuget.org versions are not considered. No pin substitution or Java release-policy change is included. Native Linux validation built the genuine stabilized Release CLI/payload and proved cold Go/Redis package restore and C# restore. This is **not a clean stabilization pass**: the unsuppressed stable pack first reproduced NU5104 on existing .NET 11 preview dependencies. The diagnostic consumer feed downgraded only NU5104 with `WarningsNotAsErrors=NU5104`; that exception is not committed. The final smoke correctly exits 1 with only Java in its failure list. The bounded shared-image fix is separate in #21849; this PR does not modify image Dockerfiles or the image-build implementation. Its `b8274fc8e0` image job and all four uploads passed, Infrastructure passed 2090/2090 including all 13 new image cases, and all 116 CLI archive E2E jobs passed. Earlier Windows extension fake-timer and DCP Docker-startup failures were investigated with same-source passing main controls; the morning live refresh reports its Final Results passing. It remains independently review-gated. Do not infer actual timeout/fallback execution from a healthy-mirror image pass. No action SHA is added or changed. No official Azure DevOps pipeline is modified. Parent's Go-minimum-version map rule/theory and the other owner's extension-e2e map block are untouched. The cache option uses existing embedded CLI inputs; no additional selector rule is required. The stabilization script remains advisory/disabled rather than being presented as a regular-PR gate. This does not close #21108: live Azure Container Apps deployment and real missing/old-toolchain E2E remain uncovered here. Cross-language lifecycle redesigns are deferred. ### CI failures diagnosed and fixed Original-head run [42116644663](https://github.com/microsoft/aspire/actions/runs/42116644663) exposed two causes: - Windows x64/ARM64 and macOS x64/ARM64 starter validation failed because Redis could not start: the Windows Docker engine pipe was absent and macOS had no Docker executable. The API correctly waited on its unhealthy dependency. Those native runners now use the supported cache-free variant; Linux and Docker E2E retain Redis coverage. No readiness assertion was removed. - `TypeScriptCodegenValidationTests` read `.codegen-hash` immediately after init, which supplies SDK sources but not restore-cache metadata. An explicit baseline restore now precedes the snapshot and stale-cache assertions. **Head `1ab3ad1e30`: all six native starter lanes passed**, including both Redis-enabled Ubuntu lanes and all four formerly failing Windows/macOS lanes, in [run 42151619597](https://github.com/microsoft/aspire/actions/runs/42151619597). Packaged CI subsequently passed [GoStarterReadinessTests](https://github.com/microsoft/aspire/actions/runs/42151619597/job/127721672515) (both cache choices), [GoStarterTemplateTests](https://github.com/microsoft/aspire/actions/runs/42151619597/job/127721672968), [JavaCodegenValidationTests](https://github.com/microsoft/aspire/actions/runs/42151619597/job/127721672216), and [TypeScriptCodegenValidationTests](https://github.com/microsoft/aspire/actions/runs/42151619597/job/127721664876). All originally failing targeted lanes/classes therefore have passing coverage. The shared image job first timed out at 45 minutes twice. Attempt 2 fetched 108 MB from Azure Ubuntu in 41 minutes 29 seconds (43.4 kB/s), versus a following GitHub download at 80.1 MB/s; GHA cache import also returned HTTP 400 service-unavailable responses. After #21849's same-input image control passed with all four uploads, one evidence-based product revalidation was started. [Attempt 3's image job passed](https://github.com/microsoft/aspire/actions/runs/42151619597/job/127718680817) and admitted packaged E2E. No blind retries were made. The successful infrastructure control used a healthy primary mirror; it does not demonstrate timeout/fallback execution in that CI run. **Head `bd30794b9e`: historical failures and final outcome are distinct.** In [run 42167868738](https://github.com/microsoft/aspire/actions/runs/42167868738), all six native starter lanes and Go/Java/TypeScript SDK validation passed. The first [shared image job](https://github.com/microsoft/aspire/actions/runs/42167868738/job/127729647462) was canceled after 45 minutes 22 seconds with Azure Ubuntu apt downloads still running and all four uploads skipped; the initial Linux tree-actions attempt also failed on a stale immutable Running snapshot. The morning live refresh subsequently shows [Final Results passing](https://github.com/microsoft/aspire/actions/runs/42167868738/job/127754717945), including [GoStarterReadinessTests](https://github.com/microsoft/aspire/actions/runs/42167868738/job/127747615122). This session did not initiate retries. A passing later attempt does not repair the tree-actions race: its separate test-only fix #21854 still needs independent approval/merge and incorporation before claiming the race fixed in this checkout. **Latest head `0bb6569d91` requires fresh checks**, now running in [42222384609](https://github.com/microsoft/aspire/actions/runs/42222384609). Both new review findings are corrected: SDK selection accepts only exact numeric MAJOR.MINOR.PATCH package filenames, and Go starter startup is inside its existing cleanup scope. Both threads were replied to and resolved with evidence. Earlier-head CI is not presented as latest-head proof. Per the user's explicit authorization, the coordinator enabled normal native **squash auto-merge** at 05:22:46Z on 2026-11-06. It remains enabled behind required fresh checks and review requirements. No admin bypass is authorized or used; the PR is not merged. ### Stabilization review corrections The smoke now publishes a Release CLI with its real stabilized bundle outside the checkout, propagates stabilization into child payload builds, and clears inherited `ASPIRE_REPO_ROOT` for every consumer invocation. Consumer packages are isolated after the driver build so build-time extraction cannot hide a missing stable package. Successful guest restores must produce metadata for the exact hosting pins, including Redis for Go; provenance is retained in the output. The real run also exposed two stale smoke assumptions: a pre-owned parent feed mapping conflicts with scaffolding's project-local channel policy, and current C# file-based templates use `#:package Aspire.Hosting.AppHost@<version>`, not the obsolete SDK directive. The smoke now lets the CLI establish its own authoritative project-scoped feed mapping and checks the current hosting-package contract. No customer NuGet policy is changed. ### Validation - `.\restore.cmd` - passed. - `dotnet build tests\Aspire.Cli.EndToEnd.Tests\Aspire.Cli.EndToEnd.Tests.csproj --no-restore --nologo -v quiet` - passed, zero warnings/errors; repeated successfully after removing temporary fault injection for `0bb6569d91`. - `dotnet test --project tests\Aspire.Cli.Tests\Aspire.Cli.Tests.csproj --no-launch-profile -- --filter-class '*.NewCommandTemplateConfigPersistenceTests' --filter-not-trait 'quarantined=true' --filter-not-trait 'outerloop=true'` - passed, 59 tests including default/true/false Go package and generated-source assertions. Existing ILCompiler NU1903 advisory was emitted. - With `ASPIRE_E2E_CLI_ARCHIVE_DIR=artifacts\readiness-ci\cli-archive-new` and `ASPIRE_E2E_DOTNET_IMAGE=aspire-e2e-polyglot-base:latest`, `dotnet test --project tests\Aspire.Cli.EndToEnd.Tests\Aspire.Cli.EndToEnd.Tests.csproj --no-launch-profile -- --filter-method '*.RestoreGeneratesSdkFiles_WithConfiguredToolchain' --filter-not-trait 'quarantined=true' --filter-not-trait 'outerloop=true'` - passed, all four npm/bun/yarn/pnpm cases against exact `1ab3ad1e30` Linux CLI/packages. The corresponding packaged CI class also passed above. - With `ASPIRE_E2E_CLI_ARCHIVE_DIR=artifacts\readiness-ci\cli-archive-new` and `ASPIRE_E2E_POLYGLOT_GO_IMAGE=aspire-go-e2e-6972:latest`, `dotnet test --project tests\Aspire.Cli.EndToEnd.Tests\Aspire.Cli.EndToEnd.Tests.csproj --no-launch-profile -- --filter-method '*.StarterShipsTidyApiAndServesRequests' --filter-not-trait 'quarantined=true' --filter-not-trait 'outerloop=true'` - 1 passed/1 failed locally against exact `1ab3ad1e30`. Cache-free passed actual scaffolding, readonly API build, tidy check, API-up, health/weather and stop without a Docker socket. Redis timed out locally. The unchanged `2859e199e2` control reproduced the same timeout on this Docker Desktop host; no workaround or weaker assertion was added. **Both cases subsequently passed packaged CI**, linked above. After the cleanup-scope correction, this local command again passed cache-free and failed Redis on API readiness; the independent CLI stop log confirmed successful shutdown, while the shared terminal helper observed the prior ERR prompt. That local run is not claimed green. - Temporary fault-injection validation used that same E2E command with only the cache-free case and a shell wrapper returning HTTP 503 for the Dashboard probe after a real packaged AppHost launch - intentionally failed. Terminal output showed `dashboard-http-failed`, then cleanup invocation; the CLI log confirmed `apphost.go stopped successfully` and exit 0. The wrapper and temporary test edit were removed before the final clean build/commit. This verifies partial-start cleanup, not a successful readiness scenario. - `docker run --rm --entrypoint bash -v "${PWD}:/workspace:ro" aspire-go-e2e-6972:latest -c "sed 's/\r$//' /workspace/artifacts/readiness-ci/validate-stable-sdk-selection.sh | bash"` - passed actual smoke-script preflight cases for missing SDK, prerelease-only, mixed stable/dev/preview/rc/build-metadata/four-component filenames, multi-digit stable versions, and feed paths with spaces. The instrumented driver received `BundleVersion=17.0.0` from the mixed feed. Invalid-only feeds exited 1 with the explicit missing-stable diagnostic. Temporary harness removed after retaining logs; not a new all-language stabilization pass. - `dotnet test --project tests\Infrastructure.Tests\Infrastructure.Tests.csproj --no-launch-profile -- --filter-class '*.TestTriggerMapTests' --filter-not-trait 'quarantined=true' --filter-not-trait 'outerloop=true'` - passed, 176 tests. - `dotnet run --project tools\SelectTests -- --changed-files artifacts\readiness-changed-files.txt --explain` - passed against the real graph/map; Go hosting/codegen, Redis, starter checksum and EF Core fixture inputs select starter-validation/polyglot with zero unattributed files. Schedule-only deployment remains advisory. - `$env:DOTNET_ROOT = Join-Path $PWD '.dotnet'; $env:ASPIRE_REPO_ROOT = $PWD; .\artifacts\bin\Aspire.Cli\Debug\net11.0\aspire.exe restore --apphost tests\PolyglotAppHosts\<integration>\Go\apphost.go --non-interactive` - passed separately for `Aspire.Hosting.EntityFrameworkCore`, `Aspire.Hosting.Foundry`, and `Aspire.Hosting`, generating actual SDK bindings. - `docker run --rm --entrypoint bash -v "${PWD}:/workspace" aspire-go-e2e-6972:latest -c 'set -e; for fixture in Aspire.Hosting.EntityFrameworkCore Aspire.Hosting.Foundry Aspire.Hosting; do cd "/workspace/tests/PolyglotAppHosts/$fixture/Go"; gofmt -w apphost.go; go build -buildvcs=false -o /tmp/fixture ./...; go vet ./...; done'` - passed for all three fixtures. - `docker run --rm --entrypoint bash -v "${PWD}:/workspace:ro" aspire-go-e2e-6972:latest -c 'set -e; bash -n /workspace/.github/workflows/polyglot-validation/test-go.sh /workspace/.github/workflows/polyglot-validation/test-go-playground.sh /workspace/eng/scripts/stabilization-smoke-init-restore.sh; work=$(mktemp -d); trap "rm -rf $work" EXIT; cp /workspace/src/Aspire.Cli/Templating/Templates/go-starter/api/* "$work/"; cd "$work"; sed -i "s/{{projectNameLower}}/starter/g" go.mod; go build -mod=readonly -o /tmp/go-ready-api .; go mod tidy -diff; go vet ./...'` - passed without a dependency-repair command. - Native Linux `./restore.sh -p:SkipTestProjects=true -p:SkipPlaygroundProjects=true` - passed with the pinned SDK. - Native Linux `./dotnet.sh pack src/Aspire.Hosting.AppHost/Aspire.Hosting.AppHost.csproj -c Release -p:StabilizePackageVersion=true -p:SkipNativeBuild=true --nologo -v quiet` - failed as expected on existing NU5104 preview dependencies; not counted as a stabilization pass. - To isolate consumer behavior, the diagnostic feed used `for project in Aspire.Hosting Aspire.Hosting.AppHost Aspire.Hosting.Dotnet Aspire.TypeSystem Aspire.Hosting.CodeGeneration.Go Aspire.Hosting.CodeGeneration.Java Aspire.Hosting.Go Aspire.Hosting.Redis Aspire.Hosting.Java Aspire.Hosting.JavaScript Aspire.AppHost.Sdk Aspire.ProjectTemplates; do ./dotnet.sh pack "src/$project/$project.csproj" -c Release -p:StabilizePackageVersion=true -p:SkipNativeBuild=true -p:WarningsNotAsErrors=NU5104 --nologo -v quiet; done` - passed. No suppression is committed; Java remains `17.0.0-dev` by existing policy. - Native Linux `STABILIZATION_SMOKE_CONFIGURATION=Release ASPIRE_REPO_ROOT=/work/source bash eng/scripts/stabilization-smoke-init-restore.sh` - built the genuine stabilized Release driver/payload and completed the consumer flow. **Exit 1 is the expected Java policy failure, not an all-language pass.** Go/Redis `17.0.0` metadata names `/work/source/artifacts/packages/Release/Shipping`; C# `17.0.0` restored; Java reached real NuGet restore and rejected its absent stable pin with no nuget.org fallback. Output assertions confirmed the final failure list is exactly `java`. Sources were `1ab3ad1e30` plus the final smoke correction; no mixed PR-stamped CLI was used. - `bash -n eng/scripts/stabilization-smoke-init-restore.sh`, PowerShell parser validation of `eng\scripts\cli-starter-validation.ps1`, and `git diff --check` - passed. - Full `Infrastructure.Tests.TestTriggerMap` namespace was not passed on this Windows host: WSL's `bash.exe` cannot find `/bin/bash`; a narrowed run also encountered readonly `.git` cleanup failures. Not counted as successful validation. ## Checklist - Is this feature complete? - [ ] Yes. Ready to ship. - [x] No. Follow-up changes expected. - Are you including unit tests for the changes and scenario tests if relevant? - [x] Yes - [ ] No - Did you add public API? - [ ] Yes - If yes, did you have an API Review for it? - [ ] Yes - [ ] No - Did you add `<remarks />` and `<code />` elements on your triple slash comments? - [ ] Yes - [ ] No - [x] No - Does the change make any security assumptions or guarantees? - [ ] Yes - If yes, have you done a threat model and had a security review? - [ ] Yes - [ ] No - [x] No --------- Co-authored-by: IEvangelist <7679720+IEvangelist@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 5138b2a)
## Description Go starters should build from their shipped files without an initial `aspire run` repairing dependencies, and Go preview validation should exercise the same runtime and native-runner paths as other AppHosts. - Ship a tidy API `go.mod` and generated `go.sum`. Add starter scenarios checking gitignore, readonly API build, tidy stability, health and weather responses; remove the Kubernetes manual-tidy workaround. - Honor the existing `--use-redis-cache false` option for Go. Omitted/true retains Redis; false omits the integration and resource. Cover all three scaffold choices with generated-source snapshots and both HTTP lifecycle variants with Docker E2E. - Add `go vet` to Go polyglot fixtures, an EF Core fixture, and Foundry tool/AKS node-pool parity. Exercise configuration-section list getters in the real Go runtime smoke. - Add Go to all six native starter-validation platforms, with official checksum-verified Go installation, timeout budget, and runtime-only selector edges for Go hosting, Go codegen, and Redis. Linux retains API/Redis readiness; containerless Windows/macOS use the cache-free starter with API readiness unchanged. - Add isolated-local-feed Go/Java scaffolding and restore to the stabilization smoke without rewriting template pins. Use a genuine Release bundled driver outside repository mode, cold consumer caches, and exact hosting-package extraction/provenance checks. Enable Java only in its subprocess. - Select languages by name rather than picker position. Restore stale Java/TypeScript SDK baselines after `aspire add` so the following `aspire restore` genuinely tests regeneration. Establish the TypeScript baseline with an actual restore before reading cache metadata. - Wait for the TypeScript Kubernetes application's actual Ready pod; exercise the Go Helm namespace/chart-version callback. Relates to #21202, #21206, #21209, #21210, #21201, and #21108. ### User-facing usage ```bash # Default retains Redis. aspire new aspire-go-starter --name GoReady --non-interactive --suppress-agent-init # Containerless starter: the Go API runs without Redis or a Docker daemon. aspire new aspire-go-starter --name GoReady --use-redis-cache false --non-interactive --suppress-agent-init ``` Both variants emit `api/go.sum` and an already-tidy API module. Its API can build with `go build -mod=readonly` without first running the AppHost's tidy resource. ### Dependencies and remaining acceptance This PR is independent of #21823. Root-module brownfield E2E #21825 merged into the #21823 feature branch, not directly into main; its delivery follows #21823. #21806 independently covers other Go init/build/vet/stable-restore scenarios. The existing stabilization gate remains disabled (`STABILIZATION_ENABLED: 'false'`); enabling it needs a separate CI/policy decision. Current `Aspire.Hosting.Java.csproj` still sets `SuppressFinalPackageVersion=true` while the Java starter pins the requested Aspire version. The corrected smoke reaches actual NuGet restore and exposes that mismatch: `Aspire.Hosting.Java (= 17.0.0)` is unavailable, the nearest local package is `17.0.0-dev`, and nuget.org versions are not considered. No pin substitution or Java release-policy change is included. Native Linux validation built the genuine stabilized Release CLI/payload and proved cold Go/Redis package restore and C# restore. This is **not a clean stabilization pass**: the unsuppressed stable pack first reproduced NU5104 on existing .NET 11 preview dependencies. The diagnostic consumer feed downgraded only NU5104 with `WarningsNotAsErrors=NU5104`; that exception is not committed. The final smoke correctly exits 1 with only Java in its failure list. The bounded shared-image fix is separate in #21849; this PR does not modify image Dockerfiles or the image-build implementation. Its `b8274fc8e0` image job and all four uploads passed, Infrastructure passed 2090/2090 including all 13 new image cases, and all 116 CLI archive E2E jobs passed. Earlier Windows extension fake-timer and DCP Docker-startup failures were investigated with same-source passing main controls; the morning live refresh reports its Final Results passing. It remains independently review-gated. Do not infer actual timeout/fallback execution from a healthy-mirror image pass. No action SHA is added or changed. No official Azure DevOps pipeline is modified. Parent's Go-minimum-version map rule/theory and the other owner's extension-e2e map block are untouched. The cache option uses existing embedded CLI inputs; no additional selector rule is required. The stabilization script remains advisory/disabled rather than being presented as a regular-PR gate. This does not close #21108: live Azure Container Apps deployment and real missing/old-toolchain E2E remain uncovered here. Cross-language lifecycle redesigns are deferred. ### CI failures diagnosed and fixed Original-head run [42116644663](https://github.com/microsoft/aspire/actions/runs/42116644663) exposed two causes: - Windows x64/ARM64 and macOS x64/ARM64 starter validation failed because Redis could not start: the Windows Docker engine pipe was absent and macOS had no Docker executable. The API correctly waited on its unhealthy dependency. Those native runners now use the supported cache-free variant; Linux and Docker E2E retain Redis coverage. No readiness assertion was removed. - `TypeScriptCodegenValidationTests` read `.codegen-hash` immediately after init, which supplies SDK sources but not restore-cache metadata. An explicit baseline restore now precedes the snapshot and stale-cache assertions. **Head `1ab3ad1e30`: all six native starter lanes passed**, including both Redis-enabled Ubuntu lanes and all four formerly failing Windows/macOS lanes, in [run 42151619597](https://github.com/microsoft/aspire/actions/runs/42151619597). Packaged CI subsequently passed [GoStarterReadinessTests](https://github.com/microsoft/aspire/actions/runs/42151619597/job/127721672515) (both cache choices), [GoStarterTemplateTests](https://github.com/microsoft/aspire/actions/runs/42151619597/job/127721672968), [JavaCodegenValidationTests](https://github.com/microsoft/aspire/actions/runs/42151619597/job/127721672216), and [TypeScriptCodegenValidationTests](https://github.com/microsoft/aspire/actions/runs/42151619597/job/127721664876). All originally failing targeted lanes/classes therefore have passing coverage. The shared image job first timed out at 45 minutes twice. Attempt 2 fetched 108 MB from Azure Ubuntu in 41 minutes 29 seconds (43.4 kB/s), versus a following GitHub download at 80.1 MB/s; GHA cache import also returned HTTP 400 service-unavailable responses. After #21849's same-input image control passed with all four uploads, one evidence-based product revalidation was started. [Attempt 3's image job passed](https://github.com/microsoft/aspire/actions/runs/42151619597/job/127718680817) and admitted packaged E2E. No blind retries were made. The successful infrastructure control used a healthy primary mirror; it does not demonstrate timeout/fallback execution in that CI run. **Head `bd30794b9e`: historical failures and final outcome are distinct.** In [run 42167868738](https://github.com/microsoft/aspire/actions/runs/42167868738), all six native starter lanes and Go/Java/TypeScript SDK validation passed. The first [shared image job](https://github.com/microsoft/aspire/actions/runs/42167868738/job/127729647462) was canceled after 45 minutes 22 seconds with Azure Ubuntu apt downloads still running and all four uploads skipped; the initial Linux tree-actions attempt also failed on a stale immutable Running snapshot. The morning live refresh subsequently shows [Final Results passing](https://github.com/microsoft/aspire/actions/runs/42167868738/job/127754717945), including [GoStarterReadinessTests](https://github.com/microsoft/aspire/actions/runs/42167868738/job/127747615122). This session did not initiate retries. A passing later attempt does not repair the tree-actions race: its separate test-only fix #21854 still needs independent approval/merge and incorporation before claiming the race fixed in this checkout. **Latest head `0bb6569d91` requires fresh checks**, now running in [42222384609](https://github.com/microsoft/aspire/actions/runs/42222384609). Both new review findings are corrected: SDK selection accepts only exact numeric MAJOR.MINOR.PATCH package filenames, and Go starter startup is inside its existing cleanup scope. Both threads were replied to and resolved with evidence. Earlier-head CI is not presented as latest-head proof. Per the user's explicit authorization, the coordinator enabled normal native **squash auto-merge** at 05:22:46Z on 2026-11-06. It remains enabled behind required fresh checks and review requirements. No admin bypass is authorized or used; the PR is not merged. ### Stabilization review corrections The smoke now publishes a Release CLI with its real stabilized bundle outside the checkout, propagates stabilization into child payload builds, and clears inherited `ASPIRE_REPO_ROOT` for every consumer invocation. Consumer packages are isolated after the driver build so build-time extraction cannot hide a missing stable package. Successful guest restores must produce metadata for the exact hosting pins, including Redis for Go; provenance is retained in the output. The real run also exposed two stale smoke assumptions: a pre-owned parent feed mapping conflicts with scaffolding's project-local channel policy, and current C# file-based templates use `#:package Aspire.Hosting.AppHost@<version>`, not the obsolete SDK directive. The smoke now lets the CLI establish its own authoritative project-scoped feed mapping and checks the current hosting-package contract. No customer NuGet policy is changed. ### Validation - `.\restore.cmd` - passed. - `dotnet build tests\Aspire.Cli.EndToEnd.Tests\Aspire.Cli.EndToEnd.Tests.csproj --no-restore --nologo -v quiet` - passed, zero warnings/errors; repeated successfully after removing temporary fault injection for `0bb6569d91`. - `dotnet test --project tests\Aspire.Cli.Tests\Aspire.Cli.Tests.csproj --no-launch-profile -- --filter-class '*.NewCommandTemplateConfigPersistenceTests' --filter-not-trait 'quarantined=true' --filter-not-trait 'outerloop=true'` - passed, 59 tests including default/true/false Go package and generated-source assertions. Existing ILCompiler NU1903 advisory was emitted. - With `ASPIRE_E2E_CLI_ARCHIVE_DIR=artifacts\readiness-ci\cli-archive-new` and `ASPIRE_E2E_DOTNET_IMAGE=aspire-e2e-polyglot-base:latest`, `dotnet test --project tests\Aspire.Cli.EndToEnd.Tests\Aspire.Cli.EndToEnd.Tests.csproj --no-launch-profile -- --filter-method '*.RestoreGeneratesSdkFiles_WithConfiguredToolchain' --filter-not-trait 'quarantined=true' --filter-not-trait 'outerloop=true'` - passed, all four npm/bun/yarn/pnpm cases against exact `1ab3ad1e30` Linux CLI/packages. The corresponding packaged CI class also passed above. - With `ASPIRE_E2E_CLI_ARCHIVE_DIR=artifacts\readiness-ci\cli-archive-new` and `ASPIRE_E2E_POLYGLOT_GO_IMAGE=aspire-go-e2e-6972:latest`, `dotnet test --project tests\Aspire.Cli.EndToEnd.Tests\Aspire.Cli.EndToEnd.Tests.csproj --no-launch-profile -- --filter-method '*.StarterShipsTidyApiAndServesRequests' --filter-not-trait 'quarantined=true' --filter-not-trait 'outerloop=true'` - 1 passed/1 failed locally against exact `1ab3ad1e30`. Cache-free passed actual scaffolding, readonly API build, tidy check, API-up, health/weather and stop without a Docker socket. Redis timed out locally. The unchanged `2859e199e2` control reproduced the same timeout on this Docker Desktop host; no workaround or weaker assertion was added. **Both cases subsequently passed packaged CI**, linked above. After the cleanup-scope correction, this local command again passed cache-free and failed Redis on API readiness; the independent CLI stop log confirmed successful shutdown, while the shared terminal helper observed the prior ERR prompt. That local run is not claimed green. - Temporary fault-injection validation used that same E2E command with only the cache-free case and a shell wrapper returning HTTP 503 for the Dashboard probe after a real packaged AppHost launch - intentionally failed. Terminal output showed `dashboard-http-failed`, then cleanup invocation; the CLI log confirmed `apphost.go stopped successfully` and exit 0. The wrapper and temporary test edit were removed before the final clean build/commit. This verifies partial-start cleanup, not a successful readiness scenario. - `docker run --rm --entrypoint bash -v "${PWD}:/workspace:ro" aspire-go-e2e-6972:latest -c "sed 's/\r$//' /workspace/artifacts/readiness-ci/validate-stable-sdk-selection.sh | bash"` - passed actual smoke-script preflight cases for missing SDK, prerelease-only, mixed stable/dev/preview/rc/build-metadata/four-component filenames, multi-digit stable versions, and feed paths with spaces. The instrumented driver received `BundleVersion=17.0.0` from the mixed feed. Invalid-only feeds exited 1 with the explicit missing-stable diagnostic. Temporary harness removed after retaining logs; not a new all-language stabilization pass. - `dotnet test --project tests\Infrastructure.Tests\Infrastructure.Tests.csproj --no-launch-profile -- --filter-class '*.TestTriggerMapTests' --filter-not-trait 'quarantined=true' --filter-not-trait 'outerloop=true'` - passed, 176 tests. - `dotnet run --project tools\SelectTests -- --changed-files artifacts\readiness-changed-files.txt --explain` - passed against the real graph/map; Go hosting/codegen, Redis, starter checksum and EF Core fixture inputs select starter-validation/polyglot with zero unattributed files. Schedule-only deployment remains advisory. - `$env:DOTNET_ROOT = Join-Path $PWD '.dotnet'; $env:ASPIRE_REPO_ROOT = $PWD; .\artifacts\bin\Aspire.Cli\Debug\net11.0\aspire.exe restore --apphost tests\PolyglotAppHosts\<integration>\Go\apphost.go --non-interactive` - passed separately for `Aspire.Hosting.EntityFrameworkCore`, `Aspire.Hosting.Foundry`, and `Aspire.Hosting`, generating actual SDK bindings. - `docker run --rm --entrypoint bash -v "${PWD}:/workspace" aspire-go-e2e-6972:latest -c 'set -e; for fixture in Aspire.Hosting.EntityFrameworkCore Aspire.Hosting.Foundry Aspire.Hosting; do cd "/workspace/tests/PolyglotAppHosts/$fixture/Go"; gofmt -w apphost.go; go build -buildvcs=false -o /tmp/fixture ./...; go vet ./...; done'` - passed for all three fixtures. - `docker run --rm --entrypoint bash -v "${PWD}:/workspace:ro" aspire-go-e2e-6972:latest -c 'set -e; bash -n /workspace/.github/workflows/polyglot-validation/test-go.sh /workspace/.github/workflows/polyglot-validation/test-go-playground.sh /workspace/eng/scripts/stabilization-smoke-init-restore.sh; work=$(mktemp -d); trap "rm -rf $work" EXIT; cp /workspace/src/Aspire.Cli/Templating/Templates/go-starter/api/* "$work/"; cd "$work"; sed -i "s/{{projectNameLower}}/starter/g" go.mod; go build -mod=readonly -o /tmp/go-ready-api .; go mod tidy -diff; go vet ./...'` - passed without a dependency-repair command. - Native Linux `./restore.sh -p:SkipTestProjects=true -p:SkipPlaygroundProjects=true` - passed with the pinned SDK. - Native Linux `./dotnet.sh pack src/Aspire.Hosting.AppHost/Aspire.Hosting.AppHost.csproj -c Release -p:StabilizePackageVersion=true -p:SkipNativeBuild=true --nologo -v quiet` - failed as expected on existing NU5104 preview dependencies; not counted as a stabilization pass. - To isolate consumer behavior, the diagnostic feed used `for project in Aspire.Hosting Aspire.Hosting.AppHost Aspire.Hosting.Dotnet Aspire.TypeSystem Aspire.Hosting.CodeGeneration.Go Aspire.Hosting.CodeGeneration.Java Aspire.Hosting.Go Aspire.Hosting.Redis Aspire.Hosting.Java Aspire.Hosting.JavaScript Aspire.AppHost.Sdk Aspire.ProjectTemplates; do ./dotnet.sh pack "src/$project/$project.csproj" -c Release -p:StabilizePackageVersion=true -p:SkipNativeBuild=true -p:WarningsNotAsErrors=NU5104 --nologo -v quiet; done` - passed. No suppression is committed; Java remains `17.0.0-dev` by existing policy. - Native Linux `STABILIZATION_SMOKE_CONFIGURATION=Release ASPIRE_REPO_ROOT=/work/source bash eng/scripts/stabilization-smoke-init-restore.sh` - built the genuine stabilized Release driver/payload and completed the consumer flow. **Exit 1 is the expected Java policy failure, not an all-language pass.** Go/Redis `17.0.0` metadata names `/work/source/artifacts/packages/Release/Shipping`; C# `17.0.0` restored; Java reached real NuGet restore and rejected its absent stable pin with no nuget.org fallback. Output assertions confirmed the final failure list is exactly `java`. Sources were `1ab3ad1e30` plus the final smoke correction; no mixed PR-stamped CLI was used. - `bash -n eng/scripts/stabilization-smoke-init-restore.sh`, PowerShell parser validation of `eng\scripts\cli-starter-validation.ps1`, and `git diff --check` - passed. - Full `Infrastructure.Tests.TestTriggerMap` namespace was not passed on this Windows host: WSL's `bash.exe` cannot find `/bin/bash`; a narrowed run also encountered readonly `.git` cleanup failures. Not counted as successful validation. ## Checklist - Is this feature complete? - [ ] Yes. Ready to ship. - [x] No. Follow-up changes expected. - Are you including unit tests for the changes and scenario tests if relevant? - [x] Yes - [ ] No - Did you add public API? - [ ] Yes - If yes, did you have an API Review for it? - [ ] Yes - [ ] No - Did you add `<remarks />` and `<code />` elements on your triple slash comments? - [ ] Yes - [ ] No - [x] No - Does the change make any security assumptions or guarantees? - [ ] Yes - If yes, have you done a threat model and had a security review? - [ ] Yes - [ ] No - [x] No --------- Co-authored-by: IEvangelist <7679720+IEvangelist@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 5138b2a)
| clearTimeout(grace); | ||
| const clientExitCode = code ?? (signal ? 128 + constants.signals[signal] : 1); | ||
| // An external kill/OOM during cleanup is not acknowledged cancellation. | ||
| process.exit(forcedKill || clientExitCode === 137 ? 137 : cancellationExitCode || (deadlineReached ? 124 : clientExitCode)); |
There was a problem hiding this comment.
[automated] [P2] Fail closed when the Buildx plugin is killed during cleanup
The supervised child is Docker, not Buildx itself. With Docker 28.0.4 (the version in this PR's CI), a SIGKILLed plugin makes Docker exit 255, not 137. Once the deadline has fired, this expression converts that to 124, so build_with_mirror_retry starts fallback and can return success without confirmed graceful cancellation.
I reproduced this through the production helper and supervisor using real Docker plus a fixture Buildx plugin that kills itself on deadline TERM: PLUGIN_KILLED_DURING_CLEANUP -> FALLBACK_RAN -> helper exit 0. This demonstrates the exit-classification bug, not actual surviving BuildKit work.
Handle the Docker/plugin termination path before converting cleanup exits to timeout, and cover that process boundary; the current direct-client SIGKILL regression only exercises exit 137.


Description
Shared CLI E2E image builds can exhaust their 45-minute job downloading from a throttled Azure Ubuntu apt mirror without reaching the existing error-only fallback. Bound primary attempts so default-source fallback also runs for slow or stalled builds.
Independent fix based on
main; relates to #21828, which is unchanged and is not a dependency. No auto-merge or bypass requested.Two-attempt root-cause evidence
Both jobs ran at #21828 head
1ab3ad1e302a110bdb4f69ad4e3ee2e05d646d9ain run 42151619597. Both have annotationThe job has exceeded the maximum execution time of 45m0s.#11 2272.6 Fetched 82.4 MB in 37min 49s (36.3 kB/s)#11 2492.6 Fetched 108 MB in 41min 29s (43.4 kB/s)Attempt 2 then downloaded from HTTPS GitHub at
#11 2497.2 Fetched 15.1 MB in 0s (80.1 MB/s), while another mirror download logged#15 92.37 Fetched 4031 kB in 1min 29s (45.4 kB/s). Both also reported GHA cache-import HTTP 400,Our services aren't available right now. Cold apt downloads, not product tests, consumed the image job. Neither attempt reached product-test execution using these images. No third rerun was triggered.Local validation
.\restore.cmd: passed, 0 errors and 18 dependency-advisory warnings.Original selector coverage passed across appropriate hosts: 299 Windows managed cases, 2 Windows native MSBuild graph cases and 79 Linux shell/Git cases:
Linux container, pinned .NET runtime, same built MTP assembly:
The initial Windows run failed in 17 shell cases because it selected the broken System32 WSL bash shim, and 5 existing Git-fixture cleanup cases with
UnauthorizedAccessException. The same cases passed on Linux. The 2 native graph cases passed on Windows; running the Windows SDK on Linux causedBadImageFormatException. Passing partitions cover every focused case without modifying unrelated tests.The 13 image cases cover success, ordinary failure, fallback failure, slow success, stall, forced kill, client cancellation, external cancellation before/during deadline cleanup, client SIGKILL during cleanup, all-image/.NET-only orchestration, and exact original apt-source restoration in real Ubuntu. Cleanup cancellation reproduces on 4bb054c as incorrect exit 0 plus fallback; the corrected supervisor exits 143 without fallback. Killed clients retain 137, never deadline 124.
Real Docker/Buildx probe re-run after the supervisor change:
build_with_mirror_retry 20s "real BuildKit cancellation probe" docker buildx build --progress plain --load -t mirror-fallback-probePassed: primary printed
PRIMARY_RUNNING, then#5 CANCELED/Canceled: context canceled; fallback printedFALLBACK_RUNNINGand loaded the image. BuildKit history recorded primaryErrorand fallbackCompleted. Process inspection before fallback and afterwards found only the BuildKit daemon, no surviving primaryRUN/sleep. Temporary builder and image removed.git diff --check: passed.Actual final-head CI
Head b8274fc, run 42170803717: image build and all four uploads passed; Infrastructure passed 2090/2090; all 116 selected CLI archive E2E jobs passed. The downloaded Infrastructure TRX confirms all 13 new image cases passed. CLI unit tests passed on Linux, macOS and Windows. This is actual final-head evidence, not evidence from superseded runs.
Image job 127739541935 ran 05:56:03–06:05:32 UTC. All four artifacts are present and unexpired:
The primary mirror was healthy:
#11 4.388 Fetched 108 MB in 1s (114 MB/s). No actual CI timeout/fallback occurred; fallback evidence is deterministic/local BuildKit evidence, not a production CI claim.Representative actual consumers confirm all four artifacts load and execute tests:
Required checks are not green; approval is still required. Two failures outside the changed image infrastructure remain, with same-source main controls rather than blind reruns:
rejects when a POSIX process group remains alive after forced terminationpassed its rejection/listener assertions but saw global fake-timer count 2 instead of 0. Test and process-helper blobs are identical to main 54b4929. Main control 127718292677 passed that exact case and all 4151 tests. The source of the two timers is not established; no extension edit or retry made.ExecutableDoesNotExistandExecutableWorkingDirectoryDoesNotExistfailed before scenario assertions because DCP'sdocker.exe --versiondependency probe timed out. Test file and entire DCP source tree are identical to main 54b4929. Main control 127718297752 TRX passed the same cases in 6.005s and 2.854s. No product edit or retry made.All four review threads have been addressed. No merge or bypass performed; #21828 remains untouched and cannot consume this fix until integration.
Security considerations
The supervisor executes existing checked-in Docker commands, not new untrusted input. Separate-session process-group signaling avoids signaling the workflow shell and permits child cleanup. A forced kill cannot confirm remote BuildKit cancellation, so it fails closed without fallback. No threat model or separate security review has been performed.
Fixes # (issue)
Checklist
<remarks />and<code />elements on your triple slash comments?