Repository navigation
Fix Go AppHost panic on callback options and accept Go numbers in unions - #21188
Conversation
Go Options structs are merged with deepUpdate before a capability call. deepUpdate converted both sides to maps, marshaled the result to JSON with the error discarded, and unmarshaled it back. encoding/json cannot marshal a func-typed field (even a nil one), so any options struct with a callback, such as AddManifestOptions.Configure, panicked with "aspire: merge error: unexpected end of JSON input". The same round trip silently dropped fields tagged json:"-", such as *CancellationToken. Merge by reflection instead. Struct fields and map entries present on both sides merge recursively, any other value comes from src, and structs with unexported fields (*CancellationToken, time.Time) are taken whole. Neither argument is modified. Add a Go codegen test that generates the SDK, writes five Go regression tests next to it and runs go test, and update the generated runtime snapshot. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c8717a87-0ee5-4296-a6fd-ea6e2d91a57e
Go boxes an untyped constant such as 3 as int, but the generated union guard
only listed the ATS types, so WithField("spec.maxReplicaCount", 3) failed at
runtime with 'parameter "value" must be one of [string, float64, bool], got
int'. When a union allows a number (float64), the guard now accepts the other
Go numeric types too; all of them serialize as JSON numbers. The error
message and the doc comment still list only the ATS types.
Add a Go-only test export with such a union and two Go tests in the generated
runtime test that call it with every numeric type and with unsupported types.
Update the two generated snapshots.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c8717a87-0ee5-4296-a6fd-ea6e2d91a57e
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 21188Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 21188" |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
🟢 Approval recommended
No blocking issues remain, and focused regression tests cover both fixes.
No open issues 🎉
What changed in this PR
Fixes Go AppHost callback-option panics from #18080 and expands numeric union support. Nested and dotted manifest fields remain out of scope.
Changes:
- Replaces JSON-based option merging with reflection to preserve callbacks and cancellation tokens.
- Accepts Go integer types and
float32in number unions. - Adds executable Go regression tests and updates generated snapshots.
| File | Description |
|---|---|
| tests/Aspire.Hosting.CodeGeneration.Go.Tests/Snapshots/TwoPassScanningGeneratedAspire.verified.go | Adds the scalar-union test export snapshot. |
| tests/Aspire.Hosting.CodeGeneration.Go.Tests/Snapshots/AtsGeneratedAspire.verified.go | Updates runtime merging and union snapshots. |
| tests/Aspire.Hosting.CodeGeneration.Go.Tests/GoTestExtensions.cs | Adds a Go-specific scalar-union fixture. |
| tests/Aspire.Hosting.CodeGeneration.Go.Tests/AtsGoCodeGeneratorTests.cs | Tests callback preservation, nested merging, and numeric guards. |
| src/Aspire.Hosting.CodeGeneration.Go/Resources/base.go | Implements reflection-based option merging. |
| src/Aspire.Hosting.CodeGeneration.Go/AtsGoCodeGenerator.cs | Expands accepted numeric union types. |
Review effort: Balanced
Sébastien Ros (sebastienros)
left a comment
There was a problem hiding this comment.
Thanks, this looks good to me. Replacing the JSON round trip in deepUpdate with a reflection merge fixes the panic on callback options and also stops dropping *CancellationToken. I exercised the new merge with nested maps, interface fields, opaque structs, nil slices and funcs, and a pointer cycle, and all of it behaves as described. The Go codegen tests pass locally (34/34) on macOS arm64.
A few minor points. Each can be a follow-up or a small update before merge:
- Additive options (unchanged behavior): a nil field in a later option still overwrites an earlier non-nil value, so
Foo(&Opts{A: x}, &Opts{B: y})sendsAas nil. You already called this out. If variadic options are meant to add up, it may deserve its own issue. - Merge order: as the description says, this overlaps #21194 (tests in
AtsGoCodeGeneratorTests.cs) and #21197 (union helpers next toGetUnionAllowedTypes, plus two snapshots). Whichever lands second will need a rebase anddotnet verify accept -y.
The union guard accepts all predeclared Go numeric types whenever float64 is an allowed member, but the generated godoc line still listed only the ATS member types, so readers could not tell that 3 or int64(3) is fine. Add that note to the 'Allowed types' godoc line and share the float64 predicate between the doc and the guard so the two cannot drift. The guard error text is unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Tests selector2 / 100 PR test projects · 2 PR jobs, from 6 changed files. Selected PR test projects (2 / 100)
Selected PR jobs (2)
How these were chosen — grouped by what changed🔧 🔧 🧪 🧪 🧪 🧪 Job reasons
Selection computed for commit |
## Description Adds CLI end-to-end coverage for Go AppHosts that mirrors what TypeScript has, plus a `PolyglotGo` image variant that carries the Go toolchain for those tests. **Tests** (`tests/Aspire.Cli.EndToEnd.Tests`, 7 classes, 10 tests): | Class | Mirrors | Covers | | --- | --- | --- | | `GoStarterTemplateTests` | `TypeScriptStarterTemplateTests` | `aspire new` Go starter, then start, wait and stop | | `GoEmptyAppHostTemplateTests` (2) | `TypeScriptEmptyAppHostTemplateTests` | empty Go AppHost from the template picker and from the direct template command | | `GoCodegenValidationTests` | `TypeScriptCodegenValidationTests` | restore generates the SDK files, and regenerates them when the SDK on disk is stale (the baseline is put back after an `aspire add`) | | `GoPolyglotApphostDirectoryTests` | `TypeScriptPolyglotApphostDirectoryTests` | stop a running Go AppHost through `--apphost <dir>` | | `GoSecretAppHostTests` | `SecretTypeScriptAppHostTests` | secret create, read, update, delete and environment-scoped secrets. `get` and `list` output is checked with `grep`, and the JSON listing after the delete must hold exactly the remaining secret | | `GoPublishTests` (3) | `TypeScriptPublishTests` | Docker Compose publish with a service callback, the default output directory, and the Dockerfile generated for `AddGoApp` | | `GoKubernetesDeployTests` | `KubernetesDeployTypeScriptTests` | Go AppHost deployed to KinD with Helm, using the Helm defaults (namespace `default`, chart version `0.1.0`); the `api` and `cache` pods must become Ready | **Image and CI plumbing** - `tests/Shared/Docker/Dockerfile.e2e-polyglot-go`, the image build, warm and load workflows, `eng/scripts/load-cli-e2e-images.sh` (`--require-go`), `reproduce-flaky-tests.yml` and `docs/ci/cli-e2e-images.md` get a `PolyglotGo` variant next to `PolyglotJava`. - The Go layer installs `docker-buildx`. `aspire deploy` builds the `api` image with `docker buildx build` and throws when buildx is missing (`DockerContainerRuntime`), while the shared polyglot base installs `docker.io` with `--no-install-recommends`, which leaves buildx out. The DotNet image (`Dockerfile.e2e`) installs it the same way. `kind`, `helm` and `kubectl` are installed by the test at runtime. - `run-tests.yml` downloads and loads that image for split jobs whose short name contains `-Go`. The classes are all named `Go*` for that reason: matching a bare `go` would also select names such as `MongoDB` and `Config`. - The three new `actions/*` pins reuse SHAs that are already pinned elsewhere in the repository, so no allow-list change is needed. - `Aspire.Hosting.Go` and `Aspire.Hosting.CodeGeneration.Go` join the CLI E2E runtime-package consumers in `eng/github-ci/test-trigger-map.yml`, next to the Java and TypeScript code generation packages. Two `InlineData` rows in `SelectTestsAcceptanceTests` cover the routing. **Helpers.** `AppHostLanguagePromptSelection.Go`, `AspireTemplate.GoStarter` and `GoEmptyAppHost`, `AspireNewGoEmptyAppHostAsync`, and `EnableExperimentalGoSupportAsync`. The last is the only place the E2E projects write `features:experimentalPolyglotGo`, so removing the flag (#21106, in draft #21183) deletes one helper and its ten call sites instead of reworking each test. The pickers type the entry's name (`Go API` for the starter, `Go` at the language prompt) instead of counting `Down` presses, and each step waits for the expected selection text before pressing Enter, so a wrong match fails with a clear timeout instead of choosing another entry. The `PolyglotGo` image has no .NET SDK, which drops the two C# starters from the template list: a `Down` count that is right with an SDK clamps onto `Empty AppHost` without one, which is how the first CI run failed. The search prompt jumps to the first entry whose text contains what was typed, ignoring case, so the selection no longer depends on what else a picker lists. Part of #21108. ### CI status The first CI run (head `d24eb6c14a`) passed the CLI E2E image build (including the `docker-buildx` line), `GoCodegenValidationTests`, `GoEmptyAppHostTemplateTests` (2 tests), `GoPolyglotApphostDirectoryTests`, `GoSecretAppHostTests` and 2 of the 3 `GoPublishTests`: 7 of the 10 tests. The other three (`GoStarterTemplateTests`, `GoPublishTests.PublishGoStarterGeneratesGoAppDockerfile` and `GoKubernetesDeployTests`) failed for one reason, the 4 `Down` presses to the Go starter. On `c788f994e7`, which replaces the `Down` presses with type-ahead, makes the Kubernetes AppHost callback-free and asserts the full secrets listing, all seven classes passed: 10 of 10 tests, including `GoStarterTemplateTests` (start, check that `cache` and `api` exist, stop) and `GoKubernetesDeployTests`. No check on that head is failing on its latest attempt (447 succeeded, 11 skipped). The three commits after it (`dad6c38807`, `5506d5944a` and `962ebc83a4`) tighten three assertions that could pass without proving anything (see the findings below). On the last of them, `962ebc83a4`, all seven classes passed again on `ubuntu-latest`: 10 of 10 tests, none skipped. That includes the stale-SDK baseline and restore flow in `GoCodegenValidationTests` and, in `GoKubernetesDeployTests`, a `kubectl wait --for=condition=Ready` that exited 0 for both the `api` and `cache` pods on a real kind cluster. Three other jobs failed on the first attempt of that run and passed on a rerun of the same commit. This PR changes nothing under `src/` or `extension/`. - `Hosting.Azure` (`AzureFunctionsTests.AddAzureFunctionsProject_WorksWithMultipleProjects`) failed on attempts 1 and 2 and passed on attempt 3. - `KubernetesDeployWithMongoDBTests` failed on attempt 1, on a runner with kernel `7.0.0-1017-azure`, and passed on attempt 2. - The `java-apphost-debug-startup-timeout` VS Code extension E2E failed on attempt 1 and passed on attempt 2. The latest attempt of every check on `962ebc83a4` is green: 447 succeeded, 11 skipped. ### Findings that shaped the tests - The Go starter's `api/go.mod` ships without a `go.sum`, and `WithModTidy` only runs in run mode, so building the starter's container fails with `missing go.sum entry` (reproduced with a scratch `go build`). `GoPublishTests` asserts the generated Dockerfile only, and `GoKubernetesDeployTests` runs `go mod tidy` in `api/` before deploying. The product fix is not in this PR (item 1 of #21206). - The Kubernetes test's AppHost passes no `WithHelm` callback. In the generated Go SDK a `Configure` func inside an options struct panics the options merge with `unexpected end of JSON input` (#18080, fix in #21188), so the test uses the Helm defaults (namespace `default`, chart version `0.1.0`) and takes the namespace and chart-version parameters back once that fix lands (item 7 of #21206). The deploy path itself (`AddKubernetesEnvironment`, `AddContainerRegistry`, `aspire deploy`) is unchanged, so the test is not marked `[ActiveIssue]` and the trigger map needs no new edge. - Three checks are written so that doing nothing cannot pass: - `secret get` and `secret list` are piped through `grep` and an `echo` marker is awaited. The key and value are already on screen from the typed `secret set` command, so waiting for that text alone always succeeds. - The codegen validation test copies the baseline `aspire.go` and `.codegen-hash` aside, runs `aspire add Aspire.Hosting.SqlServer` (which regenerates the SDK itself), puts the baseline back, checks that it is stale, and only then runs `aspire restore`. A restore that does nothing fails the test. - The Kubernetes test runs `kubectl wait --for=condition=Ready` for the `api` and `cache` pods, selected by `app.kubernetes.io/component`. `kubectl get pods` exits 0 on an empty namespace and Helm's `--wait` only waits for workloads that are in the chart, whereas `kubectl wait` exits 1 with `no matching resources found` when nothing matches. ### Not in this PR - An HTTP probe against the Go starter's API (item 6 of #21206). `GoStarterTemplateTests` starts the AppHost and checks that `cache` and `api` exist, which does not show that the API serves requests. No existing helper reads a resource URL without `jq`, which the `PolyglotGo` image does not carry, and the API probably needs host networking to reach the Redis container (an inference, not run). - The same weaknesses in the TypeScript and Java tests (#21210). `TypeScriptCodegenValidationTests` and `JavaCodegenValidationTests` run `aspire restore` right after an `aspire add` and compare the codegen hash with the one from before the add, and `KubernetesDeployTypeScriptTests` checks `kubectl get pods`, which exits 0 on an empty namespace. - An ACA deployment test for the Go starter. A new class in `Aspire.Deployment.EndToEnd.Tests` would run nightly against the real Azure subscription and cannot be verified from here. - Kubernetes `AddManifest` and `WithField` coverage. #18080 is an open product bug. - `.gitignore` assertions for the Go scaffold and starter. The templates do not write a `.gitignore` entry on `main` yet. #21193 adds one, and asserting it is item 3 of #21206. - Brownfield `aspire init` in an existing Go repository (#21201). Reading `ScaffoldingService` suggests an existing `go.mod` conflicts, but no test was run. - Go toolchain missing or too old diagnostics, which need an image without Go. - Go validation outside the CLI E2E tests, such as `go vet` in the polyglot validation and a Go scenario in `cli-starter-validation` (#21202). ## Validation Windows, repo-local SDK, head `962ebc83a4` on top of `upstream/main` `230d5ac2a8`. `main` has since gained one unrelated commit that touches only `KubernetesDeployWithCertManagerTests.cs`, so no rebase was needed. Rows 1 to 3 ran on this head. Rows 4 to 6 ran on `d24eb6c14a`: the later commits change only E2E test sources and the shared Hex1b helpers, and `Infrastructure.Tests` links neither (it links `TemporaryWorkspace.cs`, `WindowsTrayTestPayload.cs` and `PathLookupHelper.cs` from the shared directories) and does not read them, so those rows are unaffected. | Command | Result | | --- | --- | | `dotnet build tests\Aspire.Cli.EndToEnd.Tests\Aspire.Cli.EndToEnd.Tests.csproj -m:2 --no-restore` | 0 warnings, 0 errors | | `dotnet build tests\Aspire.Deployment.EndToEnd.Tests\Aspire.Deployment.EndToEnd.Tests.csproj -m:2 --no-restore` | 0 warnings, 0 errors (compiles the shared Hex1b helpers that gained the Go templates) | | `dotnet test --project tests\Aspire.Cli.EndToEnd.Tests\Aspire.Cli.EndToEnd.Tests.csproj --no-restore --no-launch-profile -- --filter-class "*.CliInstallStrategyTests" --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"` | 67 passed, 0 failed, 0 skipped. `--list-tests` (placed before the `--`) discovers the same 67, all in this class, including 4 new `PolyglotGo` cases | | `dotnet build tests\Infrastructure.Tests\Infrastructure.Tests.csproj -m:2 --no-restore` | 0 warnings, 0 errors | | `dotnet test --project tests\Infrastructure.Tests\Infrastructure.Tests.csproj --no-restore --no-launch-profile -- --filter-method "*.RealMapRepresentativeCliE2eRuntimeConsumerSelectsCliE2e"` | 4 passed, 0 failed | | `dotnet test --project tests\Infrastructure.Tests\Infrastructure.Tests.csproj --no-restore --no-launch-profile -- --filter-namespace "Infrastructure.Tests.TestTriggerMap"` | 351 total, 329 passed, 22 failed (exit 2) | The 22 `TestTriggerMap` failures are pre-existing and Windows-only: 5 `SelectTestsCliTests` and 17 `PrComparisonBaseTests` data rows. A control run on a clean `82c3b01ce1` (the `main` tip before this rebase) gave 349 total, 327 passed and the same 22 failures, caused by `bash` receiving mangled Windows paths and an `UnauthorizedAccessException` deleting a temporary git repository. This branch adds the 2 passing rows. The shell fragments of the three newest commits were also run outside CI. 27 cases ran against stub `aspire` and `kubectl` executables, with the fragments extracted from the C# sources so the dry run cannot drift from the tests: the `grep` checks accept the exact value and reject a wrong, empty or longer one, the baseline snapshot and put-back restore the stale SDK, and the `kubectl wait` command covers exactly `api` and `cache`. Real `kubectl` v1.36.1 against a throwaway fake API server confirmed the difference the Kubernetes test relies on: in an empty namespace `kubectl get pods --no-headers` exits 0 while `kubectl wait --for=condition=Ready` exits 1 with `no matching resources found`, and with only an `api` pod Ready the wait exits 0 for `api` and 1 for `cache`. Not run locally: all seven new E2E classes. They need Docker (and kind for the Kubernetes test), the CLI archive and the new `PolyglotGo` image, so CI is their only execution. All ten passed on `c788f994e7` and again on `962ebc83a4`, so the snapshot, put-back and restore flow in `GoCodegenValidationTests` and the `kubectl wait` check on a real kind cluster in `GoKubernetesDeployTests` have run end to end in CI. A crash-looping `api` or an image pull failure fails the Kubernetes test by design. ## Checklist - Is this feature complete? - [ ] Yes. Ready to ship. - [x] No. Follow-up changes expected. - Are you including unit tests for the changes and scenario tests if relevant? - [x] Yes - [ ] No - Did you add public API? - [ ] Yes - If yes, did you have an API Review for it? - [ ] Yes - [ ] No - Did you add `<remarks />` and `<code />` elements on your triple slash comments? - [ ] Yes - [ ] No - [x] No - Does the change make any security assumptions or guarantees? - [ ] Yes - If yes, have you done a threat model and had a security review? - [ ] Yes - [ ] No - [x] No --------- Co-authored-by: David Pine <7679720+IEvangelist@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c8717a87-0ee5-4296-a6fd-ea6e2d91a57e
…every resource (#21193) ## Description A freshly scaffolded Go AppHost had several rough edges: the SDK that the CLI regenerates under `.aspire/` was not ignored by git, `apphost.go` was not gofmt-clean, the scaffold output, the `go-starter` template and the generated SDK files took CRLF line endings on Windows, the launch profile ports came from a range that overlaps the Windows dynamic port range, 49 sample AppHosts used a `log.Fatalf` call that `go vet` rejects, and the starter's `builder.Err()` check could not see a failed `AddRedis` or `AddGoApp`. - **Ignore the generated SDK.** `GoLanguageSupport.Scaffold` now also writes a `.gitignore` containing `.aspire/`, where the CLI regenerates the SDK (`.aspire/modules`; it is large and rebuilt from the AppHost's integrations). The `go-starter` template gets the same one-line file, like the TypeScript, Python and Java starters, and `TemplateGitIgnoreTests` has a `go-starter` row. In a directory that already has a `.gitignore`, `ScaffoldingService` keeps it and appends only the missing entries (`MergeGitIgnoreContent`). - **gofmt-clean `apphost.go`.** The scaffold listed `"log"` before `"apphost/modules/aspire"` in one import group, which gofmt reorders, and ended without a final newline, which gofmt adds (a C# raw string literal drops the last newline). It now has two import groups and ends with a newline. - **LF line endings at the source.** gofmt rewrites a CRLF `.go` file, and three producers wrote CRLF on Windows. The scaffold's string literals take the line endings of the C# source (`apphost.go`, `go.mod`, `apphost.run.json`). The `go-starter` template is embedded and scaffolded exactly as it is checked out. The generated SDK files (`go.mod`, `transport.go`, `base.go`, `aspire.go`) mix raw string literals, `StringWriter` output and embedded resources. `Scaffold` now converts every file it returns to LF, `GenerateDistributedApplication` replaces CRLF with LF at its single exit point, and `.gitattributes` pins `*.go` and the `go-starter` directory to LF. The generator replaces only `\r\n`: `ReplaceLineEndings` would also split at U+2028, form feed and NEL, which Go allows inside comments and strings, and the generated doc comments carry the description text of the exported APIs. - **Launch profile ports.** `apphost.run.json` drew four ports from `random.Next(10000, 65000)`, which overlaps the Windows dynamic port range (49152-65535). It now uses `AppHostProfilePortGenerator`, which the CLI templates and the TypeScript scaffold already use. Its fixed ranges all end below 22300 (dashboard 15000-15299 http and 17000-17299 https, OTLP 19000-19299 and 21000-21299, resource service 20000-20299 and 22000-22299). The shared file is linked into the Go code generation project. - **Sample AppHosts pass `go vet`.** 408 `log.Fatalf(aspire.FormatError(err))` calls in 49 `tests/PolyglotAppHosts/*/Go/apphost.go` files became `log.Fatal(aspire.FormatError(err))`. `go vet` reports `non-constant format string in call to log.Fatalf` for the old form, and a `%` in an error message would be printed as a formatting verb. The edit is one token per line with line endings and BOMs unchanged, and no `Fatalf(aspire.FormatError` remains in the tree. - **Playground comment.** The example comment in `playground/GoAppHost/apphost.go` said `AddRedis("cache", nil, nil)`, which no longer matches the generated signature, where optional parameters are a trailing options struct. It now says `AddRedis("cache")`. - **Check every resource handle.** A child-returning call such as `AddRedis` or `AddGoApp` that fails records its error on the handle it returns, not on the builder, and `Build()` checks only the builder's own error, so the `builder.Err()` check in the template could not see it. The `go-starter` `apphost.go` now keeps the `api` handle and checks `Err()` on `cache` and `api` before `Build()`. The `aspire init` example comment shows the same loop, with `AddRedis` and `AddPostgres`. The template has no `{{token}}` placeholders, so `aspire new aspire-go-starter` writes this file as it is, and the `apphost.go` on the Go AppHost page in microsoft/aspire.dev#1924 is the same text. - **Tests.** `GoScaffoldTests` (new, 15 test cases) covers the exact scaffold file set, LF line endings of the scaffold output and of the `go-starter` files, a `.gitignore` that ignores only `.aspire/`, the `go.mod` module, go, require and replace directives, the launch profile ports staying inside the shared generator ranges (five port seeds, including `null`), language detection on the scaffolded files, and gofmt-cleanliness of the scaffold output and of the `go-starter` sources as checked out. `AtsGoCodeGeneratorTests` gets `GenerateDistributedApplication_WritesFilesWithLfLineEndings`. The Go code generation test project links every `go-starter` file as a test input, plus the shared `TemporaryWorkspace` helper. No `eng/github-ci/test-trigger-map.yml` change is needed: `tests/PolyglotAppHosts/**` already routes to the polyglot job, the `go-starter` files are a `None Include` that the project graph sees, and `.gitattributes` already routes to every target, so this PR selects every test target. The handle check has no test of its own: nothing pins the content of the starter `apphost.go` or of the `aspire init` comment, and `ScaffoldedAndStarterSources_AreGofmtClean` covers their formatting. Part of #21115 ### Not in this PR - Making the generated SDK (under `.aspire/modules`) gofmt-exact. About 30 emit sites write single-line blocks such as `if x != nil { ... }`, struct and const alignment differs, and the imports are space-indented, so `gofmt -l` lists the file. It is git-ignored and regenerated on every restore, and fixing it means changing those emit sites and re-accepting every generated-SDK snapshot. - Running `go vet` over the sample AppHosts in CI. - The `experimentalPolyglotGo` header comment in `playground/GoAppHost/apphost.go`, which goes away with the feature flag (#21106). ## Validation Commands ran on Windows from the worktree root, on top of `upstream/main` `5ea175ce62` at head `de7b79d7d2`, with the repo-local SDK. | Command | Exit | Result | | --- | --- | --- | | `dotnet test --project tests\Aspire.Hosting.CodeGeneration.Go.Tests\Aspire.Hosting.CodeGeneration.Go.Tests.csproj --no-restore --no-launch-profile -- --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"` | 0 | 49 passed, 0 failed, 0 skipped | | `dotnet test --project tests\Aspire.Cli.Tests\Aspire.Cli.Tests.csproj --no-restore --no-launch-profile -- --filter-class "*.TemplateGitIgnoreTests" --filter-class "*.NewCommandTests" --filter-class "*.NewCommandTemplateConfigPersistenceTests" --filter-class "*.ScaffoldingServiceTests" --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"` | 0 | 165 passed, 0 failed, 0 skipped | | `git grep -n "Fatalf(aspire.FormatError" HEAD` | 1 | no match (`git grep` exits 1 when nothing matches) | The Go run includes all of `GoScaffoldTests`. A `--list-tests` check showed the Cli run includes the new `go-starter` row of `TemplateGitIgnoreTests`. Test-first check for the scaffold LF change: the two new tests `Scaffold_WritesFilesWithLfLineEndings` and `ScaffoldedAndStarterSources_AreGofmtClean` were run on their own, in a CRLF working tree, against the scaffold as it was before the `ReplaceLineEndings` line was added (the other changes already in place). Both failed (2 run, 2 failed). The first reported `apphost.go`, `go.mod` and `apphost.run.json` as containing CR. The second reported `gofmt -l` listing the scaffolded `apphost.go` with a whole-file diff. With the change both pass. This also shows that the gofmt test really runs gofmt when the tools are present. Running `gofmt -d` (Go 1.27.0) on a minimal file with the previous single import group, and on one without a final newline, confirms the other two fixes: gofmt moves `"apphost/modules/aspire"` above `"log"` and adds the missing final newline. Test-first check for the starter and generator LF change: `GoStarterFiles_UseLfLineEndings` and `GenerateDistributedApplication_WritesFilesWithLfLineEndings` were run on their own against the source without this change, in a CRLF working tree. Both failed (2 run, 2 failed). With the change both pass, as part of the 49 above. Negative control: with the two new `.gitattributes` rules removed and the starter files re-created, `GoStarterFiles_UseLfLineEndings` and `ScaffoldedAndStarterSources_AreGofmtClean` failed again (the second with a gofmt diff of `go-starter/api/main.go`). With the rules restored and the files re-created, `git ls-files --eol` lists all 8 starter files as `i/lf w/lf attr/text eol=lf` and every test passes. The gofmt test now reads the starter files exactly as they are checked out, without converting them first. `ScaffoldedAndStarterSources_AreGofmtClean` is marked `[RequiresTools(["go", "gofmt"])]`. When a tool is missing the attribute adds the `category=failing` trait, and the CI test runner filters that trait out (`eng/Testing.targets`), so the test is left out there rather than failed. `GeneratedTransport_SupportsDuplexCommunication` in the same project already works this way with `[RequiresTools(["go"])]`. The two new LF tests need no external tool and run everywhere. `go vet` over the 53 sample AppHosts (Go 1.27.0). The first run generated one SDK from the union of all the samples' packages (80 packages) with a locally built Debug CLI and vetted every sample against it. - Before the change: 4 clean, 32 `non-constant format string in call to log.Fatalf`, 17 type errors. - After the change: 36 clean, 0 format string failures, the same 17 type errors. - The 17 type errors appear only in the shared SDK. Each of those 17 was vetted again against an SDK generated from only its own packages (in a scratch copy of `tests/PolyglotAppHosts/<Sample>/Go`: `aspire restore --non-interactive --apphost apphost.go`, then `go vet ./...`): 17 of 17 clean. - The scaffolded `apphost.go` with its `go.mod`, and the `go-starter` AppHost, were vetted the same way and also build with `go build ./...` (exit 0). The CLI for these runs was a locally built Debug CLI from an earlier `main` (`3ed7e9690b`) plus an unrelated Go `aspire docs api` commit, not a build of this branch's base. The five commits between that `main` and `5ea175ce62` touch neither the Go or TypeScript code generators, the shared code generation, the type system, the remote host, nor any Go sample. These runs also predate the LF commits and the rebase onto `5ea175ce62`; none of them changes the content of the samples or of the generated SDK, only its line endings. Handle check and end-to-end runs (head `de7b79d7d2`). The first checks used a generated SDK that has `AddRedis`, `AddGoApp` and `AddPostgres`, from an earlier CLI build (this PR does not touch those APIs or `Resource.Err()`). The `go-starter` `apphost.go` with its `go.mod` builds and vets (`go build`, `go vet`, Go 1.27.0, both exit 0) and `gofmt -l` lists nothing. The `aspire init` example comment, uncommented in a scratch `main.go` with a `[]aspire.Resource{cache, db}` slice, also builds and vets, and it is the text of the `Scaffold` raw string with the comment markers removed (compared as strings). The `apphost.go` fence on the Go AppHost page in microsoft/aspire.dev#1924 is byte-identical to the template once line endings are normalized. Then I built the CLI from this head and ran it on Windows (a Debug build that targets net11.0, run as `dotnet aspire.dll` with the repo-local runtime; Go 1.27.0 with `GOTOOLCHAIN=local`, `GOWORK=off` and `GOPROXY=off`). The Go starter is still behind `features__experimentalPolyglotGo` until #21106 lands, so I set it. `aspire new` needed `--version 17.0.0-dev`: without it the CLI looks for a local `Aspire.ProjectTemplates` package of its own version and finds none. - `aspire new aspire-go-starter --version 17.0.0-dev --name verifygo --non-interactive --suppress-agent-init` exited 0. The `apphost.go` it wrote is byte-identical to the template (1224 bytes, LF only, no BOM), `.gitignore` is exactly `.aspire/`, and all 8 starter files have no CR. In that project `gofmt -l` lists nothing, and `go build ./...` and `go vet ./...` exit 0 against the SDK the command generated. - `aspire init --language go` in an empty directory exited 0 and wrote `.gitignore` (`.aspire/`), `apphost.go` with the new handle-loop comment, `go.mod` and `aspire.config.json`. The Go files and `.gitignore` have no CR and no BOM, `gofmt -l` lists nothing, and `go build ./...` and `go vet ./...` exit 0. `aspire.config.json` has CRLF on Windows because `AspireConfigFile.Save` uses the platform newline for every language; this PR does not change that. - A failed `Add` call at run time. In a copy of the starter I renamed the Go app to `"cache"`, the name of the Redis resource, so `AddGoApp` fails on the server (`Cannot add resource of type 'Aspire.Hosting.Go.GoAppResource' with name 'cache' because resource of type 'Aspire.Hosting.ApplicationModel.RedisResource' with that name already exists`). With the new loop, `aspire run --apphost apphost.go --isolated --non-interactive` stopped before `Run()`, printed that message and `exit status 1`, and the CLI reported `The Go apphost failed`. In a second copy that skipped the loop and wrote the results to a file, `builder.Err()` was nil, `cache.Err()` was nil, `api.Err()` held that message and `Build()` returned nil, so a check of `builder.Err()` alone does not see the failure. Overlap with open PRs: this PR shares files with #21183 (`playground/GoAppHost/apphost.go`), #21187 (`GoLanguageSupport.cs`), #21188 (`AtsGoCodeGenerator.cs`, `AtsGoCodeGeneratorTests.cs`), #21194 (`AtsGoCodeGeneratorTests.cs`), #21195 and #21197 (`AtsGoCodeGenerator.cs`, `Aspire.Hosting.CodeGeneration.Go.csproj`) and #21199 (`AtsGoCodeGenerator.cs`). A trial merge (`git merge-tree --write-tree`) of this head with the head of each of those seven PRs (`0e65752d44`, `e1bb4c11ae`, `9088dfe7fb`, `ffc59f2023`, `7d33810c11`, `82a44b0723`, `4f5a7a84aa`) was clean, as was a trial merge with each of the other open Go AppHost PRs and with `upstream/main` `f4ae2cc424`. Not run: - `aspire init --language go` in a directory that already has a `.gitignore` or a `go.mod`. The run above used an empty directory. The `.gitignore` merge has two unit tests (`MergeGitIgnoreContent_AppendsMissingEntriesWithoutOverwritingExistingContent` and `MergeGitIgnoreContent_DoesNotAddDuplicateAspireEntryWhenEquivalentEntryAlreadyExists`). - The LF changes are observable only on CRLF inputs. On a Linux or macOS checkout the LF tests pass with or without them, and I ran on Windows only. A working tree checked out before the `.gitattributes` rule keeps CRLF in the `go-starter` files, because git does not rewrite existing files when an attribute changes, so `GoStarterFiles_UseLfLineEndings` fails there until the files are re-created (delete the `go-starter` directory and run `git checkout -- <directory>`). Fresh checkouts are not affected. Whether the official build checks the `go-starter` files out as LF or CRLF was not checked; the rule makes it LF either way. - The rest of `Aspire.Cli.Tests`, the CLI end-to-end tests, and the polyglot validation scripts under `.github/workflows/polyglot-validation`. `test-go.sh` there scaffolds with `aspire init --language go` and inserts code before the `builder.Build()` line; the scaffold keeps that line and both imports. - `go vet` over the 53 sample AppHosts with a CLI built from this head. Only the starter and the `aspire init` scaffold were run with it. - The output of the 36-sample union SDK vet run was not saved to a file, so only the per-sample runs above can be repeated from the description. ## Checklist - Is this feature complete? - [x] Yes. Ready to ship. - [ ] No. Follow-up changes expected. - Are you including unit tests for the changes and scenario tests if relevant? - [x] Yes - [ ] No - Did you add public API? - [ ] Yes - If yes, did you have an API Review for it? - [ ] Yes - [ ] No - Did you add `<remarks />` and `<code />` elements on your triple slash comments? - [ ] Yes - [ ] No - [x] No - Does the change make any security assumptions or guarantees? - [ ] Yes - If yes, have you done a threat model and had a security review? - [ ] Yes - [ ] No - [x] No --------- Co-authored-by: David Pine <7679720+IEvangelist@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c8717a87-0ee5-4296-a6fd-ea6e2d91a57e
> [!NOTE] > **Merge order: land this PR after the other open Go generator PRs.** It does not depend on any of them and is based on `main`, but it changes files that #21188, #21193, #21194, #21195, #21197 and #21199 also change (`AtsGoCodeGenerator.cs`, `Resources/base.go`, the Go generator tests and the two Go snapshots). > > - When this PR was opened it merged cleanly (`git merge-tree`) with every other open Go AppHost PR except #21197. The one conflict is two lines in `EmitDtoCallbackToMap`: keep #21197's `GoStringLiteral(...)` line and this PR's `clientExpression: null` argument. > - If the Go snapshots conflict or go stale after another PR lands, regenerate them by running the Go codegen test project twice, because Verify rewrites the snapshots on the first run. > - #21188 also rewrites `deepUpdate`, which fixes the options-struct panic described under "Not covered". ## Description In a Go AppHost, a callback whose parameter is a resource builder (`Action<IResourceBuilder<T>>` in C#) received a nil value, so the first call on it panicked. The host marshals callback arguments with the declared parameter type, so a closed generic `IResourceBuilder<T>` is sent with a closed-generic `$type` (`Type.FullName`, which embeds the assembly-qualified `T`), while a builder returned from a capability is sent with the plain id of `T`. The generated Go SDK registers wrappers only for plain ids, so the transport left the raw handle in place and `callbackArg[T]` fell back to the zero value of `T`, a nil interface. The generated callback shim now builds the wrapper from the declared parameter type instead of relying on `$type`, which is what the TypeScript generator already does. Fixes #21255 ### Before and after A Go AppHost from `aspire-go-empty` plus `Aspire.Hosting.Docker`, run through `aspire publish` with the Go generator and runtime from `main` versus this branch: ```go env := builder.AddDockerComposeEnvironment("env") env.ConfigureDashboard(func(dashboard aspire.DockerComposeAspireDashboardResource) { dashboard.WithHostPort(&aspire.WithHostPortOptions{Port: aspire.Float64Ptr(9003)}) }) ``` | | Before | After | |---|---|---| | Callback argument | nil | usable `DockerComposeAspireDashboardResource` | | `aspire publish` | `panic: runtime error: invalid memory address or nil pointer dereference`, the Go AppHost exits with status 2 and the CLI exits with code 6 | exit 0, all 7 steps succeed, `docker-compose.yaml` maps `9003:18888` on `env-dashboard` | ### What changed - Go runtime (`Resources/base.go`): `callbackHandleArg[T](c, args, idx, wrap)` returns `wrap(h, c)` when the argument is still a raw `*handle` and otherwise defers to `callbackArg[T]`, so values the transport already wrapped (plain resource and context ids) take the path they took before. - Generator (`AtsGoCodeGenerator`): a callback shim emits `callbackHandleArg(s.client, args, i, new<Interface>FromHandle)` for a handle-category parameter that has a generated constructor and a client in scope. DTOs, enums, `CancellationToken` and DTO property callbacks (which have no client) keep `callbackArg[T]`. - Tests: a generator test that pins the emitted wrapper for `PublishAsDockerFile`, and a Go-backed test (`[RequiresTools(["go"])]`) that compiles the generated SDK and drives a callback over an in-memory fake host with both a closed-generic and a registered `$type`. Both fail on the old generator. - Snapshots: regenerated. ### Reviewer notes - **The snapshot diff is one mechanical transform.** `callbackArg[X](args, N)` becomes `callbackHandleArg(s.client, args, N, newXFromHandle)` at 260 call sites (259 changed lines) in `TwoPassScanningGeneratedAspire.verified.go` and 11 call sites (10 changed lines) in `AtsGeneratedAspire.verified.go`, plus the 17-line helper in the latter (it embeds `base.go`). Applying the inverse transform to the new snapshots reproduces the files on `main` byte for byte. Only the four `PublishAsDockerFile` shims for `ContainerResource` (executable, dotnet tool, project and C# app) change behavior in those snapshots; every other site is a context or event type that already arrived wrapped. - **More exports are fixed than the issue lists.** Across hosting and 19 integration assemblies, 27 declared `IResourceBuilder<T>` callback parameters expand to 29 shims that were affected, for example `WithDashboard` (Docker Compose and Kubernetes), `WithRedisInsight`, `WithPgAdmin`, `WithMongoExpress`, `WithKafkaUI` and the Azure `RunAsEmulator` and `RunAsContainer` overloads. - **Not covered.** - `ConfigureEnvFile` on Docker Compose takes `Action<IDictionary<string, CapturedEnvironmentVariable>>` and has a related cause: its dictionary argument still arrives as a zero-value `*Dict` with no handle behind it, so using it panics. It is the only dictionary-typed callback parameter in the 53 generated Go validation SDKs. A small generator change that wraps it the same way worked in a local check; it is left out so this PR stays about resource builders. - DTO property callbacks have no client in scope; none of them takes a resource builder today. - Callbacks passed through an options struct (the project and C# app `PublishAsDockerFile` overloads, and `ConfigureContainer` on Redis `WithRedisInsight` and `WithRedisCommander`) panic earlier, in `deepUpdate`, on `main` with or without this change. #21188 rewrites `deepUpdate`. With a local workaround for `deepUpdate`, the Redis callbacks received nil before this change and a live wrapper after it; none of these overloads was run on a tree that contains #21188. - Java appears to have the same weakness (read, not run). TypeScript and Python already use the declared parameter type for the wrapper. - No CI job runs a Go callback at runtime: `polyglot-validation.yml` only compiles the Go validation AppHosts. The new Go-backed test covers the transport against an in-memory fake host. - Not run: Linux and macOS, `aspire run` with the dashboard UI (the checks below use `aspire publish`, and the `PublishAsDockerFile` callback does not run in run mode), `go test -race` (no C compiler on the machine), and Rust. - **Host-side alternative not taken.** Marshalling the plain resource type id would fix every guest at once, but it changes the wire contract that `CallbackProxyTests` pins. ### Validation Windows, Go 1.27.0, Docker 29.9.1. The `dotnet test` runs are on the final commit. The Go AppHost checks ran on the same change before a clean rebase onto the current `main` (the patch is identical). - `dotnet test --project tests/Aspire.Hosting.CodeGeneration.Go.Tests/Aspire.Hosting.CodeGeneration.Go.Tests.csproj --no-launch-profile -- --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"`: 35 passed, 0 failed, 0 skipped (33 on `main`). Run twice, and `git status` was clean after each run, so the snapshots are stable. - The same project with the old `AtsGoCodeGenerator.cs` and `base.go` under the new tests and snapshots: 4 failed (both new tests and both snapshot tests). - `dotnet test --project tests/Aspire.Hosting.RemoteHost.Tests/Aspire.Hosting.RemoteHost.Tests.csproj --no-launch-profile -- --filter-class "*.CodeGenerationResolverTests" --filter-class "*.IntegrationLoadContextTests" --filter-class "*.ApiReferenceExportTests" --filter-class "*.ServiceErrorMessageTests" --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"` (the classes that resolve the Go generator): 27 passed, 0 failed. - Real Go AppHosts built with `aspire new aspire-go-empty`, then `aspire restore` and `aspire publish`, before and after: - `ConfigureDashboard` with `WithHostPort(9003)`: panic and CLI exit 6, then exit 0 with all 7 steps and `9003:18888` on `env-dashboard`. - `PublishAsDockerFile(func(c aspire.ContainerResource) {...})` on an executable: panic and CLI exit 6, then a live wrapper whose calls (entrypoint, environment, and `WaitFor` with the wrapper as an argument) all appear in the compose file. - 15 callbacks whose `$type` is a registered id (`WithEnvironmentCallback`, `WithArgsCallback`, `SubscribeBeforeStart` and others): the generated compose files are identical before and after. - All 53 Go validation AppHosts under `tests/PolyglotAppHosts` pass `aspire restore` and `go build` with this generator. - A throwaway in-package test of `callbackHandleArg` (not committed): 14 observations, and 64 goroutines x 2000 calls with no bad results. ## Checklist - Is this feature complete? - [x] Yes. Ready to ship. - [ ] No. Follow-up changes expected. - Are you including unit tests for the changes and scenario tests if relevant? - [x] Yes - [ ] No - Did you add public API? - [ ] Yes - If yes, did you have an API Review for it? - [ ] Yes - [ ] No - Did you add `<remarks />` and `<code />` elements on your triple slash comments? - [ ] Yes - [ ] No - [x] No - Does the change make any security assumptions or guarantees? - [ ] Yes - If yes, have you done a threat model and had a security review? - [ ] Yes - [ ] No - [x] No Co-authored-by: David Pine <7679720+IEvangelist@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c8717a87-0ee5-4296-a6fd-ea6e2d91a57e
|
Pull request created: #1968
|
|
📝 Documentation has been drafted in microsoft/aspire.dev#1968 targeting Updated Note This draft PR needs human review before merging. |
Resolve the conflicts with the Go generator changes that landed on main (#21188 union and callback fix, #21199, #21261 accepting interfaces): - AtsGoCodeGenerator.cs: keep both sets of private fields and helpers. BuildNameMaps builds the accepting interfaces first and the documentation maps last. The enum constant name comes from main's GetEnumMemberConstantName, the doc block and GoStringLiteral escaping stay. The union allowed-types doc note also reports that float64 accepts any built-in Go numeric type, matching main's widened type switch. - Regenerate the three affected snapshots from the merged generator. Their non-comment, non-blank lines are identical to main's. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
> [!NOTE] > **Merge order:** no PR must land before this one. It overlaps #21188 in `AtsGoCodeGeneratorTests.cs`, where both add tests at the same place. Whichever lands second rebases and runs `dotnet verify accept -y`. ## Description This fixes two groups of problems in the embedded Go runtime (`Resources/transport.go` and `Resources/base.go`) that came out of the Go AppHost GA review. - A panic inside a callback terminated the whole Go AppHost process, and the transport trusted several values that come from the AppHost server: the frame length, handle envelopes and JSON-RPC error objects. - `decodeAs` could not decode a list or a map of handles, so calls such as `GetResources()` and `GetChildren()` returned an error instead of the resources. Part of #21115. No exported Go identifier was added, removed or renamed, and no wire format changed. The new error type is unexported, so callers reach `Code()` and `Data()` with `errors.As` and a small interface. ### Transport hardening A panicking callback now fails only that call, a bad frame or handle returns an error instead of panicking or allocating without limit, and the JSON-RPC error code reaches the caller. Each claim from the review was checked against the tree before anything was changed. | # | Claim | Verdict | Change | | --- | --- | --- | --- | | 1 | A panicking callback is not recovered and kills the process | Real | `recover()` in the goroutine that runs the callback. The AppHost server gets a JSON-RPC error with code `-32603` and the message `Callback execution failed: <panic value>`. The stack trace is never sent | | 2 | Cancellation ids (`ct_<UnixNano>`) can collide, and every token parks a goroutine | Real | The id comes from a per-client atomic counter. The watcher is a `context.AfterFunc` that is stopped when the capability call returns, unless the token was already canceled: then the watcher still notifies the AppHost server | | 3 | `Content-Length` is unbounded, negative or overflowing | Upper bound real. Negative and overflow not real | A frame above 64 MiB (the same cap as the generated Python runtime) is rejected before the body is allocated. `strconv.Atoi` already fails on overflow and `length <= 0` was already rejected | | 4 | Unchecked type assertions on wire values | Real, in `wrapIfHandle` only | Comma-ok assertions that return a descriptive error. The other wire assertions were already checked | | 5 | `extractResult` drops the JSON-RPC error code and data | Real | The returned error is an unexported `*rpcError` with `Error()`, `Code() int` and `Data() any` | | 6 | `client.mu` is held across the dial | Real | `connect` releases the lock around the dial and checks again afterwards. If a concurrent `connect` won, the extra connection is closed | | 7 | The callback registry is never cleaned up | By design | None. The AppHost server sends no release signal for a callback and the generated `onDisconnect` exits the process, so the registry cannot outlive the connection | | 8 | The sticky `err` on handles can race with callbacks | Documented contract | A doc comment on `handleWrapperBase`. A lock on `setErr` and `Err()` alone would be incomplete because generated code reads `.err` directly | ### Lists and maps of handles `decodeAs[T]` returned `raw.(T)` when the value already had the right type and otherwise converted it with one JSON round trip. The transport turns every handle envelope in a result into a wrapper value, including the ones inside a list or a map, so a list of handles reaches `decodeAs` as a `[]any` of wrappers. That is not a `[]Resource`, and `encoding/json` cannot fill an interface element from an object, so every non-empty list of handles failed, including a list of one: ```text json: cannot unmarshal object into .0 of type aspire.Resource ``` An empty list and a single handle outside a list worked. The Go sample audit counted 184 of the 205 `decodeAs[[]X]` call sites in the generated SDK as lists with interface elements: `DistributedApplicationModel.GetResources()`, `Configuration.GetChildren()`, about 180 `AzureResourceInfrastructure.Get<X>s()` getters and 3 `PipelineStep` lists. The polyglot fixture (`tests/PolyglotAppHosts/Aspire.Hosting/Go/apphost.go`) discards these values, so CI never ran the path. `decodeAs` now decodes slices and string-keyed maps one element at a time (`decodeInto` in `Resources/base.go`). An element that is already assignable to the element type, such as a handle wrapper, is stored as it is. Any other element is decoded with a JSON round trip of that element alone. Nested lists and maps recurse once per level of the target type, and everything else keeps the previous path. A failure names the element, for example `aspire: cannot decode element 1 of []aspire.Resource: json: cannot unmarshal number into Go value of type aspire.Resource`, where it used to be an unnamed JSON error. ### Behavior notes - `Error()` on a host error returns the same text as before. A host error with an empty message now reads `AppHost returned a JSON-RPC error (code N)` instead of an empty string. - The callback error carries the panic value formatted with `%v`. The stack trace is neither sent nor printed. The TypeScript runtime also reports only the message of a thrown callback. - Canceling a token after the capability call it was passed to has returned no longer notifies the AppHost server, like the TypeScript runtime. The cancellation id is still an opaque `ct_` string (`ct_<counter>_<unix ms>`). - A capability can also keep the token after the call, such as a generated property setter (`SetCancellationToken(&SetCancellationTokenOptions{Value: tok})`). The AppHost server keeps that token registered after the call returns, but the watcher is detached when the setter returns, so a later `tok.Cancel()` does not cancel the stored token. This is intended and matches the TypeScript and Java runtimes, which remove their cancellation listener in a `finally` after `invokeCapability`. Keeping a stored token cancelable would be a change in those runtimes as well, not a Go-only fix. - A call that returns because its own token was canceled still notifies the AppHost server. The watcher is detached only for a token that is still live. - A list of handles decodes without any JSON. The test allows at most 10 allocations for 100 handles. A list whose elements need a conversion, such as numbers into `[]int`, takes one small JSON round trip per element. ### Follow-ups Not in this PR. Proposed issue titles: - Go runtime: make the handle-wrapper sticky error race-free (generated code reads `.err` directly) - Go runtime: `List`, `Dict` and `callbackArg` swallow marshal errors - Go runtime: bound the header count and header size in `readMessage` like the Python runtime (16 headers, 8 KiB) - Go runtime: callback registry grows without a host release signal - Go runtime: `os.Exit` in the generated `onDisconnect` - Go runtime: a pre-canceled token races its `cancelToken` notification against the capability request - Go runtime: the `cancelToken` request is sent with `context.Background()` - Go codegen: a required `CancellationToken` parameter registers the token but leaves `ctx := context.Background()`, so a cancel does not unblock the local call - Go runtime: a callback that calls `runtime.Goexit` never answers the AppHost server - Go tests: share one `go` process runner across the runtime test facts - Go polyglot fixture: assert non-empty `GetResources()`, `GetChildren()` and Azure infrastructure list results ## Validation Run on the PR head, which is five commits on `main` (`5ea175ce62`). | Command | Result | | --- | --- | | `dotnet test --project tests\Aspire.Hosting.CodeGeneration.Go.Tests\Aspire.Hosting.CodeGeneration.Go.Tests.csproj --no-launch-profile -- --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"` | 34 passed, 0 failed, 0 skipped (17.4 s) | | `gofmt -l transport.go base.go` on LF copies of the two runtime files | no output | | `go vet transport.go base.go transport_hardening_test.go decode_test.go` on the same copies plus the two Go files the new fact embeds | exit 0, no findings | | `go test -count=1 -timeout 60s transport.go base.go transport_hardening_test.go decode_test.go` on the same files | ok (2.6 s) | The new `GeneratedRuntime_PassesGoRuntimeHardeningTests` writes the generated runtime for an empty model to a temp directory, adds `transport_hardening_test.go` and `decode_test.go`, and runs `go test` on it. The 19 Go tests are hermetic and fast. The transport tests drive the real client against a fake JSON-RPC host over `net.Pipe`: - a panicking callback is answered with a `-32603` error and the process survives - a callback argument with a malformed handle is answered with an error - cancellation ids are unique across concurrent tokens, a token does not park a goroutine, canceling a token while its call is in flight returns `context.Canceled`, sends `cancelToken` with the registered id and releases the registration (2000 passes, with no locking in the test), and canceling a token after its capability call returned sends nothing to the AppHost server - an oversized, overflowing, negative, zero or non-numeric `Content-Length` is rejected without a large allocation - `wrapIfHandle` reports a malformed handle instead of panicking - the JSON-RPC error code and data survive `extractResult`, and the code survives a request round trip (`ping`) over the fake host - a list of handles (one, several, with a `null` element), a map of handles, nested lists and maps, lists of values, an empty list and a nil result decode, and a decode error names the element that failed - a single handle and a DTO decode as before, and decoding 100 handles does not go through JSON The fact uses the same `[RequiresTools(["go"])]` gate as the existing `GeneratedTransport_SupportsDuplexCommunication`. The verified Go snapshot embeds the runtime, so `AtsGeneratedAspire.verified.go` is refreshed. Its changed lines are exactly the `transport.go` and `base.go` changes. Red/green for the transport changes: with `Resources\transport.go` restored from `main`, `GeneratedRuntime_PassesGoRuntimeHardeningTests` fails. The command was `dotnet test --project tests\Aspire.Hosting.CodeGeneration.Go.Tests\Aspire.Hosting.CodeGeneration.Go.Tests.csproj --no-launch-profile -- --filter-method "*.GeneratedRuntime_PassesGoRuntimeHardeningTests" --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"`, and the result was 1 failed (1.2 s) because the Go tests no longer build (`c.cancellationsMu undefined`, `undefined: maxMessageSize`, `assignment mismatch: 2 variables but wrapIfHandle returns 1 value`). A compile failure does not show the behavior, so two fixes were also removed one at a time from a scratch copy of the fixed runtime and run with `go test -timeout 30s transport.go base.go transport_hardening_test.go`: | Removed from the fixed runtime | Result | | --- | --- | | the `recover()` in `handleCallbackRequest` | FAIL, `panic: boom` and the test process dies | | the 64 MiB check in `readMessage` | FAIL, `TestReadMessageRejectsInvalidContentLength`: `Content-Length "67108865": allocated 67121408 bytes before rejecting the frame`, then a `makeslice` panic | Red/green for the cancellation race: with only `Resources\transport.go` restored from the commit before the fix, the same `--filter-method` command fails with 1 failed (9.8 s) in `TestCancellationDuringCallReturnsNotifiesHostAndReleasesRegistration`, for example `pass 623: the host was not told about the cancellation: read pipe: i/o timeout`. The race needs real parallelism, so it was also measured on a scratch module with LF copies of the runtime and the two Go test files. 40 runs of that test (2000 passes each) failed on the old runtime 0 times with 1 CPU, 33 times with 2, 40 with 4 and 39 with 8 (`go test -cpu`, with the old runtime's 5 s read deadline cut to 1 s so failing runs finish sooner). The fixed runtime failed 0 of 30 runs at each of the four settings. Red/green for the list decoding: with `Resources\base.go` reverted to the version before the list change, the same command fails with 1 failed (4.4 s), among others: ```text decode_test.go:99: one handle: json: cannot unmarshal object into .0 of type aspire.decodeTestResource decode_test.go:191: map of handles: got map[], json: cannot unmarshal object into Go struct field .a of type aspire.decodeTestResource decode_test.go:220: list of lists: got [], json: cannot unmarshal object into .0.0 of type aspire.decodeTestResource decode_test.go:323: decoding 100 handles allocated 523 times, want a handful ``` ### Not verified - `go test -race`. The race detector needs cgo, which is not set up on this Windows machine. - A real Go AppHost end to end, including the list decoding and whether the AppHost server cancels an in-flight call when it receives `cancelToken`. The tests use a fake AppHost server over `net.Pipe` and a stand-in handle interface instead of the generated `Resource` types, and the failure message above comes from that stand-in. The 184 of 205 count comes from the sample audit of the generated SDK and was not recounted. - The in-flight cancellation test only catches the old behavior with 2 or more CPUs (see the red/green numbers above), so a single-CPU runner would not catch a regression. - The polyglot fixture still discards the list results, so CI still does not execute the generated list getters. Strengthening it needs a locally generated SDK, which was not built here. - List elements that are DTOs holding handle-typed fields, and handles whose `$type` has no registered wrapper. They now fail with an error that names the element, but they were not exercised against real server data. - Item 6 (releasing `client.mu` around the dial) was checked by reading the code only, because `openConnection` cannot be replaced in a test. - A callback that calls `runtime.Goexit` still never answers the AppHost server, because `recover()` does not see `Goexit`. - Anything other than Windows with Go 1.27.0. - The full CI matrix, including the polyglot validation job. CI is the first run there. - Merge order with other Go AppHost PRs such as #21188. A trial merge with the head of #21188 conflicts only in `AtsGoCodeGeneratorTests.cs`, where both PRs add tests at the same place. `base.go` and `AtsGeneratedAspire.verified.go` merge without conflicts, but the merged tree was not built or tested. The PR that lands second needs a rebase and `dotnet verify accept -y`. ## Checklist - Is this feature complete? - [x] Yes. Ready to ship. - [ ] No. Follow-up changes expected. - Are you including unit tests for the changes and scenario tests if relevant? - [x] Yes - [ ] No - Did you add public API? - [ ] Yes - If yes, did you have an API Review for it? - [ ] Yes - [ ] No - Did you add `<remarks />` and `<code />` elements on your triple slash comments? - [ ] Yes - [ ] No - [x] No - Does the change make any security assumptions or guarantees? - [ ] Yes - If yes, have you done a threat model and had a security review? - [ ] Yes - [ ] No - [x] No --------- Co-authored-by: David Pine <7679720+IEvangelist@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c8717a87-0ee5-4296-a6fd-ea6e2d91a57e
…21197) > [!NOTE] > **Merge order:** no PR must land before this one. #21188 and #21199, which overlapped it, are on main and this branch contains them. Whichever of the remaining open PRs lands second rebases: > > - A PR that changes `TwoPassScanningGeneratedAspire.verified.go` (a new or changed `Aspire.Hosting` API reaches it) conflicts with this one there, because this PR adds doc comments throughout the file. Regenerate the snapshot with `dotnet verify accept -y` instead of merging hunks by hand. `AtsGeneratedAspire.verified.go` works the same way. > - #21195 calls `RenderGoExportedValueType`, whose signature this PR changes, so the second to land needs a one-line fix-up in `AtsGoCodeGenerator.ApiProjection.cs`. ## Description Part of #21115. The Go SDK generator documents a member only from the capability `Description`, plus a few fixed comments for enums, DTOs and handles. `Summary`, `Remarks`, `Returns`, per-parameter docs, enum value docs and the obsolete and experimental flags never reach `aspire.go`, so `go doc` and gopls hover show one thin line or nothing. This PR emits godoc from the same ATS documentation metadata the TypeScript generator already turns into JSDoc. ### What changes in the generated SDK Documentation flows to every kind of declaration: - Capabilities (handle methods, standalone functions, property getters and setters): summary, remarks, a `Parameters:` list and a `Returns:` paragraph. A trailing `...Options` parameter points at its Options struct, and each Options field carries the doc of its parameter. - Handle interfaces, DTO structs and their fields, enum types and each enum value, exported value groups and their members, and Options structs. The comments follow godoc conventions: - A comment starts with the identifier it documents, paragraphs are separated by a bare `//`, and code samples and lists from the XML docs keep an indented-block form. Lines are wrapped near 80 columns and never split a word. - `<see cref>` references become `[Name]` doc links only for generated types. Anything else stays plain text so the SDK has no dangling links. - Obsolete capabilities end with a `Deprecated:` paragraph, the one marker gopls and staticcheck recognize. - Experimental capabilities get a `# Experimental` heading and the line `Notice: This API is EXPERIMENTAL and may be changed or removed in a later release.` Go has no toolchain-recognized experimental marker, so this uses the convention `google.golang.org/grpc` documents its experimental APIs with. It renders as a heading on pkg.go.dev and in gopls hover and is stable under gofmt. TypeScript uses `@experimental`. - XML-doc text is normalized to LF and no comment line has trailing whitespace. - A one-line remarks paragraph that gofmt and `go doc` would read as a heading, such as `Requires Redis 7.2` or `Used with the resource's name`, ends with a period so it stays a paragraph. Example from the test snapshot. On main this method has only the first line: ```go // WithPreviewStatus sets the preview status. // // Parameters: // - status: The status to set. // // # Experimental // // Notice: This API is EXPERIMENTAL and may be changed or removed in a later // release. // // Deprecated: Use WithDocumentation instead. WithPreviewStatus(status DocumentedStatus) DocumentedResource ``` ### Commits 1. **Share the ATS version-skew probe between polyglot generators.** `AtsTypeSystemCompatibility` (reads the experimental flag from an `Aspire.TypeSystem` that may predate it) moves from the TypeScript generator project to `src/Shared/CodeGeneration` and is linked into the Go and TypeScript generator projects. The TypeScript generator only gains a `using`. 2. **Emit godoc from ATS documentation metadata in the Go SDK generator.** New `GoDocComment.cs` formats a comment from the metadata, and `AtsGoCodeGenerator.cs` calls it at each declaration. 3. **Escape ATS metadata at the Go generator output sinks.** Injection hardening, described below. 4. **Render exported string values as Go string literals.** The same sink class for exported values, described below. 5. **End one-line remarks that gofmt would turn into headings with a period.** See the conventions list above. 6. **Make the escaping and gofmt round-trip comments state what they cover.** Comments only. 7. **Build the Go doc maps with the name maps.** `BuildNameMaps` builds the documentation maps itself, so a caller that builds only the name maps still gets a usable renderer (review feedback). ### Escape ATS metadata at the Go generator output sinks ATS metadata (descriptions, capability ids, parameter and property names) was spliced into Go source unescaped. A multi-line description, or a name containing a quote, a newline or a percent sign, produced Go that did not compile or that carried extra statements. Now: - every doc comment line is prefixed with `//`: CR, LF and CRLF start a new comment line, and Unicode line and paragraph separators, control characters and the byte order mark are replaced or dropped, so a description cannot end the comment; - string literals (capability ids, request argument names, json struct tags, map keys, enum constant values and `%q` arguments) go through one `GoStringLiteral` helper, and a capability id inside a `fmt.Errorf` format string has its `%` doubled; - method names (`ToPascalCase`) go through the existing `SanitizeIdentifier`, as parameter and type names already did; - exported string values, enum values and dictionary keys go through `GoStringLiteral` too (see the review finding below). This is a low-severity robustness fix. An integration author already runs code on the developer's machine, so no trust boundary is crossed. Output for valid first-party names is unchanged: the hardening commits change no snapshot that exists on main. The shared scanner and the other language generators have the same pattern and are left for a follow-up. A review of this change found one more sink of the same class, rated Medium and fixed by commit 4. Exported string values, enum values and dictionary keys were written with the relaxed JSON writer, which spells a character outside the Basic Multilingual Plane (for example an emoji) as a UTF-16 surrogate pair such as `\uD83D\uDE00`. Go rejects surrogate halves in a string literal, so one such value failed `go build` for the whole SDK with `escape is invalid Unicode code point U+D83D`. These values now go through `GoStringLiteral`, which keeps such characters as UTF-8. Numbers and booleans are the same text in JSON and Go and render as before. ### Generated output and cost - Two-pass SDK (`aspire.go` for `Aspire.Hosting` plus the test types): 1,347,950 to 1,808,456 bytes (+34.2%) and 31,892 to 45,393 lines. The smaller test-types snapshot: 162,529 to 176,209 bytes (+8.4%) and 4,591 to 5,049 lines. - The sequence of non-comment, non-blank lines in both large snapshots is identical to main (27,526 and 3,730 lines compared, 0 differences). The snapshot diffs are doc comment lines and the blank lines that separate them. - Markers in the two-pass snapshot: `Deprecated:` 16, `# Experimental` 221, `Parameters:` 848, `Returns:` 789. - Generation time, scan excluded, 1 warm-up and 15 samples on a shared developer machine: median 16.0 ms on main and 104.2 ms with this change (+88 ms), min 12.8 ms and 85.7 ms. This was measured before main's latest Go generator changes were merged and not repeated. The cost is linear in the number of documented identifiers and the regular expressions are source-generated. It was not profiled further. Review tip: read `DocumentedMetadataGeneratedAspire.verified.go`, `EscapedMetadataGeneratedAspire.verified.go` and `GoDocCommentRender.verified.txt` first. The two large snapshot diffs are comments only. ### Tests and guard New tests are in `GoDocCommentTests.cs` and `GoGeneratorEscapingTests.cs`. They build their metadata in code, so the TypeScript `TestTypes` that the Go, Java, Python and Rust test projects link are untouched. - `GenerateDistributedApplication_WithDocumentedMetadata_EmitsGodoc` (Verify snapshot) covers every declaration kind above, including `Deprecated:`, `# Experimental`, enum values, special characters and CRLF input. - `Render_FormatsGodocBlocks` (Verify snapshot) covers the comment formatter, and `Render_KeepsEveryLineInsideTheComment` proves hostile text never leaves the `//` comment. `Render_EndsHeadingLookingRemarksWithAPeriod` proves that one-line remarks which `go/doc/comment` would read as a heading (gofmt rewrites them to `# Heading`) end with a period. - `GeneratedSdk_DocCommentsFollowGodocConventions` is the guard that documented APIs keep their docs: over the SDK generated from the real scanned `Aspire.Hosting` assembly, every commented declaration starts with its name, no line has trailing whitespace or a lone CR, `Deprecated:` starts its own paragraph, obsolete and experimental metadata present in the model reaches the output, and the documented enum values equal the commented constants. The two-pass snapshot also covers the whole SDK, so an XML-doc edit to a scanned `Aspire.Hosting` API changes it, the same coupling the TypeScript snapshot has. - `GeneratedSdk_PassesGoToolchainChecks` runs `go build ./...`, `go vet ./...` and `gofmt` on that SDK and asserts gofmt leaves the generated top-level doc comments unchanged. go/printer only reformats doc comments that start in column 1, so the indented interface-method and struct-field comments are covered by the heading test above instead. - `GoGeneratorEscapingTests` generates an SDK from hostile metadata: a description of `Adds Foo.\nfunc init() {}\n// ` that tries to end the comment and declare a function, capability ids, method, parameter and property names containing double quotes, percent signs, backslashes, hyphens and line separators, and exported values (a string, an enum value and a dictionary key) containing an emoji. It checks the Verify snapshot, that no injected statement appears in the output, and that the output passes `go build` and `go vet`. With the escaping commit's generator changes reverted, all three fail (the injected statement is in the output, the snapshot differs and `go build` reports syntax errors). With only the exported-value rendering reverted, the snapshot differs and `go build` reports `escape is invalid Unicode code point U+D83D`. The Go toolchain tests use the existing `[RequiresTools]` gate, like `GeneratedTransport_SupportsDuplexCommunication`, so this PR adds no test project, CI job or runtime dependency. `eng/github-ci/test-trigger-map.yml` needs no entry: the moved file is a linked `src/Shared` file, which the selector attributes to its consuming projects, and the Python, Java and Rust generators and tests do not reference it (see the `SelectTests` row below). ### Merging with the Go API reference exporter #21195 adds the Go API reference exporter to the same project. At its current head (`ec5022d3a9`) the two pull requests merge without textual conflicts (checked with `git merge-tree`), but the exporter calls `RenderGoExportedValueType`, which this change gives a required `includeDocs` parameter, so whichever merges second needs a one-line fix-up in `AtsGoCodeGenerator.ApiProjection.cs`: in `ProjectExportedValues`, pass `includeDocs: false` to `RenderGoExportedValueType`. The compiler flags that call. `BuildNameMaps` builds the documentation maps itself, so the exporter's `BuildApiModel` can render documentation without any change. The exporter's `GoApiExportTests.GoOnlySurface` snapshot then shows the ATS documentation summaries in place of the generated `X represents X.` text. A trial merge of both heads with that fix-up built, and 80 of the 81 tests in the Go codegen test project passed. The one failure was that snapshot, which differed only in those summaries and is re-accepted with `dotnet verify accept -y`. That trial ran at earlier heads of both branches (see Not verified). ### Merge with main This branch merges main (`45de07f9e3`), which now contains #21188, #21199 and #21261. The conflicts were in `AtsGoCodeGenerator.cs` and two snapshots, and the resolution keeps both sides: - Both sets of private fields and helpers are kept. `BuildNameMaps` builds the accepting interfaces from #21199 first and the documentation maps last, because the renderer resolves type names through the maps built before it. - The enum value constant takes its name from `GetEnumMemberConstantName` (#21199), with this PR's value doc block and string-literal escaping. - DTO callback conversion keeps this PR's escaped map key together with the updated `EmitGoCallbackShimBody` call from #21261. - The union allowed-types note now also says that `float64` accepts any built-in Go numeric type, matching the widened type switch from #21188. The snapshots were regenerated, not merged by hand. In the two large ones the non-comment, non-blank lines are identical to main's (see the counts above). ## Validation Run from the repository root on Windows after `restore.cmd`, with Go 1.27.0 installed so the `[RequiresTools]` tests ran and were not skipped. | Command | Exit | Result | | --- | --- | --- | | `dotnet test --project tests\Aspire.Hosting.CodeGeneration.Go.Tests\Aspire.Hosting.CodeGeneration.Go.Tests.csproj --no-launch-profile -- --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"` | 0 | 79 passed, 0 failed, 0 skipped (65 on main) | | `dotnet test --project tests\Aspire.Hosting.CodeGeneration.TypeScript.Tests\Aspire.Hosting.CodeGeneration.TypeScript.Tests.csproj --no-launch-profile -- --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"` | 0 | 153 passed, 0 failed, 0 skipped (153 on main) | | `dotnet run --project tools/SelectTests -- --from upstream/main --to HEAD --explain` | 0 | 0 unattributed changed files, no fallback. Selects 7 of 100 PR test projects: `Aspire.Cli.EndToEnd.Tests`, `Aspire.Cli.Tests`, `Aspire.Hosting.CodeGeneration.Go.Tests`, `Aspire.Hosting.CodeGeneration.TypeScript.Tests`, `Aspire.Hosting.Native.Core.Tests`, `Aspire.Hosting.RemoteHost.Tests`, `Aspire.Managed.Tests`, and triggers the `cli-starter-validation`, `extension-e2e`, `homebrew-installer`, `polyglot`, `typescript-api-compat`, `typescript-sdk` and `winget-installer` PR jobs. The Python, Java and Rust codegen test projects are not selected. | Not verified: - The `polyglot` job and the full CI matrix. Of the selected test projects, only the Go and TypeScript codegen projects and the code generation classes of `Aspire.Hosting.RemoteHost.Tests` (`CodeGenerationResolverTests`, `ApiReferenceExportTests` and `ServiceErrorMessageTests`, 22 tests) were run locally. CI is the first run for the rest. - Whether the CI runners that run the Go test project have `go` on `PATH`. The three toolchain tests depend on `[RequiresTools]`. - That the generated SDK body is gofmt-clean. It is not clean on main either, so only the comments are gofmt-checked. - Generation time was measured once on a shared, busy machine, with main and this change measured at different times. - The checks that the new tests fail without each fix were run before the two merges of main into this branch (`f06f16b7bf` and `45de07f9e3`) and were not repeated. - The Python, Java and Rust codegen test projects (29, 71 and 26 tests) passed on an earlier head of this branch and were not re-run after the exported-value and heading fixes. They do not link the moved shared file or any file those fixes changed. - The merge with #21195 was built and tested locally at its head `7e3948ad85` and an earlier head of this branch only, in the Go codegen test project and not in the full CI matrix. At its current head `ec5022d3a9` only the textual merge was checked. #21195 may change before it lands. ### Follow-up issues - Escape ATS metadata in the shared scanner and the TypeScript, Python, Java and Rust generators. - Go SDK: package doc, standard generated-code header, gofmt-clean body, LF output. - Scanner: preserve `<para>` boundaries and code indentation. - Go SDK: guard names derived with `ToCamelCase` (implementation names, list and dictionary getter fields, builder and application implementations) against Go keywords, and make `ReserveName` reject keywords and predeclared names. ## Checklist - Is this feature complete? - [ ] Yes. Ready to ship. - [x] No. Follow-up changes expected. - Are you including unit tests for the changes and scenario tests if relevant? - [x] Yes - [ ] No - Did you add public API? - [ ] Yes - If yes, did you have an API Review for it? - [ ] Yes - [ ] No - Did you add `<remarks />` and `<code />` elements on your triple slash comments? - [ ] Yes - [ ] No - [x] No - Does the change make any security assumptions or guarantees? - [ ] Yes - If yes, have you done a threat model and had a security review? - [ ] Yes - [ ] No - [x] No --------- Co-authored-by: David Pine <7679720+IEvangelist@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c8717a87-0ee5-4296-a6fd-ea6e2d91a57e
> [!IMPORTANT] > **Merge last.** This PR depends on the other open Go AppHost PRs and must land after them. > > - **Must land first**, because each adds references to the flag that this PR has to delete: #21189 (E2E helper and call sites), #21195 (`SdkExportCommandTests`) and #21196 (`GoAppHostToolingCheck` and its tests). > - **Must land first**, because it edits the same hunk of `DefaultLanguageDiscovery.cs` and this PR rebases onto it: #21191. > - **Should land first**, because removing the flag turns Go on for everyone and these fixes should ship before it: #21186, #21187, #21188, #21193, #21194, #21197 and #21199. ## Description Removes the `experimentalPolyglotGo` feature flag so Go AppHosts are available by default, the same way TypeScript AppHosts are. With no flag, **Empty (Go AppHost)** and **Starter App (Go API + Redis, Go AppHost)** are offered by interactive and non-interactive `aspire new`, and Go is offered by `aspire init` language selection and `--language go`. **This is the GA switch for the Go AppHost epic (#21115) and must merge last**, after the other children have landed: #21108 (E2E, publish, deploy), #21110 (support policy, API review, packaging), #21112 (docs and samples) and #21114 (VS Code debugging). It was opened as a draft so CI exercises the 52 Go fixtures without the flag. It is ready for review now, but not for merging. What changed: - **CLI:** `KnownFeatures` drops the Go flag and `DefaultLanguageDiscovery` no longer marks Go experimental. `LanguageInfo.IsExperimental` is kept because Python, Java and Rust still use it. - **Leftover key:** a stale `experimentalPolyglotGo` in a user's configuration is ignored with no warning, matching what happened to the TypeScript flag when it was removed. `Features` only reads named keys, and the extension schemas set `additionalProperties: false`, so VS Code flags the stale key. - **Extension schemas:** `aspire-config`, `aspire-global-settings` and `aspire-settings` are regenerated with `extension/scripts/generate-schema.js`. Only the 16-line Go block is removed from each. - **Fixtures and CI:** 52 `tests/PolyglotAppHosts/*/Go/aspire.config.json` files and `.github/workflows/polyglot-validation/Dockerfile.golang` no longer set the key. - **Java E2E picker offsets:** Go is always listed now, so with only the Java flag set the `aspire init` language picker is C#, TypeScript, Go, Java, and Java needs 3 Down presses instead of 2. The first CI run on this PR timed out waiting for `> Java` in `JavaPolyglotTests`, `JavaCodegenValidationTests` and `JavaPolyglotApphostDirectoryTests`; each now presses Down once more, with a comment stating the picker order. The order comes from `DefaultLanguageDiscovery`, `LanguageService` and `CliTemplateFactory`. TypeScript flows are unaffected because TypeScript sorts before Go. - **Tests:** `DefaultLanguageDiscoveryTests`, `NewCommandTests`, `ConfigCommandTests`, `NewCommandTemplateConfigPersistenceTests` and `InitCommandTests` now expect Go without the flag, with 6 new tests. One of them, `GetAvailableLanguagesAsync_ListsLanguagesInPickerOrder`, pins the picker order so a future ungated language fails a unit test before it breaks a Docker E2E run. - **Docs:** `docs/specs/polyglot-apphost.md`, the Go code generation README and `playground/GoAppHost/apphost.go` no longer mention the flag. Fixes #21106 ## Validation | Command | Result | | --- | --- | | `.\restore.cmd` | passed | | `dotnet build tests\Aspire.Cli.EndToEnd.Tests\Aspire.Cli.EndToEnd.Tests.csproj -m:2 --no-restore` | 0 warnings, 0 errors | | `dotnet test --project tests\Aspire.Cli.Tests\Aspire.Cli.Tests.csproj --no-launch-profile -- --filter-class "*.DefaultLanguageDiscoveryTests" --filter-class "*.NewCommandTests" --filter-class "*.ConfigCommandTests" --filter-class "*.NewCommandTemplateConfigPersistenceTests" --filter-class "*.InitCommandTests" --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"` | 312 passed, 0 failed (run before the picker-order test was added) | | `dotnet test --project tests\Aspire.Cli.Tests\Aspire.Cli.Tests.csproj --no-launch-profile -- --filter-class "*.DefaultLanguageDiscoveryTests" --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"` | 39 passed, 0 failed (38 existing plus the picker-order test) | | `node extension\scripts\generate-schema.js` | only the Go block was removed from each schema | | repository-wide search for `experimentalPolyglotGo` and its variants | 0 hits | | JSON parse of the 53 fixtures and 3 schemas | all parse, none contain the flag | Run in PR CI instead of locally, all passing on `0e65752d44`: the three Java E2E tests (`JavaPolyglotTests`, `JavaCodegenValidationTests` and `JavaPolyglotApphostDirectoryTests`; they need Docker, so CI was the first execution of the extra Down press), `Infrastructure.Tests` and the VS Code extension unit and E2E jobs. Not run: the VS Code new-project flow in an extension host (`extension/src/commands/new.ts` just runs `aspire new`). ### Before merging - [ ] Search again for `experimentalPolyglotGo`, `KnownFeatures.ExperimentalPolyglotGo`, `EnableExperimentalGoSupportAsync`, `features__experimentalPolyglotGo` and `experimentalPolyglot:go`, because open Go branches add flag setup of their own and this PR must merge after all of them. Today that covers #21189 (the `EnableExperimentalGoSupportAsync` helper and its call sites in every Go scenario), #21195 (`SdkExportCommandTests` sets `KnownFeatures.ExperimentalPolyglotGo`) and #21196 (`GoAppHostToolingCheckTests` builds its language discovery with the flag, including a `goEnabled: false` case that stops being meaningful, and `GoAppHostToolingCheck` has a comment that describes the flag). Each of them stops compiling or goes stale once the flag is removed, so delete or rewrite them here. - [ ] Rebase onto #21191, which adds `DefaultHostingPackages = ["Aspire.Hosting.Go"]` to Go's `LanguageInfo` in the same hunk of `DefaultLanguageDiscovery.cs` (#21045). Keep that initializer and drop `IsExperimental: true`. - [ ] Drop the Go experimental notes in the aspire.dev API reference and docs at the same time. ## Checklist - Is this feature complete? - [ ] Yes. Ready to ship. - [x] No. Follow-up changes expected. - Are you including unit tests for the changes and scenario tests if relevant? - [x] Yes - [ ] No - Did you add public API? - [ ] Yes - If yes, did you have an API Review for it? - [ ] Yes - [ ] No - Did you add `<remarks />` and `<code />` elements on your triple slash comments? - [ ] Yes - [ ] No - [x] No - Does the change make any security assumptions or guarantees? - [ ] Yes - If yes, have you done a threat model and had a security review? - [ ] Yes - [ ] No - [x] No --------- Co-authored-by: David Pine <7679720+IEvangelist@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c8717a87-0ee5-4296-a6fd-ea6e2d91a57e
## Description Polyglot AppHosts can now author custom Kubernetes manifests with nested objects and arrays, including CiliumNetworkPolicy ingress rules and selectors whose label keys contain dots and slashes. Remove the scalar-only `AspireUnion` restriction from `KubernetesManifestResource.WithField`. The CLR signature remains `WithField(string, object)` and the existing normalization/YAML paths handle the structured values. JSON numbers now use the same finite/range checks as CLR numeric values, including inside nested objects and arrays. Dot-separated paths and last-write-wins replacement are unchanged. Supply dotted keys inside a map subtree rather than adding path escape syntax. Manifest metadata still uses dedicated APIs. Part of #18080: addresses the remaining structured-manifest limitation (bug 2). The Go Configure callback panic and integer union handling (bug 1) were already fixed by merged #21188 and are not changed here. ### User-facing usage ```typescript await api.publishAsKubernetesService(async (service) => { const policy = await service.addManifest("cilium.io/v2", "CiliumNetworkPolicy", "api-netpol"); await policy.withField("spec.endpointSelector.matchLabels", { "app.kubernetes.io/component": "api" }); await policy.withField("spec.ingress", [{ fromEndpoints: [{ matchLabels: { "app.kubernetes.io/component": "frontend" } }] }]); }); ``` C# AppHosts continue to use `AdditionalResources` with a `BaseKubernetesResource` subclass for typed custom resources; no public CLR API is added. ### Generated SDK compatibility Focused snapshots cover the real Kubernetes capability in all five generators: Go `any` without the scalar union guard, TypeScript `any`, Python `typing.Any`, Java `Object`, and Rust `&Value`. Java's transport still unwraps existing `AspireUnion` arguments, so existing source calls remain usable after regeneration. The CLR signature and capability ID are unchanged. No generator implementation, shared runtime, or release API baseline files are edited. ### Validation - `.\restore.cmd` - passed; repository-wide restore reported 17 NuGet vulnerability warnings. - `dotnet test --project tests\Aspire.Hosting.RemoteHost.Tests\Aspire.Hosting.RemoteHost.Tests.csproj --no-launch-profile -- --filter-class "*.KubernetesManifestTests" --filter-class "*.CapabilityDispatcherTests" --filter-class "*.AtsMarshallerTests" --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"` - passed, 278 tests. - `dotnet test --project tests\Aspire.Hosting.RemoteHost.Tests\Aspire.Hosting.RemoteHost.Tests.csproj --no-launch-profile -- --filter-class "*.KubernetesManifestTests" --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"` - passed, 10 tests after normalizing trailing whitespace in generated snapshots. - `dotnet test --project tests\Aspire.Hosting.Kubernetes.Tests\Aspire.Hosting.Kubernetes.Tests.csproj --no-launch-profile -- --filter-class "*.KubernetesPublisherTests" --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"` - passed, 74 tests. - `dotnet test --project tests\Aspire.Hosting.RemoteHost.Tests\Aspire.Hosting.RemoteHost.Tests.csproj --no-launch-profile -- --filter-method "*.WithField_DispatchesStructuredValuesToPublishedYaml" --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"` with the original scalar union temporarily restored - failed as expected with `requires handle of type 'String | Double | Boolean', got 'object'`. The fix was restored before the passing runs. - `git diff --check` - passed. The behavioral test dispatches real `addManifest`/`withField` capabilities through publishing and snapshots CiliumNetworkPolicy YAML: arrays of objects, dotted/slashed keys, empty arrays, booleans, replacement semantics, and fluent return identity. Rejection tests cover resource handles, nested non-finite JSON numbers, invalid dictionary keys, unsupported CLR objects, and non-finite CLR numbers, including preservation of existing fields on failure. Evaluated MSBuild items/project references represent the new tests, snapshots, shared initializer, and Kubernetes dependency; no loose-input routing or workflow changes are needed. ### Live Go Preview validation Used the Linux x64 CLI archive and portable `built-nugets` from [CI run 42116251464](https://github.com/microsoft/aspire/actions/runs/42116251464), both matching head `3603a49c38bd7de8336c971cc16352992ffe4669`. CLI version: `17.0.0-pr.21824.g3603a49c`. Ran in the existing `aspire-go-e2e-6972:latest` Docker image with Go `1.26.10`, GCC `13.3.0`, and a live Docker engine `29.9.2`. No repository fixtures or shared Go runtime files were changed. Exact commands inside the container (all passed): ```bash /state/pr-cli/aspire new aspire-go-empty --name ManifestPreview --output /workspace/live-manifest --version 17.0.0-pr.21824.g3603a49c --source /workspace/portable-packages/Debug/Shipping --localhost-tld false --suppress-agent-init --use-config-source false --non-interactive # Added Aspire.Hosting.Kubernetes 17.0.0-pr.21824.g3603a49c to the scratch aspire.config.json. cd /workspace/live-manifest /state/pr-cli/aspire restore --apphost apphost.go --non-interactive go build ./... (cd .aspire/modules && go build ./...) /state/pr-cli/aspire publish --apphost apphost.go --output-path /workspace/published --non-interactive --include-exception-details python3 /workspace/verify-output.py ``` The compile covers the complete generated Go module, not only the focused projection snapshot. Its generated `WithField(path string, value any)` accepts native Go maps/slices. The scratch AppHost uses `PublishAsKubernetesService` and the `AddManifest` Configure callback to author a CiliumNetworkPolicy. Publishing succeeded with **10/10 pipeline steps**; a complete-output equality assertion verified `templates/api/policy.yaml`, including nested endpoint/port arrays, `app.kubernetes.io/component` and `app.kubernetes.io/name` keys, `egress: []`, `ingress: true`, and integer `revision: 3`. The initial attempt without explicit version/source selected stable `13.6.4` and failed because `Aspire.Hosting.Go` has no matching stable package; the successful run above uses explicit matching PR artifacts. This does not claim default stable Go scaffolding is validated. Not run: live TypeScript/Python/Java/Rust AppHosts or generated SDK compilation in those languages, Helm rendering, or deployment to a Kubernetes/Cilium cluster. Negative-value coverage remains the automated production dispatch/CLR tests, not live Go rejection scenarios. Callback-release and disconnect/`os.Exit` lifecycle redesigns are outside this change and this validation makes no new lifecycle guarantees. ## Checklist - Is this feature complete? - [x] Yes. Ready to ship. - [ ] No. Follow-up changes expected. - Are you including unit tests for the changes and scenario tests if relevant? - [x] Yes - [ ] No - Did you add public API? - [ ] Yes - If yes, did you have an API Review for it? - [ ] Yes - [ ] No - Did you add `<remarks />` and `<code />` elements on your triple slash comments? - [ ] Yes - [ ] No - [x] No - Does the change make any security assumptions or guarantees? - [ ] Yes - If yes, have you done a threat model and had a security review? - [ ] Yes - [ ] No - [x] No Co-authored-by: IEvangelist <7679720+IEvangelist@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Note
Merge order: no PR must land before this one. It overlaps two open PRs, and whichever lands second rebases:
AtsGoCodeGeneratorTests.cs.GetUnionAllowedTypesinAtsGoCodeGenerator.csand two snapshots. Keep both sets of helpers and re-accept the snapshots withdotnet verify accept -y.Description
A Go AppHost that passes an options struct holding a callback panics when the options are merged, for example
AddManifestOptions{Configure: ...}from #18080:Every generated Go method that takes
...*XxxOptionsmerges its options withdeepUpdatebefore serializing them.deepUpdatemerged through a JSON round trip and discarded the marshal error.encoding/jsoncannot marshal afuncvalue, even a nil one, so any options struct with a callback field failed, and*CancellationToken(json:"-") was silently dropped. This is not specific toAddManifest: the same path breaks every Configure-style option in the generated SDK (KubernetesAddManifestandWithHelm, the Azure Container Apps and App ServicePublishAs*options,InteractionProgressOptionsand the validation callbacks). The polyglot validation job onlygo builds those fixtures, so none of these paths was ever executed.deepUpdatemerges by reflection (Resources/base.go). Struct fields and map entries present on both sides merge recursively. Any other value, including nil, is taken from the later option, as before. A struct with unexported fields (such as*CancellationToken) is opaque and is taken whole. Neither argument is modified. For a small options struct this is about 6-15x faster than the JSON round trip, with about 10x fewer bytes and 17-33x fewer allocations (Go 1.27 micro-benchmark).AtsGoCodeGenerator.cs). Go boxes the constant3asint, but the generated union guard for a parameter that allows an ATS number only acceptedfloat64, soWithField(path, 3)failed withparameter "value" must be one of [string, float64, bool], got int. The type switch now accepts every Go numeric type whenfloat64is a union member. They all serialize as JSON numbers. Without this the publish in Go AppHost: AddManifest is unusable — Configure option panics on publish, and WithField can't express nested/dotted CRD fields #18080 could not set an integer such asmaxReplicaCount: 3, even after the panic was fixed.Part of #18080. This fixes Bug 1 only, so it does not close the issue.
Not in this PR
Bug 2 of #18080 (
WithFieldonly accepts scalar values and dot-notation paths, so nested specs and keys that contain dots, such asapp.kubernetes.io/component, cannot be expressed) is not Go-specific.KubernetesManifestResource.WithFieldis limited by[AspireUnion(string, double, bool)], which the server enforces for every language, whileNormalizeManifestValuealready handles arrays and objects. The proposal is to drop the union sovalueisany, with an additivewithFieldJson(path, json)overload as the fallback. That is an API-shape decision for every polyglot SDK and needs API review and a dispatch test, so it is left out here.Behavior notes
time.Durationare still rejected by the union guard. Only the predeclared numeric types are accepted.Validation
.\restore.cmddotnet test --project tests\Aspire.Hosting.CodeGeneration.Go.Tests\Aspire.Hosting.CodeGeneration.Go.Tests.csproj --no-launch-profile -- --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"The new
GeneratedRuntime_PassesGoTestswrites the generated runtime to a temp directory, adds Go tests for the merge semantics and the numeric union guard, and runsgo teston it. It uses the same[RequiresTools(["go"])]gate as the existingGeneratedTransport_SupportsDuplexCommunication. A Go-only test export (GoTestExtensions.cs,WithScalarField) exercises the union guard through the real generator. Two Verify snapshots are refreshed.Red/green: with the generator from before this change,
GeneratedRuntime_PassesGoTestsfails with 11 rejections ofgot int.End to end with a real Go AppHost, using a Debug-built CLI with
ASPIRE_REPO_ROOTpointing at this branch. The AppHost callsAddManifestwith aConfigurecallback that sets a label, an annotation, a string field andWithField("spec.maxReplicaCount", 3):aspire publishexits 0 with 10/10 pipeline steps and writestemplates\kube-service\scaler.yaml, akeda.sh/v1alpha1ScaledObjectwith the label, annotation andmaxReplicaCount: 3as an integerbase.gofrom before this changepanic: aspire: merge error: unexpected end of JSON inputfromdeepUpdateviaAddManifest, "The connection to the AppHost was lost",aspire publishexits 6Not verified:
Checklist
<remarks />and<code />elements on your triple slash comments?