Repository navigation
Build the CLI before MCP initialization in McpDocsE2ETests - #21450
Conversation
The outerloop Cli.EndToEnd job installs the CLI from an archive and never builds src/Aspire.Cli, so the first test's 'dotnet run --project' compiled the CLI inside McpClient.CreateAsync's 60s initialization timeout. Build the CLI once in a class fixture and launch it with --no-build. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 21450Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 21450" |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are recorded for the change moving CLI compilation outside the MCP initialization timeout.
No open issues 🎉
What changed in this PR
Moves CLI compilation outside the MCP initialization timeout to address intermittent failures in the documentation end-to-end tests.
Changes:
- Adds a class fixture that builds the CLI once and reports build output on failure.
- Launches each MCP server with
dotnet run --no-build, keeping compilation outside the handshake window. - Preserves existing assertions, timeouts, and retry behavior.
| File | Description |
|---|---|
| tests/Aspire.Cli.EndToEnd.Tests/McpDocsE2ETests.cs | Adds the shared CLI build fixture and skips compilation during MCP startup. |
Review effort: Balanced
Exact-head outerloop validation (head
|
|
Follow-up on outerloop run 41050874298, latest attempt 4, head
These are existing main breaks rather than flakes, so I haven't rerun anything. Required PR CI on this head is all green, with 0 unresolved threads. 🤖 Posted by AspireAgent (AI Agent) |
|
Outerloop triage for head
Main's scheduled Outerloop run has been red on all 14 of the last 14 daily runs, and each of those already went to attempt 4. I didn't rerun these jobs because the failures are deterministic on main, so another attempt can't go green until the owning PRs merge. CI ( 🤖 Posted by AspireAgent (AI Agent) |
James Newton-King (JamesNK)
left a comment
There was a problem hiding this comment.
Found 1 test-coverage issue: the MCP docs E2E tests rebuild and run the CLI from source instead of exercising the prebuilt CLI artifact.
Replace the source-build fixture with one that installs the CLI selected by CliInstallStrategy.Detect() into an isolated temp prefix using the existing get-aspire-cli*.sh acquisition scripts, then launches that executable with `agent mcp`. On CI, ASPIRE_E2E_CLI_ARCHIVE_DIR selects LocalArchive, so the tests exercise the packaged native CLI for the commit and no longer compile src/Aspire.Cli. The installed version is verified against the strategy's expected version, mirroring VerifyAspireCliVersionAsync. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Tests selector1 / 100 PR test projects · 0 PR jobs, from 1 changed file. Selected PR test projects (1 / 100)
Selected PR jobs (0)none How these were chosen — grouped by what changed🧪 Job reasonsnone Selection computed for commit |
|
Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt. |
|
✅ No documentation update needed. docs_optional → test_only. Triggered signal: |
Description
Fixes the intermittent outerloop failure of
Aspire.Cli.EndToEnd.Tests.McpDocsE2ETests.GetDoc_WithInvalidSlug_ReturnsError. It fails withSystem.TimeoutException: Initialization timed outfromMcpClient.CreateAsyncinInitializeAsync.Denominator
Outerloop Tests(tests-outerloop.yml) onmain, Oct 16–30 2026.Root cause
InitializeAsyncstarted the server withdotnet run --project src/Aspire.Cli/Aspire.Cli.csproj -- agent mcp. The Cli.EndToEnd job installs the CLI from an archive and never buildssrc/Aspire.Clifrom source; the test project doesn't reference it. So the first test in the class compiles the whole CLI before the server can answer the MCPinitializerequest, all insideMcpClient's 60s initialization timeout.TRX from 6 attempts across the two runs above:
GetDoc_WithInvalidSlug_ReturnsError) took 59.5s–68.6s, right at the limit.Fix
dotnet runthe CLI from source. A class fixture installs the CLI selected byCliInstallStrategy.Detect()into an isolated temp prefix using the existingget-aspire-cli*.shacquisition scripts (with--skip-path --skip-completions --skip-extension), then each test launches thataspireexecutable withagent mcp.ASPIRE_E2E_CLI_ARCHIVE_DIRselectsLocalArchive, so the tests exercise the packaged native CLI built for the commit.PullRequestandInstallScriptmodes are supported the same way; host-mutating or Docker-only modes fail with a clear message.aspire --versionagainst the strategy's expected version, mirroringVerifyAspireCliVersionAsync. Install happens beforeMcpClient.CreateAsync, so the 60s timeout covers only the MCP handshake.Validation (local, linux-x64)
a76b57a9, cold CLI build state)ListTools_IncludesDocsToolsfailed in 1m 05.3s:System.TimeoutException : Initialization timed out(same signature as CI)LocalArchivewith this PR'scli-native-archives-linux-x64CI artifactsrc/Aspire.Cliwas not built.PullRequestmode (GITHUB_PR_NUMBER=21450)Aspire.Cli.*.nupkgAspire CLI version mismatchas intendedInstallScriptmode couldn't be exercised locally becauseaka.msdoesn't resolve in my environment.Fixes: no tracking issue exists; found by scanning the scheduled outerloop runs.
Checklist
🤖 Posted by AspireAgent (AI Agent)