You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Bound GitRepository's post-exit output drain so git commands can't hang - #21562
GitRepositoryTests time out after 30s on macOS (#21141, #20729). In every failing log I checked, Aspire.Cli.Git.GitRepository logs Listing git-included files under: … and then never logs git ls-files returned ….
Root cause
#17079 moved GitRepository to Process.RunAndCaptureTextAsync. That API waits for EOF on stdout and stderr before it waits for the process to exit, and it has no bound. EOF only arrives when every copy of the pipe write ends is closed. That doesn't happen if another process holds an inherited copy. Two ways that can happen:
a hook or helper that git spawns and that outlives git, or
a process spawned concurrently that inherits the handles. On macOS the runtime creates pipes with pipe() followed by a separate fcntl(FD_CLOEXEC), which leaves a window where this can happen.
In either case GetIncludedFilesAsync and GetRootAsync hang forever even though git has already exited. The repo documents the same hazard in ProcessCaptureRunner, which already bounds its post-exit drain for this reason. Before #17079, GitRepository read output concurrently with WaitForExitAsync instead of requiring EOF first.
All occurrences I found are after #17079 merged (2026-10-01).
Fix
GitRepository.RunGitAsync now:
waits for git to exit, then bounds the output drain to 2s (the same bound ProcessCaptureRunner uses). In the normal case git closes its pipes on exit and the drain completes immediately.
uses the complete stdout if only stderr is held open.
returns null if stdout itself is held open, because the output may be truncated. null already means "git acceleration unavailable", so callers fall back to the filesystem walker.
still kills git on cancellation, as before.
reads each pipe's raw bytes and decodes them with Encoding.GetString, like Process.RunAndCaptureTextAsync. This keeps a leading U+FEFF in the first path; StreamReader.ReadToEndAsync would strip it.
There are no retries, no sleeps and no test timeout changes.
I didn't reuse ProcessCaptureRunner. On drain expiry it returns an empty capture with exit code 0, which ls-files would read as "zero files", and #21556 is currently changing that code.
Regression test
GetIncludedFilesAsync_DescendantHoldsInheritedPipe_ReturnsAfterGitExits (POSIX only) sets core.fsmonitor to a hook script. git runs the hook during ls-files, and the hook leaves sleep 300 >/dev/null & behind, which holds git's stderr pipe after git exits. The test kills the leftover processes in finally.
GetIncludedFilesAsync_DescendantHoldsStdoutAfterGitExits_ReturnsNull (POSIX only) covers the case where stdout is held. It uses a fake git, through a new optional internal gitExecutable constructor parameter that defaults to "git". The fake git writes partial output, leaves a process holding stdout and exits 0. The test asserts that the method returns null (fall back to the walker) instead of hanging or returning a partial list.
GetIncludedFilesAsync_FirstPathStartsWithByteOrderMark_PreservesPath (POSIX only) checks that decoding still matches the previous path. Before the fix it fails because the first path comes back with the U+FEFF stripped.
Before the fix (local DefaultTimeout is 5s; 30s in CI). The failure has the same log signature as CI:
failed GitRepositoryTests.GetIncludedFilesAsync_DescendantHoldsInheritedPipe_ReturnsAfterGitExits (5s 048ms)
System.TimeoutException : The operation at .../GitRepositoryTests.cs:184 timed out after reaching the limit of 5000
[09:09:13] GitRepository Debug: Listing git-included files under: /tmp/Aspire.Cli.Tests/Workspace/znlkum2r.2cs
[09:09:18] GitRepository Debug: git ls-files returned 2 files ... <- only after the test's finally killed the leftover process
Both new tests time out like this on the old code (5s TimeoutException).
After the fix:
Both new tests pass. The stderr test passes in about 3s and returns git's file list.
GitRepositoryTests, GitCommandRunnerTests, GitClientTests and AppHostCandidateFinderTests: 94 passed, 0 failed.
A bounded loop of 5 runs of GitRepositoryTests passed 5/5.
No leftover sleep processes after the runs.
Related
These show the same Listing git-included files signature with no returned line, through ambient AppHost discovery. They aren't closed here, so they can be confirmed separately: #20940, #20911, #20737 (and likely #20939).
GitRepository ran git through Process.RunAndCaptureTextAsync, which waits for EOF
on stdout and stderr with no bound before it waits for exit. If any other process
holds an inherited copy of git's pipe write ends (a hook or helper git spawned, or
a process spawned concurrently that inherited the handles), EOF never arrives and
GetRootAsync/GetIncludedFilesAsync hang even though git has exited.
Wait for git to exit, then bound the output drain to 2s. If only stderr is held
open, the complete stdout is still used; if stdout is held open, return null so
callers fall back to the filesystem walker instead of using a partial list.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
The regression test covers only stderr remaining open, leaving the bounded null-return fallback for open stdout unverified and partial results potentially able to silently omit AppHosts.
This PR prevents CLI Git discovery from hanging when inherited output pipes remain open after Git exits, addressing the macOS timeouts reported in #21141 and #20729.
Changes:
Bounds post-exit output draining to two seconds while preserving cancellation handling.
Keeps complete stdout when only stderr remains open; returns null for incomplete stdout so discovery can use its filesystem fallback.
Adds a POSIX regression test with a descendant holding stderr open and cleanup of leftover processes.
Add an internal git executable override so a test can run a fake git that exits
while a descendant still holds its stdout pipe, and assert GetIncludedFilesAsync
returns null within the bounded wait instead of hanging or returning a partial list.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
The redirected output and error readers are not explicitly disposed, leaving two pipe handles per invocation awaiting finalization.
No open issues 🎉
1 previously missed issue
In code that hasn't changed since last review
Dispose redirected output readers after process execution
src/Aspire.Cli/Git/GitRepository.cs:161
Please dispose both redirected readers explicitly: Process.Dispose() does not close externally accessed StandardOutput and StandardError readers, and cancelling readCts only cancels their reads. Each invocation therefore leaves two pipe handles awaiting finalization, even on success. Scope both readers with using so they are closed after the finally block cancels pending reads.
Process.Dispose doesn't close redirected streams once they've been accessed, so
dispose the readers explicitly to release the pipe handles promptly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot's review on 14b13f5 pointed out that the redirected readers weren't disposed. That's correct: once StandardOutput/StandardError are accessed, Process.Dispose no longer closes them. Fixed in bd3018e by disposing both readers explicitly. Locally, GitRepositoryTests, GitCommandRunnerTests, GitClientTests and AppHostCandidateFinderTests pass (94 passed, 0 failed), with no leftover processes.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
StandardOutput strips a leading U+FEFF from the first Git-listed filename, producing an incorrect authoritative path that breaks AppHost discovery and needs BOM-preserving decoding with a regression test.
No open issues 🎉
1 previously missed issue
In code that hasn't changed since last review
Preserve leading BOM in Git-emitted paths
src/Aspire.Cli/Git/GitRepository.cs:161
Process.StandardOutput reintroduces BOM stripping, so if the first path emitted by git ls-files -z starts with U+FEFF (for example, \uFEFFsrc/AppHost.csproj), that character is lost. The returned set then contains the wrong path, and AppHost discovery treats it as authoritative rather than falling back to the filesystem. Decode stdout without consuming a BOM or encoding preamble, and add a regression test that preserves this filename.
Closing a synchronous pipe handle on Windows can wait for a read that's still
pending on another thread, which is exactly the case when another process holds
git's pipe open. Dispose each reader from a continuation on its read instead, so
the handles are released promptly without ever blocking the caller.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cli.EndToEnd-StopNonInteractiveTests.StopAllAppHostsFromAppHostDirectory failed on attempt 1. The AppHost couldn't bind a fixed port: Failed to bind to address https://127.0.0.1:22001: address already in use. The github-actions bot reran it in attempt 2 and it passed.
VS Code extension E2E (Windows): command-palette and settings-files. Both attempt 1 jobs ended at the same moment, 47 minutes after starting, with no failed step and no log blob, which is the runner-loss signature. In attempt 2, command-palette passed. settings-files ended the same way, again after 47 minutes with no failed step and no log. Another run in the same window (41194399981) also had a Windows VS Code E2E job (debug-startup-timeout) hang for 43 minutes. On 14b13f5, the previous head of this PR, settings-files passed in about 4 minutes.
I couldn't completely rule this PR out, so I re-checked the reader disposal added in bd3018e and found a real Windows hazard. Process uses synchronous pipe handles on Windows. Closing a handle while another thread is blocked in ReadFile on it can wait for that read to finish, which is exactly what happens when another process holds git's pipe open. 898f403 now disposes each reader from a continuation on its read, so handles are still released promptly (immediately in the normal case) and disposal can never block the caller. Locally, the Git and AppHostCandidateFinder tests pass (94 passed, 0 failed). New CI is running on 898f403.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Human assessment is needed because a subtle stdout-decoding change can strip a leading U+FEFF from filenames and silently omit an AppHost from discovery.
StreamReader.ReadToEndAsync strips a leading byte order mark, but the
previous Process.RunAndCaptureTextAsync path decoded raw bytes. Read the
raw bytes and decode them with the same encoding so the first path from
git ls-files -z isn't changed.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Addressed the Copilot overview note on 898f403 in d728d53.
StreamReader.ReadToEndAsync strips a leading byte order mark, so a first git ls-files -z entry starting with U+FEFF would have lost that character. The previous Process.RunAndCaptureTextAsync path decoded the raw bytes with Encoding.GetString, which keeps it. RunGitAsync now copies the raw bytes from each pipe's BaseStream and decodes them with the same encoding, so behaviour matches the old path.
New test GetIncludedFilesAsync_FirstPathStartsWithByteOrderMark_PreservesPath:
Before: fails, because the first path comes back as AppHost.csproj with the U+FEFF stripped.
CI on 898f403: 194 checks passed. The Windows VS Code command-palette and settings-files jobs passed in about 4 minutes each, so the 47-minute hangs from bd3018e didn't recur. The only failure was VS Code extension E2E (Linux, edge-cases): three tests timed out after 180s waiting for AppHost resources or debug startup. The bot retried that job; d728d53 supersedes it.
CI on d728d53, run 41204248633, attempt 1: 192 passed and 3 failed. All three are known infrastructure flakes. The bot's automatic attempt 2 counts as the single rerun.
Aspire CLI Starter Validation (Ubuntu ARM64): the CLI found the AppHost and aspire start returned 0. After __dotnet-project-build finished, apiservice and webfrontend both went Starting -> FailedToStart, so aspire wait exited with 18. The other starter legs passed. This is the known CLI starter flake. It's in resource launch, not in the git file listing this PR changes.
AspireAgent stopped: the conflict with main is resolved locally, but pushing the merge is blocked because main's incoming changes touch .github/workflows and this token lacks the workflows permission.
The reason will be displayed to describe this comment to others. Learn more.
LGTM. The bounded post-exit drain prevents inherited Git pipes from hanging discovery while preserving complete stdout and falling back safely for incomplete output. The change is limited to internal CLI process handling, adds no public API, and preserves existing discovery behavior on normal and failure paths.
This branch has not been deployed
No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
area-clici-healthFixes CI flakiness or CI infrastructure (AspireAgent queue)
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
GitRepositoryTeststime out after 30s on macOS (#21141, #20729). In every failing log I checked,Aspire.Cli.Git.GitRepositorylogsListing git-included files under: …and then never logsgit ls-files returned ….Root cause
#17079 moved
GitRepositorytoProcess.RunAndCaptureTextAsync. That API waits for EOF on stdout and stderr before it waits for the process to exit, and it has no bound. EOF only arrives when every copy of the pipe write ends is closed. That doesn't happen if another process holds an inherited copy. Two ways that can happen:pipe()followed by a separatefcntl(FD_CLOEXEC), which leaves a window where this can happen.In either case
GetIncludedFilesAsyncandGetRootAsynchang forever even though git has already exited. The repo documents the same hazard inProcessCaptureRunner, which already bounds its post-exit drain for this reason. Before #17079,GitRepositoryread output concurrently withWaitForExitAsyncinstead of requiring EOF first.All occurrences I found are after #17079 merged (2026-10-01).
Fix
GitRepository.RunGitAsyncnow:ProcessCaptureRunneruses). In the normal case git closes its pipes on exit and the drain completes immediately.nullif stdout itself is held open, because the output may be truncated.nullalready means "git acceleration unavailable", so callers fall back to the filesystem walker.Encoding.GetString, likeProcess.RunAndCaptureTextAsync. This keeps a leading U+FEFF in the first path;StreamReader.ReadToEndAsyncwould strip it.There are no retries, no sleeps and no test timeout changes.
I didn't reuse
ProcessCaptureRunner. On drain expiry it returns an empty capture with exit code 0, whichls-fileswould read as "zero files", and #21556 is currently changing that code.Regression test
GetIncludedFilesAsync_DescendantHoldsInheritedPipe_ReturnsAfterGitExits(POSIX only) setscore.fsmonitorto a hook script. git runs the hook duringls-files, and the hook leavessleep 300 >/dev/null &behind, which holds git's stderr pipe after git exits. The test kills the leftover processes infinally.GetIncludedFilesAsync_DescendantHoldsStdoutAfterGitExits_ReturnsNull(POSIX only) covers the case where stdout is held. It uses a fake git, through a new optional internalgitExecutableconstructor parameter that defaults to"git". The fake git writes partial output, leaves a process holding stdout and exits 0. The test asserts that the method returnsnull(fall back to the walker) instead of hanging or returning a partial list.GetIncludedFilesAsync_FirstPathStartsWithByteOrderMark_PreservesPath(POSIX only) checks that decoding still matches the previous path. Before the fix it fails because the first path comes back with the U+FEFF stripped.Before the fix (local
DefaultTimeoutis 5s; 30s in CI). The failure has the same log signature as CI:Both new tests time out like this on the old code (5s
TimeoutException).After the fix:
GitRepositoryTests,GitCommandRunnerTests,GitClientTestsandAppHostCandidateFinderTests: 94 passed, 0 failed.GitRepositoryTestspassed 5/5.sleepprocesses after the runs.Related
These show the same
Listing git-included filessignature with noreturnedline, through ambient AppHost discovery. They aren't closed here, so they can be confirmed separately: #20940, #20911, #20737 (and likely #20939).Fixes #21141
Fixes #20729
Checklist
🤖 Posted by AspireAgent (AI Agent)