Skip to content

fix(git): drain upload-pack output while writing negotiation - #72

Merged
rolandjitsu merged 3 commits into
rolandjitsu:mainfrom
profiling:profiling/upload-pack-community-main
Nov 16, 2026
Merged

rolandjitsu merged 3 commits into
rolandjitsu:mainfrom
profiling:profiling/upload-pack-community-main

Conversation

@profiling

@profiling profiling commented Nov 13, 2026 •

Copy link
Copy Markdown

What kind of change does this PR introduce?

  • fix

Summary
Large upload-pack negotiations can fill stdin and stdout at the same time,
leaving clone/fetch requests blocked. Write the request while streaming stdout,
and keep the writer and child under one owner through cancellation and cleanup.
Write and child-exit errors reach the response body, including after stdout EOF.

Request writing now has a five-minute deadline. Response reads reset a
30-minute inactivity deadline; active transfers have no total duration limit
after the request is written. These limits are documented.

Also: git subprocesses that never need input no longer inherit stdin (a
test hung on a terminal), and prune_wants reports update-ref's exit
status when it stops reading early.

Fixes #71.

Tests

  • Added unit and integration coverage.

A real Git request with 30,000 haves times out before the fix and returns the
exact expected response after it. Tests also cover small v0/v2 requests, pipe
errors, deadlines, cancellation, early leader exit, buffered output and process
group cleanup. HTTP tests exercise FIN and RST with a blocked stdin writer.

All 189 tests pass on Linux. Non-Linux runtime was not tested.

Checklist

  • Local formatting, locked all-target/all-feature clippy with warnings denied,
    and locked all-feature tests pass.
  • Conventional Commit with the required Assisted-by trailer.
  • Preserves read-only, pull-only behavior.
  • Documentation updated for behavior changes.

Breaking change?
No API or configuration changes. Requests exceeding the new write or inactivity
limits now fail. Failures after headers terminate the response body.

@codecov

codecov Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.38%. Comparing base (e74fbc3) to head (2ed4286).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #72      +/-   ##
==========================================
+ Coverage   97.79%   98.38%   +0.58%     
==========================================
  Files           9        9              
  Lines        4083     5079     +996     
==========================================
+ Hits         3993     4997    +1004     
+ Misses         90       82       -8     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@profiling
profiling force-pushed the profiling/upload-pack-community-main branch from 2a347e1 to 2fb38ef Compare November 13, 2026 01:17
@profiling

Copy link
Copy Markdown
Author

The tests, lint, commit checks and both image builds pass. The remaining
codecov/project check reports 97.84% against 98.24%; patch coverage is 99.24%.

I checked the comparison: the base report is for 00b024b, using Rust 1.99.1;
this PR uses Rust 1.100.0. Codecov has no report for the actual base 859fab3.
The two intervening commits change only the version and documentation, but
the reports also differ in coverage mapping for unchanged source files.

With the same Rust 1.94.1 toolchain and full test suite, unique LCOV source-line
coverage is 3592/3657 (98.22%) on main and 4298/4367 (98.42%) with this PR.
These diagnostic results do not replace the current remote check.

Could the main coverage report be regenerated with the current CI toolchain
before assessing the project-level change? No coverage thresholds or exclusions
were changed. The follow-up adds behavior tests for owner loss, deadline error
delivery and child reaping; all 177 tests pass.

@profiling
profiling marked this pull request as draft November 13, 2026 23:47
@profiling
profiling marked this pull request as ready for review November 13, 2026 23:47

@rolandjitsu rolandjitsu left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! Mostly everything looks fine, just a couple of minor things.

Also, could you cleanup the commits?

Comment thread src/git.rs Resolved
Comment thread src/git.rs Resolved
profiling pushed a commit to profiling/git-cache-proxy that referenced this pull request Nov 14, 2026
Keep rolandjitsu#72, rolandjitsu#76 and rolandjitsu#77 with internal maintenance while deferring rolandjitsu#67.
Preserve the previous tree on its archive branch and in normal history.

Assisted-by: OpenAI:Codex
@profiling
profiling marked this pull request as draft November 15, 2026 16:23
@profiling
profiling force-pushed the profiling/upload-pack-community-main branch from 205fc39 to 469ae3e Compare November 15, 2026 17:11
@profiling

Copy link
Copy Markdown
Author

Updated, and the history is cleaned up: 4 commits rebased on main (the toolchain pin and its revert are gone, since they cancelled out).

While fixing the hang I also changed prune_wants to report update-ref's exit status when it exits before reading its input, instead of the broken pipe it caused (separate commit).

Each commit passes fmt, clippy and the tests on Linux. I could not run it on macOS, so it would be great if you could re-run the suite there.

@profiling
profiling force-pushed the profiling/upload-pack-community-main branch 2 times, most recently from 628b89b to 1d4460b Compare November 16, 2026 01:19
@profiling
profiling marked this pull request as ready for review November 16, 2026 02:46
ztzhou added 3 commits November 16, 2026 15:29
Write negotiation while streaming stdout to avoid pipe backpressure.
Own the writer and child together through cancellation and cleanup.

Assisted-by: Claude:claude-opus-4-8
Unlike std, tokio's output and spawn inherit stdin, so a child that
reads it blocks on a terminal or an open pipe.

Assisted-by: Claude:claude-opus-4-8
A child that exits before reading its input made the write fail with
a broken pipe that hid the real failure. Report the exit status first.

Assisted-by: Claude:claude-opus-4-8
@rolandjitsu
rolandjitsu force-pushed the profiling/upload-pack-community-main branch from 1d4460b to 2ed4286 Compare November 16, 2026 06:49
@rolandjitsu
rolandjitsu merged commit a44a197 into rolandjitsu:main Nov 16, 2026
13 checks passed
@profiling
profiling deleted the profiling/upload-pack-community-main branch November 16, 2026 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] upload-pack can deadlock while writing a large negotiation request

2 participants