Repository navigation
Conversation
In TunnelHTTPConnection.handle_request() (both _sync and _async), a proxy CONNECT that succeeds leaves the underlying stream registered on self._connection. If the subsequent stream.start_tls() call then raises (e.g. a certificate/handshake failure), that exception was previously left to propagate without closing self._connection first, unlike the CONNECT-status-check branch a few lines above it, which already does this correctly on a non-2xx CONNECT response. Since self._connection already completed one successful request (the CONNECT) and self._connected stays False, it keeps reporting itself as ACTIVE with Request Count: 1 to the pool - an accurate state, since nothing told it the tunnel is dead. The pool's own exception handling in handle_request() only removes the failed *request* from its queue, not the connection from self._connections, so it has no other way to notice. Each failed handshake permanently consumes a connection-pool slot; once failures reach max_connections, unrelated later requests raise PoolTimeout instead of the real ConnectError. Wrap start_tls() the same way the CONNECT status-check block already does: close the connection on any exception before re-raising. Adds a regression test (sync + async) using a MockStream whose start_tls always fails, asserting the pool has zero connections after each of several consecutive failures with a small max_connections. Verified it reproduces the exact discussion pydantic#1175 bug when the fix is reverted (a stale ACTIVE connection is left behind), and passes with it applied. ruff check/format and mypy clean on the touched files; full tests/httpcore2 suite passes except three pre-existing, unrelated keepalive_expiry timing flakes that also fail identically on main without this change.
Merging this PR will not alter performance
Comparing Footnotes
|
There was a problem hiding this comment.
All reported issues were addressed across 4 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
…nal exception - Wrap the cleanup close in AsyncShieldCancellation (sync: ShieldCancellation) so a Trio cancellation during exception handling can't interrupt it and leave the TLS socket open, matching the existing pattern used elsewhere in this codebase for exception-path cleanup. - Swallow a failure from the close itself so it can't mask the original TLS handshake exception callers are expecting. Regenerated the sync files via scripts/unasync.py. Added test_proxy_tunneling_with_tls_failure_and_failing_close: makes the close raise and asserts the original ConnectError still propagates. Fails on the pre-fix code (raises the close's RuntimeError instead).
Restores the 08cbebd (proxy SSL context) item that only lived on the old local branch, marks pydantic#1069 and the NO_PROXY CIDR work as landed, and flags overlaps with pydantic#1226, pydantic#1270, pydantic#1211, pydantic#1174 and pydantic#1169.
|
I had the same fix open as #1010 (port of encode/httpcore#1049) and have closed it in favour of this one, since this one also handles cancellation and has the better regression test. One nit: the cleanup swallows |
|
We hit this in production. A TLS reset during the handshake through our HTTP proxy (tinyproxy) left the tunnel in the pool as ACTIVE. Our client polls over a pool of one connection (python-telegram-bot’s long polling), so every later request failed with PoolTimeout until we restarted the process. Reproduced on httpcore2 2.13.1 and httpcore 1.0.9, and closing the proxy connection when start_tls fails, as this PR does, fixes it in our repro. Would be great to get this merged. |
Fixes #1175.
The bug
In
TunnelHTTPConnection.handle_request()(_sync/_asynchttp_proxy.py), a proxyCONNECTthat succeeds leaves the underlying stream registered onself._connection. If the subsequentstream.start_tls()call then raises (e.g. a certificate/handshake failure), the exception was left to propagate without closingself._connectionfirst — unlike the CONNECT-status-check branch a few lines above it, which already does exactly this on a non-2xx CONNECT response:self._connectionalready completed one successful request (the CONNECT) andself._connectedstaysFalse, so it keeps accurately reporting itself asACTIVE,Request Count: 1to the pool — nothing ever told it the tunnel died. The pool's own exception handling inhandle_request()only removes the failed request from its queue, never the connection fromself._connections, so it has no other way to notice. Each failed handshake permanently consumes a connection-pool slot; once failures reachmax_connections, unrelated later requests raisePoolTimeoutinstead of the realConnectError.The fix
Wrap
start_tls()the same way the status-check block already does:Applied to both
_sync/http_proxy.pyand_async/http_proxy.py(they're kept in sync viascripts/unasync.py, which I re-ran after editing the async source).Testing
Added
test_proxy_tunneling_with_tls_failureto bothtests/httpcore2/_sync/test_http_proxy.pyandtests/httpcore2/_async/test_http_proxy.py, following the existingMockStream/MockBackendpattern used bytest_proxy_tunneling_with_403: aFailingTLSStreamwhosestart_tlsalways raisesConnectError, run three times against a pool withmax_connections=2, assertingnot proxy.connectionsafter every failure.ACTIVETunnelHTTPConnectionleft inproxy.connections.ConnectError→ConnectError→PoolTimeoutsequence when reverted, andConnectError×3 with an empty pool after each, when fixed.ruff check/ruff format --checkandmypyclean on all four touched files.tests/httpcore2suite: 235 passed, only 3 pre-existingkeepalive_expiry=0.0timing flakes remain, which I confirmed also fail identically on unmodifiedmain(unrelated to this change).scripts/unasync.py --checkpasses (sync/async stay in lockstep).