tls: teardown skips its flush when the application left one in flight - #235
Merged
Merged
Conversation
TlsConnectionDualPipe.DisposeAsync flushes the connection unconditionally, because a handler that wrote its response and never flushed has nothing else to carry it out. Against a flush the application left in flight that call met TcpConnection.FlushAsync's one-flush-at-a-time guard and threw out of a finally, surfacing as "[r0] connection handler faulted: FlushAsync already in progress." - reported on a long-lived TLS websocket, a few times per run (#234). The guard is right and stays. The parked caller waits on _flushSignal, a single ManualResetValueTaskSourceCore that holds one continuation, so a second flush is not something the connection can serve; an application that flushes twice concurrently needs to hear about it. What is wrong is the caller: disposal is the connection's own flush, not the application's, and it has no business tripping a guard aimed at application misuse. So FlushAsync keeps the throw and grows FlushIfIdleAsync beside it, which returns instead. Skipping sends nothing because there is nothing to send: while _flushInProgress is set every GetSpan/GetMemory/Advance/Write is refused, so nothing can have entered the slab since the in-flight flush armed, and that flush snapshotted everything already there. The in-flight send owns the whole slab; a second flush would compute target == 0 and return anyway. The regression test parks a real flush the way the existing contract test does - a peer that stops draining - and stages nothing before tearing down, which is the production shape: every application write was flushed, so Complete has nothing to commit and the disposal's own flush is the only call left that can throw. Without the fix it fails with the reported message. Not covered: the same disposal throws a line earlier, out of Complete, when plaintext IS staged - GetSpan refuses it and Complete catches only IOException. That one is already tracked as the pending test above this one, and needs its own call about whether those bytes are dropped or waited for, since anything appended to the slab during an in-flight flush is zeroed by CompleteFlush. TLS suite 142 passed / 0 failed / 7 pending; Unit, Http, Chaos, File and E2E green. Tls/OpenSsl -0.6%, Tls/OpenSslPipes +1.6% at 2 reactors - inside noise.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #234.
What was happening
TlsConnectionDualPipe.DisposeAsyncflushes the connection unconditionally:That is load-bearing — a handler that wrote its response and never flushed has nothing else to carry it out (
WriterContractTestscovers exactly that shape). But against a flush the application left in flight, it met the one-flush-at-a-time guard inTcpConnection.FlushAsyncand threw out of afinally, arriving as:The
_closedearly-out cannot save it: the read side is what callsMarkClosed(), and it runs in thefinallyafter this flush. TLS-only, as reported —TcpConnectionDualPipehas no disposal at all, andHopDuplexPipedrains its send pump before closing.The call, and why it went this way
The guard is not stylistic and it stays. The parked caller waits on
_flushSignal, oneManualResetValueTaskSourceCorewith room for a single continuation — a second flush is not something the connection can serve, and an application flushing twice concurrently has a bug it needs to hear about.What was wrong is the caller. Disposal is the connection's own flush, not the application's, so it should not be tripping a guard aimed at application misuse.
FlushAsynckeeps the throw;FlushIfIdleAsyncreturns instead, and disposal calls that.Skipping sends nothing because there is nothing to send — a property of the write path, not a hope. While
_flushInProgressis set everyGetSpan/GetMemory/Advance/Writeis refused (TcpConnection.Write.cs), so nothing can have entered the slab since the in-flight flush armed, and that flush snapshotted everything already there. The in-flight send owns the whole slab; a second flush would computetarget == 0and return anyway.Considered and rejected: coalescing — returning the outstanding
ValueTaskto the second caller — which cannot work, since that hands a second awaiter the same token and the source throws on the secondOnCompleted. And quiescing (waiting the in-flight flush out before disposal's own) is the only ordering that would actually deliver bytes staged during it, but the thing that releases a flush parked on a peer that stopped draining is theMarkClosedunderneath it, so an unbounded wait turns this fault into a teardown hang.Test
tls writer: disposal does not throw while the connection's flush is in flightparks a real flush the way the existing contract test does — a peer that stops draining, 256 KB at a time — and stages nothing before tearing down. That is the production shape: every application write was flushed, soCompletehas nothing to commit and the disposal's own flush is the only call left that can throw. It asserts non-vacuity both ways (a flush genuinely in flight, and nothing staged).Proven by reverting the fix:
With it:
142 passed, 0 failed, 5 skipped, 7 pending. Unit, Http, Chaos, File and E2E green.Tls/OpenSsl-0.6%,Tls/OpenSslPipes+1.6% at 2 reactors — inside the noise floor, as expected for a teardown-path change.Not covered
The same disposal still throws a line earlier, out of
Complete, when plaintext is staged: the commit reachesTcpConnection.GetSpan, which refuses it, andCompletecatches onlyIOException. That is already tracked as the pending test directly above the new one, and it wants its own decision — those bytes cannot be delivered from that state regardless, since anything appended to the slab during an in-flight flush is zeroed byCompleteFlush, so the choice is to drop them quietly or to quiesce first and accept the hang risk. Left out of this PR deliberately.