tcp: idle and send-stall timeouts, so a connection finally has a clock

[MDA2AV]

Addresses the idle/keep-alive half of #97.

The gap

There was no clock anywhere in the TCP connection lifecycle — grep TickCount|LastSeen|Stopwatch across Connection/Tcp/ and Transport/Tcp/ returned nothing, and the only per-accepted-socket option set is TCP_NODELAY. Two shapes went unbounded:

A peer that goes quiet holds an fd, a pooled TcpConnection with its native write slab, and a recv queue for as long as it likes. In incremental mode also a registered buffer ring and a gid — capped at MaxConnections — so an idle connection at the cap converts directly into shed accepts.

A peer that stops reading is worse. Its window shuts, the socket send buffer fills, the SEND never completes, and FlushAsync parks forever holding the connection, its slab and the handler's state. TCP will not end it: a zero window is legitimate and a peer can hold one indefinitely.

What this adds

Two knobs on TcpOptions, both 60 s by default, 0 disables:

	closes a connection when
IdleTimeoutMs	nothing received or sent for this long
SendTimeoutMs	a flush has been outstanding for this long

Both ride the reactor's existing ~250 ms ticker — the same shape as TlsService.SweepHandshakes and QuicSweep — so a connection closes at the first tick past its deadline rather than exactly on it.

Why two clocks and not one

A connection with a flush outstanding is not idle, it is sending, so the send clock governs it and the idle one does not apply. Without that split, a large response to a slow peer gets reaped for making no inbound progress while working perfectly — under MSG_WAITALL the whole flush is a single completion, so nothing refreshes the activity stamp for as long as the send legitimately takes.

And the idle clock cannot cover a stall by itself: a peer that keeps sending while it has stopped reading refreshes the idle stamp on every inbound completion, so the sweep never fires while that connection's send is wedged. A websocket written from a background task — the shape reported in #234 — is exactly that, and only the send clock catches it.

Teardown

shutdown() + MarkClosed(), and nothing else, as in SweepHandshakes. shutdown() is what the peer sees and what completes the operation the reactor's ref is waiting on (a multishot recv against a silent peer; a SEND a closed window is holding). MarkClosed wakes the handler now — parked on a read, or on the very flush being timed out.

It deliberately does not clear the table slot, cancel, or DecRef. The teardown those completions already run (CloseFromRecv, and the send path's res <= 0 branch) is the one that gets the refcount right, and it runs only once the kernel is finished with the connection's slab. Releasing the reactor's ref here instead would let the connection reach zero and be recycled — slab freed or resized — while a SEND the kernel has not given back still points into it.

A performance trap worth recording

The first version stamped Environment.TickCount64 per completion. That measured -8.7% on Tcp/Raw and -4.8% on Tcp/Pipe at 4 reactors: three vDSO calls per request against a 2.4 µs budget. The sweep runs four times a second, so the reactor now caches the clock once per loop pass (one read per io_uring_enter, amortised over the batch it returned) and the stamps are a plain store. Same reasoning removed the FlushArmedMs clear from CompleteFlush, which is the hottest path in the server — the stamp is only read while a flush is outstanding and is rewritten on every arm.

Tests

Five, in E2E/Core/TcpTimeoutTests.cs, with all three negative controls:

• idle: a connection that goes quiet is closed
• idle: a connection still talking is left alone
• idle: 0 disables the sweep
• send: a flush the peer stopped draining is released, not parked forever
• send: 0 leaves a stalled flush parked

Proven by disabling the sweep, where the idle connection never closes (Connection timed out on the client read) and the stalled flush reports still parked, while all three controls keep passing.

One thing the harness forced out: every server started by TestServer also serves one connection from WaitForListen, which connects and drops without sending. A handler that reported on that one measured nothing — and on the send path it actively lied, because a flush on an already-closed connection takes FlushAsync's _closed early-out and returns instantly, so the probe's handler "absorbed" 128 MiB without a byte reaching a socket. The handlers here gate on the connection having actually delivered bytes, as WriterContractTests does.

All suites green: E2E 185, Unit 43, Http 44, Tls 142, Chaos 47, File 4.

Bench

Tcp/Raw at 4 reactors, nine alternating A/B pairs: no separable difference. Per-pair deltas flip sign both ways (-9.7% … +2.6%) and the medians sit ~1% apart, inside a baseline arm that itself spanned 1.49–1.74 M req/s over the session — the swept arm was the tighter of the two at 1.53–1.66 M. Every large negative pair is one where the baseline happened to measure high.

Not in scope

The other half of #97: no connection cap in shared mode. Track grows the table unbounded (Reactor.cs:142), MaxConnections is incremental-only, and PoolMax caps the object pool rather than live connections. That wants its own decision — shed at accept, or pause the accept re-arm for real backpressure.

Two defaults to confirm before merging

Both are on at 60 s, which is a behaviour change for existing deployments:

• IdleTimeoutMs closes a protocol that legitimately goes quiet in both directions — an idle websocket, a long-poll — unless it is raised past that protocol's keep-alive interval.
• SendTimeoutMs is a whole-flush deadline while MSG_WAITALL is on, so a very large body over a slow link is the false positive. It tightens to time-since-progress if Benchmark MSG_WAITALL on/off #230 lands and the flag goes.

Happy to flip either to 0 by default instead.