sdk/go: keep a handle's relay connection across calls - #198
Merged
Merged
Conversation
Every data-plane call dialed the owner node, upgraded and, behind an edge, shook hands with TLS, then paid a vsock connect on the node (#195). With silkd serving RPCs back to back (#196) a handle now parks its connection after a call and the next call sends on it: a Sandbox carries a small pool (8 connections, 30 s idle; WithKeepAlive tunes or disables it) and every RPC helper ends in a lease that parks the connection after a terminal frame, error frames included, and drops it on anything else. Streams (watch, pty, port_forward, lsp) still own their connection; a Run whose stdin pump is still parked in Read gives its connection up too, as the pump would write into the next RPC. Close and Hibernate drain the pool first. Before a parked connection is reused, a MSG_PEEK on the TCP socket tells a peer that hung up (hibernate, release, idle sweep) from a quiet one, so the next call redials and wakes the guest as before; platforms without the probe never park. The handle's first connection pipelines info ahead of the request: a daemon before proto 2 closes after answering it without reading the request, so the send repeats on a fresh dial and the handle dials per call from then on. logs' trailing done is consumed so the connection stays in frame. The test fake serves RPCs back to back (ServeConnOnce keeps the old shape); rpcbench gains mode C, the SDK's own path. Hot path: a pooled call costs one mutex pair and one recvfrom(MSG_PEEK) instead of a TCP dial, an HTTP upgrade, a TLS handshake and a vsock connect; the first call on a handle reads one extra info frame.
This was referenced Sep 16, 2026
The keep-alive SDK pipelines info ahead of a handle's first request, and the two hand-written agent fakes replied to it with their canned frames. A shared agentRoute answers info with proto 2 and serves the request lines back to back, dropping the connection where the test wants the guest to vanish.
dial looped though it could only run twice, and the request was sent at three sites; connect now sends it once through agentConn.sent, and the proto the probe learns lives on the Sandbox rather than the pool. The liveness peek keeps its RawConn and callback per connection instead of allocating them per call, and a parked connection reuses one timer through Reset instead of arming a new one per park. The test fake answers info in one place and drops input frames outside an RPC unconditionally; the mcp and SDK test servers share silkdtest.Upgrade for the hijack and 101, the mcp info reply comes from wire.EncodeResponse, and rpcbench pads its labels in the format string. Two comments that restated their code go; five one-line docs shrink to one fact.
A proto-1 daemon closes after every RPC, but the lease still parked its connection when the FIN had not reached MSG_PEEK yet. Gate reuse on the negotiated protocol so old daemons always dial per call, independent of close timing.
The peek type existed once per build tag to carry one bool for the RawConn.Read callback; agentConn already owns that state, so the callback becomes a method on it and the type goes away.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third step of #195: the Go SDK side of a persistent data-plane connection. Needs #196 in the guest image to take effect; against an older silkd it behaves exactly as before.
What changes
Sandboxhandle keeps its relay connection. After a call the connection is parked (up to 8 per handle, 30 s idle;WithKeepAlivetunes the window, 0 dials per call) and the next call sends its request on it. Every RPC helper ends in alease: a terminal frame, error frames included, parks the connection; a dropped connection, a cancelled ctx or a local read error drops it.CloseandHibernatedrain the pool first.Watch,OpenPty,DialPortand an LSP session own their connection as today. ARunwhose stdin pump is still parked inReadwhen the process exits gives its connection up too, since that pump would write into the next RPC. An early terminal frame during an upload (silkd rejecting it) stops the stream instead of feeding the rest into a dead RPC.recvfrom(MSG_PEEK)on the TCP socket: a peer that hung up (hibernate, release, the idle sweep) or spoke unprompted fails it and the call redials, which wakes the guest exactly as before. Platforms without the probe (!unix) never park.infoahead of the request. A daemon before proto 2 answers and closes without reading the request, so the request is sent again on a fresh dial and the handle dials per call from then on; a proto-2 daemon answers both and the connection is kept. One extra frame read on the first call, no extra round trip.logswritesexitthendone; the trailingdoneis now consumed so the connection stays in frame.Pty.Resizegoes throughdoneRPClike every other done-answering verb.wire.KeepAliveProtonames the threshold.silkdtestserves RPCs back to back like silkd (ServeConnOncekeeps the one-shot shape for the fallback test),rpcbenchgains mode C (the SDK's own path),docs/sdk.mddocuments the window and its interplay withidle_hibernate_seconds.Hot path
A pooled call: one mutex lock/unlock pair and one
recvfrom(MSG_PEEK)syscall, replacing a TCP dial, an HTTP upgrade, a TLS handshake behind an edge and a vsock connect on the node. The first call on a handle reads one extrainfoframe. Frames on the wire are unchanged. Numbers of record come fromrpcbenchon the testbed once #196's image is rebaked; mode C is there for that.Evidence
New tests in
keepalive_test.go: four RPCs share one upgrade; an old daemon costs the probe dial plus one per call;WithKeepAlive(0)dials per call; an idle connection closes after the window; a stalled stdin pump keeps its connection out; a peer that hung up is noticed before reuse; the probe leaves a peeked byte intact and notices a hang-up. Four existing tests with scripted one-shot fakes run on alegacySandboxhandle, the fallback path.Follow-ups
Python SDK (same shape);
docs/performance.mdnumbers fromrpcbenchA vs C on the testbed after the image rebake; then #34 stays open for the mux and binary bulk frames.