Skip to content

sdk/python: keep a handle's relay connection across calls - #199

Merged
CMGS merged 2 commits into
mainfrom
sdk/keepalive-py
Sep 16, 2026
Merged

CMGS merged 2 commits into
mainfrom
sdk/keepalive-py

Conversation

@CMGS

@CMGS CMGS commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Fourth step of #195: the Python SDK side of a persistent data-plane connection, the same shape as #198 for Go. Needs #196 in the guest image to take effect; against an older silkd it behaves exactly as before.

What changes

  • A Sandbox handle keeps its relay connection. After a call the connection is parked in a ConnPool (up to 8 per handle, 30 s idle; Client(..., keep_alive=...) tunes the window, 0 dials per call) and the next call sends its request on it. Every RPC helper runs under _park_after: a terminal frame, error frames included, parks the connection; a dropped connection, a timeout cut or a local error drops it. close and hibernate drain the pool first.
  • What is not reused. watch, open_pty, dial_port and an LSP session own their connection as today. run joins its stdin thread before the connection goes back, so the pump's frames never land on the next RPC (silkd drains what an exited command left unread).
  • Liveness before reuse. A parked connection is checked with a zero-timeout select: a peer that hung up (hibernate, release, the idle sweep) or spoke unprompted fails it and the call redials, which wakes the guest exactly as before. It works through TLS sockets too, since it never consumes bytes.
  • Protocol negotiation. A handle's first dial asks the daemon's proto with one info round trip. A daemon before proto 2 closes after answering, so the handle redials and dials per call from then on. One extra round trip once per handle.
  • logs writes exit then done; the trailing done is now consumed so the connection stays in frame. docs/sdk-python.md documents the window and its interplay with idle_hibernate_seconds.

Hot path

A pooled call: one lock pair and one select(2), replacing a TCP dial, an HTTP upgrade, a TLS handshake behind an edge and a vsock connect on the node. The first call on a handle pays one info round trip. Frames on the wire are unchanged.

Evidence

$ ruff format --check . && ruff check .
33 files already formatted
All checks passed!
$ (cd sdk/python && python -m pytest -q)
193 passed in 28.02s
$ (cd sdk/python && python -m mypy)
Success: no issues found in 9 source files
$ (cd sdk/python && for i in 1 2 3 4 5; do python -m pytest -q tests/test_keepalive.py | tail -1; done)
6 passed ×5
$ (cd sdk/langchain && PYTHONPATH=../python python -m pytest -q)
6 passed in 0.16s

Run on Python 3.9.6, the declared floor. The OpenAI adapter suite could not be collected locally (no agents package in this environment); CI runs it on 3.12.

New tests in tests/test_keepalive.py against an in-process fake agent that answers info, fs_stat and exec: four calls share one upgrade; an old daemon costs the probe dial plus one per call; keep_alive=0 dials per call; an idle connection closes after the window; a peer that hung up is noticed before reuse; close drains the parked connection. The fake half-closes and drains like the relay does, since a plain close with an unread stdin_close resets the reply. Scripted one-shot fakes in the existing tests run with the proto pinned to 1, the fallback path; the logs script now ends exit, done as silkd does.

Every data-plane call dialed the owner node, upgraded and, behind an
edge, shook hands with TLS, then paid a vsock connect on the node
(#195). With silkd serving RPCs back to back (#196) a handle now parks
its connection after a call and the next call sends on it: a Sandbox
carries a ConnPool (8 connections, 30 s idle; Client(keep_alive=...)
tunes or disables it) and every RPC helper runs under _park_after,
which parks the connection after a terminal frame, error frames
included, and drops it on anything else. Streams (watch, pty,
port_forward, lsp) still own their connection; run joins its stdin
thread before the connection goes back, so the pump's frames never
land on the next RPC. close and hibernate drain the pool first.

Before a parked connection is reused, a zero-timeout select tells a
peer that hung up (hibernate, release, the idle sweep) or spoke
unprompted from a quiet one, so the next call redials and wakes the
guest as before. A handle's first dial asks the daemon's proto with
one info round trip: a daemon before proto 2 closes after answering,
so the handle redials and dials per call from then on. logs' trailing
done is consumed so the connection stays in frame.

Hot path: a pooled call costs one lock pair and one select(2) instead
of a TCP dial, an HTTP upgrade, a TLS handshake and a vsock connect;
the first call on a handle pays one info round trip.
Every park armed a threading.Timer, a real OS thread, which measured
at 42.6 µs, over half the cost of a kept-connection call, and left
eight sleeping threads per handle; the pool now records a deadline
per parked connection and one self-rearming sweeper closes them as
they expire, so park and take touch no thread at all. run() sends
stdin_close inline when there is no stdin instead of spawning the
pump thread for one frame. The daemon's proto lives on the Sandbox,
a zero keep_alive window skips the probe, _park_after and _lease fold
into one context manager, _Parked goes, and logs' trailing done is a
parameter rather than a string compare. The tests share sandbox_at,
accept_upgrade and wait_until through conftest and pin the old
one-shot behaviour with keep_alive=0 instead of a private field.
@CMGS
CMGS merged commit ac42bd0 into main Sep 16, 2026
3 checks passed
@CMGS
CMGS deleted the sdk/keepalive-py branch September 16, 2026 17:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant