review: watermark rate sampling, egress door connection cap, Python bulk-frame cases

[CMGS]

Follow-up to #179 (rebased onto main after #179 merged as 58a4c86). The three items the review round classified as fix-now, each in its own commit with a test that fails on the base.

Changes

commit	area	defect	fix
c4b93cb	pool	the watermark folded 1/dt of every consecutive claim pair into its rate EWMA, so two claims a millisecond apart read as 300/s and pinned the warm target at warm_max for minutes (rateDecayTau 60 s)	arrivals accumulate in a bin that closes on the first claim after one second and folds arrivals per second; decay over silence unchanged. TestEffectiveTargetIgnoresATwoClaimBurst (fails on the base: target 20), TestEffectiveTargetTracksDemand now drives 10/s for 6 s and still reaches warm_max
0276635	pool	both egress doors accepted without bound, so a guest could hold a host goroutine and descriptor for every connection it opened, on a host shared with other tenants	each door is wrapped in a LimitListener of 256 at arm time (the pre-bind probe still sees the raw listener; close ends a blocked accept); a dial past the cap waits in the socket backlog. TestEgressDoorCapsConcurrentConnections (fails on the base: the 257th request is served). One sentence in docs/egress.md
5df4e01	pool	a bin that closed after a long silence folded its near-zero sample against the rate stored at the last close, so two claims after ten quiet minutes brought the target most of the way back to warm_max	the stored rate is decayed over the bin's span before the fold. TestEffectiveTargetForgetsRateAcrossSilence (fails on the undecayed fold: target 5)
54f57d2	egress	the door cap bounds guest-side descriptors, but the upstream transport kept eight idle connections per host with no total	MaxIdleConns 64 per sandbox; the doc sentence says what each cap bounds
8d032f6	pool	under the race detector the door-cap test dialed past the listen backlog and got a refusal	the test retries a refused dial
ac0aad4	sdk/python	the fast bulk slicer's rejection set matched the Go decoder's in three of nine malformed shapes	the other six and the tag-after-other-keys frame run on the Python side too

Behavior change worth reading: the watermark now needs about a second of arrivals before it moves, and a burst that ends within that second never raises the target. Sustained demand (10/s with a 0.5 s lead) reaches warm_max in 3–5 s instead of after the third claim; the ~60 s decay is unchanged.

Hot-path cost: noteArrival does one comparison and an increment per claim and a division once per second, in place of a division per claim; the door cap adds one semaphore acquire per accepted connection, not per byte.

Hardware

Isolated kit on the bare-metal testbed (pinned cocoon binary, sqlite meta, its own root/run/log dirs), guest images
from #179's last silkd build (this branch changes no guest code). Arms: sandboxd-base = #179 head ef9a3f6,
sandboxd = this branch (built from 857113b, content-identical to the rebased head 8d032f6). Both checks read GET /v1/info's pool row or the daemon's /proc/<pid>/fd count.

Watermark (pool warm: 2, warm_max: 8; two back-to-back claims, then four loops of claim+release for six seconds, then silence) and door cap (a guarded none-lane pool; sandbox A holds 300 idle connections to its HTTP door for 30 s while B fetches through its own door and A tries a 301st request):

sandboxd-base ready:               {"warm":2,"refilling":0,"target":2}
sandboxd-base two claims +2s:      {"warm":8,"refilling":0,"target":8}
sandboxd-base two claims +22s:     {"warm":8,"refilling":0,"target":8}
sandboxd-base sustained (18 claims in 6s): {"warm":2,"refilling":0,"target":2}
sandboxd-base sustained +5s:       {"warm":2,"refilling":0,"target":2}
sandboxd-base sustained +60s:      {"warm":2,"refilling":0,"target":2}
sandboxd-base sustained +120s:     {"warm":2,"refilling":0,"target":2}
sandboxd-base sustained +180s:     {"warm":2,"refilling":0,"target":2}
WM sandboxd-base DONE
sandboxd ready:               {"warm":2,"refilling":0,"target":2}
sandboxd two claims +2s:      {"warm":2,"refilling":0,"target":2}
sandboxd two claims +22s:     {"warm":2,"refilling":0,"target":2}
sandboxd sustained (18 claims in 6s): {"warm":2,"refilling":0,"target":2}
sandboxd sustained +5s:       {"warm":2,"refilling":0,"target":2}
sandboxd sustained +60s:      {"warm":2,"refilling":0,"target":2}
sandboxd sustained +120s:     {"warm":2,"refilling":0,"target":2}
sandboxd sustained +180s:     {"warm":2,"refilling":0,"target":2}
WM sandboxd DONE
sandboxd-base fds before flood:        16
flood spawned pid 155
sandboxd-base fds during flood (+5s):  316
B fetch during A's flood: http 200 exit 0
A 301st request:          http 200 exit 0
sandboxd-base fds after flood (+40s):  22
sandboxd-base fds after release:       18
DOOR sandboxd-base DONE
sandboxd fds before flood:        16
flood spawned pid 155
sandboxd fds during flood (+5s):  272
B fetch during A's flood: http 200 exit 0
A 301st request:          http 000 exit 28
sandboxd fds after flood (+40s):  21
sandboxd fds after release:       18
DOOR sandboxd DONE

The base pins the target at warm_max from two claims and spins up six VMs for them; the branch holds the floor, follows sustained demand, and returns to the floor as the rate decays. On the base the flood lands 300 accepted connections on the daemon and the 301st request is served; on the branch the daemon holds 256, B's fetch is unaffected, A's 301st waits unserved until the flood ends, and the descriptor count returns to its baseline either way.

An earlier pass at 3bfc0dd (before the fold-decay and idle-pool commits) gave the same door numbers and the same two-claim result; its sustained phase drove only ~3 claims/s from one sequential loop, which is why the loop was parallelised:

sandboxd-base ready:               {"warm":2,"refilling":0,"target":2}
sandboxd-base two claims +2s:      {"warm":8,"refilling":0,"target":8}
sandboxd-base two claims +22s:     {"warm":8,"refilling":0,"target":8}
sandboxd-base sustained (18 claims in 6s): {"warm":2,"refilling":0,"target":2}
sandboxd-base sustained +5s:       {"warm":2,"refilling":0,"target":2}
sandboxd-base sustained +60s:      {"warm":2,"refilling":0,"target":2}
sandboxd-base sustained +120s:     {"warm":2,"refilling":0,"target":2}
sandboxd-base sustained +180s:     {"warm":2,"refilling":0,"target":2}
WM sandboxd-base DONE
sandboxd ready:               {"warm":2,"refilling":0,"target":2}
sandboxd two claims +2s:      {"warm":2,"refilling":0,"target":2}
sandboxd two claims +22s:     {"warm":2,"refilling":0,"target":2}
sandboxd sustained (18 claims in 6s): {"warm":2,"refilling":0,"target":2}
sandboxd sustained +5s:       {"warm":2,"refilling":0,"target":2}
sandboxd sustained +60s:      {"warm":2,"refilling":0,"target":2}
sandboxd sustained +120s:     {"warm":2,"refilling":0,"target":2}
sandboxd sustained +180s:     {"warm":2,"refilling":0,"target":2}
WM sandboxd DONE
sandboxd-base fds before flood:        16
flood spawned pid 155
sandboxd-base fds during flood (+5s):  316
B fetch during A's flood: http 200 exit 0
A 301st request:          http 200 exit 0
sandboxd-base fds after flood (+40s):  22
sandboxd-base fds after release:       18
DOOR sandboxd-base DONE
sandboxd fds before flood:        16
flood spawned pid 155
sandboxd fds during flood (+5s):  272
B fetch during A's flood: http 200 exit 0
A 301st request:          http 000 exit 28
sandboxd fds after flood (+40s):  21
sandboxd fds after release:       18
DOOR sandboxd DONE

Full E2E on the branch binary (seven legs, same kit):

PASS sandboxd-e2e 22s
PASS egress-e2e-none 4s
PASS egress-e2e-egress 7s
PASS socks-e2e 14s
PASS intercept-e2e 4s
PASS archive-e2e 11s
PASS desktop-lane 34s

Codex

One round on 3bfc0dd, ac0aad4 after the rebase (the thread continued from #179): three MINOR, no MAJOR, and the closing sentence verbatim: "No blockers remain for the follow-up branch." All three are applied above: the stale stored rate across a silence (5df4e01), the door sentence's wording and the unbounded upstream idle pool (54f57d2).

Gates

sandboxd  golangci-lint run  0 issues on linux and darwin at 857113b, the pre-rebase head with the same tree; fmt --diff clean; asl clean on both GOOS; go test -race ./... exit 0 (go mod tidy: no change)
sdk/python  ruff format --check + ruff check clean; pytest tests/test_frames.py 12 passed

Not in this PR (issues): the langchain absolute deadline through the Python client's redirect walk, s3 generation GC on re-publish, a Reconcile sweep for a removed pool's golden, warm presence in the config file.