Skip to content

Coordinate web socket connects and closes - #257

Merged
natekspencer merged 3 commits into
mainfrom
serialize-ws-connect
Nov 16, 2026
Merged

natekspencer merged 3 commits into
mainfrom
serialize-ws-connect

Conversation

@natekspencer

@natekspencer natekspencer commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Found while testing #256. Both bugs are races in starting a web socket connection alongside another subscribe or close. Both are on main today.

1. Subscribes started together open two connections

When no web socket is open, two subscribes started at the same time (e.g. with asyncio.gather) both pass _ws_connect's "no web socket yet" check before either connection is up, so each calls new_connection:

  • the server gets two connections;
  • the second replaces _ws, so the first is never closed;
  • the receivers collide (RuntimeError: Concurrent call to receive() is not allowed), and a subscribe can end up on a connection that's never acknowledged.

Fix: _ws_connect holds a lock while it creates the monitor and connects. A subscribe started while another is connecting waits for that connection and then uses it.

2. A closing web socket cancels its replacement

WebSocketMonitor.close() awaits the socket's close. aiohttp (3.14.5 here; the requirement is >=3.0.0) sets closed partway through: after breaking the receiver out of receive(), and before waiting up to ws_close (10 s) for the close handshake. A subscribe started while the last unsubscribe is closing the connection:

  • before closed is set, kept the closing connection, then failed in start_subscription ("the web socket connection closed");
  • after closed is set, opened a replacement. The finishing close() then cancelled the replacement's receiver and the monitor, leaving a subscription that reported success but never got a message.

Fix:

  • new_connection() and close() hold one lifecycle lock. A replacement waits until the old connection has finished closing, so the close can't touch it. With no replacement, the cleanup works as before.
  • _ws_connect replaces any connection that isn't connected, which includes one that's closing. Before, it only replaced a missing or closed socket.
  • A new_connection() that waited for the lock behind another (e.g. a subscribe behind the monitor's reconnect) reuses the connection that one opened, instead of opening a second and leaving the first open.
  • close() cancels the monitor before taking the lock. Otherwise it would wait out a reconnect the monitor holds the lock for, which can take as long as the session's timeout (aiohttp's default is 5 minutes).

Testing

  • test_simultaneous_subscribes_share_one_connection starts two subscribes with asyncio.gather. Both must succeed, on one connection.

  • test_subscribe_while_closing_gets_a_working_connection holds the close at each point (before and after closed is set), starts a subscribe meanwhile, then lets the close finish. The subscribe must succeed on a new connection with a live receiver and monitor.

  • test_queued_new_connection_reuses_the_new_one and test_close_stops_a_stuck_reconnect cover the last two fixes.

The three race cases fail on main and passed 10 runs in a row with the fix. This merges cleanly with #256, and the full suite passes on the two combined.

🤖 Generated with Claude Code

natekspencer and others added 2 commits November 16, 2026 02:50
With no connection open, two subscribes started at the same time both
saw no web socket in _ws_connect and each opened one. The second
replaced the first, which was never closed, and the receivers collided
("Concurrent call to receive() is not allowed").

Connecting now holds a lock, so a subscribe started while another is
connecting waits for that connection and uses it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WebSocketMonitor.close() awaits the socket's close, and aiohttp marks
the socket closed partway through it. A subscribe started meanwhile
either kept the closing connection and failed, or opened a replacement
whose receiver and monitor the finishing close() then cancelled,
leaving a subscription that never got a message.

new_connection() and close() now hold one lifecycle lock, so a
replacement waits until the old connection has finished closing and
the close can't touch it. _ws_connect also replaces a connection
that's closing (no longer `connected`), not only a missing or closed
socket.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@natekspencer natekspencer changed the title Open one web socket for subscribes started together Coordinate web socket connects and closes Nov 16, 2026
close() waited for the lifecycle lock while the monitor held it for a
reconnect, which can take as long as the session's timeout (5 minutes
by default), so closing could hang behind an unreachable server. It now
cancels the monitor first, as it did before the lock.

A new_connection() queued behind another (e.g. a subscribe behind the
monitor's reconnect) opened a second connection and replaced the first,
leaving it open. It now reuses a connection opened while it waited.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@natekspencer
natekspencer merged commit 0bac7b8 into main Nov 16, 2026
6 checks passed
@natekspencer
natekspencer deleted the serialize-ws-connect branch November 16, 2026 04:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant