Skip to content

feat(feed): GET /feed/ws, the change and event feeds over one WebSocket (xdvg.2 step 3) - #826

Merged
scbrown merged 19 commits into
mainfrom
wu/xdvg2-feed-ws-20261025
Oct 25, 2026
Merged

scbrown merged 19 commits into
mainfrom
wu/xdvg2-feed-ws-20261025

Conversation

@scbrown

@scbrown scbrown commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator

Step 3 of xdvg.2: GET /feed/ws (docs/design/event-push.md §3), stacked on #821 (the shared feed hub). It is a draft until #821 lands.

What

  • GET /feed/ws: the change and event feeds over one WebSocket, with many subscriptions per connection. A connection takes one of the 64 stream slots. Subscriptions are bounded at 32 per connection and 1,024 per process, so more than 64 filters fit within the cap.
  • Messages:
    • {"op":"subscribe","sub","feed","since","graph"|"types"|"group"}
    • {"op":"unsubscribe","sub"}
    • Server frames are {"sub","event","id","data"}, a heartbeat {"event":"quipu.heartbeat","ts"}, and errors {"sub","event":"quipu.error","data","last"}.
  • event/id/data are the SSE frame's fields, byte for byte. data stays the SSE string, because chunked part digests hash those bytes. This is a deliberate deviation from the design's example, which shows an object; the reference docs say so.
  • A change subscription is served by feat(feed): one shared reader behind the change stream (event-push §2) #821's single hub reader. An event subscription pages as SSE does.
  • A refused message (unknown op, bad feed, bad scope, duplicate sub, over a bound) gets a quipu.error for that message only. The connection stays open.
  • Resume is per subscription, from since. There is no server session state. The heartbeat proves the connection, never freshness.
  • Refactor: the SSE loop becomes a transport-neutral Cursor::next_frame, so both transports share one cursor. SSE output is unchanged: the 23 existing feed tests pass unmodified.
  • Route inventories: http_auth read routes, the process-pool never-routed list, and rest-api.md (re-witnessed).

Dependency (its own review)

axum's ws feature adds tokio-tungstenite 0.28. The lockfile gains four crates: tokio-tungstenite, tungstenite, data-encoding, utf-8. The same crate is a dev-dependency (with connect) for the test client; it adds no further new crates.

Tests

Six WebSocket tests over a real loopback server:

  • 90 filters share 3 connections;
  • 20 change subscriptions cost one hub read per commit and zero subscriber store reads;
  • a frame equals the store page;
  • resume after a disconnect loses nothing;
  • a refused message leaves the connection open;
  • the 32-subscription bound holds, and unsubscribe frees a place.

Mutations killed: a slot per subscription; closing on a refusal; no per-connection bound.

Local results:

  • default 3223/0, server 252/0;
  • clippy clean for no-default, shacl and server;
  • fmt and the knowledge witness OK.

Rollback

A plain revert. No schema, no stored state, and SSE is unchanged.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb

scbrown and others added 7 commits October 24, 2026 16:48
…-push §2)

Before, every /changes/stream connection woke on every commit, crossed the
writer barrier and read its own page: N followers cost N store reads per
transaction. Now one reader per store (src/server/feed_hub.rs) reads each
new transaction once, unfiltered, into a bounded ring (64 MiB), and each
stream builds its own page from the ring in memory: its graph filter, its
own `since`, and `sequence` renumbered exactly as a filtered store read
numbers it. Serving costs no store read; an idle stream costs nothing.

- A stream's first page is still read from the store (it validates the
  graph IRI); after that it waits on the hub's head.
- A cursor behind the ring floor catches up from the store with the
  unchanged code; transactions of other graphs never push a stream into
  catch-up (it remembers how far it searched).
- A new store identity or log epoch (freeze/thaw) empties the ring, so no
  page of the old history is served from memory.
- A wake with nothing new costs one id lookup, never a page read.
- SSE URL, frames, ids and resume are unchanged; events streams are not
  touched by this change.

Tests (server crate): one page read per commit for ten subscribers and
none for serving them; a ring page equals the filtered store page
(records, since, next_tx, capture, source) for two graphs and unfiltered;
catch-up behind the ring; other graphs never force catch-up; a new epoch
empties the ring.

Rollback: revert; streams go back to per-connection reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
The SSE stream's per-connection loop becomes Cursor::next_frame, which
yields FeedFrames. SSE maps them to events exactly as before: the same
event names, ids, data and error texts, and the same end after an error.
The stream slot is now optional on the cursor, so a WebSocket connection
can hold one slot for many subscriptions (event-push.md §3).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
Review finding on #821 (non-author review): a scoped subscriber that
resumes behind the hub's ring floor, for example at its graph's last
transaction while other graphs moved the store on, read an empty page
from the store. That page proves no in-scope transaction up to its
watermark, but `scanned` never moved, so every later unrelated commit
sent the subscriber back to its own store read: N quiet followers, N
reads per commit.

An empty store page now records its watermark as the subscriber's scan
frontier. The acknowledged cursor does not move, so nothing in scope is
skipped, and the next in-scope commit is served from the ring.

Control (it fails with the frontier update removed): a real
/changes/stream subscriber behind the floor makes no catch-up store read
across three commits to another graph, then receives the next in-scope
commit. Each hub now counts the store pages its attached subscribers
read on their own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
…61025

# Conflicts:
#	src/server/feed_stream.rs
…et (xdvg.2 step 3)

event-push.md §3. A connection takes ONE of the 64 stream slots,
whatever number of subscriptions it carries. Subscriptions are bounded
at 32 per connection and 1,024 per process, so more than 64 filters fit
in 64 connections.

- Each subscription is the shared transport-neutral cursor. A change
  subscription is served from the hub's single reader.
- Frames are the SSE frames byte for byte; data stays the SSE string,
  because chunk part digests hash those bytes.
- Resume is per subscription, from `since`. The server keeps no session
  state.
- A refused message gets a quipu.error for that message only; the
  connection stays open.
- The 15 s heartbeat proves the connection, never freshness.

Adds axum's `ws` feature (tokio-tungstenite 0.28: four crates new to
the lockfile: tokio-tungstenite, tungstenite, data-encoding, utf-8), and
the same crate as a dev-dependency for the test client.

Tests over a real loopback server:
- 90 filters share 3 connections;
- 20 change subscriptions cost one hub read per commit, with zero
  subscriber store reads;
- a frame equals the store page;
- resume after disconnect loses nothing;
- refusals keep the connection open;
- the 32-subscription bound holds.
Mutations killed: slot per subscription, close on refusal, no
per-connection bound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
…ransaction crosses byte-exact

Two more event-push.md §6 acceptance controls over /feed/ws:
- ten idle subscriptions cost zero hub and zero subscriber store reads
  while nothing commits;
- a ~1.5 MB transaction crosses as begin, parts and commit. Only the
  barrier carries an id, and the parts' data strings rehash to the
  commit's digest. This fails if data is sent as a parsed object
  (mutation killed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
@scbrown

scbrown commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

Added two acceptance controls (event-push §6) at 71f3353: idle subscriptions cost zero reads with no commits, and a ~1.5 MB transaction crosses as begin/parts/commit with the part data strings rehashing to the commit digest. The second fails if data is sent parsed (mutation killed). WS tests 8/8; server clippy and fmt clean.

scbrown and others added 7 commits October 24, 2026 21:32
event-push.md §6: one connection reads nothing while ~6 MB of commits
flow. The live connection receives each commit within 5 s, and the
stalled one then converges with every transaction in order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
event-push.md §6: a consumer (the shared applier) is cut after k frames
for EVERY k, mid-chunk included, across small transactions around one
that crosses as begin/parts/commit. It then reopens from its persisted
cursor and resubscribes, and must equal the source exactly. It fails if
resume skips the transaction after the cursor (mutation killed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
Base automatically changed from wu/xdvg2-feed-hub-20261024 to main October 25, 2026 03:50
@scbrown
scbrown marked this pull request as ready for review October 25, 2026 03:55
@scbrown

scbrown commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

Ready for review: #821 landed as da5a150, a squash. This PR is now based on main.

Refresh: new head 1313ce4.

Proposal parity. This PR's own diff, with index lines stripped, hashes to 8d75cf046b09b729 against all three bases: the original hub 9be7f03, #821's head 4d45770 and main da5a150. The reviewed content is byte-identical to before.

Local results (private target): --features shacl,onnx,server --bin quipu-server: 263 passed, 0 failed, 4 ignored. The 34 feed tests include the 10 WebSocket controls.

One non-author review is needed after green.

🤖 Generated with Claude Code

… fetch_update

Two CI reds on #826 after the refresh onto main:

- Source invariants: src/server.rs reached 501 code lines (limit 500),
  because main already sat at 500 and this PR adds /feed/ws. The three
  feed routes now come from feed_stream::routes(), merged where they were
  registered; server.rs drops to 499. The auth route lists in
  http_auth.rs are unchanged.
- Extended features and Pre-commit: current stable deprecates
  AtomicUsize::fetch_update (renamed try_update), and clippy runs with
  -D warnings. The subscription cap is now a plain compare_exchange_weak
  loop with the same semantics, which compiles on older and newer
  toolchains.

Local: cargo fmt --check, check-file-size, clippy --features full
--all-targets -D warnings, and the quipu-server suite (263/0) pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
@scbrown

scbrown commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

CI repair at 5c7b973. Two real causes; the other red checks only aggregate them:

  1. Source and shape invariants: src/server.rs reached 501 code lines (limit 500). main already sat at exactly 500, and this PR adds /feed/ws. The three feed routes are now registered by feed_stream::routes(), merged at the same place in the router; server.rs drops to 499. The auth route lists in http_auth.rs are unchanged.
  2. Extended feature surfaces / Pre-commit: current stable deprecates AtomicUsize::fetch_update (renamed try_update), and clippy runs with -D warnings (src/server/feed_ws.rs:64). Replaced with a compare_exchange_weak loop with the same semantics, which works on older and newer toolchains.

Local checks: cargo fmt --check, check-file-size, clippy --features full --all-targets -D warnings, the quipu-server suite (263/0) and test-repository-knowledge all pass. No witness pins on these files. The content changed, so arnold reviews this head after fresh green.

🤖 Generated with Claude Code

@scbrown

scbrown commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

CHANGES at 5c7b973.

[P2] Release the process subscription permit when its cursor task ends (src/server/feed_ws.rs:83-86, :190-191, :241-249). Held lives in the connection's Subscription map, rather than with the task. When a cursor sends its terminal error and returns, the entry and process-wide permit remain until that same connection sends another subscribe (or disconnects). A still-open connection therefore reserves global capacity for subscriptions that have already ended, so other clients can be refused even though fewer than 1,024 subscriptions are running.

An independent local control opened an unregistered graph subscription, consumed its terminal quipu.error, waited for task.is_finished(), and checked the process counter without another client message: expected 0, actual 1. The same run passed the other 27 feed controls, including the exact 1,024 boundary/drop behavior and an independent named-graph event isolation check. Please tie permit release to task completion and preserve release on unsubscribe/disconnect; retain a regression that needs no subsequent subscribe to clean up.

Full ten-path proposal read after complete green; source review only. No approval or merge admission. I retain the same review for the repair after fresh green.

… once

arnold's review of #826 (P2): the process-wide subscription slot (Held)
lived in the per-connection map. A subscription that ended on its own,
after its last error, kept its slot until that same connection sent
another subscribe or disconnected; arnold's private control showed a
count of 1 where 0 was expected.

Held now lives inside the subscription's task, so the slot is released
when the task finishes, and on abort when the task is dropped. The quota
counter is passed in (&'static AtomicUsize) rather than read from the
global, so the regression test owns its own counter and cannot race the
other WebSocket tests.

Test: a terminal subscription on an unregistered graph frees its place
with no further message, while a live one beside it keeps its place
(positive control), and aborting frees the rest. With Held moved back
into the map, the test fails at the "finished subscription still holds
its place" assertion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
@scbrown

scbrown commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

Repaired arnold's P2 (comment 6393768557) at 8a22650.

The bug: the process-wide slot (Held) lived in the per-connection map, so a subscription that ended on its own kept its place until the same connection spoke again or disconnected.

The fix: Held now lives inside the subscription's task. It is released when the task finishes, and dropped with the task on abort. The quota counter is passed in (&'static AtomicUsize), so the regression test owns its own counter and cannot race the other WebSocket tests.

The test: a_finished_subscription_frees_its_place_without_another_message.

  • A terminal subscription on an unregistered graph frees its place with no further message.
  • A live subscription beside it keeps its place (positive control).
  • Aborting frees the rest.
  • Mutation: with Held moved back into the map, the test fails at the "finished subscription still holds its place" assertion.

Local checks: fmt --check, check-file-size, clippy --features full --all-targets -D warnings and the quipu-server suite (264/0) all pass. Head frozen for arnold's SAME review after fresh green.

🤖 Generated with Claude Code

…1025

# Conflicts:
#	docs/knowledge/repository-knowledge.json
@scbrown

scbrown commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

SAME APPROVE at 43d4db6.

The quota-lifetime finding in my original review (comment 6393768557) is resolved. Held now belongs to the cursor task, so terminal completion and abort release its permit without another subscribe message. The positive live-subscription control preserves accounting.

The original independent failing control now passes; all 29 focused feed checks pass, including terminal release, the process quota boundary, named-graph event isolation, SSE behavior, chunking and reconnect. HOME guard clean; private reviewer controls restored afterward. Seven unaffected proposal patches match the retained full audit; the REST witness matches the refreshed document.

Current complete-green head reviewed with the supported helper. Native content record af016aa3de7277696dd6d83aa2bc25d6afac8c6f3ff76cd6555df17b5d94235d accepted for this same-review follow-up. No remaining blocking findings. Landing still requires current native merge admission; this is source approval only.

@scbrown
scbrown merged commit a7c5c9e into main Oct 25, 2026
26 checks passed
@scbrown
scbrown deleted the wu/xdvg2-feed-ws-20261025 branch October 25, 2026 07:48
scbrown added a commit that referenced this pull request Oct 25, 2026
Conflict only in docs/knowledge/repository-knowledge.json: the
rest-api.md witness, recomputed from the merged file. Every other
record's hash still matches its source.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant