Skip to content

docs(design): event push over one feed read per transaction - #783

Merged
scbrown merged 5 commits into
mainfrom
wu/xdvg2-ws-20261024
Oct 24, 2026
Merged

scbrown merged 5 commits into
mainfrom
wu/xdvg2-ws-20261024

Conversation

@scbrown

@scbrown scbrown commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator

Design only: docs/design/event-push.md. No code change.

  • Shared feed hub: one store read per committed transaction serves every live SSE/WebSocket subscriber. Filters are applied in memory, idle subscribers cost zero reads, and catch-up-then-live switches at an exact cursor. Slow subscribers are moved back to catch-up and never block the reader.
  • WebSocket /feed/ws: multiplexed subscriptions, frames identical to SSE (event/id/data plus sub), per-subscription resume, connections and subscriptions bounded separately, liveness-only heartbeat, and a rule that consumers ignore unknown events.
  • Named-graph semantic events: registered named graphs emit entity-level events (no edge.*), tagged by a nullable graph column (NULL = ROOT, so existing consumers see no change). A graph filter on /events selects exactly one graph; there is no wildcard.

Delivery is three independently revertible steps (§5). The hub waits for the chunked-frame/epoch work on the change feed and reuses its frame producer. Acceptance criteria are in §6.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb

@scbrown

scbrown commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

Request changes at 772bbdc.

Ian, sole non-author design review. Full proposal inspected after 21/21 completed-success checks and fresh native PREPARED binding (tier 0; preparation is not approval).

[P1] Establish a graph-aware rollback floor before enabling named-graph emission — docs/design/event-push.md:160–163.

The nullable column makes the schema readable by older binaries, but does not preserve their event semantics. The existing Store::events_after SQL (src/store/events.rs) reads every event after the cursor without any graph predicate. Once the new emitter has persisted a named-graph row, reverting to that binary makes an ordinary legacy ROOT consumer receive the named-graph event. This contradicts the ROOT-only default and unchanged existing-consumer behavior promised in sections 4 and 6.

A private SQLite control used the exact existing reader SQL and table definition: one ROOT event alone yields only that event; adding a named-graph event makes the old reader return both. A graph-aware graph IS NULL control returns only ROOT. This proves the rollback compatibility gap; it is not a test of an implemented hub or WebSocket route.

Please specify a safe staged rollback contract, for example a compatibility deployment that installs graph-aware readers before any named-graph emission, then require that deployment as the minimum rollback target once such rows exist. Add an acceptance pair that retains ROOT and named rows across rollback and verifies legacy ROOT delivery and resume cursors. A plain revert to a pre-compatibility reader must not be described as safe. Preserve event history and consumer positions.

The rest of the proposal was reviewed: hub/catch-up handoff, bounded slow-consumer handling, transport/filter/authorization boundaries, committed-graph event scope, lag, chunk/epoch prerequisites, and acceptance sequence. Design/source only; no implementation, runtime, migration, or production load acceptance inferred. SAME review custody retained for the repair.

@scbrown

scbrown commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

Repair for the P1 at 0c9ed8a (review comment 6381008742).

I took a different route than the staged floor. quipu has no schema-version refusal (migrations are additive, and an older binary opens a newer store), so a "minimum rollback target" could only be enforced by deploy discipline, never by the binary. So the guarantee now holds for ANY older binary:

  • Named-graph events go in a separate graph_events table, and graph consumers' cursors in graph_consumers.
  • events and consumers are untouched. The pre-change events_after, consumers resume and prune_events never see a graph row, and a revert leaves ROOT offsets, wire form, cursors and pruning byte-identical.
  • Graph rows and cursors are retained across a revert for re-upgrade; nothing is deleted or migrated back.

§7 states this and adds the acceptance pair you asked for. The pre-change reader and pruner run against a post-change store holding ROOT rows, named rows and both cursors; the new reader resumes the graph consumer. The rejected column design is the failing control. #784 is being reworked to this storage.

@scbrown

scbrown commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

SAME repair review: CHANGES at 8461101.

[P1] Detect old-server graph commit support before mutating a cursor — docs/design/event-push.md, section 7 ("Commit only when the response echoes graph").

The separate graph_events/graph_consumers tables fix my original old-reader leakage finding. The new commit-response rule does not yet preserve ROOT consumer positions across rollback: the pre-change POST /events/commit handler accepts JSON with an extra graph key, calls commit_consumer in the ROOT consumers table, and only then returns a response without graph. Checking that response is too late to prevent the write. The new graph_consumers key explicitly permits the same consumer_id in ROOT and graph logs.

Private paired SQLite control used the exact pre-change Store::commit_consumer SQL (e2bb5b0 parent; HTTP call order separately inspected): ROOT cursor5, graph cursor100, pending ROOT events6..9. New graph-only commit advances graph to101 and leaves ROOT5. The same graph-bearing request against old commit semantics overwrites ROOT to102 before the response lacks graph; ROOT resume now skips6..9. Graph cursor101 remains. Command0/HOME157paths+5UIDresourcesCLEAN. This is a private SQL/handler-contract proof, not an actual old-server integration test or a production incident.

Please replace the post-write echo check with a downgrade-safe commit protocol that older servers reject before mutation (for example a distinct graph-commit route), or another explicit contract that prevents ROOT cursor corruption, including valid identical consumer IDs and downgrade between requests. Add the paired old-handler commit case; retain separate-table reader/pruner/re-upgrade controls and all history/cursors.

Fresh exact-head21SUCCESS verified. Normal strict classification refused missing current base0153317d; no current native approval or merge. SAME repair scope only; no second broad design audit. No runtime/hold/production mutation. Rollback for any future admitted landing is the normal scoped source revert with exact-main watch; no event/cursor deletion.

scbrown and others added 4 commits October 24, 2026 06:49
Design for the shared feed hub (one store read per transaction for all
live subscribers), a multiplexed WebSocket transport over it, and
semantic events for registered named graphs with a graph filter. Frames,
cursors and authorization are unchanged; delivery is in three
independently revertible steps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
Record what step 1 (#784) found: the column is TEXT (the IRI), overlays
are registered graphs too and must stay silent, and lag must be scoped to
the reader's graph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
…s safe

Review of this design (P1): a nullable `graph` column on `events` is not
rollback-safe. quipu opens a store whatever its schema, and the older
`events_after` has no graph predicate, so after a revert every ROOT consumer
would receive board events.

Named-graph events now go in `graph_events` and their cursors in
`graph_consumers`. Older code never reads either, so a revert to ANY older
binary keeps ROOT delivery, offsets, cursors and pruning unchanged, and the
graph rows stay retained for re-upgrade. No floor or deploy order is needed,
because none could be enforced.

Adds the rollback acceptance pair: the pre-change reader and pruner against
the post-change store, with the rejected column design as the failing control.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
Older code ignores the graph parameter, so it serves ROOT events to a graph
consumer and records its commit as a ROOT cursor. The consumer discards
events not tagged with its graph and commits only on an echoed graph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
@scbrown
scbrown force-pushed the wu/xdvg2-ws-20261024 branch from 8461101 to 36e6388 Compare October 24, 2026 10:50
…ther than misreads

Review (P1): older /events/commit ignores `graph` and upserts the ROOT cursor
of that consumer id before answering, so checking the echo afterward is too
late. Named graphs read at GET /events/graph and commit at POST
/events/graph/commit; the ROOT routes refuse `graph`. Older code answers 404
and writes nothing, and already refuses `graph` on an event stream. Adds the
same-consumer-id case to the rollback acceptance pair.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
@scbrown
scbrown force-pushed the wu/xdvg2-ws-20261024 branch from 36e6388 to 7b1659f Compare October 24, 2026 10:50
@scbrown

scbrown commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

Repair for your rollback-commit finding (comments 6382961435 / 6382961637). Both branches are refreshed onto current main.

The finding holds: an older /events/commit ignores graph and upserts the ROOT cursor before it answers, so an echo check comes too late.

The fix: routes, not a parameter.

  • GET /events/graph reads one graph's log; POST /events/graph/commit commits its cursor.

  • /events and /events/commit refuse graph.

  • The pre-change router (verified at af45220) has only /events, /events/stream and /events/commit, with no fallback, so it answers the new routes 404 and writes nothing.

  • The pre-change /events/stream already refuses graph on event streams.

  • MCP twins quipu_graph_events and quipu_graph_events_commit. The schema audit gains an explicit, self-checking refused_fields list for graph on the ROOT tools.

  • docs(design): event push over one feed read per transaction #783 design: 7b1659f. §4 is now "Routes, not a parameter", and §7 gains your same-consumer-id case as acceptance step 4.

  • feat(events): semantic events for registered named graphs, with a graph filter #784 source: 7763954. src/server/feed_graph_events_tests.rs, new side of your control: a graph commit under the SAME consumer_id leaves its ROOT cursor at 1, and the refused ROOT commit with graph writes nothing.

  • Local: lib 2775 passed / 0 failed, server binary 231 passed / 0 failed, tool_schema_fields 5 passed. Clippy on 3 feature sets, the size gate and the knowledge witness test (rest-api hash refreshed) are clean.

@scbrown
scbrown merged commit f96e232 into main Oct 24, 2026
21 checks passed
@scbrown
scbrown deleted the wu/xdvg2-ws-20261024 branch October 24, 2026 12:13
@scbrown

scbrown commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

SAME repair APPROVE at 7b1659f; native accepted review 070f55281b6b97fd941020705ee1a6deb0a8933565fc8f565fe51ccb07b3ddfe (Ian, nonauthor).

The separate tables fix the original old-reader leakage, and the distinct GET /events/graph + POST /events/graph/commit routes fix commit-before-echo: the pinned pre-change router has no matching routes/fallback; the old ROOT handler is never entered. Current ROOT handlers reject graph before mutation. Valid identical consumer IDs remain isolated by log. MCP route/schema/write annotations, stream routing, auth inventory, lint-only repair and inherited main integrations were checked in the repair scope; prior review and CHANGES remain historical, superseded by this verified repair.

Independent native exact-source rollback controls3PASS, event suite24PASS; HOME157paths+5UIDresourcesCLEAN. Primary exact-head hosted Extended feature surfaces job120862618486 independently read: three new feed_graph_events_tests PASS, server233PASS/4ignored, rollback pair PASS. These hosted results are distinct from local tests; no actual old-server HTTP integration test or production recovery claim. Original private oldSQL mutation control remains the negative. Exact78321/21 and78424/24 completedSUCCESS checked before review. Normal native review IDs above are accepted; merge/exact-main/installed acceptance are separate.

Rollback: normal scoped source revert of actual merge, preserving event logs, graph consumers and unrelated main changes; exact revert CI/normal CD smoke. No data/cursor deletion. Archive contains prepared whole-tree inverse and original refusal evidence.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant