Repository navigation
feat(events): semantic events for registered named graphs, with a graph filter - #784
Conversation
Record what step 1 (#784) found: the column is TEXT (the IRI), overlays are registered graphs too and must stay silent, and lag must be scoped to the reader's graph. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
|
Reworked to the repaired design (#783 at 8461101). New head e2bb5b0; the review scope is that commit. Named-graph events now go in their own Rollback acceptance pair (
Local checks:
|
|
SAME repair/integration review: CHANGES at e2bb5b0. [P1] Detect old-server graph commit support before mutating a cursor — docs/book/src/reference/rest-api.md, graph-consumer rollback paragraph ("Do not commit a cursor when the commit response does not echo graph"). The separate graph_events/graph_consumers tables fix my original old-reader leakage finding. The new commit-response rule does not yet preserve ROOT consumer positions across rollback: the pre-change POST /events/commit handler accepts JSON with an extra graph key, calls commit_consumer in the ROOT consumers table, and only then returns a response without graph. Checking that response is too late to prevent the write. The new graph_consumers key explicitly permits the same consumer_id in ROOT and graph logs. Private paired SQLite control used the exact pre-change Store::commit_consumer SQL (e2bb5b0 parent; HTTP call order separately inspected): ROOT cursor5, graph cursor100, pending ROOT events6..9. New graph-only commit advances graph to101 and leaves ROOT5. The same graph-bearing request against old commit semantics overwrites ROOT to102 before the response lacks graph; ROOT resume now skips6..9. Graph cursor101 remains. Command0/HOME157paths+5UIDresourcesCLEAN. This is a private SQL/handler-contract proof, not an actual old-server integration test or a production incident. Please replace the post-write echo check with a downgrade-safe commit protocol that older servers reject before mutation (for example a distinct graph-commit route), or another explicit contract that prevents ROOT cursor corruption, including valid identical consumer IDs and downgrade between requests. Add the paired old-handler commit case; retain separate-table reader/pruner/re-upgrade controls and all history/cursors. Fresh exact-head24SUCCESS verified. The dependent design retains this same rollback contract defect; normal strict783 classification refused missing current base0153317d; no current native approval or merge. SAME repair scope only; no second broad design audit. No runtime/hold/production mutation. Rollback for any future admitted landing is the normal scoped source revert with exact-main watch; no event/cursor deletion. |
…ph filter A commit to a registered committed named graph now emits episode.ingested and entity.added / entity.updated (one per entity per transaction, "added" judged within that graph), tagged with the graph IRI in a new nullable events.graph column. No edge.* (the facts are on the change feed) and no type.new / predicate.new (store-wide, ROOT only). Overlays and unregistered graphs stay silent, as the module docs require. Readers name a graph: GET /events and /events/stream accept graph=<iri> (400 for an unregistered or overlay graph) and combine it with types and group by AND. Without graph a read returns ROOT's events exactly as before: named-graph rows are excluded, a ROOT event's wire form has no graph key, and lag counts only the reader's own graph, so a caught-up ROOT consumer does not see its lag climb with board writes it never receives. events.graph is TEXT (the IRI, like subject), nullable with NULL = ROOT, added by an idempotent migration with a partial index; no backfill. A read-only handle on an unmigrated store treats every row as ROOT. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
reads.rs crossed the 500-code-line gate with it; it belongs next to committed_graph_registered in overlays.rs anyway. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
Review of the event-push design (P1): a `graph` column on `events` is not
rollback-safe. quipu opens a store whatever its schema, and the older
`events_after` has no graph predicate, so after a revert every ROOT consumer
would receive board events.
- Named-graph events go in `graph_events` (own offsets), and their
readers' cursors in `graph_consumers`. `events` and `consumers` are
exactly what every older binary reads; ROOT offsets no longer move with
board writes.
- `events_after_in`, lag, consumer resume and `POST /events/commit {graph}`
(and its MCP twin) read and write the graph's own log. Pruning holds each
graph's log back by that graph's consumers only.
- respace and share-completeness classify both tables.
- The named-graph half moves to `store/graph_events.rs` (size gate).
The rollback acceptance pair (`events_rollback_tests.rs`) pins the
pre-change reader, resume and pruner as their literal SQL:
- run against this store, they see only the ROOT log, at its offsets, and
never touch a board row;
- re-upgraded, the board reader resumes from its own cursor;
- control: the rejected column design leaks to the same reader.
Docs: the graph consumer contract for older servers. They ignore `graph`, so
discard events not tagged with your graph, and do not commit when the
response does not echo `graph`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
…ade-safe
Review (P1): the separate tables were not enough. A server that predates
named-graph events ignores an unknown `graph` and, on `/events/commit`,
upserts that consumer_id's ROOT cursor BEFORE answering. A client that
checked the echoed `graph` afterward was already too late; the reviewer's
paired control moved a ROOT cursor 5 -> 102, skipping 6..9.
Named graphs now use routes an older server answers 404 (writing nothing):
- GET /events/graph?graph=<IRI> reads one graph's own log;
- POST /events/graph/commit {consumer_id, graph, offset} commits its cursor.
`/events` and `/events/commit` refuse `graph`, so no client learns the unsafe
form. `/events/stream?graph=` reads the graph's log; an older stream already
refused `graph` on events. MCP twins: quipu_graph_events and
quipu_graph_events_commit, so the manifest is 81 tools by default (83 with
owl). The new write route is classified (http_auth, write_kind).
Tests: each route reads only its own log; the ROOT routes refuse `graph` and
the graph routes require it; a graph commit under the SAME consumer_id never
moves its ROOT cursor (the reviewer's control, new side). The named-graph
event stream reads the graph log.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
…OT tools The schema-field audit requires every field a handler reads to be declared in its tool's schema. /events and /events/commit now read `graph` only to refuse it, and declaring it would advertise the refused form, so the audit takes an explicit refused_fields list. It self-checks: a refused field must stay undeclared and must really be read. The two new tools map to events_graph_get and events_graph_commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
e2bb5b0 to
7763954
Compare
Record what step 1 (#784) found: the column is TEXT (the IRI), overlays are registered graphs too and must stay silent, and lag must be scoped to the reader's graph. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
|
Repair for your rollback-commit finding (comments 6382961435 / 6382961637). Both branches are refreshed onto current main. The finding holds: an older The fix: routes, not a parameter.
|
|
Repair2 pre-review gate:7837b1659fe currentOPEN/MERGEABLE but incompleteCI;78477639547 currentOPEN/MERGEABLE REALRED. Normal exactjoblogs Clippy(default)120853169869 andExtendedfeature120853169856 both show tests/tool_schema_fields/parity.rs:65 clippy::doc_markdown missingbackticks around quipu_graph_events andquipu_graph_events_commit; both exit101 compilefailure. Raw33281B/71973B retained. Initial literal error search missed ANSI styling; direct rawtail established actualerror, no emptylog inference. AuthorWu notified. No new source re-review, approval, merge, holdchange or runtime action. Original CHANGES remains until actualnewheadcompletegreen and SAMEverification; runtime/recovery unchanged. |
clippy::doc_markdown refused the bare identifiers under -D warnings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb
|
The red at 7763954 was one lint: Fixed in 1b7f71a: the two tool names are now in backticks. It is a doc comment only, with no code change. Local runs with a private target:
The repair content from 10:50Z is unchanged (routes, not a parameter). The review can self-start once 1b7f71a is fully green. |
…-20261024 # Conflicts: # docs/knowledge/repository-knowledge.json
|
Head is now 6c7d799: 1b7f71a (the lint fix) plus two merges of main.
Local checks with a private target:
The repair content is unchanged since 10:50Z. |
* docs(design): event push over one feed read per transaction Design for the shared feed hub (one store read per transaction for all live subscribers), a multiplexed WebSocket transport over it, and semantic events for registered named graphs with a graph filter. Frames, cursors and authorization are unchanged; delivery is in three independently revertible steps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb * docs(design): pin the events.graph type, committed-only scope and lag Record what step 1 (#784) found: the column is TEXT (the IRI), overlays are registered graphs too and must stay silent, and lag must be scoped to the reader's graph. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb * docs(design): named-graph events in their own tables, so any revert is safe Review of this design (P1): a nullable `graph` column on `events` is not rollback-safe. quipu opens a store whatever its schema, and the older `events_after` has no graph predicate, so after a revert every ROOT consumer would receive board events. Named-graph events now go in `graph_events` and their cursors in `graph_consumers`. Older code never reads either, so a revert to ANY older binary keeps ROOT delivery, offsets, cursors and pruning unchanged, and the graph rows stay retained for re-upgrade. No floor or deploy order is needed, because none could be enforced. Adds the rollback acceptance pair: the pre-change reader and pruner against the post-change store, with the rejected column design as the failing control. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb * docs(design): the graph consumer's contract against an older server Older code ignores the graph parameter, so it serves ROOT events to a graph consumer and records its commit as a ROOT cursor. The consumer discards events not tagged with its graph and commits only on an echoed graph. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb * docs(design): named graphs on their own routes, so older code 404s rather than misreads Review (P1): older /events/commit ignores `graph` and upserts the ROOT cursor of that consumer id before answering, so checking the echo afterward is too late. Named graphs read at GET /events/graph and commit at POST /events/graph/commit; the ROOT routes refuse `graph`. Older code answers 404 and writes nothing, and already refuses `graph` on an event stream. Adds the same-consumer-id case to the rollback acceptance pair. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
SAME repair APPROVE at 6c7d799; native accepted review a73c9ada9364f64c72adc7292faeb5669d297c5b7b73c062adbfae0bea4d2322 (Ian, nonauthor). The separate tables fix the original old-reader leakage, and the distinct GET /events/graph + POST /events/graph/commit routes fix commit-before-echo: the pinned pre-change router has no matching routes/fallback; the old ROOT handler is never entered. Current ROOT handlers reject graph before mutation. Valid identical consumer IDs remain isolated by log. MCP route/schema/write annotations, stream routing, auth inventory, lint-only repair and inherited main integrations were checked in the repair scope; prior review and CHANGES remain historical, superseded by this verified repair. Independent native exact-source rollback controls3PASS, event suite24PASS; HOME157paths+5UIDresourcesCLEAN. Primary exact-head hosted Extended feature surfaces job120862618486 independently read: three new feed_graph_events_tests PASS, server233PASS/4ignored, rollback pair PASS. These hosted results are distinct from local tests; no actual old-server HTTP integration test or production recovery claim. Original private oldSQL mutation control remains the negative. Exact78321/21 and78424/24 completedSUCCESS checked before review. Normal native review IDs above are accepted; merge/exact-main/installed acceptance are separate. Rollback: normal scoped source revert of actual merge, preserving event logs, graph consumers and unrelated main changes; exact revert CI/normal CD smoke. No data/cursor deletion. Archive contains prepared whole-tree inverse and original refusal evidence. |
Step 1 of the event-push design (#783 §4): semantic events for registered committed named graphs, plus a
graphfilter.Behaviour
episode.ingestedandentity.added/entity.updated, one per entity per transaction, with "added" judged within that graph. Events are tagged with the graph IRI in a new nullableevents.graphcolumn.edge.*events (the facts are on the change feed) and notype.new/predicate.new(store-wide, ROOT only).committed_graph_idexcludes overlay-class graphs).GET /events,GET /events/streamand the MCPquipu_eventstool acceptgraph=<iri>. An unregistered or overlay graph returns HTTP 400.graphcombines withtypesandgroupby AND, and there is no all-graphs wildcard.Unchanged for existing consumers
graph, a read returns ROOT's events exactly as before: named-graph rows are excluded, and a ROOT event's wire form has nographkey.lagcounts only the reader's own graph, so a caught-up ROOT consumer's lag does not climb with board writes it never receives.events_after(used by push subscriptions and tests) stays ROOT-only.events_after_intakes a graph.Storage
events.graph TEXTholds the IRI, likesubject, never a term id (it is listed in the respace map asNone). It is nullable with NULL = ROOT, added by an idempotent migration with a partial index(graph, offset) WHERE graph IS NOT NULL, and is not backfilled. A read-only handle on an unmigrated store treats every row as ROOT.Tests
/events/streamfiltered to one graph carries only that graph's events.cargo test --release --no-fail-fastpassed all 81 result groups. Thequipu-serverfeed tests (--features shacl,onnx,server) passed 11/0. Clippy default andshacl,onnx,serverwith-D warningsare clean, as are fmt, markdownlint, the repository-knowledge witness test (rest-api.md re-pinned) andtool_schema_fields.Rollback
Revert. The nullable column is ignored by older code.
🤖 Generated with Claude Code
https://claude.ai/code/session_01BkdpDkAkbTvXvNbNacVpJb