Commit a batch of card writes all-or-nothing, under one index job and one event - #6072
Commit a batch of card writes all-or-nothing, under one index job and one event#6072habdelra wants to merge 15 commits into
Conversation
…oordinator A batch of card writes needs to land together: several changes to several cards that either all commit or none do, under one index job and one index event. This adds the pieces that make that possible. Each base write operation becomes a pure staging function in card-operations/executors.ts: it reads the batch's pre-loaded state and returns the exact bytes every file it touches should hold, writing nothing. `stageCreate` decides a new card's URL and directory, resolves its links and serializes it — from a JSON:API document, or from a named create's lowered `of` + `fill` template. `stageUpdate` merges a patch over the stored file under the same rules the PATCH handler applies. `stageDelete` establishes that its target is there to remove. card-operations/coordinator.ts takes the realm's write lock once, resolves every `lid` up front (path math over the type a card adopts, so a create's URL is known before anything is written and a later entry can link to it), runs every executor in memory, and only then commits. A refusal therefore happens while the realm is still untouched. Realm gains `_commitBatchUnlocked`, which is the batch-write path extended with a removal leg so writes and removals commit under one index job and one index event, and `FileWriteResult.contentHash`, which is the version a caller passes back as `baseVersion`. RealmIndexUpdater gains `enqueueChanges` for a change set that mixes removals with updates; `enqueueUpdate` is that with one operation for the whole set. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBB6JdRo5XCY5WhUkgBckS
… loop's index arithmetic The staging decisions — which files a batch would write, where each local id resolves, whether the batch reaches the commit at all — are checkable without a realm, and checking them against a stub is what makes "nothing is committed" an assertion about the property itself rather than a proxy for it. The realm- driven suite keeps what only exists against a realm: bytes on disk, one index job, one index event, and byte-for-byte agreement with the PATCH handler. `stageCreate` handles its primary before its side-loaded resources rather than walking one list and branching on the index, which drops two non-null assertions and lets both it and `stageUpdate` share one side-loaded staging step. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBB6JdRo5XCY5WhUkgBckS
…dex event A created card lands under its type's directory, so a root-relative module reference in its `adoptsFrom` resolves against that directory rather than the realm — the test was asking for a module that is not there. Say where module references resolve from, in the one place that decides it, and have the test name the module the way a caller who does not want to reason about the directory would. The realm broadcasts into the Matrix room out of band from the commit, so the one-event assertion waits for the batch's own event to arrive rather than reading the room once. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBB6JdRo5XCY5WhUkgBckS
Five of the tests were refusals the coordinator makes before it commits — a duplicate local id, a link to one nothing creates, a removal with no target, two entries on one card, a type change. None of them reach the realm, and all of them are already checked against the stub, where "nothing is committed" is the property itself. What is left is what a stub cannot show: bytes on disk, one index job, one index event, and byte-for-byte agreement with the PATCH handler. Each remaining test now works on its own card, so one realm serves the whole file. Rebuilding it per test bought no isolation and cost a Matrix session room each time — enough of them in a burst to trip Synapse's room-creation limit and fail a test on the realm's inability to open a session. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBB6JdRo5XCY5WhUkgBckS
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
boxel/packages/runtime-common/realm.ts
Lines 2500 to 2501 in 45bf428
If a later adapter write or removal fails—for example because the disk becomes full or a path is unwritable—earlier iterations have already modified their files, and this method has no rollback or atomic rename strategy before rejecting commitBatch. The advertised all-or-nothing operation can therefore leave a partially applied batch on disk and out of sync with the index; the commit needs transactional staging/rollback across every write and delete.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A `lid` and a directory come from the caller and are spliced into a path, so both are checked before a URL is built from them. A `lid` of `../realm` — or `%2e%2e/realm`, which no character check catches — resolved out of the type's directory and onto the realm's own config file, and a `?` or `#` cut the stored path short so the card's id and its file stopped naming each other. Two checks, because neither covers the other: each has to be a plain path segment, and the path that comes back out of the URL has to be the path that went in. An unchanged file now records the hash it reports as well as returning it. A file written before the realm recorded hashes carries none on its row, so returning a token the row does not hold made the next write quoting it as `baseVersion` report a moved base for a file that had not moved. Refusals from the local-id pre-pass carry the entry's position, like the ones executors raise. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBB6JdRo5XCY5WhUkgBckS
…s not The guarantee is over the entries: every way an entry can be wrong is found while the realm is still untouched. It is not over the file system. The realm changes a batch's files one at a time with no rollback, so a mid-commit failure leaves the files handled before it changed — as it does for every multi-file write the realm serves. Reaching past that needs transactional staging in the write primitive, which is not something the coordinator can do above it, so the comments say where the line is rather than implying there isn't one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBB6JdRo5XCY5WhUkgBckS
|
[Claude Code 🤖] Answering the third finding from the Codex review — "Roll back files when a batch commit fails partway". It was left as the review's top-level body rather than an inline thread, so it needs a reply here. The finding is accurate about the behavior. It is not introduced here. What I have not done, and why. The remedy the finding names — transactional staging and rollback across every write and delete — is a change to the realm's write primitive, affecting every caller of it, and it is not something the coordinator can provide from above. Doing it here would widen this change well past the coordinator and would put the realm's whole write path under a rewrite that deserves its own review rather than riding along. What I have done is stop the PR from claiming more than it delivers, in Worth its own ticket, and I want to name the sharper half rather than leave it implied: when a write fails partway, the method rejects before enqueueing the index job, so the files that did land are on disk and never indexed. Disk and index then disagree until an unrelated edit or a full reindex. That divergence is the more damaging half of this and is probably cheaper to close than full transactional staging. Generated by Claude Code |
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical atomicity and overwrite risks, along with validation gaps, block approval.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds staged, all-or-nothing card batches with one realm commit, index job, and event.
Changes:
- Adds create, update, and delete staging with local-ID resolution.
- Supports mixed writes/removals, content hashes, and unified indexing.
- Adds coordinator and real-realm coverage for atomicity and indexing.
File summaries
| File | Reviewed changes and findings |
|---|---|
packages/runtime-common/tests/card-operations-batch-test.ts |
Tests staging, link resolution, validation, and refusal behavior. |
packages/runtime-common/realm.ts |
Commits mixed changes and records hashes. Critical (1 vote): sequential commit operations can partially persist on failure. |
packages/runtime-common/realm-index-updater.ts |
Supports mixed update/removal index change sets. |
packages/runtime-common/card-operations/types.ts |
Defines batch result and version metadata. |
packages/runtime-common/card-operations/index.ts |
Exports batch operations and staging types. |
packages/runtime-common/card-operations/executors.ts |
Stages card mutations and named creates. Findings: critical (1 vote) possible overwrite of existing create destinations; moderate (3 votes) missing named-create parameter validation; moderate (1 vote each) insufficient included validation for creates and updates, mutation of caller-owned resources, and inconsistent lid-less included-resource handling. |
packages/runtime-common/card-operations/coordinator.ts |
Coordinates validation, locking, staging, and commits. Moderate (1 vote each): empty lids bypass validation, and primary document.data.lid values are not indexed. |
packages/realm-server/tests/card-operations-commit-test.ts |
Tests filesystem commits, indexing, events, hashes, and no-op behavior. |
packages/realm-server/tests/card-operations-batch-test.ts |
Registers shared batch-operation coverage. |
Review details
Suppressed comments (6)
packages/runtime-common/card-operations/coordinator.ts:253
- [Claude Code 🤖] The truthiness check treats an explicitly supplied empty string as if no
lidwas supplied, socreate({ lid: '' })bypassesassertPathSegment, mints a UUID, and returns nolidinstead of rejecting the invalid caller id. This disagrees with the included-resource path, which validates any stringlid; distinguishundefinedfrom an empty string before indexing and staging (includingcreateIdentity).
let primary = entry.document?.data;
if (entry.op === 'create' && entry.lid) {
claim(
entry.lid,
createIdentity(entry, primary, paths, new Map()),
packages/runtime-common/card-operations/coordinator.ts:254
- [Claude Code 🤖] This indexes only
entry.lid, but a raw JSON:API create carries its local id ondocument.data.lid(the same shape accepted bycreateCard). A standard payload withdata.lid = "owner"therefore gets a generated URL, and any later relationship referring toowneris rejected as an unknown local id. Normalize/validate the primary resource lid into the staged identity before indexing and staging.
if (entry.op === 'create' && entry.lid) {
claim(
entry.lid,
createIdentity(entry, primary, paths, new Map()),
`entry ${index}`,
packages/runtime-common/card-operations/executors.ts:217
- [Claude Code 🤖] This loop only filters
lidand foreign-realm resources; it never validates thatincludedis an array or that each side-loaded value is a card resource. A malformed side-load can therefore reachstageSideLoaded/the serializer (or fail as a 500), unlike the existing POST validation inrealm.ts:6044-6058. Validate the container and every included resource before staging.
for (let resource of entry.document?.included ?? []) {
// A side-loaded resource with no `lid` is not staged: it has no id to be
// created under and nothing in the batch can link to it, so the client
// sent a resource the realm has no way to name. One naming another realm
// is not this batch's to write.
if (
typeof resource.lid !== 'string' ||
namesForeignRealm(resource, ctx.realmURL)
) {
continue;
packages/runtime-common/card-operations/executors.ts:396
- [Claude Code 🤖] The update path has the same validation gap: malformed
includedvalues are accepted past this loop and can be passed tostageSideLoaded/serialization, whereas the existing PATCH handler rejects a non-array or non-card side-load with 400 atrealm.ts:6204-6219. Apply the same included-resource validation before staging update side-loads.
for (let resource of included) {
if (
typeof resource.lid !== 'string' ||
namesForeignRealm(resource, ctx.realmURL)
) {
continue;
}
writes.push(await stageSideLoaded(resource, resource.lid, url.href, ctx));
packages/runtime-common/card-operations/executors.ts:245
- [Claude Code 🤖]
stageSideLoadedmutates the caller-owned included resource viapromoteStagedLinksandvisitModuleDeps. If a later entry fails, the batch has still changed its input, and retrying the same entries can stage already-resolved module/link values; this contradicts the executor's documented pure-staging contract. Clone the resource before these rewrites and serialize the clone.
let identity = stagedLid(lid, ctx);
promoteStagedLinks(resource, ctx);
visitModuleDeps(resource, (moduleId, setModuleId) => {
setModuleId(ctx.resolveModuleId(moduleId, relativeTo));
});
packages/runtime-common/card-operations/executors.ts:397
- [Claude Code 🤖] This silently drops an included resource whose
lidis absent, while the existing PATCH handler writes such an in-realm resource using a generated UUID (realm.ts:6392-6405). Consequently the same patch produces different files depending on whether it goes through the batch path, despite the stated PATCH merge parity. Either apply the same generated identity here or reject lid-less included resources consistently.
for (let resource of included) {
if (
typeof resource.lid !== 'string' ||
namesForeignRealm(resource, ctx.realmURL)
) {
continue;
}
writes.push(await stageSideLoaded(resource, resource.lid, url.href, ctx));
}
- Files reviewed: 9/9 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…ing gaps A create mints a card; it does not replace one. A caller choosing its own local id can choose one a stored card already answers to, and committing over it destroyed that card with nothing said. Each staged entry now reports the files it brings into existence, and an occupied destination refuses the batch inside the same lock the commit runs in — the answer the atomic endpoint already gives an `add` whose href is taken. Four gaps alongside it: - A named create validates every param its declaration asks for before substituting any of them. A missing value resolved to `undefined` and the field was left off the card, so a caller who forgot one got a card quietly missing it rather than being told. - `included` is held to being a list of card resources. A malformed side-load reached the serializer and came back as an internal failure, where the card endpoints answer the caller with a 400. - A side-loaded resource is rewritten on a copy. Staging is what makes a batch abandonable, and rewriting in place left the caller's own document carrying resolved links after a batch that committed nothing. - A create's local id is read from the resource as well as the entry, the way a POST body carries it. A payload naming its card only on the resource was minted under a generated id, and relationships elsewhere in the batch pointing at that local id resolved to nothing. The local-id pre-pass reads both the local id and the side-loads through the same accessors the executors use, so it claims the identities staging will ask for and refuses a malformed payload in the same terms. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBB6JdRo5XCY5WhUkgBckS
|
[Claude Code 🤖] The Copilot review listed six further findings as suppressed comments in its body, so they have no threads to answer on. Four are real and fixed in Fixed
Does not reproduce
Verification. 32 stub tests (80 assertions) and all 7 realm-driven tests pass against Generated by Claude Code |
Host Test Results 1 files ±0 1 suites ±0 2h 26m 2s ⏱️ - 6m 53s Results for commit 9de3b7a. ± Comparison against earlier commit 5e0b5b9. Realm Server Test Results 1 files ±0 216 suites ±0 1h 17m 4s ⏱️ - 2m 35s Results for commit 9de3b7a. ± Comparison against earlier commit 5e0b5b9. |
… staging gaps A batch's promise is that a refusal costs nothing: the realm untouched, no index job queued, no event out. Several checks the realm applies per file as it writes were only reaching the batch inside the commit, where a refusal already costs the files written ahead of it. - The size ceiling is applied to every staged write up front, so a payload the realm will not store is refused while nothing is written. - Indexing already in flight is drained inside the lock before anything is staged. Staging serializes a card against its type's definition, and a module written moments earlier may still be indexing. - An empty batch returns without taking the lock; broadcasting an empty index event would tell every subscriber that something changed. A `baseVersion` is now compared against a fingerprint of the bytes just read rather than the file's recorded row. A file changed out from under the realm is re-indexed without being rewritten, so the row can name a version the bytes no longer hold — the exact case a base version exists to catch. The staging gaps: - A create's side-loaded resource keeps the module references it was sent with. Resolving them against the primary resolved them against a card one directory deep, which is not where the side-load lands. - An update stages no mints. A side-load addressed by a caller-chosen local id may name a card the caller is deliberately rewriting, which is what the PATCH handler this mirrors does; treating it as a mint answered 409 where the handler succeeds. - An update carrying no `data` is refused rather than dereferenced. - A local id inside a `data` array is refused rather than silently dropped. A collection's edges are stored one key per member, so a local id written there has no key of its own to record the link on. - A template reading `actor` with none in scope is refused, and an `instance(…)` attribute is read as an own property. The id-to-path round trip refuses a path that resolves outside the realm rather than throwing out of `RealmPaths.local` while building its message. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBB6JdRo5XCY5WhUkgBckS
The realm is shared across this file's tests and index broadcasts are fire-and-forget, so a straggler from an earlier test can land inside the window and fail an assertion counting events over it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBB6JdRo5XCY5WhUkgBckS
|
[Claude Code 🤖] Requirements landed this week that this coordinator will grow into; nothing here needs to change in this PR, but four shape choices are cheap now and expensive later:
|
The card-operations barrel is the entry point a consumer imports directly rather than through the package barrel, precisely so it costs less than the package barrel does. Reaching back through `../index.ts` for one function gives that subpath an eager dependency on the whole package. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBB6JdRo5XCY5WhUkgBckS
|
[Claude Code 🤖] Treating the actor as a bare identity string here is right, and it is the shape the rest of the stack is converging on:
If this PR ends up assembling a |
|
[Claude Code 🤖] Agreed on the actor being a bare identity string — that is what this resolver already returns, and there is no code path here that builds an actor record. On the two follow-ons: the first is right about the destination but cannot land ahead of the lowering change, and the second does not reproduce. Both probed against the coordinator rather than read off the source. Dropping the So the allowance is load-bearing for the current lowered shape. It should go in the same change that stops emitting the argument, not before it — otherwise a declaration that lowers today stops staging. An actor marker on a link field already lands as a link, not an attribute. The template resolver routes a value through The actor's user id is the link's target, which is the identity a link to them needs. There is no state where a readable definition puts the id in a link's attribute slot, so there is nothing to refuse. The one path that does write it as an attribute is a type whose entry cannot be read: A 400 is not available there either — the field's declared kind is precisely what is missing. This is also not actor-specific: with no definition, every marker except What I am adding rather than changing: nothing currently covers the
Generated by Claude Code |
|
[Claude Code 🤖] Correcting my previous comment, and withdrawing the test I said I would add. I was wrong that an actor marker landing in Where the refusal belongs is still not the executor, and for the reason the probe showed rather than the one I gave. A link to the acting person is supposed to arrive through The Not adding the No code change here as a result. The actor reaches this coordinator as a bare Generated by Claude Code |
|
note that the comments above are a bunch of churn resulting from teh fact that I filed CS-12933 to narrow |
Three gaps a convergence review turned up, each a divergence from the handler the batch mirrors: - An oversized payload was reported as `500 internal-error`. `assertWriteSize` throws the realm's own `CardError` carrying 413, and `atEntry` preserved a status only for an `OperationFailure`, so the one thing that tells a caller to send less was replaced by the one thing that tells it to send again. The status is carried across under a new `payload-too-large` code; `POST`, `PATCH` and `/_atomic` all answer 413 for the same bytes. - A removal took any stored `.json` at its href, `realm.json` among them, so a batch could take the realm's own configuration with it. `DELETE` answers 404 there because the index holds no card. The staged bytes answer the same question without a read, which keeps a card written moments ago deletable where consulting the index would not. - A write into `_screenshot/` was staged. The subtree is claimed by capture serving, so a realm file stored there could never be read back; direct writes and `/_atomic` each refuse it, and a batch is the third way in. Removals stay admitted, as they are in both other paths, since they are the recovery route for anything already there. Two tests were green for the wrong reason and are now falsifiable: - The oversized-payload test stubbed `assertWriteSize` with a bare `Error`, which cannot carry a status, so the flattening above was invisible to it. The stub now throws what the realm throws. - The `realmURL` assertion could not fail: the realm stamps its own URL over whatever the patch carried, so comparing against the client's value held whether or not the strip ran. It now pins the stamp, which is the property the stored file actually depends on. The unchanged-file hash is recorded for the file's metadata resource, not for `baseVersion` — that is computed from the bytes read inside the write lock and never consults the row. The comment claiming otherwise is corrected, and its test now asserts the row it was only pretending to exercise. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBB6JdRo5XCY5WhUkgBckS
|
[Claude Code 🤖] The one-change-per-file refusal in Concretely: where |
A batch is a sequence, so two entries naming one card are meant to build on each other. They were refused instead, on the reasoning that both would be computed against the state the batch started from and the second would silently discard the first. That is a consequence of staging every entry against one fixed snapshot, not a fact about batches: feed the later entry the bytes the earlier one staged and they compose, with nothing discarded and the file written once holding the last of them. An entry's staged writes are folded into the state the next entry stages against, and the commit collapses the sequence per file. The refusal is gone; it belongs to parallel siblings, which nothing here can express yet. The corners this opens, each falling out of the sequence rather than needing a rule of its own: - A removal takes its path back out of the staged state, so a change aimed at the same card afterwards finds nothing there and refuses the way it would outside a batch. - A removal after a change drops that change's write entirely rather than writing bytes the same commit then unlinks, and both entries report no state, which is what a removal reports. - A create's bytes are not folded in, so a card the batch mints is still reached by the local id other entries link to rather than by a URL they target. Tying the batch's meaning to path math a caller has to reproduce is the coupling the local id exists to avoid. `baseMatched` now compares against the version the entry actually merged over, captured as it stages. Reading it back from the stored map at commit time was correct only while that map stood still; a second entry on one card would otherwise have been told its base matched when it merged over bytes that base never named. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBB6JdRo5XCY5WhUkgBckS
|
[Claude Code 🤖] Done in Each entry's staged writes are folded into the state the next entry stages against, and the commit collapses the sequence per file. The corners, each falling out of the sequence rather than getting a rule of its own:
One thing the change surfaced that was not in the ask. Five tests, each verified to fail without the specific mechanism it covers rather than merely passing: two entries composing with the file written once and both reporting the committed version; the base-version distinction above; the two removal corners; and a minted card refusing a later entry that targets its URL. Locally green: 188 tests, including Deliberately unchanged: a removal followed by a create at the same path is still refused, since Generated by Claude Code |
… is safe Composing two changes to one card opened three ways for a batch to lose data quietly, and the removal guard shipped alongside it was documented as protection it does not give. - A write to a path an earlier entry removed cancelled that removal, and a removal reports the same `null` whether it happened or not — so a card could be resurrected while the entry that asked to remove it reported success. Refused: the two entries are asking for opposite things, which is not a sequence to compose. - A side-loaded resource is serialized whole rather than merged, so it cannot compose over an earlier entry's change the way a second patch does. Landing it last dropped that change with nothing said while its entry still reported success and a matching base version. Refused, naming both entries. - A removal took a card under an ignored path. Ignored files are never visited, so they never get an index row, and `DELETE` — which needs one — refuses them forever; reading the bytes off disk is not the same permission. The batch would have destroyed a file no other caller can, and the realm would go on ignoring the absence. `realm.json` is not protected by the card-document check and was never going to be: a realm's config is itself a card document, and `DELETE` removes it too, so refusing it here would put the batch out of step with the endpoint rather than protect anything. The comment saying otherwise is corrected, and the test that appeared to prove it stored a bare settings object no realm writes — it now stores the config the way a realm does and pins the parity. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBB6JdRo5XCY5WhUkgBckS
Two files conflicted, both where main's read-your-writes drain — which tags every incremental index job with the matrix user whose request produced it — met this branch's refactor of the same call sites. `realm-index-updater.ts`: main added `initiatedBy` to the inline options type that this branch had extracted into `IncrementalIndexOptions` and generalized from `enqueueUpdate` to `enqueueChanges`. The field moves onto the interface, and onto the `Pick` lists `update` and `updateChanges` expose, so a caller can still scope a drain to itself. `realm.ts`: four hunks of the same shape, where main passed `delete: true` alongside `initiatedBy` and this branch had already moved the removal into the change set as `operation: 'delete'`. The change-set form stands and carries `initiatedBy` through each. The batch commit path now tags its job too. Nothing in the merge required that — the coordinator is new on this side and main could not have known to thread it — but a batch would otherwise be the one write path whose index job no reader can wait on, silently opting out of a guarantee every other path gives. `CommitBatchOptions.actor` is that user already. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBB6JdRo5XCY5WhUkgBckS
The write-route tagging test shadows the updater to capture each job's `initiatedBy`, and it shadowed `enqueueUpdate` on the reasoning — its own comment's — that the awaited path calls through it, so one spy covered both. That stopped being true when the awaited path became `updateChanges`, which enqueues a change set directly: the spy sat on a method no write reaches any more, recorded nothing, and reported every route as untagged. The tagging itself was never broken. `initiatedBy` travels from the request context through `updateIndexAndCollectInvalidations` to the deferred the drain reads, and the suite's other cases — which exercise that drain rather than the enqueue — passed throughout. Shadowing `enqueueChanges` restores the coverage and widens it: every incremental job is enqueued there, `enqueueUpdate` and both awaited forms delegate to it, so one spy now covers every route whichever form it entered by. Verified to still fail when the tag is dropped at the shared call site, rather than merely to pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBB6JdRo5XCY5WhUkgBckS
What this adds
A batch of card writes that lands all-or-nothing: several changes to several
cards that either all commit or none do, under one index job and one index
event.
Executors (
packages/runtime-common/card-operations/executors.ts) — eachbase write operation is a pure staging function. It reads the batch's
pre-loaded state, works out the exact bytes every file it touches should end
up holding, and returns them. It writes nothing, enqueues nothing, broadcasts
nothing, and rewrites only its own copies of what it was handed.
stageCreatedecides the new card's URL and directory, resolves the linksit declares, and serializes it against its type's definition — from a
JSON:API document, or from a named create's lowered
of+filltemplatewith
params()/actor()/instance()resolved by substitution.includedresources become additional creates linked bylid.stageUpdatemerges a patch over the stored file under the rules thePATCHhandler applies: the merge base is the file rather than the index, achanged
adoptsFromis refused,type/meta.realmInfo/meta.realmURL/
meta.screenshotsare stripped, arrays replace rather than merge,relationships merge, and no change means no write.
stageDeleteestablishes that its target is a card that is there to remove.A stored
.jsonthat is not a card document — a config, a fixture, anythingthe realm keeps but does not serve as a card — is refused with the 404
DELETEgives for the same URL. The bytes already read answer that, so itcosts no read and keeps a card written moments ago deletable, which
consulting the index would not. A realm's own
realm.jsonis not coveredby this and is not meant to be: it is itself a card document, and
DELETEremoves it too, so refusing it here would put the batch out of step with the
endpoint rather than protect anything the endpoint protects.
A create mints a card rather than replacing one, so a destination a stored
card already occupies refuses the batch with 409 — the answer the atomic
endpoint gives an
addwhose href is taken. A created card's id and directorycome from the caller and are spliced into a path, so both are held to naming
the file the card is stored in: each has to be a plain path segment, and the
path that comes back out of the URL has to be the path that went in. Together
those refuse a
..in any spelling walking out of the type's directory, and a?or#cutting the stored path short so the id and the file stop namingeach other.
A local id links a card only where a link can be recorded. A collection's
edges are stored one key per member —
field.0,field.1— so a local idwritten inside a single
dataarray has no key of its own to carry the linkand is refused rather than staged with the edge silently missing.
The coordinator (
card-operations/coordinator.ts) takes the realm's writelock once, drains indexing already in flight, resolves every
lidup front,reads every target's file, runs every executor in memory, and only then
commits. A
lidresolves by path math over the type a card adopts — no read,no write — so a create's URL is known before anything is written and a later
entry can link to it. An entry that cannot be carried out is therefore found
while the realm is still untouched: nothing to undo, no index job to cancel,
no event a subscriber could have acted on. A duplicate
lidand a link to oneno entry declares are each refused with the position of the entry that
produced them.
A batch is a sequence, so two entries may name one card and the second builds
on the first: it merges over the bytes the first staged, and the commit writes
the file once holding the last of them. What composes is a change over a
change. Three things do not, and are refused rather than resolved by
ordering:
opposite things, and cancelling the removal would report it as completed —
which a caller cannot tell from a removal that happened.
is serialized whole rather than merged, so it cannot compose over anything;
letting it land last would drop the earlier change with nothing said while
that entry still reported success.
visited, so it never gets an index row, and
DELETE— which needs one —refuses it permanently. Reading its bytes off disk is not the same
permission.
The corners that do fall out of the sequence: a change after a removal finds
nothing there and refuses as it would outside a batch; a removal after a
change drops that change's write rather than writing bytes the same commit
then unlinks, and both entries report no state; and a card the batch mints is
not folded in, so it is still reached by the
lidother entries link torather than by a URL they target.
Checks the realm applies per file as it writes are applied to the staged bytes
first, for the same reason. The size ceiling refuses an oversized payload
while the realm is still untouched, carrying the realm's own 413 through under
a
payload-too-largecode — the status is the remedy, since it tells a callerto send less rather than to send again.
_screenshot/is refused as a writedestination, as it is for direct writes and for
/_atomic, because a realmfile stored there could never be read back; removals stay admitted there, as
they do in both other paths, being the recovery route for anything already
stored. An empty batch takes no lock and announces nothing — an empty index
event would tell every subscriber that something changed.
That guarantee is over the entries. It is not over the file system: the realm
changes a batch's files one at a time with no rollback, as it does for every
multi-file write it serves, so a mid-commit failure leaves the files handled
before it changed. Both the coordinator's header and
_commitBatchUnlockedsay where that line falls rather than implying there isn't one.
Realm._commitBatchUnlockedis the batch-write path extended with aremoval leg, so writes and removals commit under one index job and one index
event rather than two.
FileWriteResultgainscontentHash— the file'sversion, and the token a later request passes as
baseVersion.A
baseVersionis reported against the version the entry actually mergedover, captured as that entry stages: within a batch the state moves, so the
second change to one card is compared to what the first staged rather than to
what the batch started from. That fingerprint comes from the bytes read inside
the write lock rather than from the file's recorded row — a file changed out
from under the realm is re-indexed without being rewritten, so the row can
name a version the bytes no longer hold, which is the exact case a base
version exists to catch. A file whose bytes are already what the caller staged
records its hash on the row as well as reporting it, which fills the row the
file's metadata resource reads.
RealmIndexUpdater.enqueueChangesis the general form ofenqueueUpdate: onejob over a change set that may mix removals with updates, so the invalidation
fan-out for all of them is computed against one snapshot of the realm.
enqueueUpdateis that with a single operation for the whole set, unchangedfor its callers.
Not in scope
No existing handler or route changes. The
POST/PATCH/DELETEhandlers keep their own copies of this logic; the facade that dispatches them
through the coordinator removes those copies.
Two deliberate divergences for that facade to settle. A batch create refuses
an occupied destination, where the plain
POSThandler overwrites it today.And a batch update writes a card under a
_-prefixed path, whichPOST,DELETEand/_atomicall admit and onlyPATCHrefuses — the batch sideswith the three rather than the one.
Test plan
packages/runtime-common/tests/card-operations-batch-test.ts— 52 testsagainst a stubbed realm, covering what the coordinator decides before it
commits: the paths a create stages at,
lidresolution across entries (on theentry and on the resource), side-loaded creates and side-loads over stored
cards, the patch merge rules down to relationship-map merging and the field
metadata a replaced array drops, two entries composing over one card and the
version each reports, the three conflicts that do not compose, a change after
a removal and a removal after a change, a minted card not being a later
entry's target, a realm config removed at parity with the endpoint,
realm-managed keys never reaching the file, an id that would name a file
outside the type it creates, an occupied destination, a malformed side-load,
an unlinkable local id, a named create missing a declared param or an actor,
bytes over the realm's size ceiling and the 413 they answer with, a write into
the capture subtree, a removal aimed at stored JSON that is not a card, an
empty batch, that indexing is drained before anything is serialized, that the
caller's own document is left alone, every refusal and its entry position, and
— the point of the stub — that a refused batch never calls the commit at all.
packages/realm-server/tests/card-operations-commit-test.ts— 7 tests againsta real realm, covering what only a realm can show: a create-and-link batch's
files and resolved links on disk; a failing second entry leaving the first
unwritten with no
incremental-indexrow in the jobs table and no realm event;a write and a removal producing exactly one index job and one index event
carrying both URLs;
baseMatchedtrue and false; a no-op patch leaving thefile's bytes and modification time alone with nothing queued, and recording on
the row the version it reports; and an
updateentry writing a filebyte-identical to a
PATCHof the same document through the existing handler.Locally green: 192 tests across the two new files,
card-endpoints-test,atomic-endpoints-testandatomic-batch-indexing-test— all three untouched— and
card-operations-core-testandcard-operations-dispatch-test.🤖 Generated with Claude Code
https://claude.ai/code/session_01EBB6JdRo5XCY5WhUkgBckS