Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
1a148d2
Trace peer route failures in recovery test
forhappy Sep 30, 2026
beaebd9
Pin Cellule to current main revision
forhappy Sep 30, 2026
37e052e
Record Cellule main signed API performance rerun
forhappy Sep 30, 2026
6d560fd
Clarify transaction benchmark and qualification limits
forhappy Sep 30, 2026
05bbc6d
Preserve oversized transaction read fallback across peers
forhappy Sep 30, 2026
9ab8f4a
Record peer read fix qualification and fresh release performance pair
forhappy Sep 30, 2026
319f26f
Verify signed follower acknowledgements and successor recovery
forhappy Sep 30, 2026
2ec8cf7
Wire experimental follower durability into server startup and S3 reco…
forhappy Sep 30, 2026
6a62ec8
Record follower-mode release comparison and remaining performance gaps
forhappy Sep 30, 2026
ca40e69
Cache resident handles for authenticated peer invocation
forhappy Sep 30, 2026
c2abaf7
Record peer-cache release measurements and SDK qualification failures
forhappy Sep 30, 2026
60feaf9
Keep large same-Cell transactional reads on the query path
forhappy Sep 30, 2026
83bb06a
Record compact read verification and refreshed release benchmark pair
forhappy Sep 30, 2026
a9e841d
Publish transaction prepares and commit in one coordinator command
forhappy Sep 30, 2026
e3b0113
Record prepared-commit release comparison and SDK qualification failures
forhappy Sep 30, 2026
4462bba
Expose follower enrollment and append failures for durability diagnosis
forhappy Sep 30, 2026
856a622
Retain follower diagnosis and repeated process recovery evidence
forhappy Sep 30, 2026
415cb13
Record latest-main confirmation and fresh release API benchmark pair
forhappy Sep 30, 2026
7180586
Measure runtime durability paths and settle follower recovery baseline
forhappy Sep 30, 2026
bcc7a1f
Record instrumented latest-main release verification and API benchmarks
forhappy Sep 30, 2026
c7fe25f
Upgrade Cellule routing and publication optimizations and cache backe…
forhappy Sep 30, 2026
d02ced0
Record Cellule 70bd25f release benchmarks and preserve cold recovery …
forhappy Sep 30, 2026
69957e5
Resolve resident owners without provider reads and restore refused cr…
forhappy Sep 30, 2026
96ee9cf
perf: use one fresh enrollment read for follower appends
forhappy Sep 30, 2026
7e78bf8
docs: record one-read append release verification and perf results
forhappy Sep 30, 2026
cb8cc9c
fix: recover account index before coordinator discovery
forhappy Sep 30, 2026
bfd017c
docs: record fresh release provider metrics and SQLite comparison
forhappy Sep 30, 2026
f97a197
perf: reuse durable registration for resident coordinators
forhappy Sep 30, 2026
d181348
docs: record resident admission release comparison
forhappy Sep 30, 2026
757ed22
perf: observe authenticated follower append phases
forhappy Sep 30, 2026
a0c0605
docs: record release follower phase and SQLite comparison
forhappy Sep 30, 2026
6c7ea0f
perf: bootstrap independent coordinator Cells concurrently
forhappy Sep 30, 2026
a4640b0
docs: record parallel coordinator release and qualification gaps
forhappy Sep 30, 2026
acc0352
build: upgrade Cellule to reviewed main e07670e
forhappy Sep 30, 2026
968411f
docs: record Cellule main release comparison with timeouts
forhappy Sep 30, 2026
f67390a
perf: fuse small coordinator preparation discovery
forhappy Sep 30, 2026
8a1edce
docs: record coordinator snapshot release comparison
forhappy Oct 1, 2026
110ecdd
docs: compare native-volume provider and SDK performance
forhappy Oct 1, 2026
6e5ccb7
feat: select single or partitioned Cell placement per table
forhappy Oct 1, 2026
ff74855
docs: record verified single and multi-Cell release performance
forhappy Oct 1, 2026
69cb5cf
docs: clarify placement conversion limits
forhappy Oct 1, 2026
86a3b66
perf: coalesce durable coordinator admission metadata
forhappy Oct 1, 2026
ac2a71a
docs: record coordinator admission verification and release refresh
forhappy Oct 1, 2026
2e31eb8
perf: reuse acknowledged transaction participants on first attempt
forhappy Oct 1, 2026
1d95477
docs: record acknowledged BEGIN release verification and Cell limits
forhappy Oct 1, 2026
0ada49d
perf: batch returned partition updates with compact durable results
forhappy Oct 1, 2026
fb22ddd
docs: record returned-update verification and release comparison
forhappy Oct 1, 2026
e9ccd41
Reuse acknowledged fresh write transaction completion
forhappy Oct 1, 2026
d210b30
Record acknowledged completion verification and release comparison
forhappy Oct 1, 2026
412e4ea
Record completed recovery qualification failures
forhappy Oct 1, 2026
961fd0a
perf: overlap cold coordinator catalog publication
forhappy Oct 1, 2026
230312e
docs: record cold catalog release and qualification gaps
forhappy Oct 1, 2026
6115b1f
perf: share in-flight follower fleet discovery
forhappy Oct 1, 2026
e7e41ca
Record follower discovery verification and failed release comparison
forhappy Oct 2, 2026
c8752bd
Overlap fresh remote-owner enrollment with authority reads
forhappy Oct 2, 2026
70f601e
Record fresh routing regression and release comparison
forhappy Oct 2, 2026
8dcd9d2
Bound small participant prepare replies and preserve wide failure images
forhappy Oct 2, 2026
5879bac
Record bounded prepare admission and failed release qualification
forhappy Oct 2, 2026
80a3645
Upgrade Cellule and check enrolled members before node log rotation
forhappy Oct 2, 2026
26a92f5
Record latest Cellule release verification and SQLite comparison
forhappy Oct 2, 2026
3747c54
Keep node heartbeat renewal independent of stalled log transitions
forhappy Oct 2, 2026
340c8d3
Record heartbeat regression and failed four-Cell release qualification
forhappy Oct 2, 2026
20d7216
Report node lease renewal phases when serving is fenced
forhappy Oct 2, 2026
7635051
Record renewal-phase diagnostic and unresolved transaction timeouts
forhappy Oct 2, 2026
efe1b74
perf: reuse acknowledged read metadata for routed data cells
forhappy Oct 2, 2026
c2e28af
bench: record scoped read release and remaining sqlite gap
forhappy Oct 2, 2026
e716ca6
perf: coalesce independent routed transaction prepares
forhappy Oct 2, 2026
7e9b4f7
bench: retain prepare batching release qualification failures
forhappy Oct 2, 2026
1581d33
perf: reuse completed node versions for heartbeat CAS
forhappy Oct 2, 2026
1c487b9
fix: retain directory retirement receipts across owner release
forhappy Oct 2, 2026
7177ee2
test: verify recovered index fences after idle release
forhappy Oct 2, 2026
508fda8
perf: bound saved transaction image reply reservations
forhappy Oct 3, 2026
ce4e7b7
perf: resolve active local owners and invalidate stale owner epochs
forhappy Oct 3, 2026
906c856
fix: recover credential owners and pin fused Cellule streams
forhappy Oct 3, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,6 @@
!.env.example
*.log
__pycache__/

# Local benchmark reports, logs and fixtures.
/benchmarks/
16 changes: 9 additions & 7 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 8 additions & 7 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@ repository = "https://github.com/crabbuild/beyonddb"
rust-version = "1.97"

[workspace.dependencies]
cellule-app = { git = "https://github.com/crabbuild/cellule.git", rev = "9e17746a633ca1046bd93866866074226091f81a" }
cellule-host = { git = "https://github.com/crabbuild/cellule.git", rev = "9e17746a633ca1046bd93866866074226091f81a" }
cellule-peer-http = { git = "https://github.com/crabbuild/cellule.git", rev = "9e17746a633ca1046bd93866866074226091f81a" }
cellule-runtime = { git = "https://github.com/crabbuild/cellule.git", rev = "9e17746a633ca1046bd93866866074226091f81a" }
cellule-store = { git = "https://github.com/crabbuild/cellule.git", rev = "9e17746a633ca1046bd93866866074226091f81a" }
cellule-ltx = { git = "https://github.com/crabbuild/cellule.git", rev = "9e17746a633ca1046bd93866866074226091f81a" }
cellule-app = { git = "https://github.com/crabbuild/cellule.git", rev = "1d0648b3b5ae0cca040c614b505b4d72bb769ab1" }
cellule-host = { git = "https://github.com/crabbuild/cellule.git", rev = "1d0648b3b5ae0cca040c614b505b4d72bb769ab1" }
cellule-peer-http = { git = "https://github.com/crabbuild/cellule.git", rev = "1d0648b3b5ae0cca040c614b505b4d72bb769ab1" }
cellule-runtime = { git = "https://github.com/crabbuild/cellule.git", rev = "1d0648b3b5ae0cca040c614b505b4d72bb769ab1" }
cellule-store = { git = "https://github.com/crabbuild/cellule.git", rev = "1d0648b3b5ae0cca040c614b505b4d72bb769ab1" }
cellule-ltx = { git = "https://github.com/crabbuild/cellule.git", rev = "1d0648b3b5ae0cca040c614b505b4d72bb769ab1" }
async-trait = "0.1"
blake3 = "1.8"
futures-util = "0.3"
Expand Down Expand Up @@ -61,6 +61,7 @@ extenddb-storage = { git = "https://github.com/crabbuild/extenddb.git", rev = "7
futures-util.workspace = true
fs4 = "0.13.1"
getrandom = "0.3"
object_store.workspace = true
reqwest.workspace = true
serde.workspace = true
serde_json.workspace = true
Expand All @@ -74,12 +75,12 @@ uuid.workspace = true
zeroize = "1"

[dev-dependencies]
prost = "=0.13.5"
cellule-store = { workspace = true, features = ["test-support"] }
aws-config.workspace = true
aws-credential-types.workspace = true
aws-sdk-dynamodb.workspace = true
ed25519-dalek.workspace = true
extenddb-engine = { git = "https://github.com/crabbuild/extenddb.git", rev = "7eaa89b437feed0af0f05883d3f1493f86c6fc6d" }
object_store.workspace = true
tempfile.workspace = true
tokio = { workspace = true, features = ["macros", "rt-multi-thread", "net", "time"] }
16 changes: 11 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,10 @@ partition-key ranges. A GSI is maintained asynchronously from a journal
committed with the base item. Cross-Cell transactions use a durable
coordinator decision and idempotent participant resolution.

### Choose placement per table

Set the `CreateTable` tag `beyonddb:cell-model` to `single`, `auto`, or `partitioned`. Keep one base data Cell for a table that fits its resource budgets, start at one and permit growth, or provision multiple ranges immediately. Omitting the tag preserves `initial_partitions`. GSIs use separate Cells; changing the tag later does not migrate the table. Live model conversion is not implemented. See [Cell model selection](docs/scaling.md#choose-a-tables-cell-model) and the [AWS CLI example](docs/user-guide.md#create-a-table-and-wait-for-it).

### When a write becomes durable

![Sequence of a signed PutItem: ExtendDB validates, BeyondDB routes, Cellule commits and publishes LTX, then the response returns](diagram/beyonddb-architecture/durable-write.svg)
Expand All @@ -64,11 +68,13 @@ stream intent commit in one Cell command. The successful response follows
durable publication. [PNG version](diagram/beyonddb-architecture/durable-write@2x.png) ·
[Detailed architecture and recovery design](docs/architecture.md)

The serving binary currently waits for object-store publication on each
durable write. Cellule's follower-log mode is not enabled in BeyondDB. An
opt-in persistent follower store and authenticated peer transport exist, but
the durability provider is not installed in serving and successor recovery
is unfinished. See the [follower durability design](docs/follower-durability.md).
The default serving path waits for object-store publication on each durable
write. Experimental `follower_durability_enabled` installs Cellule's node-log
provider and can acknowledge after every enrolled follower fsyncs the commit.
A three-process signed SDK test verifies item mutations, same-partition
transaction replay, and stream records after an owner kill with object uploads
withheld. Broader fault and performance qualification remains open. See the
[follower durability guide](docs/follower-durability.md).

## Current capability boundary

Expand Down
6 changes: 6 additions & 0 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,12 @@ The [implementation record](implementation-status.md#current-verified-slice)
maps these paths to tests. The full upstream protocol suite remains an
[acceptance gate](implementation-status.md#acceptance-proof-for-a-server-claim).

### Creation-time Cell model extension

`CreateTable.Tags` accepts `beyonddb:cell-model` with `single`, `auto`, or `partitioned`. `single` fixes the base table to one data Cell; `auto` starts at one and permits splitting; `partitioned` starts at the configured count, with a minimum of two. Omitting the tag retains existing behavior. The choice is durable table-generation metadata, not an AWS DynamoDB setting.

Later tag operations change metadata only. They do not change placement, and `UpdateTable` does not convert models. GSIs stay separate; LSIs stay with their base items. Read [Cell model limits](scaling.md#choose-a-tables-cell-model) and the [CLI creation example](user-guide.md#create-a-table-and-wait-for-it).

## Indexes

Local secondary indexes (LSIs) share a base Cell's atomic mutation. Global secondary indexes (GSIs) have separate owner Cells and receive base changes asynchronously.
Expand Down
4 changes: 2 additions & 2 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ sequenceDiagram
Driver->>B: PREPARE item images and locks
B-->>Driver: Durable prepare receipt
end
Driver->>Coord: Publish COMMIT or ABORT
Driver->>Coord: Record prepares + COMMIT, or publish ABORT
Coord-->>Driver: Durable decision
par Resolve owners
Driver->>A: Apply or discard decision
Expand All @@ -108,7 +108,7 @@ sequenceDiagram
Driver-->>Client: Return or replay result
```

The original coordinator and participant identities survive routing changes. If a driver dies after publishing the decision, startup or serving recovery finishes participant resolution. A caller timeout leaves the outcome unknown until the durable decision is inspected. A client token distinguishes a matching replay from a different request. Transactional reads use shared key locks and captured images. See the [transaction protocol and failure cases](cross-cell-transactions.md).
The original coordinator and participant identities survive routing changes. If a driver dies after publishing the decision, startup or serving recovery finishes participant resolution. A caller timeout leaves the outcome unknown until the durable decision is inspected. A client token distinguishes a matching replay from a different request. Cross-Cell transaction reads use shared key locks and captured images. Single-Cell reads use one atomic query with a compact internal response when it fits the query envelope. See the [transaction protocol and failure cases](cross-cell-transactions.md).

## Range split and GSI projection

Expand Down
Loading
Loading