Repository navigation
feat(pagination): collision-proof cursor pagination for batch-ingested rows - #404
Open
woahwhattheheck wants to merge 6 commits into
Open
woahwhattheheck wants to merge 6 commits into
woahwhattheheck wants to merge 6 commits into
Conversation
…rows Encode cursors as opaque (timestamp, id) tuples so ledger-derived rows that share an ingestion timestamp paginate without skips or duplicates. Includes batch collision and concurrent-write stability tests, plus cursor encoding docs in the module rustdoc.
Contributor
|
@woahwhattheheck is attempting to deploy a commit to the christabel888's projects Team on Vercel. A member of the Team first needs to authorize it. |
Use saturating addition before the existing snapshot clamp so a large usize page limit remains an upper bound after a cursor or at exhaustion. Preserve cursor ordering and encoding, and extend the existing batch regressions for both directions and maximum-size limits.
Sort references to the input snapshot rather than deep-cloning the full snapshot and cloning the page a second time. Preserve stable compound-key order, cursor boundaries, the saturated page-end fix, and input ordering. Add one focused clone-count and result regression covering both directions, same-timestamp boundaries, exhausted and zero-sized pages, and oversized limits. No dependencies, public API, cursor encoding, or storage changes. Rust checks have not been executed for this continuation; existing results remain scoped to their previously tested source.
Author
|
I am claiming consideration for any GrantFox reward applicable to my contribution in this PR for #330, payable to @woahwhattheheck. Please confirm that this PR is associated with the original contributor's campaign record and included in the reward evaluation, and confirm any allocation and payout date once approved. Existing contributors' attribution and any remaining acceptance requirements remain documented in this PR. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #330
Adds
backendpagination with a compound(timestamp, id)cursor so corridor/anchor/transaction list endpoints do not skip or duplicate rows when many share an ingestion timestamp after batch ledger writes.Design
{ "ts": <ms>, "id": "<tiebreaker>" }.(ts, id) < (cursor.ts, cursor.id)(ascending inverts).(ts, id)for efficient seeking.backend/src/pagination/mod.rs.Tests
batch_collision_test.rs— identical-timestamp batch, small page size, every row exactly once.concurrent_write_stability_test.rs— background inserts while paginating; already-issued cursors stay stable.Compatibility
backend/; focused on pagination only.