Skip to content

fix: message cursor pagination must not skip equal-timestamp rows - #16367

Open
kumarpriyanshu09 wants to merge 1 commit into
LibreChat-AI:devfrom
kumarpriyanshu09:fix/message-cursor-pagination-16310
Open

kumarpriyanshu09 wants to merge 1 commit into
LibreChat-AI:devfrom
kumarpriyanshu09:fix/message-cursor-pagination-16310

Conversation

@kumarpriyanshu09

Copy link
Copy Markdown

Problem

getMessagesByCursor filtered and sorted only by the selected timestamp field, and encoded nextCursor as that scalar alone.

When page 1 ended at timestamp T, page 2 resumed with createdAt < T (desc) or > T (asc). Any remaining rows that also had timestamp T were permanently skipped — still in Mongo, unreachable through cursor continuation.

Repro from #16310: 30 messages with the same createdAt, pageSize=25 → page 2 returned 0 instead of the remaining 5. The old equal-timestamp test never crossed a page boundary.

Solution

Reuse the composite (sort value, _id) cursor pattern already used by conversation / shared-link pagination in data-schemas:

  • Encode nextCursor as base64 JSON { primary, id }
  • Resume with $or of “past primary” and “same primary, past _id”
  • Sort by { [sortField]: sortOrder, _id: sortOrder }
  • Keep legacy scalar cursors working (ISO / date strings) for in-flight clients
  • When CLIENT_MESSAGE_SELECT strips _id, still fetch it for the tie-breaker, then remove it before returning

No route changes — api/server/routes/messages.js already calls db.getMessagesByCursor.

Tests

packages/data-schemas message cursor suite now exercises the real method (it previously reimplemented the buggy query inline), plus:

  • 30 equal-createdAt rows, pageSize=25, desc and asc → page 2 returns the remainder with no gaps/duplicates
  • Legacy scalar createdAt cursor still resumes
  • Unusable / malformed cursors start from the beginning
cd packages/data-schemas && npx jest src/methods/message.spec.ts -t "Message cursor pagination" --coverage=false

Fixes #16310

getMessagesByCursor filtered and sorted only by the timestamp field and
encoded nextCursor as that scalar alone, so rows sharing the page-boundary
timestamp were permanently skipped on the next page.

Reuse the composite (sort value, _id) cursor pattern already used by
conversation/shared-link pagination, keep legacy scalar cursor compatibility,
and add regression coverage for equal-timestamp page boundaries.

Fixes LibreChat-AI#16310

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant