Repository navigation
Cursor and offset based paginators #30
Description
Activity
The adapter interface actually already implements offset+limit rather than page-based pagination. The handling of pages is handled by the Pagerfanta object. To use a generic offset-based pagination, call
$adapter->getSlice()directly.For cursor-based pagination, I don't think this can be implemented as a generic pagination library, as I don't see how to write a generic logic turning the cursor into a filtering condition on the data source.
Seeing that a couple of the Doctrine repos are working on adding cursor pagination tools (doctrine/orm#12364 for the ORM and doctrine/mongodb-odm#2992 for the MongoDB ODM), I let Claude Code come up with a proposal for that, factoring in those PRs. Here's what I got back with commentary bits trimmed out.
(I haven't done anything to implement cursor pagination for any project I'm working on, so I have nothing to judge this against as far as whether this is feasible or even a good idea; I'm just doing my part to at least look into it all)
Feature: Cursor (Keyset) Pagination Support
Proposed Design
Cursor pagination is added as a parallel track — new classes in new namespaces, with no modifications to
AdapterInterface,Pagerfanta, or any existing code.New Packages / Locations
All cursor classes are added to existing packages (no new packages):
pagerfanta/core—lib/Core/Cursor/— shared interfaces, value objects, andCursorPaginatorpagerfanta/doctrine-orm-adapter—lib/Adapter/Doctrine/ORM/— ORM cursor adapterpagerfanta/doctrine-mongodb-odm-adapter—lib/Adapter/Doctrine/MongoDBODM/— ODM cursor adapters
Core Contracts
Pagerfanta\Cursor\CursorAdapterInterface— the unified adapter contract:/** * @template-covariant T */ interface CursorAdapterInterface { /** @param int<1, max> $length */ public function getSliceAfter(?string $cursor, int $length): CursorResult; /** @param int<1, max> $length */ public function getSliceBefore(?string $cursor, int $length): CursorResult; public function supportsCounting(): bool; /** @return int<0, max>|null */ public function getNbResults(): ?int; }
Pagerfanta\Cursor\CursorResult— value object returned by adapters:/** * @template-covariant T */ final class CursorResult { /** * @param iterable<array-key, T> $items * @param int<0, max> $count Number of items in this result page */ public function __construct( public readonly iterable $items, public readonly ?string $nextCursor, public readonly ?string $previousCursor, public readonly bool $hasMore, public readonly int $count, ) {} }
Pagerfanta\Cursor\CursorDirection— typed enum replacing string flags:enum CursorDirection { case Forward; case Backward; }
Pagerfanta\Cursor\CursorPaginator— stateful wrapper, mirrors thePagerfantaclass:/** * @template T */ final class CursorPaginator implements CursorPaginatorInterface { public function __construct(CursorAdapterInterface $adapter) {} public static function createForCursorWithMaxPerPage( CursorAdapterInterface $adapter, ?string $cursor, int $maxPerPage, CursorDirection $direction = CursorDirection::Forward, ): self {} public function setCursor(?string $cursor, CursorDirection $direction = CursorDirection::Forward): void {} public function setMaxPerPage(int $maxPerPage): void {} public function getCurrentPageResults(): iterable {} public function getNextCursor(): ?string {} public function getPreviousCursor(): ?string {} public function hasNextPage(): bool {} public function hasPreviousPage(): bool {} public function supportsCounting(): bool {} public function getNbResults(): ?int {} /** Iterate all results across all pages via forward cursor traversal */ public function autoPagingIterator(): \Generator {} }
Results are lazily fetched and cached. Calling
setCursor()orsetMaxPerPage()invalidates the cache, matching the behaviour of the offset-basedPagerfantaclass.Cursor Encoding
All adapters use the same encoding format: URL-safe base64-encoded JSON of an associative array of field values. For example, a composite cursor on
createdAt+id:{"createdAt": "2024-01-15T10:30:00+00:00", "id": 42}encoded as
base64_encode(json_encode($values, JSON_THROW_ON_ERROR)).\DateTimeInterfacevalues are serialized usingDateTimeInterface::ATOM. The associative format (keyed by field name) is more robust than a positional array and makes cursors self-describing.Utility Adapters in Core
CursorArrayAdapter— in-memory adapter for arrays. Sorts on construction, extracts cursor values via a callable. Always supports counting. Intended for tests and simple use cases.CursorTransformingAdapter— decorator that transforms items from anyCursorAdapterInterface, leaving cursor strings unchanged (cursors reference the underlying data, not the transformed output). Mirrors the existingTransformingAdapterexactly in structure.
Doctrine ORM Adapter
Phase 1 — Self-Contained (Ships Now)
Pagerfanta\Doctrine\ORM\CursorQueryAdapter— no new Composer dependencies, works with the currentdoctrine/orm: ^2.20 || ^3.5constraint./** * @template T * @implements CursorAdapterInterface<T> */ class CursorQueryAdapter implements CursorAdapterInterface { /** * @param string[] $cursorFields e.g. ['createdAt', 'id'] * @param array<string, 'ASC'|'DESC'> $directions per-field sort directions */ public function __construct( Query|QueryBuilder $query, array $cursorFields = ['id'], array $directions = [], string $rootAlias = 'e', bool $countingSupported = true, ) {} }
Implementation notes:
- Clones the
QueryBuilderinternally on construction to avoid mutating the caller's instance - Keyset
WHEREclause generated usingQueryBuilder::expr()for composite cursors:(alias.f1 > :v1) OR (alias.f1 = :v1_eq AND alias.f2 > :v2)
getSliceBefore()reverses sort directions, then reverses the result arrayhasMoredetection via the$length + 1over-fetch patterngetNbResults()clones QB, stripsORDER BY/pagination, runsCOUNT
Phase 2 — Native ORM Wrapper (Deferred)
Once doctrine/orm#12364 is released, a
NativeCursorQueryAdapterwill wrap Doctrine ORM's ownCursorPaginatordirectly. The Phase 1 adapter will remain available.Note: Doctrine ORM's
CursorPaginatoris forward-only;getSliceBefore()in the native wrapper will throw\Pagerfanta\Exception\RuntimeException.
MongoDB ODM Adapters
Phase 1 — Self-Contained (Ships Now)
Two new adapters, no new Composer dependencies, works with current
doctrine/mongodb-odm: ^2.11.CursorQueryAdapter— wraps ODMQuery\Builder:public function __construct( Builder $queryBuilder, string $cursorField = 'id', string $direction = 'ASC', ) {}
Uses
->field($cursorField)->gt($value)/->lt($value)for forward/backward navigation.CursorAggregationAdapter— wraps ODMAggregation\Builder:Uses
$matchstage with$gt/$lt,$sort, and$limit. Count query uses thecount('numResults')stage (matching the existingAggregationAdapterpattern).Phase 2 — Native ODM Wrappers (Deferred)
Once doctrine/mongodb-odm#2992 is released, native wrappers for
QueryCursorPaginatorandAggregationCursorPaginatorwill be added. Note: the upstream PR'scount()returns page count, not total results —supportsCounting()will returnfalsein these wrappers unless the upstream API is extended.
View Integration
Deferred. Cursor pagination needs a fundamentally different view contract — "previous" and "next" links only, no page number logic. A
CursorViewInterfacestub will be added tolib/Core/Cursor/View/to reserve the namespace, but no HTML-rendering implementations will ship in the initial feature.The
$routeGeneratorfor cursor views takes the opaque cursor string and returns a URL (distinct from the existing route generator infrastructure which is page-number-based).I would not bother shipping a custom implementation of an ORM paginator:
- it would duplicate upstream work
- the plan described here is flawed, as it expects being able to use QueryBuilder APIs, which won't work to support queries
- the plan described here does not account for complex DQL queries involving joins on toMany relations (also missing upstream in the initial PR, there is a second in-progress PR fixing that work to become correct), where a SQL limit would not work (unless you implement this by reusing the offset-based upstream paginator)
I suggest shipping only an adapter based on the upstream ORM paginator once it works.
All adapters use the same encoding format: URL-safe base64-encoded JSON of an associative array of field values. For example, a composite cursor on
createdAt+id:when defining the abstraction, wouldn't it be better to describe the cursor as an opaque string where each adapter is free to choose how it is built ? If not, it would be hard to use the upstream Doctrine cursor paginators as you would have no guarantee that they would actually use that format for their cursor.
#64 is probably the best shot at landing this with my own skills, knowledge, and prompt engineering techniques unless someone's got a better idea.
Just a few things to consider for v4:
Sometimes there is a need to provide an offset-limit pagination instead of human readable "page numbers". This might be useful for API endpoints. Most of the time the offset would match the
($page-1)*$perPagebut there might be situations where someone requestsoffset: 50, limit: 100. A new Adapter (sub)interface could be introduced to address that.Another idea would be to allow cursor-based navigation, where page ID is a cursor. An example would be Shopify API or AWS Cognito API.