[pixels-retina] TableUpdateData Loses Operation Order Across Delete, Insert, and Update #1398

Activity

1. gengdy1545

   assigned

   AntiO2

   on Sep 3, 2026
2. gengdy1545

   added

   bugSomething isn't workingSomething isn't working

   on Sep 3, 2026
3. gengdy1545

   added the

   Bug

   issue type on Sep 3, 2026
4. gengdy1545

   added this to the Real-time CRUD milestone on Sep 3, 2026
5. AntiO2 commented on Sep 3, 2026

   

   AntiO2

   on Sep 3, 2026

   Last edited by AntiO2

   Collaborator

   More actions

   I implemented a fix in Pixels-Sink to preserve the original operation order when sending CDC events to Retina.

   The root cause is that a single TableUpdateData stores INSERT, DELETE, and UPDATE operations in separate repeated fields. Retina processes these fields in a fixed order, so mixing different operation types in one TableUpdateData loses their original ordering. For example, an INSERT followed by a DELETE could be executed as DELETE followed by INSERT.

   The Sink now groups only adjacent rows with the same operation type into one TableUpdateData, while preserving the order of those groups:

   INSERT, INSERT, DELETE, DELETE, INSERT
   

   is encoded as:

   [INSERT, INSERT], [DELETE, DELETE], [INSERT]
   

   Implementation: pixelsdb/pixels-sink#20
6. gengdy1545 commented on Sep 4, 2026

   

   gengdy1545

   on Sep 4, 2026

   CollaboratorAuthor

   More actions

   I implemented a fix in Pixels-Sink to preserve the original operation order when sending CDC events to Retina.

   The root cause is that a single TableUpdateData stores INSERT, DELETE, and UPDATE operations in separate repeated fields. Retina processes these fields in a fixed order, so mixing different operation types in one TableUpdateData loses their original ordering. For example, an INSERT followed by a DELETE could be executed as DELETE followed by INSERT.

   The Sink now groups only adjacent rows with the same operation type into one TableUpdateData, while preserving the order of those groups:

   INSERT, INSERT, DELETE, DELETE, INSERT
   

   is encoded as:

   [INSERT, INSERT], [DELETE, DELETE], [INSERT]
   

   Implementation: pixelsdb/pixels-sink#20

   Mixing different operation types in the same TableUpdateData does not necessarily cause incorrect results. The ordering only matters when multiple operations on the same key are included in the same TableUpdateData. So the fix is mainly needed to preserve the relative order of operations that target the same key, rather than requiring a strict global ordering across all CDC events.