You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Multi-process WAL catch-up: a replay that stops at a torn record still advances the bookmark past later commits #198
Found by @mrdevrobot while reviewing #169. The defect is on main too; the code references are to the #169 branch (b721fee).
WriteAheadLog.ReadCommittedPagesSince(fromOffset, toOffset) (~L718-830) stops without an error when it hits a record it can't read: a short read, an implausible size, a decryption failure or an unknown type. Its caller, StorageEngine.TryCatchUpWithSharedWal, then still reports success and moves the bookmark to shmEnd.
So a torn record left by a crashed writer makes this engine skip every record after it. That includes later, valid commits from other processes, which this engine then never replays. Its next checkpoint can truncate the WAL that still holds them. Reproduced with a hand-truncated record.
Possible fix
Have the parser report whether it reached toOffset, for example by returning the offset it stopped at. TryCatchUpWithSharedWal should treat a short read as failure: leave the bookmark where it is and don't truncate the WAL. A torn record at the very end, written by a crashed process, needs a rule of its own. It can be skipped only once the writer lock shows that no process is still appending it.
Activity
fix(storage): don't skip WAL records past a torn one in multi-process…