Repository navigation
A reader sees the objects file as it is now before giving up on a record - #1332
Merged
Merged
Conversation
noha
force-pushed
the
a-reader-sees-the-file-grow
branch
2 times, most recently
from
October 9, 2026 11:55
55b02fa to
208adda
Compare
…cord A read-only reader of a database another process appends to can be sent by the index to a record its stream does not reach yet: over NFS the stream knows the objects file as long as the client last cached it. The metabase check of 674726be read 'receiver of < is nil' at a position 174 KB past the size its stream knew, and the record was there, intact; the check of 62398e23 has failed the same way since 2026-10-06. SoilVisitor>>recordAt:, where a read has failed already, opens the objects file of the segment anew and reads the record once more before it gives it up. Reads that succeed are not touched, and nothing outside the visitor changes. Only a read-only database is opened anew; a writable one keeps its stream, since closing a file gives up the locks of the process on it. A record that still cannot be read no longer ends the walk of the SoilTrackingVisitor: its trackers are told (#trackUnreadableRecord:in:), and the SoilDatabaseConsistencyTracker reports a SoilUnreadableRecordFinding with object id, version, position and file size. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
noha
force-pushed
the
a-reader-sees-the-file-grow
branch
from
October 9, 2026 11:56
208adda to
d5c5f23
Compare
noha
added a commit
that referenced
this pull request
Oct 9, 2026
The metabase check of 674726be, with #1332 in place, found 1:2168 at position 12324667346 although the objects file, opened anew, ended at 12324316892: the writer's index had reached the NFS server before its records, as writeback keeps no order between files. Opening the file again does not help there; only time does. So the rereading moves out of SoilVisitor, where #1332 put it for every reader and every failed read, into SoilTrackingVisitor>>visitRecordAt:, which catches the SoilUnreadableRecord anyway. It reads again only a record that is late (#isLate:): of a read-only database, at or past the size of the objects file the reader knows. It opens the file anew, reads at once, and then after the pauses of #rereadDelays, half a minute in all. A record inside the file is damaged and named at once. SoilVisitor>>recordAt: names an unreadable record again, as before #1332. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A read-only reader of a database another process appends to can be sent by the index to a record its stream does not reach yet: over NFS the stream knows the objects file as long as the client last cached it. The metabase check of 674726be read 'receiver of < is nil' at a position 174 KB past the size its stream knew, and the record was there, intact; the check of 62398e23 has failed the same way since 2026-10-06.
SoilObjectFile reads a record that starts at or beyond the end it knows, or runs off it, once more after seeing the file as it is now. A read-only database opens the file anew for that; a writable one keeps its stream, since closing a file gives up the locks of the process on it.
A record that still cannot be read no longer ends the walk of the SoilTrackingVisitor: its trackers are told (#trackUnreadableRecord:in:), and the SoilDatabaseConsistencyTracker reports a SoilUnreadableRecordFinding with object id, version, position and file size.