Skip to content

Add hi/lo positioned file IO and size64 - #1923

Closed
oxura wants to merge 2 commits into
bendlang:mainfrom
oxura:feat/1843-wide-file-offsets-20261029
Closed

oxura wants to merge 2 commits into
bendlang:mainfrom
oxura:feat/1843-wide-file-offsets-20261029

Conversation

@oxura

@oxura oxura commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

This picks up the remaining 64-bit file IO part of #1843. The r+ mode and U32 write_at already landed in #1853 and #1887.

Adds File.read_at64(file, hi, lo, max), File.write_at64(file, hi, lo, bytes) and File.size64(file). Offsets are hi * 2^32 + lo, below 2^63; sizes come back as (hi, lo). Positioned operations leave the sequential cursor alone, and the existing U32 size still reports overflow rather than truncating.

One JS trap worth calling out: Node 22 silently ignores a BigInt position in writeSync and readvSync. Wide reads use readSync instead; Bun uses typed pread/pwrite. Node returns EOVERFLOW before a write crossing its safe-integer position range, rather than writing at a rounded or ignored offset. C and Bun support the full signed offset range.

Checked 14 file cases on Bun and 12 on native C, including the new sparse-file regression up to 8 GiB; that regression also passes as standalone Node JS. Separately used real files of size 2^53 + 5 to check odd read/write offsets and size halves against Python's OS calls: exact bytes and unchanged cursor on all three backends, with Node's unsupported write leaving the file untouched. Those sparse files used 4 KiB each and were closed/unlinked. Repo gate: 54/54.

@nicolas-abril nicolas-abril self-assigned this Oct 29, 2026
@nicolas-abril

Copy link
Copy Markdown
Collaborator

Thanks, this is careful work, and the Node BigInt-position trap is a good find. We're not taking 64-bit offsets as U32 halves into Base, though. U64 is on WONTFIX's SOON list (we're adding it ourselves), and 64-bit offsets and sizes should take that type. read_at64/write_at64/size64 on halves would leave Base with a second set of names to keep after U64 lands, or a breaking change to remove them.

Until then this works as a library effect on Base's File handle (WONTFIX, "A handle type is a law of Base"). Two already exist: bend-osdev-fileio (github.com/ScottMoore0/bend-osdev) and bend-ml-binary-read on BendHub, both tested past 4 GiB on both lanes. We know a library effect costs a promise in --check-only today; we're accepting that until U64. The rest of #1843 already landed in #1853 and #1887.

@oxura

oxura commented Oct 29, 2026

Copy link
Copy Markdown
Contributor Author

Got it. I'll leave the halves API out of Base and wait for U64 rather than introduce a second set of names. Thanks for the pointers to the existing library effects.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants