Skip to content

fix(io): park the JS File and Process effects so io_wait flushes stdout - #1899

Merged
nicolas-abril merged 4 commits into
mainfrom
fix/js-park-blocking-effects
Oct 27, 2026
Merged

nicolas-abril merged 4 commits into
mainfrom
fix/js-park-blocking-effects

Conversation

@nicolas-abril

@nicolas-abril nicolas-abril commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Bug: on JS, IO.print holds stdout when it is not a terminal. File.open, File.read*, File.size, File.write* and Process.run run synchronously inside the effect (openSync, readvSync, Atomics.wait), so the loop never reaches io_wait and never writes the held text before blocking. A program that prints and then reads from a peer waiting for that output deadlocks. C runs these effects through io_work, parks the fiber, and flushes in io_wait before select.

Fix: each of these effects parks itself with io_park_on(undefined, 0, k, () => …, 0), due at once, so it runs at the next io_wait, after its io_sync, as in C. This follows the rule io_eff already enforces ("an effect that waits parks itself"), so nothing new goes into the loop:

  • io_run: unchanged.
  • io_wait: skips select (and io_sys, which needs bun:ffi) when it waits on no fd with a zero timeout, so parked files still work on Node.
  • file_write.js: its own io_sync is gone; the wait flushes first.
  • comp.ts: +11 ttok (65,985 / 66,000).

As in C, the fiber now parks, and other fibers run before its result comes back. The remaining difference is inherent to JS: the syscall runs on the main thread, so it blocks every fiber while it runs.

Supersedes #1893 (by @MattCozendey), which added an io_sync to File.read only. Its test tests/io/file_read_stdout_order is kept.

Tested (macOS):

  • bun gates/repo.ts: PASS 54 / 54.

  • All 30 tests/io tests that use File, Process.run or random_u32, plus file_read_stdout_order, pass on Bun.

  • file_read_stdout_order and stdout_file_order pass on Node.

  • Repro (IO.print("ready"), then File.read_line on /dev/stdin, fed by (sleep 3; echo hi) |): "ready" arrives at 0 s on Bun and Node.

  • New tests/io/file_open_fifo_peer: stdout is moved onto a pipe read by a shell that opens a FIFO for writing only after it sees "ready", so a flush that comes after the effect blocks hangs the run. With this PR it passes on native, Bun and through --checkup (stdout is put back for the next test). On main, the Bun lane hangs (killed at 10 s) and native passes.

  • tests/io/file_read_prompt, from fix(io): flush stdout before JS file reads #1895 (by @toprameen): IO.write("prompt> ") with no newline, then File.read_line. Passes on Bun, Node and through --checkup; on main the read runs before the prompt is written.

Not run: native, the cluster gates.

🤖 Generated with Claude Code

@Lorenzobattistela Lorenzobattistela left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merged onto main 757180e, all gates green: repo 54/54, test 1754/1754, ping 81/81, perf 124/124, safe 0 false rejects. The approach looks right to me. One must-fix from the merge with #1887, plus a few small things.

Must fix: File.write_at neither parks nor flushes after the merge. #1887 added file_write_at, which shares file_write_buffer. This PR removes the io_sync() from file_write_buffer and parks only File.write and File.write_bytes, so on JS write_at now runs before the text IO.print holds. Repro on Bun, stdout redirected to a file: open /dev/fd/1 with "w", then IO.print("hello"), File.write_at(f, 0, [88, 89]), IO.print("bye"). On main this prints XYllo then bye. With this PR it prints hello then bye: the positioned write lands first, and the held text then overwrites it. C runs write_at through io_work (file_write.c), so it should park like its siblings, keeping the Fail(22) early return for bad bytes and append files.

Small things:

  • The new comment in file_open.js says every effect C runs through io_work parks here. That becomes true once write_at parks. "As in C" is also only exact for effects that block: C's io_work hands the job to a helper thread right away and flushes later in io_wait, so C does not flush before the effect runs.
  • process_run(...a) with a.pop() reads differently from the siblings. Naming the parameters (program, args, input, max, ms, k) costs about the same.
  • tests/io/file_open_fifo_peer.c: peer_arm ignores mkfifo, pipe and fork failures. If pipe() fails, dup2(p[1], 1) uses uninitialised fds, and the test hangs instead of failing at setup.

Non-blocking notes:

  • Every effect now has to remember the io_park_on(undefined, 0, k, …, 0) wrapper (11 copies so far), and write_at already slipped through. Before, file_write_buffer flushed for every caller. A shared helper next to io_park_on, or a flag on io_eff, would make it hard to forget, if comp.ts ever has the room.
  • Cost on JS (not measured; no bench uses File): every File or Process op now ends the fiber's run and goes through a full io_wait, io_sync included. A read-a-line, print-a-line loop on a pipe makes one write(2) per line instead of one per 64 KiB. That's what C does too, so I'm noting it, not asking for a change.

I closed #1895 in favour of this PR. Its file_read_prompt test (a prompt written with IO.write, no newline, before a read_line) passes here and fails on main. It could be added alongside file_read_stdout_order if you like, but that's optional.

nicolas-abril and others added 3 commits October 27, 2026 20:53
On JS, File.open/read*/size/write* and Process.run ran synchronously
inside the effect, so the loop never reached io_wait and never wrote
what IO.print holds before blocking: a program that prints and then
reads from a peer waiting for that output deadlocked. C runs these
through io_work, parks the fiber, and flushes in io_wait.

Each of these effects now parks itself with io_park_on, due at once, so
it runs at the next io_wait, after its io_sync, as in C. Other fibers
run before its result, as in C. io_wait skips select when it waits on
no fd with a zero timeout, so parked files need no bun:ffi on node.
File.write's own io_sync goes away.

Supersedes #1893; its test file_read_stdout_order is kept.

Co-authored-by: MattCozendey <MattCozendey@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
file_read_stdout_order reads a regular file, which never blocks, so it
pins the order of the output but not that the flush comes before the
effect blocks. Here stdout is moved onto a pipe read by a shell that
opens the FIFO for writing only after it sees "ready", so a late flush
hangs the run. On main the JS lane hangs; native passes on both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1887 added File.write_at on file_write_buffer, which no longer flushes
held stdout itself, so on JS the positioned write ran before the text
IO.print held. It now parks like File.write and File.write_bytes,
keeping its Fail(22) for bad bytes and append files.

Also: process_run names its parameters, the comment above file_open
goes, and the FIFO test fails at setup if mkfifo, pipe or fork fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nicolas-abril
nicolas-abril force-pushed the fix/js-park-blocking-effects branch from b25fd63 to b8f361b Compare October 27, 2026 19:54
@nicolas-abril

Copy link
Copy Markdown
Collaborator Author

Thanks for the review! Rebased onto main and pushed b8f361b:

  • File.write_at now parks like write and write_bytes, and keeps its Fail(22) early return for bad bytes and append files. Your repro (/dev/fd/1 opened with "w", stdout redirected to a file) now prints XYllo then bye on Bun, the same as native. No JS effect that C runs through io_work is left unparked; random_u32 stays as it is on purpose, since WebCrypto never blocks.
  • The comment above file_open is gone.
  • process_run names its parameters.
  • file_open_fifo_peer fails at setup if mkfifo, pipe or fork fails (err_fail in C, a throw in JS).

Tested on macOS: all 33 tests/io tests that use File, Process.run or random_u32 pass on Bun, file_open_fifo_peer passes on native and Bun, and the repo gate passes 54/54.

I left the shared helper for now: comp.ts has 15 ttok of room. And thanks for the pointer to file_read_prompt from #1895.

🤖 Generated with Claude Code

From #1895: IO.write("prompt> ") with no newline, then File.read_line.
The JS twin checks that the prompt was written before the read ran.
Passes with this PR; on main the read runs first.

Co-authored-by: toprameen <11931132+toprameen@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nicolas-abril
nicolas-abril merged commit 7a7e650 into main Oct 27, 2026
1 check passed
@nicolas-abril
nicolas-abril deleted the fix/js-park-blocking-effects branch October 27, 2026 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants