Skip to content

fix(tests): the FIFO peer gives up when its run dies first - #1917

Merged
Lorenzobattistela merged 1 commit into
mainfrom
fix/fifo-peer-leak
Oct 28, 2026
Merged

Lorenzobattistela merged 1 commit into
mainfrom
fix/fifo-peer-leak

Conversation

@Lorenzobattistela

Copy link
Copy Markdown
Collaborator

tests/io/file_open_fifo_peer (from #1899) starts a peer shell that echoes the run's first line to the real stdout, then writes "hi" into a FIFO the run opens. The peer inherits the shard's stdout, so if the run dies before opening the FIFO, the peer blocks forever on printf hi > fifo, orphaned, holding the node's ssh session open. The test gate then waits out the 20 min shard timeout instead of reporting the failure, which looks like a hung cluster node.

Seen with #1906, whose JS runtime throws ReferenceError: str_in is not defined at peer_arm's return: every test gate run hung on the node holding this test's shard and left two orphaned peers there (checkup lane and JS lane).

The fix, in both the C and JS peers:

  • IFS= read -r l || { rm -f "$1"; exit 1; }: a run that dies before its first line closes the pipe, and the peer exits.
  • (sleep 10; [ -p "$1" ] && rm -f "$1" && kill $$) >/dev/null 2>&1 &: in the checkup lane the error text itself becomes the first line, so the peer also needs a bound on the FIFO wait. The watchdog only acts if the FIFO is still there (the normal path removes it within milliseconds), so it never kills the cat the peer execs into, and it holds no stdout.

Checked:

  • main + this: test gate PASS: 1781 / 1781; repo PASS: 54 / 54. The test prints exactly ready, hi on the C, JS and checkup lanes, with no peer, watchdog or FIFO left after.
  • refactor(comp): name clz and popcount after Metal's builtins #1906 + this: the gate finishes in 15 s with FAIL io_file_open_fifo_peer [interp] / [js] (PASS: 1746 / 1781, the rest of that shard's checkup lane fails with it) instead of hanging, and no peer is left on the node.

🤖 Generated with Claude Code

file_open_fifo_peer's peer shell held the shard's stdout while it blocked
on the FIFO. A run that died before opening it (a JS ReferenceError, an
alarm) left the peer orphaned, so the node's session never closed and the
test gate waited out its 20 min shard timeout instead of reporting the
failure. Now the peer exits when its pipe closes before a line, and a
watchdog removes a FIFO still unread after 10 s and kills the peer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Lorenzobattistela
Lorenzobattistela merged commit 8407907 into main Oct 28, 2026
1 check passed
@Lorenzobattistela
Lorenzobattistela deleted the fix/fifo-peer-leak branch October 28, 2026 17:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant