Repository navigation
fix(ts): fail the bench when Chrome stalls - #35
Conversation
Quest-Claim: 1d26b1d5180756e7
Chrome's page reports each step it starts by beacon, and `main.ts` fails the run, naming the step and with Chrome's stderr, once 30 s pass without one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
…ome-timeout # Conflicts: # ts/bench/entry.ts
A beacon per sample slowed Chrome's JS samples by about 10%; one per round is within noise. The stall limit grows with the sample length to cover a round. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
…ome-timeout # Conflicts: # quest/m2/README.md # ts/bench/entry.ts
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a3d1d25ddc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| proc.kill(); | ||
| reject(new Error(`Chrome ${why}:\n${await stderr}`)); |
There was a problem hiding this comment.
Reject the stall without waiting for Chromium's stderr to close
If the stalled browser does not terminate on SIGTERM, or a surviving child retains its stderr pipe, await stderr never completes. The stall timer therefore never rejects promise, and chrome() remains stuck instead of entering its cleanup block. I reproduced this with the same failure handler and a Bun subprocess that ignores SIGTERM: the promise remained pending after the timeout handler ran and settled only after SIGKILL. Collect stderr incrementally so the failure can report what is available without waiting for EOF, and ensure process cleanup cannot block indefinitely.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in e607917. Chrome's stderr now goes to target/bench/chrome.log instead of a pipe, so a child process left behind cannot hold anything open, and the failure path reads the file and does not wait for EOF. A stall also sends SIGKILL now, because a hung browser may ignore SIGTERM. I re-ran the injected hang: it failed after 30 s with Chrome's stderr in the error, and no Chromium processes were left.
(written by Opus 5.5)
A child Chrome leaves behind can hold a stderr pipe open, so waiting for the pipe's end could hang the failure itself. A hung browser may also ignore SIGTERM. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
|
Merging. Summary:
(written by Opus 5.5) |
Problem
A hung Chromium renderer stalled
just ts benchuntil the job timed out. It was seen twice overnight, first in #15, and a rerun passed.Approach
Chrome's page now sends a
/progressbeacon at the start of each step: loading each file, checking each route on each workload, warming up each body, and each timing round.main.tsfails the run once a step goes 30 s without a beacon. The error names the step and includes Chrome's stderr, for example:When Chrome exits early, it fails through the same path. Nothing retries. On a stall the browser gets SIGKILL, since a hung process may ignore SIGTERM. Chrome's stderr now goes to
target/bench/chrome.loginstead of a pipe, because a child process left behind could hold the pipe open and keep the failure itself waiting forever.During timing there is one beacon per round, not one per sample. A beacon per sample slowed Chrome's JS samples by about 10%: over 3 alternating run pairs, beacons-off/beacons-on was 0.88 to 0.90 for
hand-written,translated, andA/A. With one beacon per round, the same comparison gives 1.00 to 1.03, which is within noise. Because a round is the longest step, the limit ismax(30 s, 1000 × --sample-ms).I checked the guard by injecting a never-resolving
awaitin round 2, before and after the stderr change. Each time the run failed 30 s later with the error above and left no Chromium processes behind.What the hang is
I caught one live: another agent's
just ts benchhad sat in Chrome for 1 h 22 min.target/bench/chromepersists across runs and Chrome uses the network in the background.The hang did not come back in 13 Chrome-only runs plus 5 full runs here. If it does, the error will name the loading step and include Chrome's stderr.
Impact
bench/time.ts'smeasuretakes aprogresscallback.Alternatives
--roundsand the workload count.fetch: the page would handle a response for each post.SharedArrayBufferand a worker: no main-thread cost, but a lot of machinery for an XS guard. Check and warm-up run the same code first and already name both.Follow-ups
--disable-background-networking --disable-component-updateand a fresh profile per run. Today the browser downloads components and talks to GCM during the measurement. That is noise whether or not it causes the hang, but it is unproven as the cause, so it is not in this PR.🤖 Generated with Claude Code
https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn