Windows smoke test: fix the crash-loop flake

[wenkaifan0720]

The Windows smoke test's crash-loop case (added in #51) failed about once in ten runs. The failing run was windows-build 35989453550, GPU mode only. The victim tile got no processGone after 8 chrome://kill navigations; the check said "the victim loaded 1 times". The host logs only to OutputDebugString, so the run showed nothing about why.

Making it diagnosable

• Log file. FLUTTER_CEF_LOG_FILE=<path> makes the plugin also append its own and every host's log lines to that file, each with a millisecond tick. CI sets it for each compositing mode and prints the host's crash-loop lines ("renderer terminated … reloading" and "giving up on this browser"), or the whole log when a mode fails. It is documented in the Windows README.
• Timeline. The probe logs a timeline of the victim: each kill, then its page-finished, load-error and processGone events.
• Rounds. The case runs FLUTTER_CEF_SMOKE_CRASH_ROUNDS rounds, 3 in CI, each on a fresh host.

What the logs and experiments showed

• The failure message said nothing. The victim was an authored https page. The navigate that sends a kill clears a tile's authored document, so every reload after a kill went to DNS for a name that doesn't exist and came back as an error page. Error pages report no pageStarted, so "loaded 1 times" is what a passing run shows too.
• The host counted every kill in every experiment. I ran the old kill loop 70 times on CI, instrumented, and all 70 passed; the host logged 3 "renderer terminated" lines and then "giving up" on the 4th kill every time. The runs were:
  • Fresh DNS names: 5 rounds × 2 modes. Each kill's reload failed with ERR_NAME_NOT_RESOLVED in 60–260 ms.
  • A reload that hangs connecting: a TEST-NET address, 2 × 2. Kills sent while the frame was dead with its reload still in flight still counted.
  • Real pages from a local server: held 0–2500 ms, including just before and just after the next kill, 12 × 2.
  • Error pages from a local server: the connection dropped after 0–1500 ms, 14 × 2.
• The one failure didn't reproduce. What the old case did have were two sources of nondeterminism: each reload's pace depended on the runner's DNS, and kills went out once a second whatever the page was doing.

The fix (test only)

• Offline victim. The victim is a data: page, so its reload after a kill never goes to the network. The sentinel stays at an https origin, so the two tiles are still different sites with separate renderers.
• Event-driven loop. Each kill waits for the reload it causes to finish, or for processGone, before the next goes out. A kill that shows neither within 5 s is sent again, and the whole loop is bounded at 90 s.
• No host or policy change. The host behaved correctly in every timing tried.

Separate flake seen along the way (not fixed here)

In one run (35995242385) the runner's network was very slow: example.com took 35 s to load. After kOpShutdown, the software-mode pipe_probe host then didn't exit within the probe's 20 s. The host had closed its pipe, so it was stuck inside CefShutdown.

• Product impact: none. The plugin's reaper ends the host's process tree 3 s after shutdown.
• Gap with macOS: macOS has a hard-exit watchdog that bounds teardown at 30 s, and Windows has none. That bound wouldn't fit pipe_probe's 20 s either, and pipe_probe depends on the public internet.
• Suggested follow-up: port the watchdog, and serve pipe_probe's pages locally.

Verification

• windows-build passes on this head repeatedly; the pass count is in the thread below.
• Each run exercises the crash loop 6 times: 3 rounds in each compositing mode.
• flutter test (280) passes, flutter analyze is clean, and tool/check_comment_tags.sh is clean.

🤖 Generated with Claude Code

[wenkaifan0720]

Rerun results for windows-build on 46992b3 (run 36000564911):

attempt	job	windows-build	crash-loop rounds passed	FAIL lines
1	107635905579	success	6/6	0
2	107638999452	success	6/6	0
3	107641963464	success	6/6	0
4	107644387043	success	6/6	0
5	107647324290	success	6/6	0
6	107649490936	success	6/6	0

That is 6/6 attempts green and 36/36 crash-loop rounds (3 rounds per compositing mode, 2 modes, per attempt). Each round's host log shows three "renderer terminated … reloading" lines and then "giving up on this browser", with each kill sent after the previous reload's pageFinished.

🤖 Generated with Claude Code