Skip to content

poll_tcp_oracle's late_window_missed retry ladder cannot influence the race it retries, so it emits a false FAIL on 17% of beast KVM x86 boots #694

Description

@ryanbreen

Summary

poll_tcp_oracle's stage 3 has a precondition it retries: if the parent's
poll-entry stamp is later than the peer's token-receipt stamp
(entry > token_ms), nothing about wakes can be concluded from that trial, so
the oracle widens the peer's delay and tries again — up to LATE_MAX_ATTEMPTS
(5), after which it emits a FAIL:

[POLL_TCP_ORACLE:FAIL:late_window_missed:attempts=5 overshoot_ms=20 delay_ms=1280]

The retry cannot influence the event it retries. The race is between the
parent stamping entry (just after it sends the token) and the child being
scheduled to stamp token_ms on receipt. delay_ms controls how long the
child waits after token_ms before writing — it does not move token_ms
relative to entry at all. Doubling it 80 → 160 → 320 → 640 → 1280 changes
nothing about the odds; the ladder is five independent flips of the same coin,
and on a loaded KVM host with the child free to run on another CPU, losing five
in a row is ordinary.

Rate at the merge candidate

4 of 24 boots (17%) on beast breenix-x86 KVM at fix/568-poll-wedge @
efe08b17, all four with overshoot_ms of 19–23 ms. The attempts
distribution across the 24 boots reaches the ceiling repeatedly (five boots at
attempts=5, four of which exhausted). The 20-boot aarch64 gate on the Mac
never exhausts, so this is x86-KVM-shaped.

This is a false red: it fails a gate for parent-side scheduling latency,
which is exactly the error two earlier revisions of this bound made and which
the current code's own comment says it exists to avoid. It fails loudly rather
than silently, which is why it is filed rather than treated as blocking.

The repair that removes the race by construction

Anchor the peer's clock to the parent's own instant instead of to the peer's
receipt:

  • parent stamps entry, then sends a token containing entry, then polls;
  • peer sleeps until entry + delay_ms (absolute, same kernel clock) and writes;
  • peer echoes entry back with write_ms; the parent checks the echo matches,
    that write_ms >= entry + delay_ms, and that returned >= write_ms.

park_ms = write_ms - entry >= delay_ms then holds unconditionally instead of
only when the coin lands right, entry > token_ms becomes unrepresentable, and
the whole retry ladder can be deleted. The one thing it gives up is that
entry moves one send() syscall earlier, which widens the already-disclosed
gap between the entry stamp and the poll() syscall itself (review's N1).

Why it was not fixed in round 3

Round 3's mandate was to prove the reinstated scheduler reorder at the
merge-candidate bytes, and the proof is a 40-boot A/B on those bytes. Re-cutting
the oracle after that measurement would have made the shipped bytes differ from
the measured ones — the exact objection round 2's review raised (R-3). The
finding is disclosed in docs/planning/green-program/sockets/EVIDENCE-2026-08-29.md
and filed here instead.

Serial (in-repo)

docs/planning/green-program/sockets/serials/x86-r3-CAND-late-window-missed-boot8-user-20260829.txt

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions