Skip to content

test: bound the writer-rollover follow test's wait on what it emitted - #26732

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
fix/24662-follow-rollover-bounded
Oct 31, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
fix/24662-follow-rollover-bounded

Conversation

@JLargent42

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: test_follow_allows_real_writer_rollover either passes or fails fast, and never sits on the 180 s per-test timeout.

On a macOS run the [reader-open] case hit Failed: Timeout >180.0s (run 40420314931, job 121913820382, PR #24648 head 0fbb51667d9c). That log's captured stdout already holds xxx…\nafter-1\nafter-2\n, so follow had read the rolled-over file. What never came was the KeyboardInterrupt that ends the test.

Why it matters

A hung test costs a macOS shard 3 minutes, turns the shard red on PRs that do not touch this code, and gets re-run as a flake.

Not a goal

  • No change to _tail_log_file or open_log_file_for_tail. The follow code did what it should in the failing run (it emitted both rolled-over records), so this is a test defect, not a follow bug.
  • No change to the other follow tests in this file (see Sibling sweep).

What changed (motivation → approach → change)

  • Symptom: follow kept polling after it had printed the rollover, until the per-test timeout.
  • Root cause: the test's tick raised KeyboardInterrupt only when len(ticks) == 3. When the run failed (Oct 26) the test patched cli_server.time.sleep, which is the stdlib module, so any other thread in the worker that slept also called tick. One of those calls took tick refactor: remove dead legacy compatibility shims #3 (and swallowed its KeyboardInterrupt), the count went past 3, and the equality never held again. Follow then looped on a zero-cost fake sleep for good. test: keep two more test files' clocks off the shared stdlib module #24928 (68fd509f6, merged Oct 30) moved these tests onto a private ManualClock, which removes the neighbour thread, but the stop is still an exact count: any missed or extra tick brings the hang back.
  • Change: tick now collects what follow has emitted and stops follow once after-2\n is out, which is the event the test waits for. It raises AssertionError with the emitted text after 1000 ticks, so a missed rollover fails at once instead of hanging (sleep is the fake, so the cap costs no wall-clock time). The rotation trigger points are unchanged, and the test still asserts that the writer's rolled file holds after-1\nafter-2\n and that follow printed the seed plus both rolled records.

Backwards compatibility

Compatible: test-only change; no production code or contract is touched.

Removes nothing: the diff changes one test's stop condition and keeps every assertion it had.

Tests

test/test_windows_logs_and_diagnostics.py::test_follow_allows_real_writer_rollover (both cases): stops on the emitted rollover with a 1000-tick cap, and keeps its assertions.

Proof: not provable -- test-only change

Rate, whole test file run alone at -n 0 on Linux, each run killed at 30 s if it hangs. "Neighbour" loads a local-only pytest plugin that starts one daemon thread calling time.sleep(0.0005) in a loop, standing in for the background threads (skill catalog, sel writer, pools) seen in the failing job's stack dump:

Test file version No neighbour With neighbour
Before #24928 (68fd509f6^) 0/30 failed 10/10 hung ([reader-open])
origin/main 271808883 0/30 failed 0/30 failed
This PR 0/30 failed 0/30 failed

The plugin is not committed. test/test_flake_pattern_ratchet.py and test/test_wait_discipline_ratchet.py pass locally with this change (137 passed together with this file).

Manual verification

N/A — unit coverage sufficient: the change is a test's stop condition, measured by the repeated runs above.

Related Issues

Closes #24662

Sibling sweep

Swept: git grep -nE "len\((ticks|waits)\) == [0-9]+:" -- test/test_windows_logs_and_diagnostics.py
Sibling: test/test_windows_logs_and_diagnostics.py:153 -- not affected -- test_windows_follow_handles_truncation also stops on len(waits) == 3, but since #24928 its sleep is a private clock that only the follow loop calls and no writer races its reader, so no tick can be missed; left for its own change if it ever flakes

Pattern harvest

Rule candidate: review-prompt
Pattern: a fake-sleep test that ends an endless poll loop only on an exact tick count, with no cap, hangs to the per-test timeout when one tick is missed

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

@JLargent42
JLargent42 requested review from a team and KiroCrewCaptain as code owners October 31, 2026 13:11
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 31, 2026
@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision e50eaf4d0c22b4c9449c1b87f4e488ea77e12425 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision e50eaf4d0c22b4c9449c1b87f4e488ea77e12425 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of e50eaf4d0c22b4c9449c1b87f4e488ea77e12425 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks. This is a test-only change: the test now stops when follow emits after-2\n and fails after 1000 ticks, which removes the hang without touching production code.

[DESIGN-REVIEWED] e50eaf4

Description read

[DESCRIPTION-READ] aaf35ff4fae425992236c893b720628dc7d8183cfc659249eb5418337e0127e4

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed e50eaf4d0c22b4c9449c1b87f4e488ea77e12425 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] e50eaf4

Verdict parsed from the review's SHA-scoped output markers for commit e50eaf4d0c22b4c9449c1b87f4e488ea77e12425.

False positive or not applicable? A repository writer can comment:
/ai-review override fable e50eaf4d0c22b4c9449c1b87f4e488ea77e12425 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of e50eaf4d0c22b4c9449c1b87f4e488ea77e12425 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] e50eaf4

False positive or not applicable? A repository writer can comment:
/ai-review override gpt e50eaf4d0c22b4c9449c1b87f4e488ea77e12425 false-positive: <reason, 20 to 500 characters>

test_follow_allows_real_writer_rollover stopped follow only when the
fake sleep saw exactly its third tick. On a macOS run (#24662) one tick
went to a neighbour thread through the then-shared stdlib sleep, the
count passed 3 for good, and follow polled until the 180s per-test
timeout, though it had already printed the rolled-over records.

#24928 moved the sleep onto a private ManualClock, which removes that
neighbour. The stop is still an exact count, so any missed or extra
tick hangs again. Stop once follow has emitted the rollover, and fail
with the emitted text after 1000 ticks (no wall-clock cost). The
assertion that the real writer's rollover is followed is unchanged.

Closes #24662
@JLargent42
JLargent42 force-pushed the fix/24662-follow-rollover-bounded branch from 8ff37a9 to e50eaf4 Compare October 31, 2026 13:28
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 31, 2026
@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) October 31, 2026 15:03

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (1 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: test-only change: the writer-rollover follow test now stops on what follow emitted instead of an exact tick count, with a 1000-tick cap so a missing tick fails fast instead of hanging.

@KiroCrewCaptain
KiroCrewCaptain merged commit ba6ece9 into main Oct 31, 2026
94 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the fix/24662-follow-rollover-bounded branch October 31, 2026 15:03
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky: test_follow_allows_real_writer_rollover times out on macOS

2 participants