Skip to content

test: keep two more test files' clocks off the shared stdlib module - #24928

Merged
mbriones98 merged 1 commit into
mainfrom
fix/clock-isolation-20648
Oct 30, 2026
Merged

mbriones98 merged 1 commit into
mainfrom
fix/clock-isolation-20648

Conversation

@mbriones98

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: the last two test files from #20648 stop sharing their fake clock with every other thread in the test worker, so they no longer fail at random under load.

test/test_sandbox_backend_cache.py::test_off_loop_transient_retry_sleeps recorded the probe's retry sleep with monkeypatch.setattr(sb.time, "sleep", ...), and seven log-follow tests in test/test_windows_logs_and_diagnostics.py drove the follow loop with monkeypatch.setattr(cli_server.time, "sleep", tick). sb.time and cli_server.time are the stdlib time module, so every thread in the xdist worker used the fake while it was set: a neighbour's sleep landed in the recorded list, or ran tick, which writes the log file and raises KeyboardInterrupt. With a sleeping neighbour thread (#20607's spinner plugin), the two files on main failed 8 of 15 runs (test_off_loop_transient_retry_sleeps, test_windows_follow_reads_appended_lines_and_stops).

9 of the 11 sites in the issue were already fixed on main by #21038 and earlier PRs. These are the last two.

Why it matters

A random red on a shared backend shard blocks unrelated PRs and costs a rerun. The tick sites could also end another test's loop early, by sending its thread a KeyboardInterrupt.

Not a goal

What changed (motivation → approach → change)

Symptom: recorder assertions and follow tests failed under a sleeping neighbour thread. Root cause: the fake was set on the shared stdlib module. Change: the same fix #20608 and #20664 used.

  • test_sandbox_backend_cache.py: a _pin_clock helper installs a ManualClock on sb's own time binding. The facade mirror passes that to every sandbox owner, including sandbox_wrap, which makes the retry sleep. The retry test now asserts clock.sleeps.
  • test_windows_logs_and_diagnostics.py: a _pin_sleep helper installs a ManualClock on cli_server and sets tick as the sleep of that clock's private stand-in module. The seven sites call the helper.
  • Each file adds a neighbour-thread test. In the Windows logs file it is assert_clock_is_private (test(pod): keep the remaining stdlib sleep patches off the shared module #20664). In the sandbox file it is a local check, because the unchanged autouse fixture already swaps the stdlib sleep for a no-op: a neighbour's sleep must not be recorded and its monotonic() must not be the clock's. Moving either helper back onto the stdlib module fails that test on every run.

Backwards compatibility

Compatible: test-only change; no product code or test helper API changes.

Removes nothing: only the bodies of two test files change; no test, fixture or helper is removed.

Tests

Proof: not provable -- test-only change

  • Forced condition (a daemon thread looping time.sleep(0.002) plus sys.setswitchinterval(1e-6), from Windows flake: pod health-wait tests patch the worker-wide time.sleep #20607): the two files at the parent commit failed 8 of 15 runs. With this change they failed 0 of 15, and 0 of 20 more runs in 20 shuffled orders (seeds 1-20).
  • Reverting _pin_clock to monkeypatch.setattr(sb.time, ...) and _pin_sleep to monkeypatch.setattr(cli_server.time, "sleep", tick) makes test_another_thread_does_not_reach_the_retry_clock and test_another_thread_keeps_the_real_sleep_and_clock fail 3 of 3 runs.
  • test/test_sandbox_backend_cache.py, test/test_windows_logs_and_diagnostics.py, test/test_flake_pattern_ratchet.py and test/test_facade_mirror_guards.py: 197 passed. isort and flake8 are clean. mypy on these two files shows the same 5 errors as on main, and mypy src/kiro_crew/ is clean.

Manual verification

N/A — unit coverage sufficient: the change is to tests only, and the neighbour-thread repro above was run locally.

Related Issues

Fixes #20648

Sibling sweep

Swept: git grep -nE "setattr([A-Za-z_][A-Za-z_0-9.]*.time, *"'["']" -- test/
Sibling: test/test_sandbox_backend_cache.py:63 -- not affected -- a no-op sleep records nothing, and its region is changed by open PR #20822
Sibling: test/test_computer_use_overlay.py:1026 -- not changed here -- same recorder shape, outside #20648's classified sample; left for a follow-up issue
Sibling: test/test_platform_compat_windows_tree.py:349 -- not changed here -- same recorder shape, outside #20648's classified sample; left for a follow-up issue
Sibling: test/test_readiness_sweep_scan.py:316 -- not changed here -- same recorder shape, outside #20648's classified sample; left for a follow-up issue
Sibling: test/test_apps_backend_coverage_supervision.py:1087 -- not changed here -- scripted sleep, outside #20648's classified sample; left for a follow-up issue
Sibling: test/test_wheel_engine.py:799 -- not changed here -- scripted sleep, outside #20648's classified sample; left for a follow-up issue
Sibling: test/test_pod_runtime_refactor_facade.py:484 -- not changed here -- raising sleep, outside #20648's classified sample; left for a follow-up issue

Pattern harvest

Rule candidate: semgrep
Pattern: a test patching <module>.time.<clock> replaces the stdlib clock for the whole worker. kirocrew.test-stdlib-clock-rebound already refuses new sites; this PR removes two existing ones.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

The sandbox off-loop retry test and the Windows log-follow tests still
patched `sb.time.sleep` / `cli_server.time.sleep`. That binding is the
stdlib module, so every other thread in the worker shared the fake: its
sleeps landed in the retry test's recorded list, and the follow tests'
`tick` (which writes the log and raises KeyboardInterrupt) ran for any
neighbour that slept.

Both files now install a ManualClock on the facade's own `time` binding
(the mirror passes it to every owner). The retry test asserts
`clock.sleeps`; the follow tests set `tick` as the sleep of that private
stand-in. A neighbour-thread check in each file fails on every run if the
helper is put back on the stdlib module.

Fixes #20648
@mbriones98
mbriones98 requested review from a team and KiroCrewCaptain as code owners October 27, 2026 17:21
@mbriones98
mbriones98 enabled auto-merge (squash) October 27, 2026 17:21
@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision cc7ab2bb9c10b951fc4b57e9364c00efa9113acd touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision cc7ab2bb9c10b951fc4b57e9364c00efa9113acd ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of cc7ab2bb9c10b951fc4b57e9364c00efa9113acd — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: a test-only change that reuses the repo's existing private-clock pattern (#20608/#20664) on the last two sites from #20648, with a neighbour-thread test for each file.

[DESIGN-REVIEWED] cc7ab2b

Description read

[DESCRIPTION-READ] 5ae6bcfe2bf58600cf4d5a6b8044d81a2c270bcf96a6960ba80ff16f8b499227

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of cc7ab2bb9c10b951fc4b57e9364c00efa9113acd and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] cc7ab2b

False positive or not applicable? A repository writer can comment:
/ai-review override gpt cc7ab2bb9c10b951fc4b57e9364c00efa9113acd false-positive: <reason, 20 to 500 characters>

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed cc7ab2bb9c10b951fc4b57e9364c00efa9113acd — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] cc7ab2b

Verdict parsed from the review's SHA-scoped output markers for commit cc7ab2bb9c10b951fc4b57e9364c00efa9113acd.

False positive or not applicable? A repository writer can comment:
/ai-review override fable cc7ab2bb9c10b951fc4b57e9364c00efa9113acd false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 27, 2026

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (2 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: test-only: moves two test files' sleep and clock patches onto a private ManualClock instead of the shared stdlib module.

@mbriones98
mbriones98 merged commit 68fd509 into main Oct 30, 2026
206 of 216 checks passed
@mbriones98
mbriones98 deleted the fix/clock-isolation-20648 branch October 30, 2026 21:31
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 30, 2026
JLargent42 added a commit that referenced this pull request Oct 31, 2026
test_follow_allows_real_writer_rollover stopped follow only when the
fake sleep saw exactly its third tick. On a macOS run (#24662) one tick
went to a neighbour thread through the then-shared stdlib sleep, the
count passed 3 for good, and follow polled until the 180s per-test
timeout, though it had already printed the rolled-over records.

#24928 moved the sleep onto a private ManualClock, which removes that
neighbour. The stop is still an exact count, so any missed or extra
tick hangs again. Stop once follow has emitted the rollover, and fail
with the emitted text after 1000 ticks (no wall-clock cost). The
assertion that the real writer's rollover is followed is unchanged.

Closes #24662
KiroCrewCaptain pushed a commit that referenced this pull request Oct 31, 2026
…#26732)

test_follow_allows_real_writer_rollover stopped follow only when the
fake sleep saw exactly its third tick. On a macOS run (#24662) one tick
went to a neighbour thread through the then-shared stdlib sleep, the
count passed 3 for good, and follow polled until the 180s per-test
timeout, though it had already printed the rolled-over records.

#24928 moved the sleep onto a private ManualClock, which removes that
neighbour. The stop is still an exact count, so any missed or extra
tick hangs again. Stop once follow has emitted the rollover, and fail
with the emitted text after 1000 ticks (no wall-clock cost). The
assertion that the real writer's rollover is followed is unchanged.

Closes #24662
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky-capable stdlib clock patches outside the pod-health files

2 participants