Repository navigation
test: raise per-test timeout for the whole-tree facade mirror guard scan - #22320
bolichen97 merged 1 commit into
Conversation
5421fb9 to
a1d0486
Compare
First Principles Review (Opus 5.5, fork) — ⏭️ skippedRevision |
UX Review (Opus 5.5, fork) — ⏭️ skippedRevision |
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Check before merge: the parse is said to be cached across the module, yet three tests each still take 30–40s. Confirm the cost is not repeated work that a shared cache would remove. [DESIGN-REVIEWED] e014aba Description read[DESCRIPTION-READ] 56f2bacbfa88b53c1176b5295be37e62ef52390afc3506f1c40c0ac269342dcf That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
test_every_name_the_test_trees_patch_through_a_facade_reaches_its_readers (and the sibling corpus-scan guards in this module) build _corpus_patched_names / _corpus_events, which read every test-tree file and ast.parse each one that needs the syntax tree -- across all mirrored facades at once. That whole-tree, all-facades scan is ~35s locally and scales with the runner, so on a slower CI host the first test to trigger the cold build (they share an lru_cache) can exceed the 120s per-test default and fail with "Timeout >120.0s". kirodotdev#21640 removed this cost from the single-facade sibling (facade_patched_names, with a per-file leaf short-circuit), but the all-facades builder here inherently scans the whole corpus and has no single-facade path to switch to. Raise the per-test ceiling for this module to 300s (matching the repo's existing pytest.mark.timeout convention) so a legitimately large scan is not read as a hang; a genuine runaway still trips at 300s. Closes kirodotdev#22281
a1d0486 to
e014aba
Compare
|
Thanks for this. Could you mark it ready for review when you get a chance? The same timeout keeps turning unrelated PRs red on The change looks right to me: one module-level |
bolichen97
left a comment
There was a problem hiding this comment.
Approving at head e014aba.
The change is one module-level pytest.mark.timeout(300) added next to the existing xdist_group mark in test/test_facade_mirror_guards.py. pytest-timeout gives a marker precedence over the command-line and ini default, so this lifts the ceiling only for this module and leaves the global 120s default in place everywhere else. A real hang in this module still fails at 300s, and your local run (slowest test about 40s) leaves plenty of headroom on slower runners.
Gates checked on this head:
readiness: passedlabel is on the current head and PR Readiness is green.- No failing or pending required checks; Backend Tests (3.12, 2) passed.
- Design, Opus and GPT review lanes report no blocking findings; First Principles and UX were skipped as not applicable.
- The linked issue is tier T1.
- Test-only diff, no security-sensitive paths, no conflicts with main, and main has not changed this module's marks.
The design lane asked whether the 30 to 40s per test is repeated work a shared cache could remove. The description explains why the all-facades scan cannot skip files the way the single-facade reader does, so that is a fair follow-up optimization, not a blocker here.
Thanks for the fix. Enabling squash auto-merge.
Problem / Motivation
Goal: Stop the whole-tree facade mirror guard tests from timing out on slower CI runners.
test/test_facade_mirror_guards.py::test_every_name_the_test_trees_patch_through_a_facade_reaches_its_readersscans the whole test tree across all mirrored facades and can exceed CI's 120s per-test timeout (Failed: Timeout >120.0s), seen onBackend Tests (3.12, 2)for #19599. Locally it takes ~35s; slower runners cross the limit.Why it matters
The failure is a flake unrelated to the PR under test. It turns unrelated PRs red on
Backend Testsand costs a re-run (or a misdiagnosis) each time it hits.Not a goal
--timeoutdefault or any other module's timeout.What changed (motivation → approach → change)
_corpus_patched_names()/_corpus_events()and the create-true scan. Those read every test-tree file (1000+) andast.parseeach one that needs the syntax tree, across every mirrored facade at once. Measured locally (-n0, this branch), three tests each spend 30-40s:test_every_name_the_test_trees_patch_through_a_facade_reaches_its_readers40.3s,test_no_test_writes_a_mirrored_facade_namespace_directly39.7s,test_no_test_patches_a_mirrored_facade_name_with_create_true32.2s. Scaled to a slow runner, any of them can cross 120s.test_session_allocation_bindingswithfacade_patched_names(a per-facade reader with a per-file leaf short-circuit). The all-facades builder here has to scan the whole corpus for every facade, and the parse is already shared across this module's scans, so "share the parsed corpus" removes no work from this test. The issue's other sanctioned option fits: this is a legitimately large scan, not a pathological one.pytest.mark.timeout(300)to the module'spytestmarklist next to the existingxdist_groupmark. This follows the repo's existing convention (e.g.test_adaptive_controller.py,test/integration/test_streaming.py). It applies module-wide because more than one test in the module carries this cost. A genuine runaway still trips at 300s.Backwards compatibility
Compatible: only raises a test-only timeout ceiling for one module; no runtime code changes.
Removes nothing: the diff only adds a pytest mark and a comment in a test module.
Tests
No new tests. The change is a test-configuration edit, and CI's
Backend Testsjob exercises the module under the new ceiling.Manual verification
Run locally on Python 3.12 with the repo's
.[dev]deps before pushing:pytest test/test_facade_mirror_guards.py -n0: 61 passed in 139.9s (slowest 40.3s, so 300s leaves ~7x headroom).COMMENT_HISTORY_BASE_REF=origin/main python scripts/check_comment_history.py: passed.black --check,flake8,isort --check-onlyon the changed file: clean.Related Issues
Closes #22281
Pattern harvest
Not generalizable: tree-scan guards already get a per-module timeout or a shared corpus case by case, and this one is a legitimately large all-facades scan, not a recurring defect shape.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributed under the project's Apache-2.0 license.