Skip to content

test: raise per-test timeout for the whole-tree facade mirror guard scan - #22320

Merged
bolichen97 merged 1 commit into
kirodotdev:mainfrom
JLargent42:fix/facade-mirror-guards-timeout-22281
Oct 20, 2026
Merged

bolichen97 merged 1 commit into
kirodotdev:mainfrom
JLargent42:fix/facade-mirror-guards-timeout-22281

Conversation

@JLargent42

@JLargent42 JLargent42 commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Stop the whole-tree facade mirror guard tests from timing out on slower CI runners.

test/test_facade_mirror_guards.py::test_every_name_the_test_trees_patch_through_a_facade_reaches_its_readers scans the whole test tree across all mirrored facades and can exceed CI's 120s per-test timeout (Failed: Timeout >120.0s), seen on Backend Tests (3.12, 2) for #19599. Locally it takes ~35s; slower runners cross the limit.

Why it matters

The failure is a flake unrelated to the PR under test. It turns unrelated PRs red on Backend Tests and costs a re-run (or a misdiagnosis) each time it hits.

Not a goal

What changed (motivation → approach → change)

  • Symptom: the reported test times out at 120s on slow runners.
  • Root cause: the corpus-scan guards in this module build _corpus_patched_names() / _corpus_events() and the create-true scan. Those read every test-tree file (1000+) and ast.parse each one that needs the syntax tree, across every mirrored facade at once. Measured locally (-n0, this branch), three tests each spend 30-40s: test_every_name_the_test_trees_patch_through_a_facade_reaches_its_readers 40.3s, test_no_test_writes_a_mirrored_facade_namespace_directly 39.7s, test_no_test_patches_a_mirrored_facade_name_with_create_true 32.2s. Scaled to a slow runner, any of them can cross 120s.
  • Why not the test: read only the facade's own files in the session allocation patch scan #21640 approach: test: read only the facade's own files in the session allocation patch scan #21640 removed this cost from the single-facade sibling in test_session_allocation_bindings with facade_patched_names (a per-facade reader with a per-file leaf short-circuit). The all-facades builder here has to scan the whole corpus for every facade, and the parse is already shared across this module's scans, so "share the parsed corpus" removes no work from this test. The issue's other sanctioned option fits: this is a legitimately large scan, not a pathological one.
  • Change: add pytest.mark.timeout(300) to the module's pytestmark list next to the existing xdist_group mark. This follows the repo's existing convention (e.g. test_adaptive_controller.py, test/integration/test_streaming.py). It applies module-wide because more than one test in the module carries this cost. A genuine runaway still trips at 300s.

Backwards compatibility

Compatible: only raises a test-only timeout ceiling for one module; no runtime code changes.

Removes nothing: the diff only adds a pytest mark and a comment in a test module.

Tests

No new tests. The change is a test-configuration edit, and CI's Backend Tests job exercises the module under the new ceiling.

Manual verification

Run locally on Python 3.12 with the repo's .[dev] deps before pushing:

  • pytest test/test_facade_mirror_guards.py -n0: 61 passed in 139.9s (slowest 40.3s, so 300s leaves ~7x headroom).
  • COMMENT_HISTORY_BASE_REF=origin/main python scripts/check_comment_history.py: passed.
  • black --check, flake8, isort --check-only on the changed file: clean.

Related Issues

Closes #22281

Pattern harvest

Not generalizable: tree-scan guards already get a per-module timeout or a shared corpus case by case, and this one is a legitimately large all-facades scan, not a recurring defect shape.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: checking Automated validation is still running labels Oct 20, 2026
@JLargent42
JLargent42 force-pushed the fix/facade-mirror-guards-timeout-22281 branch from 5421fb9 to a1d0486 Compare October 20, 2026 00:02
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 20, 2026
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ⏭️ skipped

Revision e014aba7538bd9055575ad44a3625f4f8e75b738 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision e014aba7538bd9055575ad44a3625f4f8e75b738 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of e014aba7538bd9055575ad44a3625f4f8e75b738 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check before merge: the parse is said to be cached across the module, yet three tests each still take 30–40s. Confirm the cost is not repeated work that a shared cache would remove.

[DESIGN-REVIEWED] e014aba

Description read

[DESCRIPTION-READ] 56f2bacbfa88b53c1176b5295be37e62ef52390afc3506f1c40c0ac269342dcf

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed e014aba7538bd9055575ad44a3625f4f8e75b738 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] e014aba

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed e014aba7538bd9055575ad44a3625f4f8e75b738 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] e014aba

test_every_name_the_test_trees_patch_through_a_facade_reaches_its_readers
(and the sibling corpus-scan guards in this module) build _corpus_patched_names
/ _corpus_events, which read every test-tree file and ast.parse each one that
needs the syntax tree -- across all mirrored facades at once. That whole-tree,
all-facades scan is ~35s locally and scales with the runner, so on a slower CI
host the first test to trigger the cold build (they share an lru_cache) can
exceed the 120s per-test default and fail with "Timeout >120.0s".

kirodotdev#21640 removed this cost from the single-facade sibling (facade_patched_names,
with a per-file leaf short-circuit), but the all-facades builder here inherently
scans the whole corpus and has no single-facade path to switch to. Raise the
per-test ceiling for this module to 300s (matching the repo's existing
pytest.mark.timeout convention) so a legitimately large scan is not read as a
hang; a genuine runaway still trips at 300s.

Closes kirodotdev#22281
@bolichen97

Copy link
Copy Markdown
Contributor

Thanks for this. Could you mark it ready for review when you get a chance?

The same timeout keeps turning unrelated PRs red on Backend Tests (3.12, 2) (tracked in #22004, #22007 and #22281). Right now it is the only red check on several PRs that are otherwise ready to merge.

The change looks right to me: one module-level pytest.mark.timeout(300) next to the existing xdist_group mark, and a real runaway still fails at 300s. The Design, GPT and Opus review lanes all pass on e014aba. Once it is out of draft and CI is green, I can review and merge it. It may also be worth adding Closes #22004 and Closes #22007 to the description, since they report the same timeout.

@JLargent42
JLargent42 marked this pull request as ready for review October 20, 2026 16:19
@JLargent42
JLargent42 requested review from a team and KiroCrewCaptain as code owners October 20, 2026 16:19
@JLargent42
JLargent42 requested a review from buluoray October 20, 2026 16:19
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 20, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at head e014aba.

The change is one module-level pytest.mark.timeout(300) added next to the existing xdist_group mark in test/test_facade_mirror_guards.py. pytest-timeout gives a marker precedence over the command-line and ini default, so this lifts the ceiling only for this module and leaves the global 120s default in place everywhere else. A real hang in this module still fails at 300s, and your local run (slowest test about 40s) leaves plenty of headroom on slower runners.

Gates checked on this head:

  • readiness: passed label is on the current head and PR Readiness is green.
  • No failing or pending required checks; Backend Tests (3.12, 2) passed.
  • Design, Opus and GPT review lanes report no blocking findings; First Principles and UX were skipped as not applicable.
  • The linked issue is tier T1.
  • Test-only diff, no security-sensitive paths, no conflicts with main, and main has not changed this module's marks.

The design lane asked whether the 30 to 40s per test is repeated work a shared cache could remove. The description explains why the all-facades scan cannot skip files the way the single-facade reader does, so that is a fair follow-up optimization, not a blocker here.

Thanks for the fix. Enabling squash auto-merge.

@bolichen97
bolichen97 merged commit ef52170 into kirodotdev:main Oct 20, 2026
107 of 113 checks passed
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test_facade_mirror_guards tree scan can exceed the 120s CI test timeout

2 participants