Skip to content

test(windows): collect five suites whose Windows blockers are gone - #24037

Merged
KiroCrewCaptain merged 1 commit into
kirodotdev:mainfrom
leonlaiyc:claude/windows-collect-stale-suites
Oct 25, 2026
Merged

KiroCrewCaptain merged 1 commit into
kirodotdev:mainfrom
leonlaiyc:claude/windows-collect-stale-suites

Conversation

@leonlaiyc

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: the Windows shards run the stop/kill/cancel, theme-install and Office-preview suites, whose Windows blockers are already fixed on main, instead of skipping all 338 of their tests at collection.

test/windows-collect-ignore.txt keeps whole files from being collected on Windows. Five entries are still there for reasons that no longer hold:

Named on the command line on a Windows host, which bypasses the hook, all five pass.

Why it matters

The list's own rule is that removing an entry re-enables the suite on Windows and that residual gaps go into windows-expected-failures.txt. Leaving these entries means a Windows regression in theme install, Office preview or the gateway's kill path never shows up in CI, including the kill-path regression the stop/kill tests were written for.

Not a goal

  • The other entries in the list. test_snapshot.py and its two topic parts, both test_outbox_*.py files, test_file_raw.py, test_file_download.py, test_dashboard_file_io.py, test_webapp_preview.py, test_env.py and test_app_backend_stale_reap.py still fail on Windows here, need a symlink privilege this host lacks, or are being edited by another open PR (fix(dashboard): answer Range requests on /api/outbox so chat videos can seek #23601 adds tests to test_outbox_binary.py).
  • Any product code.

What changed (motivation → approach → change)

  • Symptom: 338 tests that pass on Windows are never collected on the Windows shards.
  • Root cause: their files are still listed in test/windows-collect-ignore.txt, which test/conftest.py's pytest_ignore_collect hook and scripts/ci-surface-tests.py both read, for blockers fixed since they were listed.
  • test/windows-collect-ignore.txt: the five entries go.
  • test/test_ci_surface_tests.py: the same five names leave the exact set test_ignore_list_matches_the_names_conftest_previously_inlined pins. Its docstring records why, as it already does for test_harness.py. None of the five is in _OBSERVED_WINDOWS_FAILURES.

Backwards compatibility

Compatible: test-only; no product code changes, and POSIX collection is unchanged.

Removes nothing: five suites are collected on Windows that were not; nothing is skipped or loosened.

Tests

  • The five files on a Windows host, run together as the Windows shards run them (-n auto --dist loadgroup --timeout=180 --max-worker-restart=0): 338 passed, 9 skipped on Python 3.12.13. Each file alone also passes on 3.12.13 and 3.13.5: office preview 40, theme install 139 + 104 + 28 (7 skipped), stop/kill/cancel 27 (2 skipped).
  • The list's readers on that host: test_ci_surface_tests.py, test_platform_collect_ignore.py, test_windows_fail_closed_optin.py and test_windows_gap_list.py, 89 passed on 3.12.13. On 3.13.5, test_windows_fail_closed_optin.py::TestFreshWindowsGatewayDelegates::test_health_then_one_session_and_one_prompt_succeed times out in wait_until on this branch and identically on clean main.
  • black, isort, flake8, check_comment_history, check_brand_name and test_flake_pattern_ratchet.py (60 passed) pass.
  • Trial merges with fix(sandbox): pin sealed paths' parent directories #22969, the other open PR that edits test_ci_surface_tests.py, and with the PRs that edit test/windows-expected-failures.txt leave both files clean. fix(sandbox): pin sealed paths' parent directories #22969 conflicts with main in .github/workflows/ci.yml on its own.

Proof: not provable -- test-only change

Manual verification

N/A — test-only change; the Windows runs above are the check, and the Windows matrix collects these suites on this PR.

Related Issues

Fixes #24036

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

🤖 Generated with Claude Code

test_stop_kill_cancel.py, the three test_theme_install*.py parts and
test_file_office_preview.py sit in windows-collect-ignore.txt for
reasons that no longer hold: the kill-path tests skip their POSIX-only
cases per test (kirodotdev#6963), theme packs are served on Windows (kirodotdev#5943), and
the file schema accepts a drive-letter path (kirodotdev#4647). The 338 tests in
them pass on Windows, so drop the five entries and the same names from
the set test_ci_surface_tests.py pins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leonlaiyc
leonlaiyc requested review from a team and KiroCrewCaptain as code owners October 25, 2026 15:13
@leonlaiyc
leonlaiyc requested a review from iamwhatever October 25, 2026 15:13
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Oct 25, 2026
@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision fa6be0baea0234e1858e22b9d77d2cafcedc2f8c touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ⏭️ skipped

Revision fa6be0baea0234e1858e22b9d77d2cafcedc2f8c ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of fa6be0baea0234e1858e22b9d77d2cafcedc2f8c via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the change only re-enables collection of five test files, the author ran them on a Windows host, and the Windows CI shards exercise them on this PR.

[DESIGN-REVIEWED] fa6be0b

Description read

[DESCRIPTION-READ] ce52d7726e7a2ebd839cf0e4cd7582ad2cf7f24e5aa94b04fbbbf10219b3b8bc

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed fa6be0baea0234e1858e22b9d77d2cafcedc2f8c via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] fa6be0b

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed fa6be0baea0234e1858e22b9d77d2cafcedc2f8c via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] fa6be0b

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 25, 2026
@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) October 25, 2026 17:03

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (2 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: test-only change that removes five suites from the Windows collect-ignore list and updates the matching inventory test.

@KiroCrewCaptain
KiroCrewCaptain merged commit 0b9a470 into kirodotdev:main Oct 25, 2026
109 checks passed
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Five test suites stay uncollected on Windows after their Windows blockers were removed

2 participants