Repository navigation
ci: run the windows backend shards with four workers again - #36667
bolichen97 wants to merge 1 commit into
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: this changes the worker count of one CI job, a revert undoes it cleanly, and 4 workers is the count the same runner used, and passed with, before #33084. Notes
[DESIGN-REVIEWED] 3cd50f4 Description read[DESCRIPTION-READ] 6b224ddd88914467be315df8849b6e3bc69f71da15202b34dd5b76bb1c78424b That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks: this is a CI-only worker count. The 2-worker grant is shown in CI logs and reproduced from the budget code, and the Linux lane already passes an explicit What this change shipsInventory (3 items) — 3 justifiedIntent: get PR CI verdicts back about 12 minutes sooner by restoring 4 test workers on each Windows backend shard. FIX.
[FIRST-PRINCIPLES-REVIEWED] 3cd50f4 Description read[DESCRIPTION-READ] b71dad788bf33dde103d38c9eed32c7af476a666dbb87170538b398db4cfa956 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
The Windows backend shards ran `pytest -n auto`, which goes through the xdist memory budget. At its 5 GiB per-worker reservation (#33084) the 15 GiB / 8-core Windows runner is granted 2 workers instead of 4, which roughly doubled each shard's run time and made those shards the last job of every PR's CI run. Pass an explicit `-n 4`, the count that runner ran green with, as the Linux lane already passes an explicit count. An explicit `-n` still claims its slots from the shared pool, and a run alone on its host keeps all N. The two tests that pin the Windows command now pin `-n 4`, and one also asserts `-n auto` is absent. Part of #30257 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2c9d93d to
3cd50f4
Compare
|
Rebased: main moved in a landed fix for [operator: bolichen97] |
|
Review passed on this head. Please rebase onto the latest main so CI can run fresh; this head is 439 commits behind main and its only CI run is from 11-13. A fresh Windows run will also confirm that four workers still hold with the test files added since then. The approval will follow once the rebased head is green. |
Code change approver recommendation: ApproveWhy: the decider estimates a 7% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes. CI-only change: backend-test-windows shards pass explicit An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
|
[operator: bolichen97] @buluoray the rebase is ready (the same +7/-4 change, no conflict), but I am holding the push until #37964 lands. Main is red right now on |
|
This PR and #38921 make the same change: both replace Please keep one of #36667 and #38921 and close the other, carrying over anything only this one has. |
|
[operator: bolichen97] Closing this as superseded by #38921, which merged into main as 577c105 and makes the same change in a fuller form: the Windows backend shard step sizes its own xdist workers (one per 3 GiB after 2 GiB for the host, capped at the core count, so 4 on the 15 GiB fleet runner and 4 on a 4-core hosted runner) instead of Rebasing this branch onto main conflicts in all three of its files ( |
Pull request was closed
Problem / Motivation
Goal: Windows backend test shards get back the 4 xdist workers the memory budget took away, so they stop setting every PR's CI wall clock.
backend-test-windowsruns its 8 file shards withpytest -n auto, which resolves throughxdist_budget.resolve_workers(). #33084 raised the Linux/Windows per-worker reservation (_GIB_PER_WORKER,_GIB_PER_WORKER_AVAILABLE) from 3 to 5 GiB, for good reasons on developer hosts. A side effect is that the 15 GiB / 8-core Windows runner now gets 2 workers instead of 4:Each Windows shard's test step went from 22.9-25.6 min to about 42 min.
Why it matters
Those Windows shards are now the last job to finish on every CodeBuild-routed PR run. The median PR wall clock went from 36.6 to 51.1 min, so every contributor waits about 15 extra minutes for a verdict on each push. The Linux shards are unaffected because they already pass an explicit
-n "$WORKERS".Not a goal
xdist_budget.py, its 5 GiB per-worker constants).SHARD_COUNT, anyneeds:, orlint-gate.-n autojob (new-test-stabilityand the rest; none of them is on the critical path).What changed (motivation → approach → change)
A PR's CI verdict should arrive about 12 minutes sooner (51 min to about 39): the Windows backend test shards get back their 4 test workers, so they finish before the Linux path again instead of last. The rest of the 15 minutes is Linux suite growth, which this PR does not touch.
-n autoon that runner goes through the memory budget, and at 5 GiB per worker 15 GiB installed / 12.2 GiB free allows 2 workers..github/workflows/ci.yml, step "Run tests (Windows shard N/8)" ofbackend-test-windows,-n autobecomes-n 4. That is the count the same runner got, and ran green with, before fix: guard host memory against parallel test runs #33084. This follows the Linux lane, which already passes an explicit count. An explicit-nstill claims its slots from the shared pool throughxdist_budget.claim_explicit_workers, which keeps N for a run that is alone on its host (a CI runner always is). A two-line comment above the pytest call says why the count is explicit. The comment block above the step said "-n autoalready saturates each runner", which is no longer accurate, so it now says "The shard's workers already saturate each runner".test/test_ci_windows_canary.py::test_shard_selection_and_checks_stay_intact(exact command string) andtest/test_ci_pytest_progress.py::TestCIProgress::test_windows_wiring_keeps_acceptance_and_other_jobs_unchanged(flag list) now pin-n 4. The latter also asserts-n autois absent from the Windows command, as the Linux wiring test already does.Backwards compatibility
Compatible: a CI-only worker count on one job; no product code, config key or API changes.
Removes nothing: the step still runs every test it ran, only with an explicit worker count.
Tests
test_shard_selection_and_checks_stay_intactpins the Windows step's full pytest command line, now with-n 4.test_windows_wiring_keeps_acceptance_and_other_jobs_unchangedpins-n 4in the flag list and that-n autois absent.ci.yml(assert ... -n 4 ... == ... -n auto ...) and pass against this branch's.Proof: not provable -- docs or config only
Manual verification
Host: Linux (
Linux 7.0.0-1014-aws x86_64). The real Windows-runner proof is this PR's own CI run: its Windows shard logs must showcreated: 4/4 workers.How it was verified
KIROCREW_TEST_SLOT_DIR):The warning line matches the CI shard logs exactly.
-q -n 4 --timeout=180 --no-cov --max-worker-restart=0 -m "not ipv6_required and not real_venv" --file-shards 8 --file-shard N) on this host overtest/test_ci_windows_canary.py test/test_ci_pytest_progress.py test/test_ci_file_shards.py test/test_xdist_host_budget.py, for every N:Shards 2, 4, 5 and 8 also hold none of the four files (exit 5), and each one created 4/4 workers.
claim_explicit_workers(4)under the Windows-runner readings, after the fix:4.python -m pytest -n0 test/test_ci_windows_canary.py test/test_ci_pytest_progress.py test/test_xdist_host_budget.py test/test_workflow_matrix_max_parallel.py -q:788 passed. All 12 test files that readbackend-test-windowsfromci.yml, plustest_workflow_matrix_max_parallel.py:1016 passed.mypy src/kiro_crew, comment-history, brand and file-line-cap all pass;scripts/local-gate.py: backend related 3659 passed, 2 skipped; frontend related 14760 passed (519 files); "all selected gates passed".python -m pytest -n0 test/test_flake_pattern_ratchet.py(68 passed),semgrep scan --config semgrep/ --baseline-commit <merge-base> --error(0 findings),npm --prefix website run bench:render -- --fail-on dispatches,selectorCalls(exit 0, its CI filter includesci.yml).Related Issues
Part of #30257
The issue stays open for one more measured CI window.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.