Repository navigation
ci: give New-test stability 150 min on Windows, 90 on Ubuntu - #36724
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. Both limit raises cite measured pass times from #31863 and are tied to #36722. The test change fixes a real blind spot: the budget check could not see the per-leg limits. Notes
What this change shipsInventory (5 items) — 4 justifiedIntent: let a large test PR finish New-test stability's three passes instead of being cancelled at the time limit. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] 831c750 Description read[DESCRIPTION-READ] a7c4577de08e762d792ffc033223cb66380f0b2aaba7ac2887a0382ec2c8dbf7 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks. The new limits come from measured pass times, and the merge-queue budget test now counts each leg's own limit, so a future increase that overruns the 220-minute budget will fail the test. Notes
[DESIGN-REVIEWED] 831c750 Description read[DESCRIPTION-READ] 55962e6c79790c31cb4c38a4fd8472019be8467c143718eb7bd1506df3689063 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
1835106 to
c704701
Compare
Code change approver recommendation: ApproveWhy: the decider estimates a 6% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes. The PR raises the New-test stability time limits: Windows from 90 to 150 minutes, and Ubuntu gets its own 90 instead of the An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
c704701 to
c0925f4
Compare
c0925f4 to
ea382cc
Compare
A large test PR still ran past the job's limits with no failing test: one Windows pass over about 400 files takes about 32 minutes, so three do not fit in 90, and a slow Ubuntu runner needs about 72 minutes, past 60. The Windows leg's matrix limit goes to 150 and the Ubuntu leg gets an explicit 90. The merge-queue budget test now counts each leg's own limit rather than only the shared fallback, and a new test pins both legs. Closes #36722
ea382cc to
831c750
Compare
Problem / Motivation
Goal: A large test PR can pass New-test stability on both legs instead of being cancelled at the job's time limit.
New-test stabilityis still cancelled at itstimeout-minuteson #31863 (head 7c763a5, 403 modified test files), with no failing test:Why it matters
PR Readiness counts a cancelled leg as blocking, so a large test PR whose tests all pass cannot become ready, and a re-run repeats the same cancel.
Not a goal
LEAF_REPEAT= 3), its per-test timeouts or its runners.What changed (motivation → approach → change)
Three passes need about 100 minutes on Windows and about 72 on a slow Ubuntu runner, so each leg gets a limit with room above that. In
.github/workflows/ci.yml, jobnew-test-stability, the Windows matrix entry'stimeout_minutesgoes from 90 to 150 and the Ubuntu entry gets an explicittimeout_minutes: 90(it used the|| 60default). The comment above the limit states the measured pass times; the run they come from is the one cited above.The merge-queue budget test in
test/test_ci_fleet_routing_expression_parity.pycounted only the|| 60fallback for this job, so it could not see either leg's own limit. It now counts the largest of the fallback and every leg's value, and a new test pins both legs at 90 and 150.Cost: a stalled leg can now hold its runner 60 minutes longer on Windows and 30 on Ubuntu. A run that finishes today finishes the same, because the limit is only a ceiling.
Backwards compatibility
Compatible: only two job time limits rise; every run that finished under the old limits runs the same way.
Removes nothing: the diff raises two limits, rewrites one comment and widens what one test counts.
Tests
test/test_ci_fleet_routing_expression_parity.py::test_new_test_stability_sets_each_legs_own_limit(new) pins the Ubuntu leg at 90 and the Windows leg at 150.test_merge_queue_readiness_budget_covers_ci_critical_pathnow counts each leg's owntimeout_minutes, not only the|| 60fallback. With the Windows leg temporarily set to 300 it fails; before this change it passed. At 150 this job's longestneedschain is about 195 minutes, under the 220-minute merge-queue budget, and ci.yml's critical path (210, throughcoverage-gate) does not change.test/test_flake_pattern_ratchet.pypasses.docs/ci/ci-and-reviews.mddescribes the job without one).ci.yml(4,808 passed); flake8, black and isort are clean on the changed test file.Proof: not provable -- docs or config only
Manual verification
N/A — CI config and a test only. This PR modifies a test file, so its own New-test stability run exercises the new limits on a small scope; the large case takes effect on the next large test PR.
Related Issues
Closes #36722
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)🤖 Opened by the Crew Tasks Bot for mdbrio.