Repository navigation
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the change reuses the Linux pin mechanism, writes macOS timings to a separate pin file, and pins change only which shard runs a file, so one revert walks it back. Notes
[DESIGN-REVIEWED] a80580a Description read[DESCRIPTION-READ] 85c58f180092583ee04e60908ed05463f8ba66cdfd8200501647bb29a63c61e3 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. Every item reuses the existing Linux pin mechanism ( What this change shipsInventory (6 items) — 6 justifiedIntent: keep the nightly macOS shards under the timeout so publication isn't cancelled; this is a FIX (CI balance).
[FIRST-PRINCIPLES-REVIEWED] a80580a Description read[DESCRIPTION-READ] 4cca4563cfdb2b53f95460f38621d7e1f919822cf8b25b362501e01e0b84d875 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Code change approver recommendation: A person should decideWhy: the decider estimates a 12% chance that this change introduces a defect, within the 12% limit for approving on its own, but it changes 902 lines, more than the 400 it may approve on its own. The PR records per-file macOS test timings with the existing ci_pytest_progress plugin, uploads them as macos-pytest-progress-N, adds --platform to the pin generator (defaults stay linux, and the macOS default output can't overwrite the Linux pin file), and wires a generated 8-shard macOS pin file into the macOS shard command. The pin loader strictly validates shard count, owner range and duplicates, and the new gate test checks the committed file against SHARD_COUNT=8 with every shard getting pins. The timing plugin is payload-only and can't change test selection or outcomes. Pins only move which shard owns a file, so every file still has exactly one owner. Release, Windows and the reduced/leaf scopes are untouched, and the existing tests that limit the progress plugin and pin flags to certain jobs only read ci.yml, so nothing conflicts. Tests cover both new behaviours. Remaining concerns (thin timing data behind the pins, stale comment in test-durations.yml, path filter, description drift) are non-blocking. An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
ff7d960 to
2ad1ad8
Compare
2ad1ad8 to
a80580a
Compare
bolichen97
left a comment
There was a problem hiding this comment.
@CrysisDeu 改动本身没问题:权限、secrets、触发条件都没动,新测试本地全过。合并前请做一件事:
-
Rebase 到最新的 main(
git merge-tree显示没有冲突)。这次 macOS 的 2 片红都已经在 main 上修好了:- 分片 4 由 #37476 修(只读源目录在 macOS 上 rename 失败,和本 PR 无关)。
- 分片 8 由 #37478 修。这是新分片表把
test/test_harness_parity.py挪到第 8 片后暴露出来的泄漏:同一个 worker 里,前面一个 dashboard 测试在kiro_crewlogger 上留了 handler。
rebase 后
ci:macos会再跑一次全套,请确认 8 片全绿。这是新分片表第一次在合并后的树上跑,邻居又变了,可能还有别的泄漏。 -
可选:
.github/workflows/test-durations.yml头部注释(第 4-8 行)还写着只有 Linux 用 pin 表,顺手改一下。
另外:Linux Backend Tests (3.12, 7) 是 test_denied_commands_security_cost.py:595 的墙钟时间 flake,Coverage Gate 跟着它红,两者都已重跑转绿。
macOS 全绿后我就批准。
|
No blocking issues were found on this head. Approval waits on the open maintainer request: rebase onto current main and show a What the review checked on a80580a:
Done well: the PR reuses the existing Linux pin machinery and adds only a Optional:
|
|
@CrysisDeu 提醒一下:这个 PR 还需要 rebase 到最新的 main(没有冲突)。rebase 后 |
Problem / Motivation
Goal: Let the nightly macOS backend shards be balanced by measured macOS test time, so the slowest shard drops back under 32 minutes.
The Nightly macOS backend shards (Platform Tests / Backend Tests (macOS) 1..8) have crept up from a 31-minute slowest shard (11/7) to 39 minutes (11/14, run 43564614563), and shard 5 hit the old 40-minute cap on 11/12. pytest time per shard, from the job logs:
Two causes:
scripts/ci_file_shard_pins.txt(ci: balance Linux backend test shards with a measured pin file #30278), but that file is built from Linux timings, and those do not predict macOS cost: the same hash split is within 1.11x on Linux. The heavy macOS files are git/subprocess-heavy ones that are cheap on Linux, e.g.test/test_crewmate_worktree.py(19 s on Linux, its top 3 tests alone 208 s on macOS, shard 5) andtest/test_remote_command_lifetime.py(2.6 s on Linux, 2 x 60 s on macOS, shard 5).A balanced split would put every shard near the 26.7-minute mean, under the 32-minute target, without changing the shard count,
max-parallel: 4or the timeout.Why it matters
The nightly gates publication on these shards. One shard over the cap cancels the run and skips every publish job (11/12). The lane is two waves of 4, so the slowest shard of each wave sets the whole lane's time.
Not a goal
SHARD_COUNT,max-parallel(4) ortimeout-minutes.What changed (motivation → approach → change)
Symptom: the slowest macOS shard is ~40% over the mean -> cause: path-hash ownership plus macOS-only heavy files that Linux timings cannot see -> change: record per-file macOS seconds the same way backend-test already does, and pin the heaviest files from those measurements.
.github/workflows/platform-tests.yml: the macOS shard pytest command loads-p scripts.ci_pytest_progress --ci-progress-dir="$RUNNER_TEMP/pytest-progress"(the same plugin and flagci.ymlbackend-test uses), passes--file-shard-pins scripts/ci_file_shard_pins_macos.txt, and a newalways()step uploadsmacos-pytest-progress-<N>(7-day retention) after the existing shard-log upload.scripts/ci_file_shard_pins.py: new--platform {linux,macos}(defaultlinux) choosing which<platform>-pytest-progress-*artifacts to read; the default--outfollows it (ci_file_shard_pins.txt/ci_file_shard_pins_macos.txt), so a macOS refresh cannot overwrite the Linux pins. Linux behaviour is unchanged.docs/system-specs/common/testing-conventions.md(Shard balance): macOS now uses its own measured pin file; Windows, release, reduced and leaf keep pure hash ownership.scripts/ci_file_shard_pins_macos.txt: generated, 800 pins, from this PR's two macOS runs (43657254686 hash-only, 43664027179 with a one-run pin file):python scripts/ci_file_shard_pins.py <run-a> <run-b> --platform macos. Replayed on each run's data the pinned worker-seconds per shard are 2763-2998 and 2671-2913, against 2413-3622 for the hash split. Shard count,max-parallel: 4andtimeout-minutes: 60are unchanged.Backwards compatibility
Compatible: the plugin only writes timing JSONL and never changes test selection or outcomes; pins change only WHICH macOS shard runs a file, every file still has exactly one owner; the generator's default stays
linux.Removes nothing: the diff only adds a pytest plugin flag, a pin file, an artifact upload step and a CLI option.
Tests
test/test_macos_platform_tests_gate.py::test_the_shards_record_per_file_seconds_for_the_pin_generatorlocks in the plugin flags on the macOS shard command and themacos-pytest-progress-${{ matrix.group }}upload withif: always().It also asserts the macOS pin file is wired in, made for
SHARD_COUNT, and pins files to every shard.test/test_ci_file_shards.py::test_pin_generator_reads_macos_artifacts_into_the_macos_pin_filelocks in that--platform macosreadsmacos-pytest-progress-*, linux mode refuses them, and the macOS default output is not the Linux pin file.test_the_shard_log_is_uploaded_even_when_the_shard_is_cancelledstill finds the shard-log upload first.Proof: not provable -- test-only change
Manual verification
Labelled
ci:macossomacos-on-demand.ymlruns the full platform-tests.yml suite on this head; run 43657254686 uploaded all 8macos-pytest-progress-*artifacts (the pin file's input). Its red shards (1, 2, 4, 8) are main regressions also red on other PRs' macOS runs (test_readonly_source_skill_install.py,test_autonudge_row_scrub.py,test_one_conversation_one_session.py,test_history.py,test_telegram_sessions_resume.py), not this diff. pytest was not run locally.Related Issues
N/A — found by Build Watcher on the 11/14 nightly (run 43564614563).
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributed under the project's Apache-2.0 license.