Repository navigation
fix(test): Keep the installer stand-in's step out of the TERM group - #38268
Conversation
test_a_cancel_while_the_child_is_starting_lets_the_apply_command_restore went red on macOS with no `outcome` file. Two causes, both reproduced on a loaded Mac: 1. The cli.sh stand-in ran its long step (`sleep 30 &`) inside the process group the stop signals. The group SIGTERM killed the waited step at the same moment, and bash 3.2 (macOS /bin/sh) then returned from `wait` with 143 and exited without running the TERM trap (~1 in 150 runs under load). The step now runs in a group of its own (`set -m`), as cli.sh's _run_step does on macOS; teardown reaps it through `steppid`. 2. The CommandProvider tests stopped the stand-in with the production 4 s TERM grace; a ~7 s stall SIGKILLed the rollback mid-trap. The class now raises the grace to FALLBACK_TIMEOUT_SECS, which needs _stop_installer to read UPDATE_INSTALLER_TERM_GRACE_SECS at call time instead of binding it as a default at def time. Product behaviour is unchanged. Adds the rule to testing-conventions and AUTOSDE tests-are-deterministic (item 15).
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the fix removes both causes at the test seam. The one product edit, reading the grace when [DESIGN-REVIEWED] b83cd3d Description read[DESCRIPTION-READ] cec0f0a54fe0a58c0c3ccf11117b5940820be694f98e9605b3ea25557fabb27d That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Both causes are fixed where they start (the stand-in's process group and the grace default bound when the function is defined), and the same-file stand-in at Notes
What this change shipsInventory (5 items) — 4 justifiedIntent: stop a macOS flake where the installer stand-in's rollback is sometimes never recorded. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] b83cd3d Description read[DESCRIPTION-READ] 5d78f2d9fa0eb8e9498fac59fdde7795a0dbe6053332a8724796c43fd8f70ecf That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Problem / Motivation
Goal: Stop
test_update_installer_graceful_stop.pyfrom going red at random on macOS.TestCommandProviderRollsBack::test_a_cancel_while_the_child_is_starting_lets_the_apply_command_restorefailed on #38209 (job) withFileNotFoundError: .../case/outcome: the stand-in installer's TERM-trap rollback never ran. #38209 touches no update or kill code.Why it matters
A flaky macOS leg reds unrelated pull requests. Under load the same flake hits
TestTerminateAndReap::test_a_child_that_honours_term_is_never_killedtoo.Not a goal
cli.sh: its_run_stepalready keeps the step in its own group.What changed (motivation → approach → change)
Symptom → cause, both reproduced on a loaded Mac (30 ×
yes,-n 6,--count 300):sleep 30 &) inside the process group the stop SIGTERMs. The same signal killed the waited step, and macOS/bin/sh(bash 3.2.57) sometimes leftwaitwith 143 and exited without running the trap. Aset -xtrace ends atwait <pid>; exit status 143;venv.pre-rebuildleft behind. About 1 in 150 runs.→ Change: the step runs under
set -m(a group of its own, as_run_stepdoes on macOS); its pid goes tosteppid, which the existingreap_groupsfixture reaps._stop_installerbound that grace as adef-time default, so no fixture could raise it.→ Change:
_stop_installer(grace=None)readsUPDATE_INSTALLER_TERM_GRACE_SECSat call time (as_start_installeralready does).TestCommandProviderRollsBackraises it toFALLBACK_TIMEOUT_SECSthrough_floor_monkeypatch; the settle bound uses the same net.Docs:
testing-conventions.md§ Shell stand-ins that trap a group signal;AUTOSDE.yamltests-are-deterministicitem 15.Backwards compatibility
Compatible:
_stop_installeris private;grace=Noneresolves to the same 4 s constant, and every caller that passes a grace is unchanged.Removes nothing: only a default's binding time and test code change.
Tests
test_update_installer_graceful_stop.py: stand-in step in its own group; generous grace for the CommandProvider class.test_update_provider.py+test_coder_shutdown_budget.py(105 passed);test_flake_pattern_ratchet.py(71 passed).Proof: not provable -- other: the flake needs a loaded macOS runner; reverting the production hunk passes on an idle host
Manual verification
N/A — reproduced and verified by the load loops above.
Related Issues
Seen on #38209.
Sibling sweep
Swept: git grep -lE "trap [a-z_'"]+.*TERM" -- test | xargs grep -lE '& $|&\swait|wait "$'
Sibling: test/test_installer_distro.py -- not affected -- it reads cli.sh text and signals no stand-in
Sibling: test/test_mcp_gateway_backend_record.py -- not affected -- its child ignores TERM and relies on no trap
Pattern harvest
Rule candidate: review-prompt
Pattern: shell stand-in relies on a TERM trap while the step it waits on stays in the signalled process group
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributed under the project's Apache-2.0 license.