Repository navigation
test: fix two Windows-red tests and write down why - #16083
Conversation
test_app_script_cron_sibling_import: the command-cron recorder case claimed to run on every host, but Windows refuses every command cron before it spawns (no POSIX shell), so the recorder saw nothing. Gate the command cases on the product's own probe, cron_script._resolve_command_shell(). test_context_composition_contract: the byte-exact build_message signature pin went red on main when a branch added the keyword thread_replies_text beside the branch that froze the list. Compare through _frozen_view, which tolerates only a new keyword-only parameter with a default, and add a negative control proving every caller-visible change still reds. testing-conventions.md: two new entries under 'A gate that reds on someone else's pull request'.
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify the Linux/macOS shards of this round are green — the author tested only Windows locally and the probe-gated skip must not fire there. [DESIGN-REVIEWED] 49e1d34 Description read[DESCRIPTION-READ] 845857546763a11cde7c8f7ae985d73dfcf65972b6306ac3ae400473f3b6b152 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Opus 5 Review — 🔴 changes requested (blocking)Reviewed Two blocking issues: both changed test files loosen a ratchet rather than re-pin it — the context-surface signature pin stops catching the exact class of drift already present on main, and the new cron-test skip stands the pin down on hosts that do run command crons. [BLOCK-MERGE] 49e1d34 BLOCKING — test/test_context_composition_contract.py:1568 or re.search(rf"(?:\(|, ){re.escape(p.name)}(?=[:=,)])", frozen)
BLOCKING — test/test_app_script_cron_sibling_import.py:155 if cron_script._resolve_command_shell() is None:
pytest.skip("this host refuses command crons: no POSIX shell to run them")On a POSIX host with [OPUS-REVIEWED] 49e1d34 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
|
Re the Opus review (posted after merge), both findings handled in #16096:
|
) * test: keep the command-cron pin live on sandbox-less POSIX hosts Address Opus review of #16083: probe the command shell after the recorder patches wrap_argv (unpatched, the probe raises on a POSIX host with no sandbox backend and wrongly skips), and re-record thread_replies_text in the frozen build_message signature so the snapshot describes main exactly. * test: gate the real-sandbox command-cron case on the shell probe too
Problem / Motivation
Two backend tests are red on the Windows shards of every open pull request (seen on #15823, shards 1 and 5). Neither is a flake: both fail the same way every run, locally too.
test_app_script_cron_sibling_import.py::TestACommandCronRunningBundleCode::test_the_call_site_leaves_the_bundle_readableassert len(spawns) == 1fails with[]. The file claimed this case runs "on every host".test_context_composition_contract.py::TestSurface::test_builder_members_keep_their_kind_and_signaturebuild_message's full signature byte for byte. #15998 recorded the list; #15761, cut from an older base, added the keywordthread_replies_text. Each was green alone; on main together the pin is red on every platform.What changed
cron_script._resolve_command_shell() is None, not on the platform name. A host that runs command crons is still held to the pin._frozen_view, which drops only a NEW keyword-only parameter that has a default (no caller can feel it). A removed, renamed, reordered or retyped parameter, a new positional or required one, or a changed return annotation still reds.test_only_a_compatible_addition_is_toleratedis the negative control for each.docs/system-specs/common/testing-conventions.md: two entries under "A gate that reds on someone else's pull request" -- gate on the product's refusal probe and run an "every host" test on Windows before merge; pin what callers depend on, not every byte, for a surface feature work keeps growing.#16078 bumps the frozen string for the same red. The two do not touch the same lines and either can merge first.
Testing
On Windows (this host): both files
125 passed, 3 skipped;--noconfteston the cron file passes; flake8, black and the comment-history gate are clean. The doc-reading tests (test_docs_mock_pid_convention,test_black_gate_contract,test_symbols_manifest_contract,test_harness) pass. Linux/macOS rely on CI.Pattern harvest
Rule candidate: a test whose name or docstring says "every host" must gate on the product's own refusal probe when the product refuses that feature on some platform, and a byte-exact snapshot of a surface other branches routinely grow should tolerate compatible additions.