Repository navigation
test: clear main's four reds left by #21753, #21598 and later backend edits - #22178
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that [DESIGN-REVIEWED] 01f8ca6 Description read[DESCRIPTION-READ] 41b7e989fd7860b9314eb98c9fc80fee121ac97ab34d357510f56556f9115970 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Check that the What this change shipsInventory (4 items) — 4 justifiedIntent: get main's CI green again by clearing four stale pins left behind by merged PRs. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] 01f8ca6 Description read[DESCRIPTION-READ] a7407b257ff1522ad5154edf56c49afe3214532b29b1d9d812081456f76d0491 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
780b537 to
8a99388
Compare
… edits #21753 committed a live claude saved-member-spec capture that a test replays, so the x-capture gate fails main with "allowlisted but now proven; delete the entry". Delete that row from scripts/check_harness_parity.py. The same commit's three claude captures carry agent_version but no agent_name in their _meta header, so test_acp_frame_replay's live-header check fails against their own initialize agentInfo. Add "agent_name": "@agentclientprotocol/claude-agent-acp" to each header. test_backend_cards_advertised's SET_GATE_ALLOWLIST overstates three counts after recent backend edits (its shrink half fails): drop the acp_cli_overlay.py and model_choice.py KIRO_SLASH_COMMANDS rows (now 0) and lower acp_model_effort.py to 2. #21598 split llm_helpers.py, and its facade now serves name_grant from llm_helpers_parts/tool_approval.py through its TYPE_CHECKING manifest. test_name_grant_facade's spelling scan read only module-level imports, so it lost the llm_helpers.name_grant spelling (which test_llm_helpers_deny_notice.py patches) and its helper-binding case fails. Count a module-level if TYPE_CHECKING: block as module level. Part of #20192
buluoray
left a comment
There was a problem hiding this comment.
Approving at 8a99388. Applied onto current main and verified.
Harness Parity Gate: with HARNESS_BASE_REF=origin/main, scripts/check_harness_parity.py exits 1 on main (ACP_BACKENDS_MEMBER_CAPABILITIES [claude]: allowlisted but now proven) and exits 0 with this change; the --test self-test passes. The removed row is genuinely proven: backends_members.py cites test/fixtures/acp_frames/claude/saved-spec-projection-live.jsonl, the capture and its .expected.json are committed, and test/test_claude_live_captures.py replays it.
name_grant patch scan: test_the_patch_scan_reads_every_spelling_and_nothing_else[helper-binding] is red on main because the llm_helpers split moved its name_grant import under if TYPE_CHECKING:, while llm_helpers.name_grant still resolves to the facade at runtime. Folding top-level TYPE_CHECKING blocks into the module-level walk restores that spelling. test/test_harness_parity.py plus test/test_name_grant_facade.py: 94 passed with -n0 (main: 2 failed).
Not covered here, and still red on main: test_acp_frame_replay.py::test_a_live_header_names_the_agent_its_capture_answered_as. The three claude live captures from #21753 (compact-live, mcp-stdio-mount-live, saved-spec-projection-live) carry no _meta.agent_name, which #22064 now requires to match the initialize agentInfo name (@agentclientprotocol/claude-agent-acp). That is the Backend Tests shard 3 failure on this head, so this cannot go green until those headers are fixed; adding it here would let this PR clear all three reds at once. Small follow-up: the TYPE_CHECKING fold matches a bare TYPE_CHECKING name only, not typing.TYPE_CHECKING.
8a99388 to
01f8ca6
Compare
buluoray
left a comment
There was a problem hiding this comment.
Approving at 01f8ca6. This supersedes my approval at 8a99388.
Verified each of the four main reds on current origin/main (6257f69) and confirmed this head (origin/main plus one commit) clears all of them. With -n0 on test/test_harness_parity.py, test/test_name_grant_facade.py, test/test_acp_frame_replay.py, test/test_claude_live_captures.py and test/test_backend_cards_advertised.py, plus HARNESS_BASE_REF=origin/main python3 scripts/check_harness_parity.py: main has 4 failed / 151 passed and the gate reports ACP_BACKENDS_MEMBER_CAPABILITIES [claude] as allowlisted but now proven; this head has 155 passed and the gate passes (17 cited plus 3 allowlisted).
- Harness parity: deletes the (ACP_BACKENDS_MEMBER_CAPABILITIES, claude) row; the claude live captures prove it, so the allowlist shrinks.
- name_grant facade: the scan now reads the body of a module-level
if TYPE_CHECKING:block (not its else). llm_helpers.py lists name_grant there, and at runtime llm_helpers.name_grant is kiro_crew.name_grant, so the spelling is real. - Capture headers: in compact-live, mcp-stdio-mount-live and saved-spec-projection-live,
_meta.agent_name@agentclientprotocol/claude-agent-acp and agent_version 0.93.0 equal the agentInfo name and version in each file own initialize result, so the header states what the capture answered as. A wrong agent_name reddens test_a_live_header_names_the_agent_its_capture_answered_as. - SET_GATE_ALLOWLIST: new counts equal the runtime gates in source (acp_cli_overlay.py 0, model_choice.py 0, acp_model_effort.py 2 at lines 172 and 611; acp.py and acp_turns.py still 1). Every edit lowers or removes a row; raising acp_model_effort.py back to 3 reddens test_no_new_runtime_site_gates_a_field_backed_line_on_a_set.
CI at this sha when checked: 54 success, 10 skipped, 0 failed, 33 still running. Design and First Principles PASS; Opus and GPT no blocking findings. Auto-merge is on, so this lands once the required checks are green.
Small follow-up from before: the TYPE_CHECKING fold matches a bare TYPE_CHECKING name only, not typing.TYPE_CHECKING.
Problem / Motivation
Goal: Turn main green again with one PR that clears all four of its current reds. Each red is a stale entry that a merged PR left behind. Two of them are hidden today: Await Fast Gate fails first, so main's backend shards never run them.
ACP_BACKENDS_MEMBER_CAPABILITIES [claude]: allowlisted but now proven; delete the entry. Main push run 39266672407 on 8e686e5, and again on later headsscripts/check_harness_parity.pystill lists("ACP_BACKENDS_MEMBER_CAPABILITIES", "claude")in the x-capture allowlisttest/test_name_grant_facade.py::test_the_patch_scan_reads_every_spelling_and_nothing_else[helper-binding](assert set() == {'D'})llm_helpers.py. Its facade now servesname_grantfromllm_helpers_parts/tool_approval.pythrough its module-levelif TYPE_CHECKING:manifest. The test's spelling scan read only top-level imports, so it stopped seeing thellm_helpers.name_grantspelling.test/test_llm_helpers_deny_notice.py:240really patches through itif TYPE_CHECKING:block in the scantest/test_acp_frame_replay.py::test_a_live_header_names_the_agent_its_capture_answered_as(claude/compact-live.jsonl: _meta disagrees with its own initialize agentInfo,(None, '0.93.0'))agent_versionbut noagent_namein their_metaheader"agent_name": "@agentclientprotocol/claude-agent-acp", the name each capture's owninitializeanswer gives, to the three headerstest/test_backend_cards_advertised.py::test_no_new_runtime_site_gates_a_field_backed_line_on_a_set, shrink half (lower SET_GATE_ALLOWLIST ...)ACP_BACKENDS_KIRO_SLASH_COMMANDSrows overstate their files' membership-test counts:providers/acp_cli_overlay.py1 (now 0),providers/acp_model_effort.py3 (now 2),subagent_parts/model_choice.py1 (now 0)Why it matters
Fast Gate runs on every PR. While it is red, Await Fast Gate fails, the heavy matrix is skipped, and the Coverage Gate fails closed. So no PR rebased onto main can go green until both entries are fixed.
Not a goal
_metaheader gains the agent name), orllm_helpers' facade.What changed
scripts/check_harness_parity.py: delete the("ACP_BACKENDS_MEMBER_CAPABILITIES", "claude")x-capture allowlist row. This is a 1-line list edit, and it is exactly what the gate's own error message asks for.test/fixtures/acp_frames/claude/{compact,mcp-stdio-mount,saved-spec-projection}-live.jsonl: add"agent_name": "@agentclientprotocol/claude-agent-acp"to the_metaheader line, afteragent_version, which is the key order every other backend's live capture uses. The frames are unchanged.test/test_backend_cards_advertised.py:SET_GATE_ALLOWLISTloses theacp_cli_overlay.pyandmodel_choice.pyACP_BACKENDS_KIRO_SLASH_COMMANDSrows (their counts are now 0), andacp_model_effort.pygoes from 3 to 2. The real counts come from the test's own_set_gates()on main; note that its shrink message prints the old values, not the new counts.test/test_name_grant_facade.py:_spellings()now also counts a module-levelif TYPE_CHECKING:block as module level. That block is where a split module's facade lists the names it serves from an owner. The docstring says so. Thellm_helpers.name_grantspelling comes back, so the corpus patch scan sees the real seam intest_llm_helpers_deny_notice.pyagain. The extra spellings can only add patched names, never hide one.Backwards compatibility
Compatible. One CI list row, one test allowlist, three fixture header lines and one test helper change; no runtime code, path, key or field changes.
Tests
No new tests. The two reds are the proof:
python3 scripts/check_harness_parity.pyallowlisted but now proven)every X-feature member cites a replayed live capture (17 + 3 allowlisted))test/test_name_grant_facade.py[helper-binding])test/test_acp_frame_replay.pytest/test_backend_cards_advertised.pytest_backend_cards_advertised.py,test_acp_frame_replay.py,test_real_adapter_fixture_pins.py,test_x_feature_captures.py,test_harness_parity.py,test_name_grant_facade.py,test_llm_helpers_deny_notice.pyflake8 and black are clean on the changed test files.
Manual verification
N/A: every change is checked by the gate or test that was red.
Related Issues
Part of #20192
Pattern harvest
The second red is a semantic merge race between two refactor PRs. #22069 pinned a spelling that #21598 had just moved behind a facade manifest. Facade-aware spelling discovery is now in this one test; other
*_facade.pyspelling scans that read only top-level imports could miss served names the same way. That is a follow-up, not part of this fix.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.