Repository navigation
Conversation
kirodotdev#22056's ratchet recorded sites kirodotdev#21376 removed 75s later. Three claude captures from kirodotdev#21753 omit the agent_name kirodotdev#22064 then began requiring.
4dfd1f2 to
60d936f
Compare
UX Review (Opus 5.5, fork) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5, fork) — ⏭️ skippedRevision |
Design Review (Opus 5.5, fork) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS This PR is already redundant: base Watch
[DESIGN-REVIEWED] 60d936f Description read[DESCRIPTION-READ] b2181d0c207919105d04b15df7e7d6b77db16cd9033c7e2fd828311bf4ccf22c That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
mbriones98
left a comment
There was a problem hiding this comment.
Checks green and readiness passed at 60d936f; no behavior change.
|
HOLD at 60d936f. The Design Review's CONCERNS is still unanswered, and it is correct. #22178 (3e39e7a) already landed the same SET_GATE_ALLOWLIST lowering and the To clear this: close it as superseded by #22178. Or, if a red still remains, rebase onto main, show the non-empty diff that clears it, and reply to the Design Review. The change itself is sound test data, and CI is green on this head. — crew-pr-reviewer (r113) |
|
Closing as superseded by #22178, which merged on 2026-10-19 with the same changes:
Rebasing head |
Problem / Motivation
Goal: Main's CI stops failing on two test-side defects that no source change can
satisfy, so PRs can be merged again.
Two tests fail on every commit on main, for the same underlying reason: a gate landed and
the tree it measured moved out from under it.
test_backend_cards_advertised.py::test_no_new_runtime_site_gates_a_field_backed_line_on_a_set#22056 added the ratchet at
79caee664f(01:09:41) with a baseline measured against thetree as it then stood. #21376 landed at
f161e2734a(01:10:56), 75 seconds later and withthe ratchet already in its ancestry, and removed three of the very gate sites the baseline
had just recorded. Its branch predated the ratchet, so nothing told it to re-measure.
test_acp_frame_replay.py::test_a_live_header_names_the_agent_its_capture_answered_as#21753 committed three live claude captures whose
_metaomitsagent_name. #22064 thenadded the assertion that a live header must name the agent its own capture answered as. The
versions already agree at
0.93.0; only the name is absent.Both baselines live as literals in test data, so neither failure can be cleared from the
source tree.
Why it matters
PR Readinessis the only required status check, it monitorsci.yml, andci.ymlrunsboth tests. So every PR that reaches CI reds and cannot merge, whatever its own content.
Main has failed CI on each of
88914fd0d1,2cf3d0d69f,6257f698feand602ba3c1d4.Not a goal
test_x_feature_captures.py(2 cases) andtest_harness_parity.py(1) also red on main; fix(ci): drop the proven claude member-capabilities allowlist entry #22186 fixes those with a one-lineallowlist deletion, and this PR deliberately does not duplicate it. Verified by applying
fix(ci): drop the proven claude member-capabilities allowlist entry #22186's deletion here as a throwaway probe: it clears exactly those three and nothing
else. Merge fix(ci): drop the proven claude member-capabilities allowlist entry #22186 first and this PR goes green on its own.
initializeanswer at0.93.0, matching its header'sagent_version. Only a headerfield is missing, and it is filled from that same answer.
What changed (motivation → approach → change)
The ratchet. Symptom:
shrunkfires with three entries. Root cause: the baselinerecords gate sites #21376 deleted. Change: lower it to what the tree actually contains,
measured with the test's own
_set_gates()at88914fd0d1.ACP_BACKENDS_KIRO_SLASH_COMMANDS)providers/acp_cli_overlay.pyproviders/acp_model_effort.pysubagent_parts/model_choice.pyRemoved rather than set to 0, because an entry means "this site may gate on the set"; a site
that no longer does has nothing to allow, and a 0 would silently re-permit the next one.
The capture headers. Symptom: a live header's
agent_nameisNone. Root cause: threecaptures were committed without the field. Change: add it to each of the three, read off
that capture's OWN
initializeanswer, which is what the test's docstring prescribes("read off the wire, not typed") and what
claude/session.jsonlalready does. Placedimmediately after
agent_versionto match that file's field order. All three answer as@agentclientprotocol/claude-agent-acp. The edit is a textual insert of one field into theheader line, so no other byte of any capture moves, asserted per file against the parsed
object before and after.
Backwards compatibility
Compatible: test and fixture data only, and both changes tighten rather than relax. No
runtime code, config, wire field or public function is touched.
The diff removes two keys from
SET_GATE_ALLOWLIST, so its readers are listed:test/test_backend_cards_advertised.py:86-- release (CI) -- thegrownassertion reads the allowlist with.get(k, 0), so a removed key becomes a 0 ceiling and a re-introduced gate site at that path now fails. Proven below.test/test_backend_cards_advertised.py:91-- release (CI) -- theshrunkassertion, the one that was failing; it passes once the baseline matches the tree.SET_GATE_ALLOWLISTappears only in that one file, 4 occurrences, all of them its definition and those two assertions.The capture change adds a key and removes none. Its readers are
test_acp_frame_replay.py::test_a_live_header_names_the_agent_its_capture_answered_asandthe pin gate in
test_real_adapter_fixture_pins.py, which the docstring notes keys on thesetwo fields; both pass (8 passed in the pin gate).
Tests
No new test. Both changes correct baselines inside existing tests, so those tests ARE the
coverage, and each was exercised in both directions.
Ratchet:
88914fd0d1unmodified the file reds: 1 failed, 1 passed. With this change: 2 passed.backend in ACP_BACKENDS_KIRO_SLASH_COMMANDSsite toproviders/acp_cli_overlay.pynowfails the
grownassertion, naming{('providers/acp_cli_overlay.py', …): 1}-- the entrythis PR removes. Before the change that same site was permitted. Probe reverted.
Capture headers:
88914fd0d1unmodified the suite reds on the assertion quoted above. With this change:55 passed.
agent_versionto0.99.0while its wire still answers
0.93.0reds the same assertion. So the missing field isfilled without blinding the check test(acp): red the real-adapter lane when a pin bump leaves live captures behind #22064 added. Probe reverted.
test_real_adapter_fixture_pins.py: 8 passed.Neighbouring suites run together:
test_ratchet_scope.py,test_main_ratchet_audit.py,test_harness_parity.py,test_x_feature_captures.py,test_acp_frame_replay.py,test_backend_cards_advertised.py,test_real_adapter_fixture_pins.py-- 211 passed, 3failed, the 3 being the pre-existing x-capture failures named under "Not a goal". They fail
identically at
88914fd0d1with this change reverted.Manual verification
N/A -- both changes are test baselines, and both assertion directions are exercised above.
Screenshots / video
N/A -- no user-visible frontend surface is touched.
Related Issues
Corrects the baseline added by #22056 after #21376 removed the sites it recorded, and the
headers of the captures added by #21753 against the assertion added by #22064. Unblocks
#22186, which reds on these two tests rather than on its own change.
Pattern harvest
Rule candidate: review-prompt
Pattern: a gate whose baseline is a literal in test code or test data goes stale the moment a
concurrently-approved PR changes what it measures, and the staleness cannot be cleared from
the source tree. Both failures here are that shape, 75 seconds apart in one case. Two signals
a reviewer can act on: a ratchet or pin landing while PRs touching the measured sites are
already approved, and a "lower the baseline" style assertion with no instruction to
re-measure. Prefer deriving the baseline from the tree at a pinned ref, or land the gate
behind the PRs it measures.
Checklist
Contribution License Agreement
I confirm this contribution is made under the terms of the project's contribution licence.