Repository navigation
ci(design-review): verify readers of anything a PR takes away - #16092
Conversation
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS The replay table is the only evidence the new BLOCK fires correctly — author-asserted against #13273/#12798 base trees, never exercised by this PR's own run; verify it. What this change shipsInventory (7 items) — 7 justifiedIntent: Design Review itself verifies every reader of anything a PR removes, hides or prunes, and blocks on an unlisted broken one — an ADDITION, derived from two linked incidents (#13273→#16072, #12798→#16016/#16033).
No prior mechanism does this job: grepped [FIRST-PRINCIPLES-REVIEWED] 909dc46 Description read[DESCRIPTION-READ] 0523c33b2682d12907c966830bf0629d5895bb1445eaa47f1fd8ced5aea1a29b That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Land the sibling PR-template PR promptly, so authors learn the [DESIGN-REVIEWED] 909dc46 Description read[DESCRIPTION-READ] 27389d37b76aa2c9e544f7bdae9cfecc087b08425b19a65f78fd05abb413bb63 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
b33012d to
d9c3753
Compare
|
|
|
|
d9c3753 to
45d11da
Compare
|
45d11da to
448191d
Compare
|
|
|
448191d to
a4925d7
Compare
|
|
a4925d7 to
9d40419
Compare
|
|
|
Design Review now runs a TAKE-AWAY CHECK: it lists what the patch removes, renames, hides, tightens or migrates, greps the tree for readers across every entry point (crew page, chat, subagent, cron, app bundles, prompt builder, release), compares them with the description's Backwards compatibility list, and BLOCKs on a broken reader the author did not list, or on 'Removes nothing' when a take-away has readers outside the patch. A weakly justified listed reader is CONCERNS. The new trigger joins the falsification rule and the one-BLOCK budget. The fork twin carries the same text. #13273 and #12798 both passed every lane while breaking readers their authors never checked (app crons importing their own bundle; chat resume, subagent continue and the prompt builder reading pruned crewmate rows).
9d40419 to
909dc46
Compare
|
|
|
|
Problem / Motivation
Goal: Design Review checks, against the code, every reader of anything a PR takes away, and blocks on a broken reader the author did not list.
#13273 hid
<config_dir>/appsfrom cron children. Every app cron that runs code out of its ownapps/<app>/bundle then failed before its first line (reverted by #16072). #12798 pruned synced crewmateconfig.agentsrows after checking only the Crew page. Chat resume, subagent continue and spawn, and the prompt builder still read those records (fixed by #16016 and #16033). Both PRs passed every review lane. The author's "Compatible" claim was never checked against code.Why it matters
A take-away with an unchecked reader ships a break on a path the PR never mentions. Both cases above broke core paths (app crons, chat resume, subagents) for every user who had the data, and needed follow-up PRs to repair.
Not a goal
Reader:line format there)..github/review-prompts/first-principles.md): it guards over-building, and this check is out of its scope.What changed (motivation → approach → change)
The Design Review lane took the author's compatibility line on trust, so a reader outside the PR's own surface was never looked at.
The reviewer now does the reader sweep itself. It lists what the patch takes away: a removed or renamed key or row, a migration that deletes or rewrites records, a newly hidden path, a stricter validator, a removed public function. It greps the tree for each one across every entry point: crew page, chat, subagent, cron, app bundles, prompt builder, release. Then it compares with the description's
## Backwards compatibilitylist, oneReader: <path>:<symbol> -- <entry> -- <why | test>line each, orRemoves nothing: <why>. A reader aCompatible:line names bypath:symbolcounts as listed, and one aBreaking:line names counts as listed and accepted; aCompatible:naming no reader lists nothing. When a description cut at the 8000-byte capture cap is missing the section, or the section is the last one before the cut, the check caps at CONCERNS.The prompt in
.github/workflows/design-review.ymland its fork twin.github/workflows/fork-design-review.yml(same text in both) gains this TAKE-AWAY CHECK and a new BLOCK TRIGGER. A reader that breaks under the patch and is not listed blocks, named aspath:symbolplus its entry point.Removes nothinglists no reader, so under it every breaking reader is unlisted. A listed reader with a weak reason is CONCERNS. The trigger joins the existing falsification rule with its own three items, and the one-BLOCK budget is unchanged.docs/ci/ci-and-reviews.mdrecords the trigger and the section shape it reads.Replay over the two diffs. The same-repo lane runs this PR's own prompt, but this PR takes nothing away, so its own review never exercises the new BLOCK; the replay is the evidence:
<config_dir>/appspassed asextra_hidden_dirsinrun_script_sandboxed/run_command_sandboxedsrc/kiro_crew/apps/bridges.py:_cron_defs_from_manifesthands the app'sscript/command, which live in and import fromapps/<app>/, tosrc/kiro_crew/cron_script.py:resolve_script_pathcrewmate_prune_migration.pydeletes syncedconfig.agentsrowssrc/kiro_crew/config/loader.py:_resolve_agent_selection(chat resume),src/kiro_crew/subagent_persistence.pyandsubagent_manager/admission/gate.py("selected member is unavailable"),src/kiro_crew/execution_context.py:execution_from_record(prompt builder member lookup)Neither PR listed those readers, so both would have been a BLOCK naming them.
Backwards compatibility
Compatible: only review-prompt text changes. A PR that takes nothing away is reviewed as before. One that does now needs its readers listed, or it gets a Design Review BLOCK, which a human can still override.
Tests
test_ai_review_workflows.py::TestDesignTakeAwayCheckpins the TAKE-AWAY CHECK byte-identical across both design lanes (the fork lane is the one outside contributors get), and pins that an unlisted broken reader is BLOCK, a weak reason is CONCERNS, theReader:shape, and the truncated-description cap to CONCERNS. It also checks that every entry-point path the sweep names still exists, so a moved file fails a test instead of leaving the sweep grepping a ghost path.test_ai_review_workflows.py,test_fork_stage2_checkout.py,test_pr_quality_gates.pyandtest_ci_fleet_routing_expression_parity.pypass locally.Manual verification
Both workflow files parse as YAML, and the added lines are identical in the two. The replay above was checked against the base trees of #13273 (
0496784bd1~1) and #12798 (9abfda6fe2~1).Related Issues
Refs #13273, #12798, #16072, #16016, #16033
Checklist