Skip to content

ci(pr-hygiene): require a Reader: list on take-away diffs - #16115

Merged
chenmingwei23 merged 1 commit into
mainfrom
ci/take-away-reader-list
Oct 2, 2026
Merged

chenmingwei23 merged 1 commit into
mainfrom
ci/take-away-reader-list

Conversation

@iamwhatever

@iamwhatever iamwhatever commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A PR that hides, deletes, tightens or migrates something must list every reader of that thing on main, so a reader on another entry point is not broken unnoticed.

Two take-away PRs passed CI and broke readers their authors never listed:

Why it matters

Both breaks reached main and hit every user of the affected entry point (all app script crons; resumed chats and subagent runs of a pruned crewmate). The template's ## Backwards compatibility section let an author answer from the Goal; nothing asked for the code readers.

Not a goal

  • Changing Design Review (design-review.yml); a separate item parses the same Reader: format there.
  • Judging reader-list content in CI. The gate checks presence and format only; completeness is the reviewers' job.
  • Detecting every take-away shape. The detector starts with a small set of shapes and can grow.
  • Making ## Backwards compatibility a required section for every PR.

What changed (motivation → approach → change)

Readers were missed because nothing asked for them → make the author list them in a fixed, parseable line, and fail PR Hygiene when the diff has a take-away shape but the body lists none.

  • .github/PULL_REQUEST_TEMPLATE.md: ## Backwards compatibility now asks, for a take-away diff, for one line per reader: Reader: <path>:<symbol> -- <entry: chat|cron|subagent|app|crew page|release> -- <why it still works | test name>, found by grepping origin/main across every entry point; otherwise exactly one Removes nothing: <why> line.
  • .github/scripts/take_away_check.py (new): reads the merge-base diff and PR_BODY (env, data only). Four shapes, each visible on one diff line, in product code (src/) only: a *_migration.py file with a changed code line (a comment-only edit is not), a new extra_hidden_dirs= argument (an exact pass-through extra_hidden_dirs=extra_hidden_dirs is not), a config.agents row del/.pop, a removed module-level public def (column 0, not re-added in the same file; a def moved to another module counts, its old import path breaks). Test files are skipped; a re-wrapped call is not a new hit. Removed config keys and stricter validators are left out: a line diff cannot tell a removed field from docstring prose or a parameter, so those shapes hard-failed unrelated edits. The doc says the rule still applies to them. On a hit, at least one well-formed Reader: line must sit in ## Backwards compatibility (outside comments and fences); Removes nothing: does not satisfy a hit; malformed lines are named.
  • .github/workflows/code-review.yml, PR Hygiene: a merge-base step (copied from the sibling jobs) and "Require a reader list on take-away diffs", after the Pattern harvest step. The body reaches the script only via env. Like the template-sections step above it, the new step skips dependabot[bot], which writes its own body.
  • docs/system-specs/common/take-away-changes.md (new): the rule, what CI checks, fix(cron): mask the apps tree from every cron child that runs model-supplied code #13273 and feat(members): prune sync-generated crewmates once at startup #12798 as worked examples with the reader lines they should have carried. Indexed in docs/system-specs/common/README.md.
  • AGENTS.md: one routing row to that doc.

Hit rate on the last 150 main commits: 4 flagged (the #13273 revert, a removed public search_skills, two crewmate_prune_migration.py follow-ups) -- each does take something away.

Backwards compatibility

Removes nothing: adds a template paragraph, a new PR Hygiene step and a script; no existing step, key or function changes. The new step fails only PRs whose diff has a take-away shape and lists no Reader: line (this PR's own diff has none -- verified by running the script on it).

Tests

test/test_take_away_check.py (new):

Locally: test_take_away_check.py, test_pr_hygiene_guidance.py, test_prepare_pr_body_register.py, test_fix_loop_discipline.py -- 95 passed. flake8 and isort clean on the new files.

Manual verification

Ran the script on the full #13273 and #12798 diffs with their real bodies (both exit 1) and on this PR's own diff (exit 0).

Related Issues

Follows the #13273 / #12798 regressions (fixes #16072, #16016, #16033).

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@iamwhatever
iamwhatever requested a review from a team as a code owner October 2, 2026 08:42
@iamwhatever
iamwhatever requested a review from CrysisDeu October 2, 2026 08:42
@github-actions github-actions Bot added the readiness: action required A blocking check or review needs attention label Oct 2, 2026
@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 0e052701eecd6aa744fee25c363873435c226c42 and found no blocking issues.

This comment is updated in place on each push.

Review details

FINDING -- .github/workflows/code-review.yml:534 -- "git diff" encodes a pure module rename without removed lines, so removed import paths evade the reader check -> Fix: add --no-renames.
FINDING -- .github/scripts/take_away_check.py:135 -- "m.group(0)" differs when an existing tuple or call is merely reformatted across lines, falsely failing PR Hygiene -> Fix: compare stable assignment/call prefixes.
[GPT-REVIEWED] 0e05270

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 0e052701eecd6aa744fee25c363873435c226c42: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🟡 CONCERNS

Design-level review of 0e052701eecd6aa744fee25c363873435c226c42 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Detector hardcodes today's seam names with no drift pin; a rename in src/ silently de-fangs the gate it was built for.

Watch

The shapes match literal strings (extra_hidden_dirs=, config.agents row delete) that live in product code the detector never verifies. The doc itself concedes the coupling ("a PR that renames one updates the detector in the same commit"), but nothing enforces it: a sandbox refactor that renames extra_hidden_dirs makes HIDDEN_DIR_RE match nothing forever, the step stays green, and the exact #13273 regression class reopens with no signal — the failure direction is permanent silence, not a visible red.
Clears when: test_take_away_check.py gains a seam-existence pin (grep src/ at HEAD for each shape's anchor string, e.g. extra_hidden_dirs and a config.agents write site) so renaming a seam reddens the detector's own suite.

Suggestions

  • The pr-hygiene job now carries its second merge-base resolution and the repo its fourth copy of that step; a follow-up extracting one composite action (or reusing the job's existing diffbase output at line 32 of code-review.yml instead of adding a second resolver in the same job) removes the drift surface.

[DESIGN-REVIEWED] 0e05270

Description read

[DESCRIPTION-READ] 49cb6a396dd0df13b975165006d4df575cc7442a8c761ad4daf66fdbe4627aec

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ PASS

Premise-level review of 0e052701eecd6aa744fee25c363873435c226c42 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Verify the author's "4 flagged in the last 150 main commits" false-positive run — it is the one claim this gate's blast radius rests on and CI never re-measures it.

What this change ships

Inventory (8 items) — 8 justified

Intent: stop a PR that takes something away from merging without naming the readers it breaks — an ADDITION (new CI gate), with provenance in two linked regressions (#13273 reverted by #16072; #12798 fixed by #16016/#16033).

  1. PR template asks take-away diffs for Reader: lines, else one Removes nothing: line — justified
  2. PR Hygiene fails a take-away-shaped diff whose body lists no Reader: line — justified
  3. New detector script with four shapes; the shapes it cannot see are declared in the doc — justified
  4. pr-hygiene gains its own merge-base step (4th copy; grepped merge-base in code-review.yml: 3 pre-existing per-job copies, and GHA jobs cannot share steps) — justified
  5. The new step skips dependabot, whose body is machine-written — justified
  6. New spec take-away-changes.md + common/README row (same-commit doc invariant) — justified
  7. AGENTS.md routing row to that spec — justified
  8. Verbatim fix(cron): mask the apps tree from every cron child that runs model-supplied code #13273/feat(members): prune sync-generated crewmates once at startup #12798 diff+body slices as test fixtures — justified

Checked for a second spelling: pr-description-check.sh reads body only, never the diff; the Breaking: writer sweep covers writers of intentional breaks, not readers of claimed-compatible removals. The Reader: parser has exactly 1 consumer (the new step; grepped Reader: under .github — design-review carries none).

[FIRST-PRINCIPLES-REVIEWED] 0e05270

Description read

[DESCRIPTION-READ] 2dc70f6a3ae9b07f947932b9bc84f3f34388778fc2e667aba15d7cac9530f954

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed 0e052701eecd6aa744fee25c363873435c226c42 — this comment is updated in place on each push.

Review details

The take-away gate's detector both misses a real take-away (a git mv of a module) and fires on unrelated code (del agents[k]).

FINDING — .github/workflows/code-review.yml:534 — git diff "$MERGE_BASE" "$HEAD_SHA" keeps git's default rename detection, so a module move emits only similarity index/rename from/rename to and no hunks (verified on this repo's own src/kiro_crew/crew_log/tree.py → session_tree.py rename); parse_diff records empty added/removed, the remove: ... removes public def shape never fires, and a PR that deletes every kiro_crew.<old> import path reports "No take-away shape in the diff" → Fix: pass --no-renames to that git diff so a move arrives as delete+add under the old path.

FINDING — .github/scripts/take_away_check.py:65 — AGENTS_ROW_DELETE_RE matches a bare agents identifier (\b[\w.]*\bagents), so a new src/ line over an unrelated dict — the shape already on main at src/kiro_crew/context_management.py:192 (del agents[k], a sub-agent eviction map) — fails PR Hygiene with "deletes config.agents rows" and demands readers of something that has none → Fix: require the qualified receiver the doc names, e.g. (?:cfg|config|doc|overlay)\s*\.\s*agents.

[OPUS-REVIEWED] 0e05270

Verdict parsed from the review's SHA-scoped output markers for commit 0e052701eecd6aa744fee25c363873435c226c42.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 0e052701eecd6aa744fee25c363873435c226c42: <one-sentence reason>

@iamwhatever
iamwhatever force-pushed the ci/take-away-reader-list branch from 85f7cdf to 7600103 Compare October 2, 2026 09:50
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 2, 2026
@iamwhatever
iamwhatever force-pushed the ci/take-away-reader-list branch from 7600103 to 54e3f65 Compare October 2, 2026 10:08
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 2, 2026
@iamwhatever
iamwhatever force-pushed the ci/take-away-reader-list branch from 54e3f65 to 1af9485 Compare October 2, 2026 10:46
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 2, 2026
A diff that hides, deletes, tightens or migrates something must list every reader of it under ## Backwards compatibility. #13273 and #12798 both passed CI and broke readers nobody listed. Adds .github/scripts/take_away_check.py (presence/format only), the template format, a convention doc and one AGENTS.md route.
@iamwhatever
iamwhatever force-pushed the ci/take-away-reader-list branch from 1af9485 to 0e05270 Compare October 2, 2026 11:11
@iamwhatever

Copy link
Copy Markdown
Contributor Author
  • Second merge-base resolver in pr-hygiene -- rebutted span=45c2a4decf80

The diffbase output at line 32 belongs to the autosde-rules job; steps in one job cannot read another job's step outputs.
pr-hygiene had no merge-base step before this PR, so this is its first, copied verbatim from the siblings.
Extracting a composite action touches four jobs and is outside this PR's Goal.

@iamwhatever

Copy link
Copy Markdown
Contributor Author
  • Seam rename leaves a shape silently dead -- rebutted span=98a3a1a87ac7

Disproportional here. A dead shape only weakens a presence-only gate; it never fails a PR.
The doc already tells a seam renamer to update the detector, and the shape list is documented as incomplete.
A seam-existence pin would tie a CI-tooling test to product internals; scope is frozen to the Goal.

@iamwhatever

Copy link
Copy Markdown
Contributor Author
  • Re-wrapped tuple across lines reads as a new hide -- rebutted span=b36c01ad9b53

Single-line re-wraps are already neutral (pinned by the re-wrapped-call test).
A multi-line re-wrap of an existing extra_hidden_dirs value is rare; the cost of the false hit is one Reader: line, which is the answer the doc asks for on a mask edit anyway.
Scope is frozen; no push for an advisory.

@iamwhatever

Copy link
Copy Markdown
Contributor Author
  • Pure module rename evades the reader check -- rebutted span=b472e0e37f30

The gate checks presence of a small, documented shape set; take-away-changes.md says it misses take-aways it cannot see in one line and the rule still applies.
A rename is a reviewers' call under that rule; the gate never passes a PR that would otherwise fail.
Under scope freeze this PR pushes only for its own red checks or a BLOCK.

@iamwhatever

Copy link
Copy Markdown
Contributor Author
  • git diff rename detection hides a module move -- rebutted span=2285112cc7b6

Same class as the GPT rename item: a moved module is outside the documented shape set, and the doc states the rule still applies to unlisted take-aways.
The gate fails closed only on listed shapes; missing a shape never fails a correct PR.
Scope frozen to the Goal; no push for an advisory.

@iamwhatever

Copy link
Copy Markdown
Contributor Author
  • Bare agents dict delete reads as a config.agents row delete -- rebutted span=4da00ac777f0

The false hit costs one Reader: line, never a blocked merge with no remedy.
The one other del agents[k] on main (context_management.py) is an existing line and is not re-flagged; only a NEW such delete in src/ hits.
Narrowing to config-typed receivers needs type information a line diff lacks; scope is frozen.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 2, 2026
@chenmingwei23
chenmingwei23 merged commit da93399 into main Oct 2, 2026
97 checks passed
@chenmingwei23
chenmingwei23 deleted the ci/take-away-reader-list branch October 2, 2026 15:54
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants