Skip to content

ci(sensitive-review): let the PR author's template comment clear the check - #17865

Merged
iamwhatever merged 1 commit into
mainfrom
ci/sensitive-review-owner-comment
Oct 8, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
ci/sensitive-review-owner-comment

Conversation

@iamwhatever

@iamwhatever iamwhatever commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The PR author can turn Sensitive Change Review green by posting the filled template, without waiting for an approver.

Today only an approver's review body can clear the check. The author did the sandbox checks, but has to wait for someone to paste the template into an Approve review. A plain comment does nothing.

Why it matters

Sensitive PRs sit red until a reviewer copies text the author already wrote. That slows every sandbox, floor and redaction change, and it makes the approver's job paperwork instead of review.

Not a goal

What changed (motivation → approach → change)

This reverses one rule from #16862: "the PR author cannot clear the check". The gate owner (@iamwhatever, author of #16862) decided the author, who ran the sandbox, should write the record. The approver's job becomes the review, not copying text.

The second person stays, enforced by the main ruleset (read today): required_approving_review_count: 1, require_last_push_approval: true, dismiss_stale_reviews_on_push: true, require_code_owner_review: true. So no sensitive head merges without a fresh approval from someone else.

Part Change
.github/scripts/sensitive_change_review.py Counts a PR-author comment with a complete template that names the current head SHA (7+ chars), if the author has write access. Bots and other users never count. A comment read failure exits 2. The sticky comment pre-fills - Head commit: <sha>. The glob covers sensitive-change-review*.yml.
.github/workflows/sensitive-change-review-comment.yml (new) On the author's template comment, re-runs the newest check run for the head. issue_comment runs from main, so a PR cannot edit it. Job-level concurrency, so other comments cannot cancel it. Hosted, no checkout, forks skipped.
.github/workflows/sensitive-change-review.yml Header comment only.
docs/ci/ci-and-reviews.md Names both records and the ruleset's second-person rule.

Backwards compatibility

Compatible: a reviewer's template approval clears the check exactly as before.

Loosens one rule: the author's own comment now clears the check, where #16862 said the author does not count. The reviewer path, fork refusal, head match and sticky comment are unchanged.

Tests

test/test_sensitive_change_review.py (101 passed):

  • author comment naming the head passes; 7-char SHA passes
  • no SHA, an old SHA, or an incomplete template fails
  • another user's or a bot's comment fails; author without write fails
  • newer comment wins over a stale one; forks still refused; approver path still passes
  • main() passes on an author comment; a comment read error exits 2
  • summary pre-fills the head SHA
  • comment workflow: only issue_comment, author-only if, hosted, no checkout, minimal permissions, job-level non-cancelling concurrency, re-runs the check workflow

Manual verification

Still required after merge (the comment workflow only runs from main): on a test PR touching src/kiro_crew/sandbox.py, post the filled template as the author and confirm the check re-runs and goes green; then push and confirm it goes red again.

This PR changes the gate's own files, and main's checker does not know the author path yet. So this PR itself needs a reviewer's template approval.

Related Issues

Reverses part of #16862. Related to #16870.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@iamwhatever
iamwhatever requested a review from a team as a code owner October 7, 2026 21:06
@iamwhatever
iamwhatever requested a review from CrysisDeu October 7, 2026 21:06
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Sensitive Change Review

Sensitive files changed:

  • .github/scripts/sensitive_change_review.py

  • .github/workflows/sensitive-change-review.yml

  • @chenmingwei23: missing the ## Sensitive change review heading

How to clear this check (a plain PR comment does NOT count):

  1. Someone other than the PR author, with write access, opens
    "Files changed" -> "Review changes".
  2. They pick "Approve" (not "Comment").
  3. They paste the template below into the review box, fill every
    field (N/A needs a reason), then click "Submit review".
  4. A new push resets this check: approve the new head again.

The author may draft the filled template as a comment for the
reviewer to copy, but only the reviewer's approval clears the check.

## Sensitive change review
- What rule changed:
- Before -> after (what was allowed/blocked, now):
- Worst case if wrong (breaks commands / leaks secret / locks user out):
- Evidence checked (which sandbox result proves it):
- No regression:
  - Tools still work (which tools/commands were run, result):
  - Backward compatible (old config / old data / old callers still work, how checked):
  - Existing tests (which suites passed on this head):
- How to undo:

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 8d5f547158d1a2e8787cf4ff23302169eb72c7b1 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Before relying on the new path, run the post-merge check: an author comment on a sandbox.py test PR turns the check green, and a new push turns it red again.

[DESIGN-REVIEWED] 8d5f547

Description read

[DESCRIPTION-READ] b1a672bd4a8bf32234b831695def77c518f5cd00e501c422825ac9ea25b2c9b3

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 8d5f547158d1a2e8787cf4ff23302169eb72c7b1 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 8d5f547

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 8d5f547158d1a2e8787cf4ff23302169eb72c7b1: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — 🟡 CONCERNS

Premise-level review of 8d5f547158d1a2e8787cf4ff23302169eb72c7b1 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The PR reverses a pinned #16862 rule, and the only thing it rests on is the description saying the gate owner agreed. Nothing on record shows that decision.

Not justified as shipped

  • Item 1 — inherited: the deleted pin said "Operators pasted the template as a PR comment, or approved their own PR. The summary must name the real action: a non-author Approve review." The description rests the reversal on "The gate owner (@iamwhatever, author of ci: require a reasoned human approval for sensitive-path changes #16862) decided…". That decision is asserted, not linked. The other support is that the second person still exists, because "the main ruleset (read today)" requires a fresh approval from someone else. That is a repo-settings claim the diff cannot show.
    Clears when: the gate owner's decision is linked from this PR (an issue or a comment on ci: require a reasoned human approval for sensitive-path changes #16862 or this PR), or a maintainer records /ai-review override first-principles 8d5f547158d1a2e8787cf4ff23302169eb72c7b1: <decision>.

What this change ships

Inventory (8 items) — 7 justified

Intent: let a sensitive PR's author clear Sensitive Change Review with their own filled template, without waiting for an approver. This is an ADDITION that loosens a gate.

  1. The author's template comment naming the head SHA now clears the check (reverses a pinned ci: require a reasoned human approval for sensitive-path changes #16862 rule) — inherited: the decision is asserted, not linked
  2. Posting that comment re-runs the check on its own, with no push — justified
  3. The sticky comment's template now pre-fills a Head commit: line — justified
  4. The "How to clear" steps now lead with the author's comment, and "a plain PR comment does NOT count" is gone — justified
  5. Edits to the new comment workflow now count as sensitive too (the path match widened) — justified
  6. If the comment list can't be read, the check fails closed (exit 2) — justified
  7. Failure text changed from "No approval" to "No reasoning template" — justified
  8. The author's comment counts only if the author has write access — justified

[FIRST-PRINCIPLES-REVIEWED] 8d5f547

Description read

[DESCRIPTION-READ] bc3b399e00d97042fcfbabe028ac12cc03f028fc2db8f29842d08c89a56541eb

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 8d5f547158d1a2e8787cf4ff23302169eb72c7b1 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 8d5f547

Verdict parsed from the review's SHA-scoped output markers for commit 8d5f547158d1a2e8787cf4ff23302169eb72c7b1.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 8d5f547158d1a2e8787cf4ff23302169eb72c7b1: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 7, 2026
@iamwhatever
iamwhatever force-pushed the ci/sensitive-review-owner-comment branch from 7bef946 to 0ac8b7e Compare October 7, 2026 22:20
@iamwhatever

iamwhatever commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor Author

self-added: no

  • span=e11a9b1f962e: The second-person check moves out of the gate

rebutted: the main ruleset keeps the head-bound second person. Read today: required_approving_review_count=1, require_last_push_approval=true, dismiss_stale_reviews_on_push=true, require_code_owner_review=true.
So no sensitive head merges without a fresh non-author approval on that exact push; the gate no longer duplicates it.
The gate owner (author of #16862) chose this split; the body now says so.

@iamwhatever

iamwhatever commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor Author

self-added: no

  • span=f411bd4ac6fe: Count the author's comment only with a non-author approval on the head

rebutted: the owner weighed this exact option and declined it.
require_last_push_approval + dismiss_stale_reviews_on_push already force a non-author approval on the current head before merge, so the extra gate check adds no protection, only a second wait.

@iamwhatever

iamwhatever commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor Author

self-added: no

  • span=5f08ebc79d3b: Framing contradicted: Removes nothing

fixed in 0ac8b7e: Backwards compatibility now says this loosens one #16862 rule (the author does not count), and drops 'Removes nothing'.
What changed opens by naming the reversal, who decided it, and the ruleset that keeps the second person.

@iamwhatever

iamwhatever commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor Author

self-added: no

  • span=7294559b1cc6: Reversal of a two-day-old decision presented as purely additive

fixed in 0ac8b7e: the body names the reversal of #16862 and links it. Evidence: the gate owner (#16862's author) decided the author who ran the sandbox writes the record.
The second-person guarantee moves to the main ruleset, quoted with its live settings.

@iamwhatever

iamwhatever commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: job-level concurrency on the comment re-run workflow

  • span=f78e1f6c0c6d: cancel-in-progress before the job if

fixed in 0ac8b7e: concurrency moved from workflow to job level with cancel-in-progress=false, so a skipped run from another commenter never joins the group.
Pinned by test_other_comments_cannot_cancel_the_authors_rerun.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

Draft for the approver to paste into an Approve review (main's checker does not read author comments yet, so this PR still needs a reviewer).

Sensitive change review

  • Head commit: 0ac8b7e
  • What rule changed: the PR author's own template comment naming the head SHA now clears Sensitive Change Review.
  • Before -> after (what was allowed/blocked, now): before only a non-author approval body cleared it; now the author's comment also clears it.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): an author self-certifies a weak sandbox change; the main ruleset still demands a fresh non-author approval to merge.
  • Evidence checked (which sandbox result proves it): no runtime sandbox path changes; CI-only diff, unit tests cover every accept and refuse case.
  • No regression:
    • Tools still work (which tools/commands were run, result): no product code touched; the checker ran 101 unit tests green locally.
    • Backward compatible (old config / old data / old callers still work, how checked): reviewer template approvals still pass, pinned by test_reviewer_path_still_passes_without_comment.
    • Existing tests (which suites passed on this head): test/test_sensitive_change_review.py passed, plus full CI on this head.
  • How to undo: revert this single commit on main.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 8d5f547158d1a2e8787cf4ff23302169eb72c7b1 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 7, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Backend Tests shards 2/6 (+ Coverage Gate) are red from main, not this diff: test_http_server_bind_ratchet and test_session_control_doc fail on main's own push CI too (tracked in #17900 / #17902). Fixes in flight: #17907, #17906, #17903. This PR waits and rebases once one lands.

@iamwhatever
iamwhatever force-pushed the ci/sensitive-review-owner-comment branch from 0ac8b7e to 8d5f547 Compare October 8, 2026 00:13
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Rebased: main's red backend tests are fixed by #17909, so this head moves onto it (73b43b9 -> 1ad2d6d); CI re-running on 8d5f547.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 8, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

@chenmingwei23 thanks for the approval. Sensitive Change Review is still red because the approval body is empty. This PR changes the gate itself, so main's checker still reads the template only from an Approve review. Please submit one more Approve review on head 8d5f547 with the filled template from the draft comment above, with the head SHA set to 8d5f547.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 8, 2026
@iamwhatever
iamwhatever merged commit 1548bfb into main Oct 8, 2026
146 of 148 checks passed
@iamwhatever
iamwhatever deleted the ci/sensitive-review-owner-comment branch October 8, 2026 01:13
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants