Skip to content

ci(issue-gate): require exactly one tier label on the issue - #16634

Merged
chenmingwei23 merged 2 commits into
mainfrom
ci/issue-gate-require-tier
Oct 4, 2026
Merged

chenmingwei23 merged 2 commits into
mainfrom
ci/issue-gate-require-tier

Conversation

@bolichen97

@bolichen97 bolichen97 commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A PR can only pass Issue Gate when every issue it points at has a settled size, meaning exactly one tier:T1..tier:T4 label.

Issue Gate (#16241) checks that a referenced issue carries a triage verdict (auto-fixable, needs-investigation, needs-human), but never its tier. Triage already writes a tier on almost every issue, yet a PR can still merge against an issue whose size nobody settled, or whose size is ambiguous (two tiers).

Why it matters

The tier decides how much process the work needs (T3 needs a one-pager, T4 a design review). Without the gate, a T3/T4-sized change can be built and merged as if it were a small fix, and nothing records that anyone sized it.

Not a goal

  • Writing tier labels. The maintainer-operated triage pipeline writes them, as it writes the verdict.
  • Enforcing what each tier requires (one-pager, design review). This PR only requires that a tier exists.
  • Backfilling the 41 triaged open issues that have no tier yet.

What changed (motivation → approach → change)

Gap: the gate reads each issue's labels but only checks the verdict set → approach: reuse that same label read and add one more check beside the verdict check, so no extra API call → change:

  • .github/workflows/issue-gate.yml: new TIER_LABELS env (tier:T1 tier:T2 tier:T3 tier:T4). For each declared issue, after the verdict check, count exact-match tier labels: zero → finding "has no tier label", two or more → finding "carries more than one tier label". Header rule text and both job-summary messages name the new requirement.
  • docs/ci/ci-and-reviews.md: "The rule, in full" names the tier requirement and what each tier means.
  • test/test_issue_gate_refs.py: TestLabelContract pins the tier label set, like it pins the verdict labels.
  • src/kiro_crew/builtin_skills/kirocrew-dev/prepare-pr/SKILL.md: the gate's failure conditions now name the tier requirement (Opus review advisory).

Backwards compatibility

Breaking: a PR whose referenced issue has no tier label (or two) now goes red on Issue Gate where it passed before. Writer sweep on origin/main @ a60fda2: tier labels are written only by the out-of-repo triage pipeline; of 1,482 triaged open issues, 41 have no tier and 0 have two. A PR pointing at one of those 41 turns green once triage adds a tier and the PR description is edited (the edited trigger re-runs the gate); the issue-gate: waived label is unchanged.

Reader: .github/workflows/pr-readiness.yml:workflow_specs "issue-gate.yml|Issue Gate" -- entry: release -- still reads the same job name and conclusion; the verdict can now be red in one more case
Reader: src/kiro_crew/builtin_skills/kirocrew-dev/prepare-pr/SKILL.md:Issue Gate section -- entry: chat -- updated in this PR to name the tier requirement; grammar and waiver unchanged
Reader: test/test_ai_review_workflows.py:"issue-gate.yml|Issue Gate" -- entry: release -- lane name unchanged, test passes

Tests

  • test/test_issue_gate_refs.py::TestLabelContract::test_verdict_and_pending_labels_are_the_documented_set now also pins TIER_LABELS to exactly tier:T1..tier:T4, so a rename on either side reds a test instead of every PR.
  • pytest test/test_issue_gate_refs.py test/test_ai_review_workflows.py: 738 passed.

Manual verification

Ran the new tier check in bash against label sets: no tier → "no tier label"; tier:T1 → pass; tier:T1 + tier:T3 → "more than one tier label"; look-alikes tier:T10 / tier:T → "no tier label" (exact-line match). This PR's own Issue Gate run is the live check: #16633 carries needs-human and tier:T1.

Related Issues

Closes #16633

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

@bolichen97
bolichen97 requested a review from a team as a code owner October 4, 2026 03:41
@bolichen97
bolichen97 requested a review from CrysisDeu October 4, 2026 03:41
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 4, 2026
@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 9e23ab4482b0f97f6b84d514de0eb9070e274eb7 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Check that the out-of-repo triage pipeline really writes tier:T* beside every verdict. Otherwise the 41 untiered issues turn into red PRs.

What this change ships

Inventory (5 items) — 5 justified

Intent: a maintainer wants every merged PR to point at an issue whose size someone has settled. ADDITION: a stricter CI gate.

  1. Issue Gate fails when a referenced issue has no tier label — justified
  2. Issue Gate fails when a referenced issue has two or more tier labels — justified
  3. The gate's header comment and job-summary messages name the tier requirement — justified
  4. The ci-and-reviews doc and the prepare-pr skill name the tier requirement and define each tier — justified
  5. The label-contract test pins TIER_LABELS to exactly T1..T4 — justified

[FIRST-PRINCIPLES-REVIEWED] 9e23ab4

Description read

[DESCRIPTION-READ] bf720234a41dee943dc5afe0a4de238fde68534cb402e0e296c521eb8a0817ae

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 9e23ab4482b0f97f6b84d514de0eb9070e274eb7 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Before merging, check that the out-of-repo triage pipeline writes the tier at the same time as the verdict. If it writes it later, auto-fixable bot PRs will fail the gate in the gap between the two.

[DESIGN-REVIEWED] 9e23ab4

Description read

[DESCRIPTION-READ] 15269ccd7651aa35888e63bf0512f59a622ba53f1cc087d560fb2eb5ab8d2b8e

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 9e23ab4482b0f97f6b84d514de0eb9070e274eb7 — this comment is updated in place on each push.

Review details

Advisory only: when just the tier label is missing, the failure summary gives the wrong recovery instruction.

FINDING — .github/workflows/issue-gate.yml:373 — if an issue already has its verdict label but no tier label (or two of them), the summary still says "Once the verdict label lands," so the author waits for a label that's already on the issue, and in the two-tier case waiting will never fix it → Fix: change it to "Once the verdict and tier labels are in place," and add that a maintainer must remove the extra tier label when there are two.

[OPUS-REVIEWED] 9e23ab4

Verdict parsed from the review's SHA-scoped output markers for commit 9e23ab4482b0f97f6b84d514de0eb9070e274eb7.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 9e23ab4482b0f97f6b84d514de0eb9070e274eb7: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 9e23ab4482b0f97f6b84d514de0eb9070e274eb7 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 9e23ab4

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 9e23ab4482b0f97f6b84d514de0eb9070e274eb7: <one-sentence reason>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 4, 2026
@bolichen97
bolichen97 enabled auto-merge (squash) October 4, 2026 04:47
@chenmingwei23
chenmingwei23 merged commit ca23a16 into main Oct 4, 2026
101 of 108 checks passed
@chenmingwei23
chenmingwei23 deleted the ci/issue-gate-require-tier branch October 4, 2026 06:16
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 4, 2026

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed (read-only check of diff, tests and CI). Security looks fine. The workflow is still pull_request with read-only permissions. Label names are only compared with grep -F against a fixed list, never run as shell. Approving. Before merge, please confirm that triage writes the tier at the same time as the verdict, otherwise auto-fixable bot PRs fail in the gap between the two. Please also change the summary line "Once the verdict label lands" to name both labels, and say that a second tier label must be removed by a maintainer. The 41 open issues with no tier will make some PRs go red until triage adds one or a maintainer applies the waiver.

bolichen97 added a commit that referenced this pull request Oct 5, 2026
PR Readiness reads main's lane list, which waits on an Issue Gate check.
Issue Gate is a pull_request workflow, so GitHub reads it from the PR's
merge ref, and release/0.8.0 has no issue-gate.yml. Every PR into the
release branch therefore waits on a check that never starts.

Copy .github/workflows/issue-gate.yml from main unchanged. Its grammar
script is read from a checkout of the default branch at run time, so the
workflow file is the only piece the release branch needs.

Source: origin/main 2359397 (file last changed there in #16740;
introduced by #16241, #16634, #16697).

Fixes #16868
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: Issue Gate should require exactly one tier label on the referenced issue

3 participants