Skip to content

ci: skip the AI review after maintainer activity and add /ai commands - #4679

Merged
dolfim-ibm merged 1 commit into
mainfrom
ci/ai-pr-commands
Oct 8, 2026
Merged

dolfim-ibm merged 1 commit into
mainfrom
ci/ai-pr-commands

Conversation

@dolfim-ibm

Copy link
Copy Markdown
Member

The AI review ran on every push, also on PRs that a maintainer already reviews. That costs money and adds noise. This PR stops the automatic review after a maintainer is involved, and gives maintainers /ai commands instead.

Automatic review only before maintainer activity

Before each push-triggered review, the job reads the issue comments, the reviews, and the review comments of the PR. The review is skipped if one of them comes from a maintainer: OWNER, MEMBER, or COLLABORATOR, not a bot, not the PR author, and not an /ai command. The triage still runs on every push, because it is cheap and keeps the CI-safety label current.

/ai commands

A PR comment from a maintainer that starts with one of these commands:

  • /ai review (or /ai): a new, full review of the current commit. It ignores the maintainer check, the "already reviewed" check, and the duplicate gate. The size limits still apply.
  • /ai triage: runs the triage again.

The bot adds a 👀 reaction to confirm the command. Comments from other users and other text are ignored. Only an exact command on the first line counts.

Workflow changes

  • New trigger: issue_comment (created). A cheap if: on the new context job skips all other comments without running the script.
  • The context job runs ai_pr_event.py. It validates the event again and sets the outputs pr, run, and force_review. The other jobs use these outputs and no longer read github.event.pull_request, which does not exist for comment events.
  • Concurrency: only pushes and maintainer /ai comments share the group of the PR. Any other comment gets its own group, so it cannot cancel a running triage.
  • The comment text never goes into a shell. It is only read from the event file by the script.

Checklist:

  • Documentation has been updated, if necessary. (.github/CI_LABELS.md)
  • Examples have been added, if necessary.
  • Tests have been added, if necessary.

Review evidence:

…mmands

- A push runs the AI review only while no maintainer (OWNER, MEMBER, or
  COLLABORATOR, not a bot and not the PR author) has commented on or reviewed
  the PR. The triage still runs on every push.
- Maintainers can comment `/ai review` (or `/ai`) for a new full review of the
  current commit, and `/ai triage` to run the triage again. The bot confirms
  a command with a reaction. Commands from other users are ignored.
- A new `context` job resolves the PR and the mode from the
  pull_request_target or issue_comment event, so the other jobs no longer read
  the event payload. Other comments get their own concurrency group and do not
  cancel a running triage.

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

✅ DCO Check Passed

Thanks @dolfim-ibm, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@github-actions github-actions Bot added ai:ci-needs-care AI triage: check CI-safety findings before you approve CI enhancement New feature or request labels Oct 8, 2026
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

AI triage (advisory)

Checked commit f91424db14. The analysis read the diff only. It did not run code from this pull request.

This PR reduces AI review costs by skipping the automatic review after a maintainer gets involved. It also adds /ai review and /ai triage slash commands for maintainers, implemented via a new issue_comment trigger and a new ai_pr_event.py script.

Topic labels added: enhancement.

CI safety: needs maintainer care before you approve CI.

  • .github/CI_LABELS.md: changes CI workflows, actions, or CI scripts
  • .github/scripts/ai_pr_event.py: changes CI workflows, actions, or CI scripts
  • .github/scripts/ai_pr_review.py: changes CI workflows, actions, or CI scripts
  • .github/workflows/ai-pr-triage.yml: changes CI workflows, actions, or CI scripts
  • .github/workflows/ai-pr-triage.yml: The concurrency group: key and the if: condition on the context job both evaluate github.event.comment.body directly as a GitHub Actions expression; a crafted comment body could alter expression parsing (expression injection), even though the text never reaches a shell.
1 related pull request(s) in the same code
PR State Signal Assessment
#4655 merged referenced as #4655 related: PR 4655 changed checkout refs in the same workflow file from base.sha to github.sha. PR 4679 builds on that merged change and restructures the workflow further; the two do different things.

The ai:* labels are advisory. They do not approve CI and they do not add tests:full. A new push runs this check again.

@dolfim-ibm dolfim-ibm changed the title feat(ci): skip the AI review after maintainer activity and add /ai commands ci: skip the AI review after maintainer activity and add /ai commands Oct 8, 2026
@dolfim-ibm
dolfim-ibm requested review from cau-git and ceberam October 8, 2026 08:12
@codecov

codecov Bot commented Oct 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@PeterStaar-IBM PeterStaar-IBM left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@dolfim-ibm
dolfim-ibm merged commit 1c08724 into main Oct 8, 2026
52 checks passed
@dolfim-ibm
dolfim-ibm deleted the ci/ai-pr-commands branch October 8, 2026 08:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai:ci-needs-care AI triage: check CI-safety findings before you approve CI enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants