Skip to content

ci: restore advanced CodeQL setup so fork PRs are not blocked - #633

Merged
Kateřina Churanová (kate-shine) merged 2 commits into
mainfrom
kchuranov/codeql-advanced-fork-prs
Aug 3, 2026
Merged

ci: restore advanced CodeQL setup so fork PRs are not blocked#633
Kateřina Churanová (kate-shine) merged 2 commits into
mainfrom
kchuranov/codeql-advanced-fork-prs

Conversation

@kate-shine

@kate-shine Kateřina Churanová (kate-shine) commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Problem

Pull requests from forks can never merge. They sit forever on:

Code scanning is waiting for results from CodeQL for the commits <sha> or <sha>.

Current example: #624 (from sandersaares/oxidizer).

Root cause: this repo uses CodeQL default setup, and default setup does not run for pull requests from forks — that exclusion is documented behaviour, not a bug. The code-scanning merge-protection rule still expects a CodeQL result, so the check stays pending indefinitely and the PR is permanently blocked.

Evidence gathered on this repo:

PR Head repo Fork? CodeQL run CodeQL check
#621, #632, #568, #570-#573 microsoft/oxidizer no yes yes
#624 sandersaares/oxidizer yes none none, blocked
#622 kate-shine/oxidizer yes none none, blocked

Why advanced setup fixes it

An advanced-setup workflow triggered by pull_request does run for fork PRs, and GitHub accepts its SARIF upload on public repositories even though the fork's GITHUB_TOKEN is read-only.

Verified empirically against prettier/prettier, a public repo on advanced setup. On fork PR head bf2849cee9d467aedf3fbd42a95201ea4393f855 (from splincode/prettier):

  • the only workflow runs are pull_request ones, including .github/workflows/codeql.yml; there is no dynamic (default setup) run at all;
  • yet github-advanced-security posted check CodeQL with conclusion success.

History

.github/workflows/codeql.yml already exists and is correct, but it has been in the disabled_manually state since 2026-01-22 and has not run since.

It was introduced by #219 ("ci: Switch to advanced CodeQL mode"), whose stated goal was, verbatim, "Once this is checked in, then we can tweak to improve permissions on forks" — exactly the problem above. It ran 33 times, all successful, and roughly 4.5 hours after that PR merged, default setup was switched back on in the repository settings, which automatically disabled this workflow. No PR, issue or comment records a reason. Since then #232, #237, #462, #470, #484, #543, #563 and #574 have all been maintaining a workflow that never runs.

Changes

  • Document why advanced setup is preferred, so this does not get silently reverted to default setup a second time.
  • Add the merge_group trigger. main.yml and anvil-pr.yml both have it and codeql.yml did not; without it the merge queue stalls on this workflow once it is required.
  • Bump github/codeql-action to v4.37.3 (confirmed "immutable": true via the releases API, consistent with the existing tag-pinning rationale in the file).

Required admin action, this PR alone is not sufficient

Merging this changes nothing by itself, because the workflow is still disabled and GitHub will keep it disabled while default setup is on. A repository admin must, in this order:

  1. Settings, Advanced Security, disable CodeQL default setup.
  2. Re-enable this workflow: gh api -X PUT repos/microsoft/oxidizer/actions/workflows/codeql.yml/enable.

Doing step 2 first does not work.

Trade-off

Fork PRs from first-time contributors will show CodeQL as action_required until a maintainer clicks "Approve and run". That is standard GitHub Actions fork policy and it also applies to every other pull_request workflow in this repo today. It is a single click, versus the current situation where fork PRs cannot merge at all.

Draft

Left as a draft until an admin confirms they will make the settings change, since merging without it is a no-op.

CodeQL default setup never runs for pull requests from forks. The
code-scanning merge-protection rule therefore waits forever on a check
that will never report, and fork PRs (for example #624) can never merge.

Advanced setup triggered by `pull_request` does run for fork PRs, and
GitHub accepts the SARIF upload on public repositories. Verified against
prettier/prettier, which uses advanced setup: fork PR head bf2849ce had
no `dynamic` run at all, yet github-advanced-security reported the
`CodeQL` check as successful from the `pull_request` codeql.yml run.

This restores .github/workflows/codeql.yml, which has been in the
`disabled_manually` state since 2026-01-22 and so has not run since,
despite continued dependency maintenance.

Changes:
- Document why advanced setup is preferred, so this is not silently
  reverted to default setup again.
- Add the `merge_group` trigger, which main.yml and anvil-pr.yml already
  have and codeql.yml was missing; without it the merge queue stalls.
- Bump github/codeql-action to v4.37.3 (verified immutable).

Note: landing this file is not sufficient on its own. A repository admin
must first disable CodeQL default setup, then re-enable this workflow,
otherwise GitHub keeps it disabled.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 3, 2026 09:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Restores/modernizes the repository’s advanced CodeQL workflow configuration so CodeQL can run on events (including fork PRs and merge queue) where default setup won’t produce the required CodeQL check, preventing merge-protection from blocking indefinitely.

Changes:

  • Adds an explicit rationale comment documenting why advanced CodeQL setup is required for fork PR mergeability.
  • Adds the merge_group trigger to support merge queue usage.
  • Bumps github/codeql-action usage to v4.37.3 (init/analyze), maintaining the “immutable release tag” rationale.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@codecov

codecov Bot commented Aug 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.0%. Comparing base (70eccfa) to head (0f77a08).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #633   +/-   ##
=======================================
  Coverage   100.0%   100.0%           
=======================================
  Files         473      473           
  Lines       45493    45493           
=======================================
  Hits        45493    45493           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kate-shine
Kateřina Churanová (kate-shine) marked this pull request as ready for review August 3, 2026 10:14
Copilot AI review requested due to automatic review settings August 3, 2026 14:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@kate-shine
Kateřina Churanová (kate-shine) merged commit c44554c into main Aug 3, 2026
52 checks passed
@kate-shine
Kateřina Churanová (kate-shine) deleted the kchuranov/codeql-advanced-fork-prs branch August 3, 2026 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants