Skip to content

ci: report required Scorecard analysis on pull requests - #47

Merged
PeterGuy326 merged 1 commit into
mainfrom
ci/report-scorecard-on-pull-requests
Sep 12, 2026
Merged

PeterGuy326 merged 1 commit into
mainfrom
ci/report-scorecard-on-pull-requests

Conversation

@PeterGuy326

Copy link
Copy Markdown
Contributor

What is broken

main branch protection requires the OpenSSF Scorecard status context:

required_checks: ['test', 'docker-build', 'CodeQL (javascript-typescript)', 'Dependency review', 'OpenSSF Scorecard']

.github/workflows/bytefolk-scorecard.yml triggers only on push(main) / schedule / workflow_dispatch. It has no pull_request trigger, so on a pull request the required context is never produced. GitHub reports it as Expected forever and mergeStateStatus stays BLOCKED for every PR in this repository, regardless of reviews, and including for admins, because enforce_admins is true.

Measured on #45, which is green on all four contexts that do report (test, docker-build, CodeQL (javascript-typescript), Dependency review) and still cannot merge. This is a chicken-and-egg defect: the required check can only start reporting once a workflow that reports it reaches main, and no PR can reach main while the check is missing.

The same trap exists in bytefolk/digital-employee, where the repair is #263. bytefolk/roleweave already carries an equivalent scorecard-pr.yml and is not affected. bytefolk/mem does not require the context and is not affected.

What this does

Adds .github/workflows/scorecard-pr.yml, a separate PR-triggered analysis producing the exact required context name OpenSSF Scorecard.

Because pull_request events use the workflow definitions from the head branch, this PR reports the required context on its own head. It is self-unblocking: it needs no protection change, no admin override, and no removal of the required context to land.

Security properties

This is a copy of the file already running in bytefolk/roleweave, byte for byte — the committed blob is e401e67b61fc33b6e938dc337ecb79497ffbf243 in both repositories. It is not a new or independently drafted workflow.

  • permissions: contents: read at the workflow level, and no job-level permission escalation. No security-events: write, no id-token: write.
  • No run: step anywhere in the file, so the entire class of ${{ github.event.* }} script-injection findings is structurally absent rather than mitigated.
  • persist-credentials: false on checkout — the PR merge commit is fetched without a token left in the runner.
  • publish_results: false — no attempt to publish PR results to the scorecard API, which is what makes the job safe on untrusted fork code.
  • Trigger is pull_request, not pull_request_target, so the job gets no access to base-repository secrets.
  • All three actions pinned to full commit SHAs, matching the versions bytefolk-scorecard.yml already uses: actions/checkout v7.0.1 -> 3d3c42e5aac5ba805825da76410c181273ba90b1, ossf/scorecard-action v2.4.4 -> 2d1146689b8cda280b9bc96326124645441f03bc, actions/upload-artifact v7.0.1 -> 043fb46d1a93c77aae656e7c1c64a875d1fc6a0a.
  • timeout-minutes: 15, if-no-files-found: error, retention-days: 14.

Scope

One new file, +38/-0. bytefolk-scorecard.yml is untouched — repository-wide analysis, badge, and SARIF publication on main continue exactly as before. No runtime, dependency, or lockfile change.

Known limitations

Upstream describes Scorecard's PR support as experimental. Local-checkout mode analyses the merge commit rather than the whole repository, so its governance coverage is narrower than the main run and it is not evidence that findings are resolved. The report is uploaded as a workflow artifact; it is not a synthetic status and does not waive anything.

Verification

This PR's own head is the verification: once CI runs, OpenSSF Scorecard appears on this PR as a real reported context. If it reports and is green, the defect is fixed and mergeable_state should move off BLOCKED as soon as a non-author CODEOWNER approval is present.

Review

Requesting a non-author CODEOWNER approval — .github/CODEOWNERS lists @PeterGuy326 @Bindy-lbb and I am the author, so I cannot supply it. @Bindy-lbb this is the same repair you reviewed on bytefolk/digital-employee#263, and the file is byte-identical to the one already merged in bytefolk/roleweave.

No automatic close keywords: this PR intentionally closes no issue.

main requires the `OpenSSF Scorecard` context, but bytefolk-scorecard.yml
triggers only on push(main)/schedule/workflow_dispatch, so the required
context never reports on any pull request. Every PR in this repository is
therefore permanently BLOCKED regardless of approvals -- measured on #45,
which is green on all four reporting contexts and still cannot merge.

Add a separate PR-triggered local analysis. It grants contents:read only,
contains no run: step, persists no checkout credentials, sets
publish_results: false, and pins all three actions to the same commits
bytefolk-scorecard.yml already uses. Repository-wide reporting and badge
publication on main are unchanged.

Mirrors bytefolk/roleweave's existing scorecard-pr.yml and
bytefolk/digital-employee#263.

@Bindy-lbb Bindy-lbb left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

正式复审通过(head 5ed0baa)。

新增的 PR Scorecard workflow 仅使用 pull_request + local checkout,权限为 contents: read,不使用 pull_request_target、不授予发布/SARIF 写权限,Scorecard 与 checkout action 均 SHA-pinned;报告只上传为短期 artifact,默认分支的发布/badge workflow 未被改动。当前 test、docker-build、CodeQL、Dependency review 和 OpenSSF Scorecard 共 6 个 check 全部成功。批准合入。

@PeterGuy326
PeterGuy326 merged commit e566396 into main Sep 12, 2026
6 checks passed
@waterbro-8

Copy link
Copy Markdown

This is not a review, carries no vote, and does not ask to merge or close anything. I am filing this here because this PR is the one that changed the check set, and its author is the person most likely to sweep the queue.

Reading for the current main tip e566396d1d6, i.e. this PR's merge commit (merged_at 2026-09-12T15:58:49Z), posted at the moment of this comment.

1. What this PR changed for everyone else

pull_request events evaluate the workflow files from refs/pull/N/merge, i.e. from the branch's own base — so merging a new workflow into main does not retroactively give it to branches that were cut earlier. The file split is directly visible:

.github/workflows/ at 65068136a1e  (old base): bytefolk-scorecard.yml  bytefolk-security.yml  ci.yml
.github/workflows/ at e566396d1d6  (this merge): bytefolk-scorecard.yml  bytefolk-security.yml  ci.yml  scorecard-pr.yml

2. Seven open PRs are still on the old base and therefore have no OpenSSF Scorecard context

All readings are current-head, check-runs deduped per check name by latest suite id.

PR base OpenSSF Scorecard on head other failing checks reviews
#50 e566396d1d6 success 0
#49 e566396d1d6 success test, docker-build 0
#48 e566396d1d6 success docker-build 0
#46 65068136a1e absent 0 (draft)
#45 65068136a1e absent 0
#43 65068136a1e absent 0
#42 65068136a1e absent 0
#41 65068136a1e absent 0
#40 65068136a1e absent 0
#39 65068136a1e absent test, docker-build 0

3. Being honest about what is and isn't proven

Every one of the ten is currently blocked with zero reviews, so the missing review already explains today's blocked on all of them. I am not claiming Scorecard is what blocks these seven right now. The claim is narrower: after a review lands, these seven may still not merge, because they don't carry a check the three up-to-date branches do — and I cannot read whether it is configured as required: rulesets on this repo returns [], branches/main/protection returns 404 for me, and org-level rulesets need admin:org, which I don't have.

The reason I raise it rather than drop it is a control experiment in the same org, which I ran myself and is now closed end to end: digital-employee#260 had 15/15 checks success with no OpenSSF Scorecard context, its base commit did not contain scorecard-pr.yml, and it sat blocked. Its base was updated, the context appeared as success, it went clean, and it merged at 2026-09-12T03:42:24Z on head 80a886dd314. Same shape, one variable moved, outcome changed.

4. Update branch is unusually cheap right now, and that window is the point

reviews is 0 on all ten — nobody has submitted an approving review on any of them. So clicking Update branch today costs no existing vote.

That is not a durable property, and a push can cost someone a vote — but the org's behaviour here is not consistent across my own two measurements, so I'll give both rather than a rule. On mem#140 a push moved the head and the previous APPROVED came back DISMISSED. On digital-employee#260 an Update-branch push moved the head and the reviewer's APPROVED was re-pointed at the new head with submitted_at unchanged, i.e. it survived. I don't know which GitHub uses for a UI-driven Update branch versus a real push, and I'm not going to guess. Either way the ordering advice is free: update the branch first, then request the review — doing it after a vote lands spends a ticket on an avoidable risk.

One caveat worth stating: for #49, #48 and #39 the update does not help, because those already fail build legs on their own content (Next 16 migration for #49; nodemailer peer-resolution / lock for #48 and #39 — those two are covered on their own threads, and #45 is currently the only nodemailer PR green on docker-build). For #46, #45, #43, #42, #41, #40 the branch update is the only outstanding mechanical thing standing between them and "waiting purely on a review".

5. What I did not do

No review submitted, no vote, no merge, no close, no label/assignee change, nothing pushed. I did not click Update branch on any of them — I'm read-only on bytefolk/doc (viewerPermission: READ, both viewerCanEnableAutoMerge and viewerCanDisableAutoMerge false), so I can't, and I'd flag before doing it if that changed anyway, because "who pushed the head" is exactly the field the last-push-approval rule keys off. §3 is deliberately labelled as an inference from a control group rather than a ruleset reading, and §2's table is from live API responses, not from any cached or inherited number.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants