Skip to content

fix(source)+feat(product): bind PR merge evidence and publish scoped read-only job summaries - #227

Draft
fraware wants to merge 98 commits into
mainfrom
product/read-only-source-scoped-pr-summary-v1
Draft

fraware wants to merge 98 commits into
mainfrom
product/read-only-source-scoped-pr-summary-v1

Conversation

@fraware

@fraware fraware commented Oct 16, 2026 •

Copy link
Copy Markdown
Owner

Scope: trustworthy, reusable read-only PR observation (cumulative draft)

Source: cumulative development from qualified #225, security draft #226, and this distribution feature. This PR remains draft/unmerged and targets main for repository CI; it does not merge or close predecessors. The current exact-head workflow results must be assessed afresh before review or promotion.

Newly confirmed real-PR source-identity bug

During genuine PR dogfood of the first summary renderer, run 38830701134 FAILED CLOSED. Its uploaded Action artifact proves two different subjects:

  • ovk-source-binding.json: checked-out merge-preview 6c706febb13c32b75ae9cd6ab13a2023b3daa961, with contributor PR head 90f0a4492056149fd57c84502d2ebea80d2d8231.
  • ovk-evidence.json subject.head_sha: 90f0a4492056149fd57c84502d2ebea80d2d8231, rather than the observed merge preview.
  • Root cause: build_repository_context unconditionally replaced the Action's explicit github.sha argument with the event's pull_request.head.sha when an event file was supplied. Previous binding tests checked the diff/receipt, not this final bundle correspondence.

The observed mismatch is logged under #222. It invalidates earlier informal statements that #223/#224's source-binding receipt always matched emitted evidence; those historical CI runs remain useful software tests, not proof of the missing invariant.

Bounded correction

  1. A new ovk check --source-binding path checks the locally generated receipt against actual checked-out Git HEAD and tree, current GitHub event base/head/repository, requested merge SHA, and SHA-256 of the exact diff bytes. Invalid or missing material fails closed before emitting evidence.
  2. Only with the explicit locally checked input does build_repository_context preserve the merge-preview subject, retaining original PR-head identity separately in event metadata. Existing callers without --source-binding preserve historical behavior; no global subject-convention rewrite.
  3. Composite Action passes this opt-in only for its ordinary automatically collected pull_request diff; caller-supplied diffs and multi-lane manifests retain the unbound, non-publication contract established in security(action): prevent unbound manifest results becoming PR-wide checks #226. An empty genuine PR diff is admitted as an observation without inventing supported semantics or allow outcomes.
  4. Real-Git adversarial tests cover two-parent PR merge, stale checkout, modified diff, mismatched base/head/receipt and CLI evidence subject. The Action dogfood test itself must exercise the full evidence path at the final candidate head.

Distribution feature

  • A Python-standard-library-only renderer reads the existing ovk-evidence.json, ovk-source-binding.json, and exact .ovk-pr.diff; rehashes the diff and evidence file, checks all source/tree/event identities and the subject match, and rejects inconsistent inputs.
  • For internally prepared advisory default PR runs, appends a short GitHub native job summary with observed merge-preview SHA, tree SHA, PR head/base SHAs, diff/evidence SHA-256, and scoped checker decision. Requires no Checks-write token, PR-comment permission, web service, or hosted account. No arbitrary user source excerpts are copied into the summary.
  • The report explicitly warns that none of these local observations authenticate the GitHub event/runner, prove source-to-IR correspondence, bind into the signed evidence envelope, authorize merging, or establish generalized formal verification.
  • docs/SOURCE_SCOPED_JOB_SUMMARY.md defines a consent-based evaluation of first correct result, reviewer utility, repeat PR use and organic teammate adoption.

Qualification / release NO-GO

Obtain exact-head CI, six-platform reproducibility and native backend workflows where triggered; inspect actual emitted evidence and receipt from the real Action dogfood (not merely isolated unit tests). Distinct governance #25, source-model soundness #195, historical evidence transport #177 and consenting external pilot #20 remain open. Do not merge or publish a release, claim strict reliance, or contact external maintainers without explicit approval.

Development status is provisional until these gates close.

fraware commented Oct 16, 2026

Copy link
Copy Markdown
Owner Author

Exact-head technical qualification checkpoint — 2026-10-15

Current draft head: 90f0a4492056149fd57c84502d2ebea80d2d8231. CI 38830701134 and six-platform repro 38830701116 are queued and must complete successfully at this exact head before any review promotion. Parent security draft #226 has a separate exact-head gate and remains unmerged.

Key reuse/distribution contract: the renderer is Python-stdlib-only, checks the actual binary diff's SHA-256 against the observed source-binding receipt, validates complete base/head/merge/tree SHAs and their agreement with the evidence subject, and attaches the SHA-256 of exact local evidence bytes. It emits a concise advisory observation through GITHUB_STEP_SUMMARY (no PR Comment, Check-write token, hosted service, or code snippets) only on the internally constructed default PR diff. Caller-supplied diffs/manifests have no publication path or downstream allow Action output under parent #226. A digest is not a signature, source-model proof, or transfer authorization.

The source/target parent integration is preserved: tests/test_action_combined_identity_contract.py has identical Git blob SHA in current #226 and #227 heads (e49a592dbbe3188aa5c9c59e8fb85c72ce6c5e26); differences relative to parent are the scoped Action summary step, stdlib renderer, documentation and regression tests. No independent user activation or production release is claimed.

Proceed only after completed green full CI + all six repro cells and independent security review, with #25 and #195 still blocking broader reliance.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant