Repository navigation
fix: restore a genuinely running, verified CICD Squabbler check - #989
Conversation
|
Caution Review failedThe pull request is closed.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
hyperpolymath
changed the title
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
hyperpolymath
changed the title
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4442def6c3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| '.github/workflows/cicd-squabbler.yml': | ||
| - 'actions/checkout@v7.0.1' | ||
| - 'hyperpolymath/cicd-squabbler@1ab1507a424c9edfe6cb53dea4e33a2f3cc558e3' | ||
| '.github/workflows/cicd-squabbler.yml': [] |
There was a problem hiding this comment.
This lock update records only the Squabbler dependency removal, while .github/workflows/governance.yml:19 and the corresponding lock entry remain pinned to 333b06a01db1874ffedb2de1bfabb195640927c7. Consequently, every Governance run still uses the revision known to lack the reusable-workflow allowlist and continues reporting the five gaps this change is intended to resolve; update the workflow and lock to a runnable standards revision/policy combination that contains that allowlist.
Useful? React with 👍 / 👎.
| - name: Check out exact event commit without external action | ||
| env: | ||
| REPO: ${{ github.repository }} | ||
| SHA: ${{ github.event.pull_request.head.sha || github.sha }} |
There was a problem hiding this comment.
When a pull request branch is behind its base branch, this explicitly selects pull_request.head.sha instead of github.sha, which is the pull request's synthetic merge commit and was what the previous default checkout audited. The hard gate can therefore pass against the standalone branch even when combining the base and PR changes creates a dependency cycle or dead upstream; use the event merge SHA, or construct the merge before running Squabbler, so the check covers what will actually land.
Useful? React with 👍 / 👎.
…ision (#992) ## Investigation: what was actually failing (current runs, 2026-10-31) Estate sampling showed many workflows red with zero jobs, including Hypatia-related ones. Those are population observations, not proof of a Hypatia defect. Classified from this repo's current runs: | Run | Category | Verdict | |---|---|---| | Governance `41196057086` — `startup_failure`, 0s, zero jobs | **1 · Admission/source resolution** | No job ever started; no analysis happened. Intermediate pin change on the branch. | | Governance `41196138008` / `41196238803` — *Allowlist Preflight* only | **3 · Tool/configuration (pinned-policy mismatch)** | Callers pinned standards `333b06a…`, whose allowlist (verified at that ref, both copies) has **no** `metadatastician/standards/…` pattern — so the five canon reusables the caller itself uses were flagged as gaps. Standards `1725beb…` adds exactly `metadatastician/standards/.github/workflows/*@*`. Not five findings, and not a reason to weaken the gate. | | Mirror `41196238787` | **2 · Runner setup / prerequisites** | At `333b06a…`, `Verify <forge> SSH host` ran *before* the not-configured skip: forges with no `*_SSH_KEY` failed on host verification, then printed "Skipped". At `1725beb…` verification is gated on the key; unconfigured forges skip explicitly. Remainder is owner-side (register deploy keys or leave the forge disabled — the run then honestly says *skipped*). | | Hypatia Security Scan `41196238773` | **4 · Actual analysis channel** | `Hypatia Neurosymbolic Analysis` job ran and succeeded — the only green shape that means "the scanner ran". | Note: #989's body claimed the Governance pin had moved to `1725beb…`, but the merged tree still pinned `333b06a…` in all five callers and `actions.lock`. This PR makes the tree match the claim and adds a contract test so the claim is checked, not trusted. ## Change - Pin all five standards reusable callers (`governance`, `hypatia-scan`, `mirror`, `scorecard`, `secret-scanner`) to immutable standards `1725bebd3773ea25627479f813283abf1c497a85` (standards `main` at time of writing; its reusables carry upstream contract tests and stage policy from `job.workflow_sha` — the caller's exact pin, never a floating HEAD clone). - `actions.lock` updated consistently, including the new transitive `actions/download-artifact@9000827…` resolution from `hypatia-scan-reusable@1725beb…`. (The `gh actions-lock` binary could not be fetched in the sandbox — release-asset host unreachable — so the Governance *Actions lockfile verify* job on this PR is the authoritative regeneration check; will fix forward if it disagrees.) - New checker `scripts/standards-integration/check-standards-integration.sh` + `test/standards-integration-test.sh`, wired into a zero-third-party-action **Standards Integration** workflow (same pattern as the squabbler canary, so the gate can run even when action resolution is the thing that's broken). - Consumer handoff, prerequisites, taxonomy and bump checklist: `docs/consumers/standards-governance.adoc`. Hypatia is **not** disabled and no genuine finding is dismissed: the scan lane keeps its base/event-SHA scanner source (never the PR head, never floating) and its existing gating. ## Outcome semantics (never collapsed into "green") - checker exit `0` / `outcome=pass` = integration channel ready — **not** a scan result - exit `1` / `outcome=fail category=config` = category-3 configuration failure - exit `3` / `outcome=unavailable category=tool-unavailable` = tool absent — reported as *not verified*, fails closed in CI; absent tools, skipped jobs and `startup_failure` are never successful analysis ## Actual execution evidence Local, before push: ``` ok(positive) outcome=pass standards_pin=1725bebd3773ea25627479f813283abf1c497a85 callers=5 ok(floating-pin) rc=1 outcome=fail (governance.yml @main rejected) ok(lock-drift) rc=1 outcome=fail (actions.lock ≠ workflow pin rejected) ok(divergent-pins) rc=1 outcome=fail (scorecard.yml on a different SHA rejected) ok(pr-head-scanner) rc=1 outcome=fail (hypatia-ref = PR head rejected) ok(unavailable-tool) rc=3 outcome=unavailable (gh absent for --verify-remote; not a pass) PASS: standards-integration positive, negative and unavailable-tool cases --verify-remote with gh present: remote=verified repo=metadatastician/standards commit=1725beb… test/ci-health-test.sh: PASS (unchanged suite) YAML: all six touched workflow files parse ``` CI on this PR is the remaining evidence: Governance must start real jobs and pass Allowlist Preflight (or each remaining gap gets explained here individually), Hypatia Security Scan's analysis job must run, Mirror must print explicit per-forge lines, and Standards Integration must pass. **I will notify @hyperpolymath here only when those checks are genuinely clean** — a merged PR, a skipped job, or a zero-job run will not be reported as clean. Consumer handoff (summary; full version in the doc): copy the five `uses:` pins + `hypatia-ref: ${{ github.event.pull_request.base.sha || github.sha }}`, pass only declared secrets (never `secrets: inherit`), ensure the repo Actions allowlist permits `metadatastician/standards/.github/workflows/*`, regenerate `actions.lock` with `gh actions-lock` in the same PR, and run `bash test/standards-integration-test.sh`.
Why this PR remains draft
The previous Squabbler integration was merged in PR #988 although its workflow was
startup_failurewith zero jobs. First a cross-owner reusable call, then a same-owner composite action failed to start. A successful GitHub merge is not proof that required checks ran.This PR replaces it with a workflow having no external
uses:. It checks out the exact event commit without a marketplace action, fetches squabbler v0.1.0, verifies sha2566cbeb4577d83ccf2dea3405a3afb0d34b7fee422af73f2113d413ea7edba026cand attestation (signer workflow, pinned source SHAb854d17abc0dce296719a349ebe856ff945cf03eand subject digest), then runschains .=<repo> --follow 1with hard failure on all nonzero outcomes.Live evidence: on PR #989's first canary commit, GitHub ran all three steps successfully, including attestation and chains. Squabbler inspected 37 transitive repositories, 290 edges, 17 nonblocking findings and zero blocking ones. It reported 202 tag/branch-based upstream references (advisory work, not auto-bypass).
The
actions.lockentry for this new zero-usesworkflow is[]. No token with write privileges, no administration token, no--apply, no auto-merge and no fleet quarantine change.Separately discovered Governance blocker
The existing Governance reusable at standards SHA
333b06a...has an allowlist missing fivemetadatastician/standards/.github/workflows/*refs; live Allowlist Preflight fails. Current standards SHA1725bebd...contains the policy correction, but calling that new revision from Hypatia produced a zero-job startup_failure (effective caller Actions policy or reusable-resolution issue). This PR briefly tested the new pin, then reverted it to the last runnable revision. Do not silently replace a visible failing gate with an invisible missing gate. Restoring that permitted pin/allowlist is a separate blocker requiring effective Actions policy investigation/authorised administration access; the connector currently returns 403 on classic protection reads.Remaining acceptance
This remains a draft until the required checks can genuinely complete. Keep squabbler independent of Hypatia.