Skip to content

CI: reproduction check + guard self-tests (this repo had no CI) #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
BTCBoyd merged 1 commit into from
Aug 9, 2026
Merged

CI: reproduction check + guard self-tests (this repo had no CI) #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 104 additions & 0 deletions .github/workflows/reproduction.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# Does each package's tree reproduce the published artifact it claims?
#
# This repo had NO CI at all until this workflow. The publish guard
# (scripts/refuse-dirty-publish.sh) runs only in prepublishOnly, so nothing checked a tree between
# releases — which is exactly when x402-op-authorize drifted from the 0.4.0 it declared.
name: reproduction

on:
push:
branches: [main]
pull_request:
workflow_dispatch:

jobs:
reproduce:
runs-on: ubuntu-latest
strategy:
fail-fast: false # one drifted package must not hide the state of the other six
matrix:
package:
- ap2-op-authorize
- fireblocks-op-authorize
- l402-op-authorize
- mppx-op-account
- ows-op-verify
- wdk-op-policy
- x402-op-authorize
steps:
- uses: actions/checkout@v4

# ─── NODE IS PINNED HERE BECAUSE NOTHING ELSE PINS IT ────────────────────────────────
# Reproduction is relative to a toolchain. esbuild and typescript ARE pinned, by each
# package's committed package-lock.json, so they are not the risk. Node is pinned by
# NOTHING in the published artifact: no `engines` floor is a build guarantee, and a
# consumer on a different Node major may legitimately fail to reproduce a tarball we
# consider correct.
#
# 22.22.3 is the version under which byte-identical reproduction was MEASURED on
# 2026-08-08 (x402-op-authorize@0.4.0, all 14 files and the .tgz itself). Changing this
# line changes what "reproduces" means. If you bump it, re-measure against a known-good
# package first and record the result here — a green run under an unmeasured toolchain
# proves less than it appears to.
- uses: actions/setup-node@v4
with:
node-version: 22.22.3

- name: Reproduce ${{ matrix.package }}
run: bash scripts/verify-reproduction.sh packages/${{ matrix.package }}

# The publish guard is exercised here too, so its own failure modes stay live rather than being
# discovered at publish time by whoever is publishing.
guard-self-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }
- uses: actions/setup-node@v4
with:
node-version: 22.22.3
# The guard refuses a branch with no upstream BEFORE it reaches any later check, and
# actions/checkout does not configure upstream tracking. Without this the self-tests below
# would go red for the wrong reason and read as "the guard is broken".
- name: Give HEAD an upstream, so the guard reaches the checks under test
run: |
git fetch origin main
git checkout -B guard-selftest origin/main
git branch --set-upstream-to=origin/main guard-selftest

- name: The guard must refuse an already-published version FOR THAT REASON
working-directory: packages/x402-op-authorize
run: |
npm ci --silent
node -e '
const fs=require("fs"); const p=JSON.parse(fs.readFileSync("package.json","utf8"));
p.version="0.4.0"; // a version that IS published
fs.writeFileSync("package.json", JSON.stringify(p,null,2));
'
# ASSERT THE REASON, NOT THE REFUSAL. Editing package.json makes the tree dirty, so the
# guard refuses either way — the dirty-tree check alone would satisfy a bare "it exited
# non-zero" assertion, and this job would stay green with the registry check deleted.
# An earlier gate in a fail-fast chain hides every later one; a refusal is evidence about
# whichever check fired, never about the one under test.
OUT="$(bash ../../scripts/refuse-dirty-publish.sh 2>&1 || true)"
echo "$OUT"
if ! printf '%s' "$OUT" | grep -q "IS ALREADY PUBLISHED"; then
echo "::error::the guard did not refuse for the registry reason — it refused for something else, or not at all"
exit 1
fi
echo "guard refused with the already-published reason, as intended"
git checkout -- package.json

- name: The guard's negative control must itself be able to fail
working-directory: packages/x402-op-authorize
run: |
# Break registry_state so a missing version reads as present, and require the guard to
# refuse naming its own instrument. Without this, a guard whose registry lookup silently
# stopped working would wave every publish through and this workflow would stay green.
sed 's/ echo free; return/ echo taken; return/' ../../scripts/refuse-dirty-publish.sh > /tmp/broken-guard.sh
OUT="$(bash /tmp/broken-guard.sh 2>&1 || true)"
if ! printf '%s' "$OUT" | grep -q "CONTROL FAILED"; then
echo "::error::the guard's own negative control did not fire when registry_state was broken"
exit 1
fi
echo "negative control fired, naming the instrument"
83 changes: 83 additions & 0 deletions scripts/verify-reproduction.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
#!/bin/bash
# Does the tree that CLAIMS version V actually reproduce the published V?
#
# WHY THIS IS HEAD-BASED AND NOT gitHead-BASED. The obvious formulation — "check out the commit
# each published version names and confirm it rebuilds" — PASSES on the case that went wrong.
# Measured 2026-08-08: x402-op-authorize@0.4.0 rebuilds BYTE-IDENTICALLY from its gitHead in the
# archived repo. Meanwhile this monorepo's tree declared 0.4.0 while carrying four changed files,
# including a widened exported union. The gitHead-based check would have reported everything fine.
#
# The question that catches it is about the WORKING TREE: if package.json's version is already
# published, this tree must reproduce that published artifact exactly. If it is not published,
# there is nothing to compare against and we skip — which after the publish guard's registry check
# is the normal state of a tree between releases.
#
# Together the two are total: a tree either declares an unpublished version (publishable, skipped
# here) or a published one (must reproduce it). There is no third state.
set -uo pipefail
PKG_DIR="${1:?usage: verify-reproduction.sh <package-dir>}"
cd "$PKG_DIR" || exit 1
NAME="$(node -p 'require("./package.json").name')"
VER="$(node -p 'require("./package.json").version')"

# Is this version published? Key on EXIT CODE plus response shape, never on output being non-empty:
# npm prints its E404 body to STDOUT and exits 1, so an emptiness test scores MISSING as PRESENT.
OUT="$(npm view "${NAME}@${VER}" version --json 2>/dev/null)"; RC=$?
if [ $RC -ne 0 ] && printf '%s' "$OUT" | grep -q '"code": *"E404"'; then
echo "SKIP ${NAME}@${VER} — not published, nothing to reproduce"
exit 0
fi
if [ $RC -ne 0 ] || ! printf '%s' "$OUT" | grep -q "\"${VER}\""; then
echo "FAIL ${NAME}@${VER} — could not establish whether this version is published (fail-closed)" >&2
exit 1
fi

WORK="$(mktemp -d)"; trap 'rm -rf "$WORK"' EXIT
( cd "$WORK" && npm pack "${NAME}@${VER}" >/dev/null 2>&1 ) || {
echo "FAIL ${NAME}@${VER} — could not fetch the published tarball (fail-closed)" >&2; exit 1; }
PUBLISHED="$(ls "$WORK"/*.tgz | head -1)"

npm ci --silent >/dev/null 2>&1 || { echo "FAIL ${NAME}@${VER} — npm ci failed" >&2; exit 1; }
npm run build --silent >/dev/null 2>&1 || { echo "FAIL ${NAME}@${VER} — build failed" >&2; exit 1; }
REBUILT="$(npm pack --silent 2>/dev/null | tail -1)"
[ -f "$REBUILT" ] || { echo "FAIL ${NAME}@${VER} — npm pack produced nothing" >&2; exit 1; }

# WHAT IS COMPARED, AND WHY IT IS NOT THE WHOLE TARBALL.
#
# The first version of this check compared the tarballs byte-for-byte. Running it against all seven
# packages showed why that is the wrong subject: FIVE failed, and four of those differed only in
# README.md and package.json — because the provenance boundary notes were added to the READMEs after
# those versions were published, and package.json's repository field changed when the packages were
# rehomed into this monorepo. Under whole-tarball comparison, fixing a typo in a README would break
# CI until the next release. That is not a defect worth a red build.
#
# THE SUBJECT IS THE SHIPPED CODE. dist/ is what consumers execute, and it is what diverged in the
# case this check exists for: x402-op-authorize declared 0.4.0 while its dist/ differed in four
# files, including a widened exported union. A dist/ difference means the code we ship is not the
# code this tree builds. Everything else is reported and does not fail the build.
mkdir -p "$WORK/a" "$WORK/b"
tar xzf "$PUBLISHED" -C "$WORK/a" 2>/dev/null
tar xzf "$REBUILT" -C "$WORK/b" 2>/dev/null
rm -f "$REBUILT"

if ! diff -rq "$WORK/a/package/dist" "$WORK/b/package/dist" >/dev/null 2>&1; then
echo "FAIL ${NAME}@${VER} — SHIPPED CODE DIFFERS FROM WHAT THIS TREE BUILDS" >&2
echo " dist/ is what consumers execute. This tree claims ${VER} and does not produce its" >&2
echo " published dist/. Either bump the version (npm version prerelease --preid rc) or" >&2
echo " restore the tree." >&2
diff -rq "$WORK/a/package/dist" "$WORK/b/package/dist" 2>&1 \
| sed "s|$WORK/a/package/dist|published:|g; s|$WORK/b/package/dist|rebuilt:|g" | sed 's/^/ /' >&2
exit 1
fi

# Non-dist differences are reported, never fatal. Read them: a changed dependency range in
# package.json is a real signal even though it does not fail this check, whereas an edited README is
# expected between releases.
OTHER="$(diff -rq "$WORK/a/package" "$WORK/b/package" 2>/dev/null | grep -v "/dist" || true)"
if [ -n "$OTHER" ]; then
echo "OK ${NAME}@${VER} — shipped dist/ reproduces exactly; non-code files differ (expected between releases):"
printf '%s\n' "$OTHER" | sed "s|$WORK/a/package/||g; s|$WORK/b/package/||g" | sed 's/^/ /'
else
echo "OK ${NAME}@${VER} — reproduces the published tarball byte-for-byte"
fi
exit 0
Loading