From 5353ff25c947dfdf90b8836648de254ce8d72505 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 06:16:46 -0700 Subject: [PATCH 01/12] chore(quest): claim quest/m1/release-versions.md Quest-Claim: 83d31f10c591e387 From 95eaaaea68268d3490142adc1a227ade770fd5d2 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 06:26:03 -0700 Subject: [PATCH 02/12] feat(gh): bot PR patch-bumps hand-versioned packages on release A backport touching only a JS, Python, Swift, or Kotlin package landed on `release` unchanged in version, so its release workflow skipped it as already published. The Release bump workflow now keeps one `release-bump` PR patch-bumping each such package whose sources changed since its current version first reached `release`, with its bun.lock or uv.lock entry. Co-Authored-By: Claude Opus 5.5 --- .github/justfile | 6 + .github/workflows/alert.yml | 1 + .github/workflows/release-bump.yml | 63 +++++++ CONTRIBUTING.md | 4 +- quest/m1/README.md | 1 - quest/m1/release-versions.md | 34 ---- sh/gh/bump.py | 276 +++++++++++++++++++++++++++++ sh/gh/bump.test.py | 190 ++++++++++++++++++++ 8 files changed, 538 insertions(+), 37 deletions(-) create mode 100644 .github/workflows/release-bump.yml delete mode 100644 quest/m1/release-versions.md create mode 100755 sh/gh/bump.py create mode 100644 sh/gh/bump.test.py diff --git a/.github/justfile b/.github/justfile index 1d5c0d883b6..2ad2eed4ea4 100644 --- a/.github/justfile +++ b/.github/justfile @@ -49,6 +49,7 @@ test: bun sh/gh/queue.test.ts python3 sh/gh/install-formula.test.py sh/gh/release.test.sh + python3 sh/gh/bump.test.py python3 sh/gh/ffi.test.py bun test sh/gh/ffi.test.ts sh/gh/cpp.test.ts python3 sh/rs/package-rename.test.py @@ -70,6 +71,11 @@ test: release *args: sh/gh/release.sh "$@" +# Patch-bump hand-versioned packages with unreleased changes: `just gh bump check|apply|pr`. +[positional-arguments] +bump *args: + python3 sh/gh/bump.py "$@" + # Post a failed run to Discord, or check cron and publishing alert coverage. [positional-arguments] alert *args: diff --git a/.github/workflows/alert.yml b/.github/workflows/alert.yml index 65a66afd87a..ac323e62dbc 100644 --- a/.github/workflows/alert.yml +++ b/.github/workflows/alert.yml @@ -22,6 +22,7 @@ on: - moq.sh - Nightly - release-brew + - Release bump - Release C - Release Cpp - Release Dart diff --git a/.github/workflows/release-bump.yml b/.github/workflows/release-bump.yml new file mode 100644 index 00000000000..aa042e3bbda --- /dev/null +++ b/.github/workflows/release-bump.yml @@ -0,0 +1,63 @@ +name: Release bump + +# release-plz bumps Rust crates on `release`. This opens one PR that +# patch-bumps every other package whose version a human owns and whose sources +# changed since that version shipped, so a backport publishes once the PR +# merges. See sh/gh/bump.py for which packages it covers. +permissions: + contents: read + +on: + push: + branches: [release] + paths: + - "js/**" + - "py/moq-rs/**" + - "swift/**" + - "kt/**" + - "bun.lock" + - "uv.lock" + - "sh/gh/bump.py" + - ".github/workflows/release-bump.yml" + # Regenerating the PR by hand. Dispatch it on `release`; the job refuses any + # other ref. + workflow_dispatch: + +# Each run rebuilds the PR from the pushed `release`, so a newer push makes an +# in-flight run moot. +concurrency: + group: release-bump + cancel-in-progress: true + +jobs: + bump: + name: Bump + runs-on: ubuntu-latest + if: ${{ github.repository_owner == 'moq-dev' && github.ref == 'refs/heads/release' }} + + steps: + # The app's token, as release-plz uses, so the PR it opens runs the + # Release gate. + - name: Generate GitHub token + uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3 + id: generate-token + with: + client-id: ${{ secrets.APP_CLIENT_ID }} + private-key: ${{ secrets.APP_PRIVATE_KEY }} + + - name: Checkout repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 0 + token: ${{ steps.generate-token.outputs.token }} + + - uses: ./.github/actions/just + + - name: Open or update the bump PR + env: + GH_TOKEN: ${{ steps.generate-token.outputs.token }} + GIT_AUTHOR_NAME: moq-bot + GIT_AUTHOR_EMAIL: moq-bot[bot]@users.noreply.github.com + GIT_COMMITTER_NAME: moq-bot + GIT_COMMITTER_EMAIL: moq-bot[bot]@users.noreply.github.com + run: just gh bump pr diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 46ddba07058..d2abb609053 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -96,13 +96,13 @@ For an advisory against Quinn, follow the advisory triage in [`rs/moq-quic/READM - Smoke clients call only the latest published APIs. A release cut updates `test/smoke/clients` to the APIs being published, then verifies the affected smoke lanes. - A release is cut by hand: a PR merging `main` into `release`, with a merge commit. - A breaking cut first writes, on `main`, each changed package's version-transition section in its `MIGRATE.md` (`## 0.3 -> 0.4`) from the migration audit: the `!` subjects since the `release` merge base, each of those PRs' Impact section, and the API diff. -- Suggest a `release` backport for any serious bug fix (crash, security, data loss, broken interop), whether found on `main` or reported against a release. Land it on `main` first, then port it onto `release` as a separate PR that keeps the published API, so it ships as a patch. The Release gate's API diff (`just api-diff --deny`) fails a backport that changes one; on the cut PR, whose head is `main`, it only reports. +- Suggest a `release` backport for any serious bug fix (crash, security, data loss, broken interop), whether found on `main` or reported against a release. Land it on `main` first, then port it onto `release` as a separate PR that keeps the published API, so it ships as a patch. The Release gate's API diff (`just api-diff --deny`) fails a backport that changes one; on the cut PR, whose head is `main`, it only reports. Don't bump versions in it: release-plz bumps the crates, and the Release bump workflow opens a `release-bump` PR patch-bumping every JavaScript, Python, Swift, and Kotlin package whose sources changed; merging both publishes the fix. - After any release publishes, patch or breaking, run `just test wire-compat --remeasure` on `main` and land the rewritten `test/interop/compat/planned-breaks.json` with refreshed reasons. Otherwise the nightly wire-compat check fails on the stale pins. - Nothing merges `release` into `main` automatically. Before cutting a minor or breaking release, the maintainer merges `release` into `main` once, so trunk carries the published versions, CHANGELOGs, and any `release`-only commits. It lands as a merge commit with `--admin`, never squashed, so the cut starts from an advanced merge base. # Versions -Releases are cut separately; bump only when asked. Each package's version lives in one place: +Releases are cut separately; bump only when asked. On `release`, the Release bump workflow (`just gh bump check` previews it) keeps one `release-bump` PR patch-bumping every JavaScript, Python, Swift, and Kotlin package whose sources changed since its current version first reached `release`, with the matching lockfile entries. Go derives its own patch, and Dart, C++, and OBS release from a tag pushed by hand. Each package's version lives in one place: - **Rust**: release-plz owns crate versions and Rust dependency requirements. - **JavaScript**: `js/*/package.json` packages with a `scripts.release` entry (skip private ones like `@moq/wasm`), plus the matching workspace version in `bun.lock`. diff --git a/quest/m1/README.md b/quest/m1/README.md index 3f93058e5c5..dac6681ae32 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -183,7 +183,6 @@ removed). - [JS SETUP rejection ownership](/quest/m1/js-setup-rejection.md) - reproduce an unhandled version-refusal rejection without weakening the refusal - [moq-clock moves into kio::clock](/quest/m1/kio-clock.md) - moq-clock and its test macro become `kio::clock` and `#[kio::test]` before the cut publishes them - [Poll-first async](/quest/m1/poll-first/README.md) - library crates compile no coroutines by default: async wrappers are generated from `poll_*` and gated behind an `async` feature -- [Every package's release backport ships](/quest/m1/release-versions.md) - a bot bumps every hand-versioned package changed on `release`, so backports publish without a manual bump ## Related diff --git a/quest/m1/release-versions.md b/quest/m1/release-versions.md deleted file mode 100644 index 7888c2d6315..00000000000 --- a/quest/m1/release-versions.md +++ /dev/null @@ -1,34 +0,0 @@ -# [M] Every package's release backport ships - -## Goal - -A fix merged into `release` for any published package publishes without a -hand-made version bump. Today release-plz bumps only Rust crates, so a -backport touching only a JS, Python, Dart, Kotlin, Swift, Go, C, or C++ -package lands on `release` unchanged in version and its release workflow -skips it as already published. - -## Plan - -Found 2026-10-28: the @moq/flate and @moq/net backports -([moq-dev/moq#6961](https://github.com/moq-dev/moq/pull/6961), -[moq-dev/moq#6959](https://github.com/moq-dev/moq/pull/6959), -[moq-dev/moq#6967](https://github.com/moq-dev/moq/pull/6967)) merged and the -release PR merged, but Release JS logged "already published, skipping" and a -separate hand bump was needed. - -Decided 2026-10-28 (maintainer): - -- Scope: every hand-versioned package listed in CONTRIBUTING.md's Versions - section, not just JS. -- Mechanism: a release-plz-like bot. On each push to `release`, it opens or - updates one PR that patch-bumps every package whose sources changed since - its last published version (with the matching lockfile entries, e.g. - `bun.lock`, `uv.lock`); merging it publishes through the existing release - workflows. Prefer an existing maintained tool if one covers these - ecosystems; extending release-plz's own PR is also fine. -- Until this lands, a backport author bumps by hand. - -Update CONTRIBUTING.md's Versions and backport bullets to describe the result. - -Public API: none. Wire: none. diff --git a/sh/gh/bump.py b/sh/gh/bump.py new file mode 100755 index 00000000000..90f577a4df3 --- /dev/null +++ b/sh/gh/bump.py @@ -0,0 +1,276 @@ +#!/usr/bin/env python3 +"""Patch-bump hand-versioned packages whose sources changed since they last shipped. + +release-plz bumps Rust crates on `release`; this does the same for the packages +whose version a human owns and whose release workflow publishes on a push to +`release` once the registry lacks that version (JS, Python, Swift, Kotlin). +Without a bump, a backport touching only one of them lands unchanged in version +and its release skips it as already published. + +A package last shipped at the oldest first-parent commit on `release` that +carries its current version: each first-parent commit is a push its release +workflow saw. Any change to its sources between that commit and HEAD is +unreleased, so the package needs a patch bump. + +Usage: + bump.py check [--ref ] List the packages that need a bump. + bump.py apply Bump them in the working tree. + bump.py pr Bump them on the `release-bump` branch and open + or update its PR into `release`; close it when + nothing is left. Needs GH_TOKEN. +""" + +import argparse +from dataclasses import dataclass, field +import json +from pathlib import Path +import re +import subprocess +import sys + +BASE = "release" +BRANCH = "release-bump" +TITLE = "chore: patch-bump hand-versioned packages" +REPO = "moq-dev/moq" + + +def git(*args: str, check: bool = True) -> subprocess.CompletedProcess: + return subprocess.run(["git", *args], capture_output=True, text=True, check=check) + + +def show(rev: str, path: str) -> str | None: + result = git("show", f"{rev}:{path}", check=False) + return result.stdout if result.returncode == 0 else None + + +def patch(version: str) -> str: + match = re.fullmatch(r"(\d+)\.(\d+)\.(\d+)", version) + if not match: + raise ValueError(f"cannot patch-bump non-release version {version!r}") + major, minor, micro = match.groups() + return f"{major}.{minor}.{int(micro) + 1}" + + +def substitute(pattern: str, text: str, version: str, path: str) -> str: + """Replace group 2 of the first match of `pattern` with `version`.""" + out, count = re.subn(pattern, lambda m: m.group(1) + version + m.group(3), text, count=1, flags=re.M) + if count != 1: + raise ValueError(f"version not found in {path}") + return out + + +@dataclass +class Edit: + """A version string in a file: group 2 of `pattern` is the version.""" + + path: str + pattern: str + + def read(self, text: str) -> str | None: + match = re.search(self.pattern, text, flags=re.M) + return match.group(2) if match else None + + +@dataclass +class Package: + name: str + # The version's source of truth; its value at a commit says what shipped. + manifest: Edit + # Shipped sources, as git pathspecs. + paths: list[str] + # Copies of the version that must move with it, such as a lockfile entry. + locks: list[Edit] = field(default_factory=list) + + def version(self, rev: str) -> str | None: + text = show(rev, self.manifest.path) + return None if text is None else self.manifest.read(text) + + +# Files none of these packages ship, so changing one alone is not a release. +UNSHIPPED = [ + ":(exclude,glob)**/*.test.*", + ":(exclude,glob)**/tests/**", + ":(exclude,glob)**/Tests/**", + ":(exclude,glob)**/MIGRATE.md", +] + + +def packages(rev: str) -> list[Package]: + """Every hand-versioned package that publishes on a push to `release`. + + Go derives its patch in CI, and Dart, C, C++, and OBS release from a tag a + human pushes, so a version bump alone would not publish any of them. + """ + found = [] + listing = git("ls-tree", "--name-only", rev, "js/").stdout.split() + for directory in sorted(listing): + text = show(rev, f"{directory}/package.json") + if text is None: + continue + manifest = json.loads(text) + # CONTRIBUTING.md's rule: a release script, and not private. + if manifest.get("private") or not manifest.get("scripts", {}).get("release"): + continue + name = manifest["name"] + found.append(Package( + name=name, + manifest=Edit(f"{directory}/package.json", r'^(\t"version": ")([^"]+)(")'), + # API reports, benches, and examples stay out of the built package. + paths=[directory, *(f":(exclude){directory}/{d}" for d in ("api", "bench", "examples"))], + locks=[Edit("bun.lock", rf'^( "{re.escape(directory)}": \{{\n "name": "{re.escape(name)}",\n "version": ")([^"]+)(")')], + )) + found.append(Package( + name="moq-rs (Python)", + manifest=Edit("py/moq-rs/pyproject.toml", r'^(\[project\]\n(?:(?!\[).*\n)*?version = ")([^"]+)(")'), + paths=["py/moq-rs", ":(exclude)py/moq-rs/benches", ":(exclude)py/moq-rs/examples"], + locks=[Edit("uv.lock", r'^(\[\[package\]\]\nname = "moq-rs"\nversion = ")([^"]+)(")')], + )) + found.append(Package( + name="Moq (Swift)", + manifest=Edit("swift/VERSION", r"\A()(\S+)(\s*)\Z"), + paths=["swift/Sources/Moq", "swift/Package.swift.template"], + )) + found.append(Package( + name="dev.moq:moq (Kotlin)", + manifest=Edit("kt/gradle.properties", r"^(moq\.version=)(\S+)()$"), + paths=["kt/moq", ":(exclude,glob)kt/moq/src/*Test/**", "kt/build.gradle.kts", "kt/settings.gradle.kts", "kt/gradle.properties"], + )) + return found + + +def shipped(pkg: Package, head: str) -> str | None: + """The oldest first-parent commit since which `pkg` has had its current version.""" + current = pkg.version(head) + if current is None: + raise ValueError(f"no version in {pkg.manifest.path} at {head}") + log = git("log", "--first-parent", "--format=%H", head, "--", pkg.manifest.path).stdout.split() + for rev in log: + if pkg.version(f"{rev}^") != current: + return rev + return None + + +@dataclass +class Bump: + package: Package + old: str + new: str + since: str + changed: list[str] + + +def plan(head: str) -> list[Bump]: + bumps = [] + for pkg in packages(head): + since = shipped(pkg, head) + if since is None: + continue + changed = git("diff", "--name-only", since, head, "--", *pkg.paths, *UNSHIPPED).stdout.split() + if changed: + old = pkg.version(head) + bumps.append(Bump(pkg, old, patch(old), since, changed)) + return bumps + + +def apply(bumps: list[Bump], root: Path) -> None: + # Every edit is checked before any file is written, so a stale lockfile + # leaves the tree untouched. Packages share lockfiles, so edits chain. + texts: dict[str, str] = {} + for bump in bumps: + for edit in [bump.package.manifest, *bump.package.locks]: + text = texts.get(edit.path) or (root / edit.path).read_text() + if edit.read(text) != bump.old: + raise ValueError(f"{edit.path} does not carry {bump.package.name} {bump.old}") + texts[edit.path] = substitute(edit.pattern, text, bump.new, edit.path) + for path, text in texts.items(): + (root / path).write_text(text) + + +def body(bumps: list[Bump], head: str) -> str: + lines = [ + "Patch-bumps every hand-versioned package whose sources changed on `release` since its current version shipped, " + "so merging this publishes them through their release workflows.", + "", + "| Package | Version | Unreleased changes |", + "| --- | --- | --- |", + ] + for bump in bumps: + files = "file" if len(bump.changed) == 1 else "files" + compare = f"[{len(bump.changed)} {files}](https://github.com/{REPO}/compare/{bump.since[:10]}...{head[:10]})" + lines.append(f"| {bump.package.name} | {bump.old} → {bump.new} | {compare} |") + lines += [ + "", + "Each bump is a patch, as a backport keeping the published API needs. " + "The next push to `release` regenerates this branch from scratch, dropping any edit to it.", + "", + f"Generated by `just gh bump pr` at {head[:10]}.", + ] + return "\n".join(lines) + "\n" + + +def gh(*args: str, stdin: str | None = None) -> str: + return subprocess.run(["gh", *args], input=stdin, capture_output=True, text=True, check=True).stdout.strip() + + +def open_pr() -> str | None: + number = gh("pr", "list", "--repo", REPO, "--base", BASE, "--head", BRANCH, "--state", "open", + "--json", "number", "--jq", ".[0].number // empty") + return number or None + + +def pr(root: Path) -> None: + head = git("rev-parse", "HEAD").stdout.strip() + bumps = plan(head) + existing = open_pr() + if not bumps: + if existing: + gh("pr", "close", existing, "--repo", REPO, "--delete-branch", + "--comment", f"Every hand-versioned package on `release` is bumped as of {head[:10]}.") + print(f"closed #{existing}: nothing left to bump") + else: + print("nothing to bump") + return + + apply(bumps, root) + names = ", ".join(f"{b.package.name} {b.new}" for b in bumps) + git("checkout", "-B", BRANCH) + git("commit", "--all", "--message", f"{TITLE}\n\n{names}") + # The branch is regenerated from `release` on every run, like release-plz's. + git("push", "--force", "origin", f"HEAD:refs/heads/{BRANCH}") + text = body(bumps, head) + if existing: + gh("pr", "edit", existing, "--repo", REPO, "--title", TITLE, "--body-file", "-", stdin=text) + print(f"updated #{existing}: {names}") + else: + url = gh("pr", "create", "--repo", REPO, "--base", BASE, "--head", BRANCH, "--title", TITLE, + "--body-file", "-", stdin=text) + print(f"opened {url}: {names}") + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + sub = parser.add_subparsers(dest="command", required=True) + check = sub.add_parser("check") + check.add_argument("--ref", default="HEAD") + sub.add_parser("apply") + sub.add_parser("pr") + args = parser.parse_args() + + root = Path(git("rev-parse", "--show-toplevel").stdout.strip()) + if args.command == "check": + bumps = plan(args.ref) + for bump in bumps: + print(f"{bump.package.name}: {bump.old} -> {bump.new} ({len(bump.changed)} files since {bump.since[:10]})") + for path in bump.changed: + print(f" {path}") + if not bumps: + print("nothing to bump") + elif args.command == "apply": + apply(plan("HEAD"), root) + else: + pr(root) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/sh/gh/bump.test.py b/sh/gh/bump.test.py new file mode 100644 index 00000000000..af6b790e109 --- /dev/null +++ b/sh/gh/bump.test.py @@ -0,0 +1,190 @@ +"""Find hand-versioned packages with unreleased changes on a first-parent history.""" + +import importlib.util +import json +import os +from pathlib import Path +import subprocess +import tempfile +import unittest + +spec = importlib.util.spec_from_file_location("bump", Path(__file__).with_name("bump.py")) +bump = importlib.util.module_from_spec(spec) +spec.loader.exec_module(bump) + + +def package_json(name: str, version: str, private: bool = False) -> str: + manifest = {"name": name, "version": version, "scripts": {"release": "bun ../common/release.ts"}} + if private: + manifest["private"] = True + return json.dumps(manifest, indent="\t") + "\n" + + +BUN_LOCK = """{ + "workspaces": { + "js/a": { + "name": "@moq/a", + "version": "0.1.0", + }, + "js/b": { + "name": "@moq/b", + "version": "0.2.0", + }, + }, +} +""" + +PYPROJECT = """[build-system] +requires = ["hatchling"] + +[project] +name = "moq-rs" +# Bumped by hand. +version = "0.5.0" +""" + +UV_LOCK = """[[package]] +name = "moq-ffi" +version = "0.4.0" + +[[package]] +name = "moq-rs" +version = "0.5.0" +source = { editable = "py/moq-rs" } +""" + + +class BumpTest(unittest.TestCase): + def setUp(self): + self.dir = tempfile.TemporaryDirectory() + self.root = Path(self.dir.name) + self.cwd = os.getcwd() + os.chdir(self.root) + self.git("init", "--quiet", "--initial-branch", "release") + self.git("config", "user.name", "test") + self.git("config", "user.email", "test@example.com") + self.write({ + "js/a/package.json": package_json("@moq/a", "0.1.0"), + "js/a/src/index.ts": "export {};\n", + "js/b/package.json": package_json("@moq/b", "0.2.0"), + "js/b/src/index.ts": "export {};\n", + "js/wasm/package.json": package_json("@moq/wasm", "0.0.0", private=True), + "js/wasm/src/index.ts": "export {};\n", + "bun.lock": BUN_LOCK, + "py/moq-rs/pyproject.toml": PYPROJECT, + "py/moq-rs/moq/__init__.py": "", + "uv.lock": UV_LOCK, + "swift/VERSION": "0.3.0\n", + "swift/Sources/Moq/Moq.swift": "", + "kt/gradle.properties": "moqffi.version=0.0.0-dev\nmoq.version=0.4.0\n", + "kt/moq/src/jvmMain/Moq.kt": "", + }) + self.commit("initial") + + def tearDown(self): + os.chdir(self.cwd) + self.dir.cleanup() + + def git(self, *args: str) -> str: + return subprocess.run(["git", *args], capture_output=True, text=True, check=True).stdout.strip() + + def write(self, files: dict[str, str]): + for path, text in files.items(): + (self.root / path).parent.mkdir(parents=True, exist_ok=True) + (self.root / path).write_text(text) + + def commit(self, message: str, files: dict[str, str] | None = None) -> str: + if files: + self.write(files) + self.git("add", "--all") + self.git("commit", "--quiet", "--message", message) + return self.git("rev-parse", "HEAD") + + def planned(self) -> dict[str, str]: + return {b.package.name: b.new for b in bump.plan("HEAD")} + + def test_nothing_changed(self): + self.assertEqual(self.planned(), {}) + + def test_backport_bumps_only_its_package(self): + self.commit("fix(a): a backport", {"js/a/src/index.ts": "export const fix = 1;\n"}) + self.assertEqual(self.planned(), {"@moq/a": "0.1.1"}) + + def test_unshipped_files_are_not_a_release(self): + self.commit("test(a): more tests", { + "js/a/src/index.test.ts": "", + "js/a/api/index.api.md": "", + "js/a/MIGRATE.md": "", + "kt/moq/src/jvmTest/MoqTest.kt": "", + "swift/Tests/MoqTests/MoqTests.swift": "", + }) + self.assertEqual(self.planned(), {}) + + def test_private_package_is_skipped(self): + self.commit("fix(wasm): private", {"js/wasm/src/index.ts": "export const fix = 1;\n"}) + self.assertEqual(self.planned(), {}) + + def test_every_ecosystem(self): + self.commit("fix: everywhere", { + "py/moq-rs/moq/__init__.py": "fix = 1\n", + "swift/Sources/Moq/Moq.swift": "// fix\n", + "kt/moq/src/jvmMain/Moq.kt": "// fix\n", + }) + self.assertEqual(self.planned(), { + "moq-rs (Python)": "0.5.1", + "Moq (Swift)": "0.3.1", + "dev.moq:moq (Kotlin)": "0.4.1", + }) + + def test_bump_resets_the_baseline(self): + self.commit("fix(a): a backport", {"js/a/src/index.ts": "export const fix = 1;\n"}) + bump.apply(bump.plan("HEAD"), self.root) + self.commit("chore: bump") + self.assertEqual(self.planned(), {}) + + def test_cut_merge_ships_main_changes_made_after_its_bump(self): + # On main: bump @moq/a, then land another change, then cut into release. + self.git("checkout", "--quiet", "-b", "main") + self.commit("chore: bump a", {"js/a/package.json": package_json("@moq/a", "0.2.0")}) + self.commit("feat(a): after the bump", {"js/a/src/index.ts": "export const feature = 1;\n"}) + self.git("checkout", "--quiet", "release") + self.git("merge", "--quiet", "--no-ff", "--message", "cut", "main") + # The cut's tree is what Release JS published, so nothing is pending. + self.assertEqual(self.planned(), {}) + self.commit("fix(a): a backport", {"js/a/src/index.ts": "export const fix = 1;\n"}) + self.assertEqual(self.planned(), {"@moq/a": "0.2.1"}) + + def test_apply_edits_manifest_and_lockfiles(self): + self.commit("fix: a and python", { + "js/a/src/index.ts": "export const fix = 1;\n", + "py/moq-rs/moq/__init__.py": "fix = 1\n", + }) + bump.apply(bump.plan("HEAD"), self.root) + self.assertEqual(json.loads((self.root / "js/a/package.json").read_text())["version"], "0.1.1") + self.assertEqual( + (self.root / "bun.lock").read_text(), + BUN_LOCK.replace('"version": "0.1.0"', '"version": "0.1.1"'), + ) + self.assertIn('version = "0.5.1"', (self.root / "py/moq-rs/pyproject.toml").read_text()) + self.assertEqual( + (self.root / "uv.lock").read_text(), + UV_LOCK.replace('version = "0.5.0"', 'version = "0.5.1"'), + ) + + def test_apply_refuses_a_stale_lockfile(self): + self.commit("fix(a): a backport", { + "js/a/src/index.ts": "export const fix = 1;\n", + "bun.lock": BUN_LOCK.replace('"version": "0.1.0"', '"version": "0.0.9"'), + }) + with self.assertRaisesRegex(ValueError, "bun.lock does not carry @moq/a 0.1.0"): + bump.apply(bump.plan("HEAD"), self.root) + self.assertEqual(self.git("status", "--porcelain"), "") + + def test_patch(self): + self.assertEqual(bump.patch("0.9.9"), "0.9.10") + with self.assertRaises(ValueError): + bump.patch("1.0.0-rc.1") + + +if __name__ == "__main__": + unittest.main() From 5e91735bf3a175426335a10b95fb9c4b66e9df17 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 08:32:17 -0700 Subject: [PATCH 03/12] feat(gh): bump and tag the tag-released packages too Dart, C, and C++ publish only from a tag, so a backport touching one of them never shipped. The Release bump workflow now patch-bumps them in the same `release-bump` PR when their sources changed since their current version's tag, bumps OBS and C++ together since one tag releases both, and pushes each tag a version on `release` lacks. C and C++ tags wait until the moq-ffi release `cpp/moq/FFI_VERSION` pins is published, so the workflow now runs on every push to `release` to retry them. Co-Authored-By: Claude Opus 5.5 --- .github/justfile | 2 +- .github/workflows/release-bump.yml | 28 ++-- CONTRIBUTING.md | 7 +- quest/m2/release/cpp-release.md | 18 +-- quest/m2/release/dart-publish.md | 2 +- sh/gh/bump.py | 201 +++++++++++++++++++++++++---- sh/gh/bump.test.py | 99 ++++++++++++++ 7 files changed, 307 insertions(+), 50 deletions(-) diff --git a/.github/justfile b/.github/justfile index 2ad2eed4ea4..88170c6366b 100644 --- a/.github/justfile +++ b/.github/justfile @@ -71,7 +71,7 @@ test: release *args: sh/gh/release.sh "$@" -# Patch-bump hand-versioned packages with unreleased changes: `just gh bump check|apply|pr`. +# Patch-bump and tag hand-versioned packages with unreleased changes: `just gh bump check|apply|pr|tag`. [positional-arguments] bump *args: python3 sh/gh/bump.py "$@" diff --git a/.github/workflows/release-bump.yml b/.github/workflows/release-bump.yml index aa042e3bbda..82c542570fc 100644 --- a/.github/workflows/release-bump.yml +++ b/.github/workflows/release-bump.yml @@ -3,24 +3,19 @@ name: Release bump # release-plz bumps Rust crates on `release`. This opens one PR that # patch-bumps every other package whose version a human owns and whose sources # changed since that version shipped, so a backport publishes once the PR -# merges. See sh/gh/bump.py for which packages it covers. +# merges. For packages a tag publishes (Dart, C, C++), it then pushes each tag +# the versions on `release` lack. See sh/gh/bump.py for which packages it +# covers. permissions: contents: read on: + # Every push, not only those touching a package: a C or C++ tag waiting on + # its pinned moq-ffi release retries on the next one. push: branches: [release] - paths: - - "js/**" - - "py/moq-rs/**" - - "swift/**" - - "kt/**" - - "bun.lock" - - "uv.lock" - - "sh/gh/bump.py" - - ".github/workflows/release-bump.yml" - # Regenerating the PR by hand. Dispatch it on `release`; the job refuses any - # other ref. + # Regenerating the PR, or retrying a tag, by hand. Dispatch it on `release`; + # the job refuses any other ref. workflow_dispatch: # Each run rebuilds the PR from the pushed `release`, so a newer push makes an @@ -37,7 +32,7 @@ jobs: steps: # The app's token, as release-plz uses, so the PR it opens runs the - # Release gate. + # Release gate and the tags it pushes run their release workflows. - name: Generate GitHub token uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3 id: generate-token @@ -53,6 +48,13 @@ jobs: - uses: ./.github/actions/just + # Before the PR step, which moves HEAD to the bump commit. A version the + # bump PR raises is tagged on the push its merge makes. + - name: Push release tags + env: + GH_TOKEN: ${{ steps.generate-token.outputs.token }} + run: just gh bump tag + - name: Open or update the bump PR env: GH_TOKEN: ${{ steps.generate-token.outputs.token }} diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d2abb609053..4e7194f8696 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -96,18 +96,19 @@ For an advisory against Quinn, follow the advisory triage in [`rs/moq-quic/READM - Smoke clients call only the latest published APIs. A release cut updates `test/smoke/clients` to the APIs being published, then verifies the affected smoke lanes. - A release is cut by hand: a PR merging `main` into `release`, with a merge commit. - A breaking cut first writes, on `main`, each changed package's version-transition section in its `MIGRATE.md` (`## 0.3 -> 0.4`) from the migration audit: the `!` subjects since the `release` merge base, each of those PRs' Impact section, and the API diff. -- Suggest a `release` backport for any serious bug fix (crash, security, data loss, broken interop), whether found on `main` or reported against a release. Land it on `main` first, then port it onto `release` as a separate PR that keeps the published API, so it ships as a patch. The Release gate's API diff (`just api-diff --deny`) fails a backport that changes one; on the cut PR, whose head is `main`, it only reports. Don't bump versions in it: release-plz bumps the crates, and the Release bump workflow opens a `release-bump` PR patch-bumping every JavaScript, Python, Swift, and Kotlin package whose sources changed; merging both publishes the fix. +- Suggest a `release` backport for any serious bug fix (crash, security, data loss, broken interop), whether found on `main` or reported against a release. Land it on `main` first, then port it onto `release` as a separate PR that keeps the published API, so it ships as a patch. The Release gate's API diff (`just api-diff --deny`) fails a backport that changes one; on the cut PR, whose head is `main`, it only reports. Don't bump versions in it: release-plz bumps the crates, and the Release bump workflow opens a `release-bump` PR patch-bumping every other hand-versioned package whose sources changed, then pushes the Dart, C, and C++ tags those bumps need; merging both publishes the fix. - After any release publishes, patch or breaking, run `just test wire-compat --remeasure` on `main` and land the rewritten `test/interop/compat/planned-breaks.json` with refreshed reasons. Otherwise the nightly wire-compat check fails on the stale pins. - Nothing merges `release` into `main` automatically. Before cutting a minor or breaking release, the maintainer merges `release` into `main` once, so trunk carries the published versions, CHANGELOGs, and any `release`-only commits. It lands as a merge commit with `--admin`, never squashed, so the cut starts from an advanced merge base. # Versions -Releases are cut separately; bump only when asked. On `release`, the Release bump workflow (`just gh bump check` previews it) keeps one `release-bump` PR patch-bumping every JavaScript, Python, Swift, and Kotlin package whose sources changed since its current version first reached `release`, with the matching lockfile entries. Go derives its own patch, and Dart, C++, and OBS release from a tag pushed by hand. Each package's version lives in one place: +Releases are cut separately; bump only when asked. On `release`, the Release bump workflow (`just gh bump check` previews it) keeps one `release-bump` PR patch-bumping every hand-versioned package whose sources changed since its current version shipped, with the matching lockfile entries. A JavaScript, Python, Swift, or Kotlin version ships when it first reaches `release`. A Dart, C, or C++ version ships when its tag is pushed, and the workflow pushes the tag for each such version on `release` that lacks one; C and C++ wait until the moq-ffi release `cpp/moq/FFI_VERSION` pins is published, retrying on each later push. Go derives its own patch. Each package's version lives in one place: - **Rust**: release-plz owns crate versions and Rust dependency requirements. - **JavaScript**: `js/*/package.json` packages with a `scripts.release` entry (skip private ones like `@moq/wasm`), plus the matching workspace version in `bun.lock`. - **Python**: `py/moq-rs/pyproject.toml`, plus the matching `moq-rs` entry in the root `uv.lock`; `py/moq-ffi` follows the `moq-ffi-v*` tag and Rust crate. - **Swift**: `swift/VERSION`. **Kotlin**: `moq.version` in `kt/gradle.properties`. **Dart**: `version` in `dart/moq/pubspec.yaml`. Their FFI counterparts track the Rust crate. - **Go**: `go/wrapper/VERSION` holds a human-owned `MAJOR.MINOR` line; CI derives the patch, so only edit it for a breaking API. Leave the placeholder FFI version in `go.mod` alone. +- **C**: `c/moq/VERSION`, human-owned; a `c-v` tag releases it. - **C++**: `cpp/moq/VERSION`, human-owned; a `cpp-v` tag releases it. -- **OBS**: `cpp/obs/VERSION`, human-owned; each C++ release also cuts `obs-moq-v`, and refuses one whose version already names an earlier release. +- **OBS**: `cpp/obs/VERSION`, human-owned; each C++ release also cuts `obs-moq-v`, and refuses one whose version already names an earlier release. So the Release bump workflow bumps the plugin with every C++ release, and C++ with every plugin change. diff --git a/quest/m2/release/cpp-release.md b/quest/m2/release/cpp-release.md index 23fe95f1a4a..57495d52031 100644 --- a/quest/m2/release/cpp-release.md +++ b/quest/m2/release/cpp-release.md @@ -5,9 +5,8 @@ The OBS release path is dry-run nightly, then the first `cpp-v` tag is pushed and `release-cpp.yml` publishes one `moq-cpp--` archive per target to its GitHub release, plus the -matching `obs-moq-v` plugin release. A maintainer cuts it by hand -once `main` reaches `release`; check with -`gh release list --repo moq-dev/moq | grep cpp-v`. +matching `obs-moq-v` plugin release, once `main` reaches `release`; +check with `gh release list --repo moq-dev/moq | grep cpp-v`. ## Plan @@ -26,11 +25,14 @@ The nightly C++ and OBS release checks pass: the which failed only in unrelated lanes, checked the installed consumers of all four packaged archives and built both OBS plugins against them. It packages this run's prebuilt moq-ffi input, while a tag with `cpp/moq/FFI_VERSION` unset builds moq-ffi from source, a -path no nightly checks. So if the pin is still unset, first dispatch -`release-cpp.yml` on the ref to be tagged without `prebuilt-test`: it runs -the tag's source build and archive check on all four hosts, then both OBS -builds, and publishes nothing. This quest owns that dry run, tagging, and verifying the -published downloads. +path no nightly checks. The Release bump workflow pushes the `cpp-v` tag +itself on the first push to `release` after the pin names a published +moq-ffi release, and never while it is unset. So if the pin is still unset, +first dispatch `release-cpp.yml` on the ref to be tagged without +`prebuilt-test`: it runs the tag's source build and archive check on all four +hosts, then both OBS builds, and publishes nothing. Then tag it by hand. This +quest owns that dry run, any hand tagging, and verifying the published +downloads. ## Required diff --git a/quest/m2/release/dart-publish.md b/quest/m2/release/dart-publish.md index 23fb5f21536..b7b113b0870 100644 --- a/quest/m2/release/dart-publish.md +++ b/quest/m2/release/dart-publish.md @@ -19,7 +19,7 @@ can proceed independently; cutting the tag cannot. - Confirm trusted publishing is configured for `moq` (repository `moq-dev/moq`, tag pattern `moq-dart-v{{version}}`). - Raise `dart/moq/pubspec.yaml`'s `moq_ffi` floor to the release the wrapper now needs (the media namespace and codec types), since `^0.4.3` resolves bindings without them. -- Cut the next `moq-dart-v*` release with the binding breaks and confirm the workflow publishes it. +- The Release bump workflow pushes each `moq-dart-v*` tag a version on `release` lacks. Its first run on `release` tags `moq-dart-v0.1.1`, the drain backport ([335e0b5f29](https://github.com/moq-dev/moq/commit/335e0b5f29)); confirm Release Dart publishes it unattended, and that the cut's binding-break release publishes the same way. - Verify the published `moq_ffi` resolves its native asset from a clean machine with no monorepo checkout, since that download path is the one CI never exercises. diff --git a/sh/gh/bump.py b/sh/gh/bump.py index 90f577a4df3..c97af38af03 100755 --- a/sh/gh/bump.py +++ b/sh/gh/bump.py @@ -2,26 +2,36 @@ """Patch-bump hand-versioned packages whose sources changed since they last shipped. release-plz bumps Rust crates on `release`; this does the same for the packages -whose version a human owns and whose release workflow publishes on a push to -`release` once the registry lacks that version (JS, Python, Swift, Kotlin). -Without a bump, a backport touching only one of them lands unchanged in version -and its release skips it as already published. +whose version a human owns. Without a bump, a backport touching only one of +them lands unchanged in version and never publishes. -A package last shipped at the oldest first-parent commit on `release` that -carries its current version: each first-parent commit is a push its release -workflow saw. Any change to its sources between that commit and HEAD is -unreleased, so the package needs a patch bump. +Two kinds of package publish differently: + +- JS, Python, Swift, and Kotlin publish on a push to `release` once the registry + lacks their version. Each first-parent commit is a push their release workflow + saw, so one last shipped at the oldest first-parent commit carrying its + current version. +- Dart, C, and C++ publish from a `` tag, and OBS rides each + C++ tag. One last shipped at its current version's tag; a current version + with no tag is unreleased, so `tag` pushes it, while C and C++ wait until the + moq-ffi release `cpp/moq/FFI_VERSION` pins is published. + +Any change to a package's sources since it last shipped is unreleased, so it +needs a patch bump. Usage: - bump.py check [--ref ] List the packages that need a bump. + bump.py check [--ref ] List the packages that need a bump or a tag. bump.py apply Bump them in the working tree. bump.py pr Bump them on the `release-bump` branch and open or update its PR into `release`; close it when nothing is left. Needs GH_TOKEN. + bump.py tag Push each release tag HEAD's versions lack. + Needs GH_TOKEN. """ import argparse from dataclasses import dataclass, field +import importlib.util import json from pathlib import Path import re @@ -33,6 +43,13 @@ TITLE = "chore: patch-bump hand-versioned packages" REPO = "moq-dev/moq" +# The C and C++ packages' moq-ffi pin, and its asset names. +FFI_VERSION = "cpp/moq/FFI_VERSION" +FFI_SHA256 = "cpp/moq/FFI_SHA256.json" +_spec = importlib.util.spec_from_file_location("ffi", Path(__file__).parents[1] / "cpp/ffi.py") +ffi = importlib.util.module_from_spec(_spec) +_spec.loader.exec_module(ffi) + def git(*args: str, check: bool = True) -> subprocess.CompletedProcess: return subprocess.run(["git", *args], capture_output=True, text=True, check=check) @@ -80,6 +97,12 @@ class Package: paths: list[str] # Copies of the version that must move with it, such as a lockfile entry. locks: list[Edit] = field(default_factory=list) + # The prefix of the tags naming its releases, for a package a tag publishes. + tag: str | None = None + # The package whose tag publishes this one, when it is not its own. + rides: str | None = None + # Whether its tag waits for the pinned moq-ffi release. + ffi: bool = False def version(self, rev: str) -> str | None: text = show(rev, self.manifest.path) @@ -96,11 +119,7 @@ def version(self, rev: str) -> str | None: def packages(rev: str) -> list[Package]: - """Every hand-versioned package that publishes on a push to `release`. - - Go derives its patch in CI, and Dart, C, C++, and OBS release from a tag a - human pushes, so a version bump alone would not publish any of them. - """ + """Every hand-versioned package present at `rev`. Go derives its patch in CI.""" found = [] listing = git("ls-tree", "--name-only", rev, "js/").stdout.split() for directory in sorted(listing): @@ -135,14 +154,52 @@ def packages(rev: str) -> list[Package]: manifest=Edit("kt/gradle.properties", r"^(moq\.version=)(\S+)()$"), paths=["kt/moq", ":(exclude,glob)kt/moq/src/*Test/**", "kt/build.gradle.kts", "kt/settings.gradle.kts", "kt/gradle.properties"], )) + version_file = r"\A()(\S+)(\s*)\Z" + tagged = [ + Package( + name="moq (Dart)", + manifest=Edit("dart/moq/pubspec.yaml", r"^(version: )(\S+)()$"), + paths=["dart/moq", ":(exclude)dart/moq/test", ":(exclude)dart/moq/bench"], + tag="moq-dart-v", + ), + # The C package links the same pinned moq-ffi input as C++. + Package( + name="moq (C)", + manifest=Edit("c/moq/VERSION", version_file), + paths=["c/moq", ":(exclude)c/moq/test", FFI_VERSION, FFI_SHA256], + tag="c-v", + ffi=True, + ), + Package( + name="moq (C++)", + manifest=Edit("cpp/moq/VERSION", version_file), + paths=["cpp/moq", ":(exclude)cpp/moq/test", ":(exclude)cpp/moq/bench"], + tag="cpp-v", + ffi=True, + ), + # Release Cpp builds the plugin on every C++ tag and cuts its obs-moq tag. + Package( + name="obs-moq (OBS)", + manifest=Edit("cpp/obs/VERSION", version_file), + paths=["cpp/obs", ":(exclude)cpp/obs/test"], + tag="obs-moq-v", + rides="moq (C++)", + ), + ] + # `release` gains a package only when a cut brings it. + found += [pkg for pkg in tagged if show(rev, pkg.manifest.path) is not None] return found -def shipped(pkg: Package, head: str) -> str | None: - """The oldest first-parent commit since which `pkg` has had its current version.""" +def released(pkg: Package, head: str) -> str | None: + """The commit that shipped `pkg`'s current version, or None while it is unreleased.""" current = pkg.version(head) if current is None: raise ValueError(f"no version in {pkg.manifest.path} at {head}") + if pkg.tag: + result = git("rev-parse", "--quiet", "--verify", f"refs/tags/{pkg.tag}{current}^{{commit}}", check=False) + return result.stdout.strip() or None + # The oldest first-parent commit since which it has had its current version. log = git("log", "--first-parent", "--format=%H", head, "--", pkg.manifest.path).stdout.split() for rev in log: if pkg.version(f"{rev}^") != current: @@ -157,19 +214,86 @@ class Bump: new: str since: str changed: list[str] + # The package it releases alongside, when its own sources did not change. + alongside: str | None = None def plan(head: str) -> list[Bump]: - bumps = [] - for pkg in packages(head): - since = shipped(pkg, head) - if since is None: + found = packages(head) + since = {pkg.name: released(pkg, head) for pkg in found} + bumps: dict[str, Bump] = {} + for pkg in found: + if since[pkg.name] is None: continue - changed = git("diff", "--name-only", since, head, "--", *pkg.paths, *UNSHIPPED).stdout.split() + changed = git("diff", "--name-only", since[pkg.name], head, "--", *pkg.paths, *UNSHIPPED).stdout.split() if changed: old = pkg.version(head) - bumps.append(Bump(pkg, old, patch(old), since, changed)) - return bumps + bumps[pkg.name] = Bump(pkg, old, patch(old), since[pkg.name], changed) + + # One tag publishes a package and every package riding it, so once any of + # them releases, each other one still at a released version needs a bump. + for rider in found: + group = [pkg for pkg in found if pkg.name == rider.rides] + if not group: + continue + group.append(rider) + releasing = [pkg.name for pkg in group if pkg.name in bumps or since[pkg.name] is None] + if not releasing: + continue + for pkg in group: + if pkg.name not in bumps and since[pkg.name] is not None: + old = pkg.version(head) + bumps[pkg.name] = Bump(pkg, old, patch(old), since[pkg.name], [], alongside=releasing[0]) + return [bumps[pkg.name] for pkg in found if pkg.name in bumps] + + +def release_assets(tag: str) -> set[str]: + """The asset names on the GitHub release for `tag`, empty when it has none.""" + result = subprocess.run(["gh", "release", "view", tag, "--repo", REPO, "--json", "assets", "--jq", ".assets[].name"], + capture_output=True, text=True) + if result.returncode: + if "release not found" in result.stderr.lower(): + return set() + raise RuntimeError(result.stderr) + return set(result.stdout.split()) + + +def ffi_blocker(head: str) -> str | None: + """Why a C or C++ tag on `head` must wait for moq-ffi, or None once its pin is published.""" + text = show(head, FFI_VERSION) or "" + pins = [line.strip() for line in text.splitlines() if line.strip() and not line.strip().startswith("#")] + if len(pins) != 1: + return f"{FFI_VERSION} pins no single moq-ffi release" + release = pins[0] + hashes = json.loads(show(head, FFI_SHA256) or "{}") + published = release_assets(f"moq-ffi-v{release}") + missing = [asset for asset in (ffi.name(release, target) + ".tar.gz" for target in ffi.TARGETS) + if asset not in hashes or asset not in published] + if missing: + return f"moq-ffi v{release} lacks the published, pinned {', '.join(missing)}" + return None + + +@dataclass +class Tag: + name: str + package: Package + # Why it cannot be pushed yet, or None. + blocker: str | None + + +def pending(head: str) -> list[Tag]: + """Each release tag HEAD's versions lack, for the packages that push their own.""" + tags = [] + blocker: str | None = None + checked = False + for pkg in packages(head): + if not pkg.tag or pkg.rides or released(pkg, head) is not None: + continue + if pkg.ffi and not checked: + blocker, checked = ffi_blocker(head), True + tags.append(Tag(f"{pkg.tag}{pkg.version(head)}", pkg, blocker if pkg.ffi else None)) + return tags def apply(bumps: list[Bump], root: Path) -> None: @@ -197,7 +321,16 @@ def body(bumps: list[Bump], head: str) -> str: for bump in bumps: files = "file" if len(bump.changed) == 1 else "files" compare = f"[{len(bump.changed)} {files}](https://github.com/{REPO}/compare/{bump.since[:10]}...{head[:10]})" + if bump.alongside: + compare = f"none; releases with {bump.alongside}" lines.append(f"| {bump.package.name} | {bump.old} → {bump.new} | {compare} |") + tags = [f"`{b.package.tag}{b.new}`" for b in bumps if b.package.tag and not b.package.rides] + if tags: + lines += [ + "", + f"Once this merges, the Release bump workflow pushes {', '.join(tags)} to publish them; " + "a C or C++ tag waits until the moq-ffi release `cpp/moq/FFI_VERSION` pins is published.", + ] lines += [ "", "Each bump is a patch, as a backport keeping the published API needs. " @@ -247,6 +380,20 @@ def pr(root: Path) -> None: print(f"opened {url}: {names}") +def push_tags() -> None: + head = git("rev-parse", "HEAD").stdout.strip() + tags = pending(head) + for tag in tags: + if tag.blocker: + # The next push to `release` retries it. + print(f"waiting to push {tag.name}: {tag.blocker}") + continue + git("push", "origin", f"{head}:refs/tags/{tag.name}") + print(f"pushed {tag.name} at {head[:10]}") + if not tags: + print("nothing to tag") + + def main() -> int: parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) sub = parser.add_subparsers(dest="command", required=True) @@ -254,19 +401,25 @@ def main() -> int: check.add_argument("--ref", default="HEAD") sub.add_parser("apply") sub.add_parser("pr") + sub.add_parser("tag") args = parser.parse_args() root = Path(git("rev-parse", "--show-toplevel").stdout.strip()) if args.command == "check": bumps = plan(args.ref) for bump in bumps: - print(f"{bump.package.name}: {bump.old} -> {bump.new} ({len(bump.changed)} files since {bump.since[:10]})") + reason = f"releases with {bump.alongside}" if bump.alongside else f"{len(bump.changed)} files" + print(f"{bump.package.name}: {bump.old} -> {bump.new} ({reason} since {bump.since[:10]})") for path in bump.changed: print(f" {path}") + for tag in pending(args.ref): + print(f"{tag.package.name}: tag {tag.name}" + (f" (waiting: {tag.blocker})" if tag.blocker else "")) if not bumps: print("nothing to bump") elif args.command == "apply": apply(plan("HEAD"), root) + elif args.command == "tag": + push_tags() else: pr(root) return 0 diff --git a/sh/gh/bump.test.py b/sh/gh/bump.test.py index af6b790e109..1bfd2886568 100644 --- a/sh/gh/bump.test.py +++ b/sh/gh/bump.test.py @@ -7,6 +7,7 @@ import subprocess import tempfile import unittest +from unittest import mock spec = importlib.util.spec_from_file_location("bump", Path(__file__).with_name("bump.py")) bump = importlib.util.module_from_spec(spec) @@ -78,8 +79,21 @@ def setUp(self): "swift/Sources/Moq/Moq.swift": "", "kt/gradle.properties": "moqffi.version=0.0.0-dev\nmoq.version=0.4.0\n", "kt/moq/src/jvmMain/Moq.kt": "", + "dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.0\n", + "dart/moq/lib/moq.dart": "", + "c/moq/VERSION": "0.8.0\n", + "c/moq/src/moq.c": "", + "cpp/moq/VERSION": "0.1.0\n", + "cpp/moq/src/moq.cpp": "", + "cpp/moq/FFI_VERSION": "# Unset until published.\n", + "cpp/moq/FFI_SHA256.json": "{}\n", + "cpp/obs/VERSION": "0.7.0\n", + "cpp/obs/src/plugin.cpp": "", }) self.commit("initial") + # Every tag-published package has released its current version. + for tag in ("moq-dart-v0.1.0", "c-v0.8.0", "cpp-v0.1.0", "obs-moq-v0.7.0"): + self.git("tag", tag) def tearDown(self): os.chdir(self.cwd) @@ -103,6 +117,19 @@ def commit(self, message: str, files: dict[str, str] | None = None) -> str: def planned(self) -> dict[str, str]: return {b.package.name: b.new for b in bump.plan("HEAD")} + def pending(self, assets: set[str] = frozenset()) -> dict[str, str | None]: + with mock.patch.object(bump, "release_assets", return_value=assets): + return {t.name: t.blocker for t in bump.pending("HEAD")} + + def pin(self, version: str) -> set[str]: + """Pin the C and C++ packages to moq-ffi `version`, returning its asset names.""" + assets = {bump.ffi.name(version, target) + ".tar.gz" for target in bump.ffi.TARGETS} + self.commit("chore: pin moq-ffi", { + "cpp/moq/FFI_VERSION": f"# Pinned.\n{version}\n", + "cpp/moq/FFI_SHA256.json": json.dumps({asset: "0" * 64 for asset in assets}) + "\n", + }) + return assets + def test_nothing_changed(self): self.assertEqual(self.planned(), {}) @@ -180,6 +207,78 @@ def test_apply_refuses_a_stale_lockfile(self): bump.apply(bump.plan("HEAD"), self.root) self.assertEqual(self.git("status", "--porcelain"), "") + def test_tagged_package_bumps_after_a_change_since_its_tag(self): + self.commit("test(dart): more tests", {"dart/moq/test/moq_test.dart": ""}) + self.assertEqual(self.planned(), {}) + self.commit("fix(dart): a backport", {"dart/moq/lib/moq.dart": "// fix\n"}) + self.assertEqual(self.planned(), {"moq (Dart)": "0.1.1"}) + self.assertEqual(self.pending(), {}) + + def test_untagged_version_awaits_its_tag_instead_of_a_bump(self): + self.commit("chore: bump dart", {"dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.1\n"}) + self.commit("fix(dart): after the bump", {"dart/moq/lib/moq.dart": "// fix\n"}) + self.assertEqual(self.planned(), {}) + self.assertEqual(self.pending(), {"moq-dart-v0.1.1": None}) + + def test_cpp_and_obs_release_together(self): + self.commit("fix(obs): plugin only", {"cpp/obs/src/plugin.cpp": "// fix\n"}) + self.assertEqual(self.planned(), {"moq (C++)": "0.1.1", "obs-moq (OBS)": "0.7.1"}) + alongside = {b.package.name: b.alongside for b in bump.plan("HEAD")} + self.assertEqual(alongside, {"moq (C++)": "obs-moq (OBS)", "obs-moq (OBS)": None}) + + def test_cpp_release_bumps_obs(self): + self.commit("fix(cpp): library only", {"cpp/moq/src/moq.cpp": "// fix\n"}) + self.assertEqual(self.planned(), {"moq (C++)": "0.1.1", "obs-moq (OBS)": "0.7.1"}) + + def test_untagged_cpp_bumps_a_released_obs(self): + self.commit("chore: bump cpp", {"cpp/moq/VERSION": "0.2.0\n"}) + self.assertEqual(self.planned(), {"obs-moq (OBS)": "0.7.1"}) + + def test_obs_never_pushes_its_own_tag(self): + self.commit("chore: bump obs", {"cpp/obs/VERSION": "0.8.0\n"}) + self.assertEqual(self.planned(), {"moq (C++)": "0.1.1"}) + self.assertEqual(self.pending(), {}) + + def test_ffi_pin_change_releases_c_and_cpp(self): + self.pin("0.4.20") + self.assertEqual(self.planned(), {"moq (C)": "0.8.1", "moq (C++)": "0.1.1", "obs-moq (OBS)": "0.7.1"}) + + def test_c_and_cpp_tags_wait_for_the_pinned_ffi_release(self): + self.commit("chore: bump", { + "dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.1\n", + "c/moq/VERSION": "0.8.1\n", + "cpp/moq/VERSION": "0.1.1\n", + }) + self.assertEqual(self.pending(), { + "moq-dart-v0.1.1": None, + "c-v0.8.1": "cpp/moq/FFI_VERSION pins no single moq-ffi release", + "cpp-v0.1.1": "cpp/moq/FFI_VERSION pins no single moq-ffi release", + }) + assets = self.pin("0.4.20") + missing = sorted(assets)[-1] + blocked = self.pending(assets - {missing}) + self.assertIn(missing, blocked["c-v0.8.1"]) + self.assertEqual(self.pending(assets), {"moq-dart-v0.1.1": None, "c-v0.8.1": None, "cpp-v0.1.1": None}) + + def test_tag_pushes_only_unblocked_tags(self): + remote_dir = tempfile.TemporaryDirectory() + self.addCleanup(remote_dir.cleanup) + remote = Path(remote_dir.name) + subprocess.run(["git", "init", "--quiet", "--bare", str(remote)], check=True) + self.git("remote", "add", "origin", str(remote)) + head = self.commit("chore: bump", {"dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.1\n", "c/moq/VERSION": "0.8.1\n"}) + with mock.patch.object(bump, "release_assets", return_value=set()): + bump.push_tags() + pushed = subprocess.run(["git", "ls-remote", "--tags", str(remote)], capture_output=True, text=True, check=True) + self.assertEqual(pushed.stdout, f"{head}\trefs/tags/moq-dart-v0.1.1\n") + + def test_missing_tagged_packages_are_skipped(self): + # A `release` before the cut that brings C, C++, and OBS. + self.git("rm", "--quiet", "-r", "c", "cpp") + self.commit("chore: drop c and cpp") + self.assertEqual(self.planned(), {}) + self.assertEqual(self.pending(), {}) + def test_patch(self): self.assertEqual(bump.patch("0.9.9"), "0.9.10") with self.assertRaises(ValueError): From 94d73bc2394b7ade522d17fb69108b12e17bde32 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 08:45:38 -0700 Subject: [PATCH 04/12] fix(gh): hold a C++ tag until OBS is bumped, load ffi.py lazily A C++ tag pushed while OBS still sat at a released version would publish that plugin version again, which Release Cpp refuses, and the tag cannot be retracted. `tag` now waits until the bump PR moves OBS. sh/cpp/ffi.py also loads only once C or C++ is present, since `release` before its cut has neither, so the tooling-only backport needs only the bump files. Co-Authored-By: Claude Opus 5.5 --- sh/gh/bump.py | 34 +++++++++++++++++++++++++--------- sh/gh/bump.test.py | 20 +++++++++++++++----- 2 files changed, 40 insertions(+), 14 deletions(-) diff --git a/sh/gh/bump.py b/sh/gh/bump.py index c97af38af03..960b1907bb0 100755 --- a/sh/gh/bump.py +++ b/sh/gh/bump.py @@ -43,12 +43,18 @@ TITLE = "chore: patch-bump hand-versioned packages" REPO = "moq-dev/moq" -# The C and C++ packages' moq-ffi pin, and its asset names. +# The C and C++ packages' moq-ffi pin. FFI_VERSION = "cpp/moq/FFI_VERSION" FFI_SHA256 = "cpp/moq/FFI_SHA256.json" -_spec = importlib.util.spec_from_file_location("ffi", Path(__file__).parents[1] / "cpp/ffi.py") -ffi = importlib.util.module_from_spec(_spec) -_spec.loader.exec_module(ffi) + + +def load_ffi(): + """sh/cpp/ffi.py, which names the pinned assets. Loaded only once C or C++ is + present, since a `release` from before their cut lacks it.""" + spec = importlib.util.spec_from_file_location("ffi", Path(__file__).parents[1] / "cpp/ffi.py") + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module def git(*args: str, check: bool = True) -> subprocess.CompletedProcess: @@ -265,6 +271,7 @@ def ffi_blocker(head: str) -> str | None: if len(pins) != 1: return f"{FFI_VERSION} pins no single moq-ffi release" release = pins[0] + ffi = load_ffi() hashes = json.loads(show(head, FFI_SHA256) or "{}") published = release_assets(f"moq-ffi-v{release}") missing = [asset for asset in (ffi.name(release, target) + ".tar.gz" for target in ffi.TARGETS) @@ -284,15 +291,24 @@ class Tag: def pending(head: str) -> list[Tag]: """Each release tag HEAD's versions lack, for the packages that push their own.""" + found = packages(head) tags = [] - blocker: str | None = None + ffi: str | None = None checked = False - for pkg in packages(head): + for pkg in found: if not pkg.tag or pkg.rides or released(pkg, head) is not None: continue - if pkg.ffi and not checked: - blocker, checked = ffi_blocker(head), True - tags.append(Tag(f"{pkg.tag}{pkg.version(head)}", pkg, blocker if pkg.ffi else None)) + blocker = None + # A rider still at a released version needs its bump merged first: the + # tag would publish that version again, which its release refuses. + stale = [rider for rider in found if rider.rides == pkg.name and released(rider, head) is not None] + if stale: + blocker = f"{stale[0].name} {stale[0].version(head)} is already released; merge its bump first" + elif pkg.ffi: + if not checked: + ffi, checked = ffi_blocker(head), True + blocker = ffi + tags.append(Tag(f"{pkg.tag}{pkg.version(head)}", pkg, blocker)) return tags diff --git a/sh/gh/bump.test.py b/sh/gh/bump.test.py index 1bfd2886568..a0a71536734 100644 --- a/sh/gh/bump.test.py +++ b/sh/gh/bump.test.py @@ -123,7 +123,8 @@ def pending(self, assets: set[str] = frozenset()) -> dict[str, str | None]: def pin(self, version: str) -> set[str]: """Pin the C and C++ packages to moq-ffi `version`, returning its asset names.""" - assets = {bump.ffi.name(version, target) + ".tar.gz" for target in bump.ffi.TARGETS} + ffi = bump.load_ffi() + assets = {ffi.name(version, target) + ".tar.gz" for target in ffi.TARGETS} self.commit("chore: pin moq-ffi", { "cpp/moq/FFI_VERSION": f"# Pinned.\n{version}\n", "cpp/moq/FFI_SHA256.json": json.dumps({asset: "0" * 64 for asset in assets}) + "\n", @@ -230,9 +231,15 @@ def test_cpp_release_bumps_obs(self): self.commit("fix(cpp): library only", {"cpp/moq/src/moq.cpp": "// fix\n"}) self.assertEqual(self.planned(), {"moq (C++)": "0.1.1", "obs-moq (OBS)": "0.7.1"}) - def test_untagged_cpp_bumps_a_released_obs(self): + def test_untagged_cpp_bumps_a_released_obs_before_its_tag(self): + assets = self.pin("0.4.20") self.commit("chore: bump cpp", {"cpp/moq/VERSION": "0.2.0\n"}) - self.assertEqual(self.planned(), {"obs-moq (OBS)": "0.7.1"}) + self.assertEqual(self.planned(), {"moq (C)": "0.8.1", "obs-moq (OBS)": "0.7.1"}) + # Tagging now would publish OBS 0.7.0 again. + self.assertIn("obs-moq (OBS) 0.7.0 is already released", self.pending(assets)["cpp-v0.2.0"]) + bump.apply(bump.plan("HEAD"), self.root) + self.commit("chore: bump") + self.assertEqual(self.pending(assets), {"c-v0.8.1": None, "cpp-v0.2.0": None}) def test_obs_never_pushes_its_own_tag(self): self.commit("chore: bump obs", {"cpp/obs/VERSION": "0.8.0\n"}) @@ -248,6 +255,7 @@ def test_c_and_cpp_tags_wait_for_the_pinned_ffi_release(self): "dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.1\n", "c/moq/VERSION": "0.8.1\n", "cpp/moq/VERSION": "0.1.1\n", + "cpp/obs/VERSION": "0.7.1\n", }) self.assertEqual(self.pending(), { "moq-dart-v0.1.1": None, @@ -276,8 +284,10 @@ def test_missing_tagged_packages_are_skipped(self): # A `release` before the cut that brings C, C++, and OBS. self.git("rm", "--quiet", "-r", "c", "cpp") self.commit("chore: drop c and cpp") - self.assertEqual(self.planned(), {}) - self.assertEqual(self.pending(), {}) + # That `release` lacks sh/cpp/ffi.py too. + with mock.patch.object(bump, "load_ffi", side_effect=AssertionError("loaded ffi.py")): + self.assertEqual(self.planned(), {}) + self.assertEqual(self.pending(), {}) def test_patch(self): self.assertEqual(bump.patch("0.9.9"), "0.9.10") From f128621d63d202ceae9d53f6d624453b69ffde06 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 09:17:55 -0700 Subject: [PATCH 05/12] fix(gh): treat OBS as shipped while its C++ tag's release runs Release Cpp cuts the obs-moq tag only after the plugin builds, so a push in between saw OBS untagged and asked for another C++ bump. A rider whose version matches its host's tag now counts as released at that tag. Co-Authored-By: Claude Opus 5.5 --- sh/gh/bump.py | 19 ++++++++++++++++--- sh/gh/bump.test.py | 10 ++++++++++ 2 files changed, 26 insertions(+), 3 deletions(-) diff --git a/sh/gh/bump.py b/sh/gh/bump.py index 960b1907bb0..8c1539163d4 100755 --- a/sh/gh/bump.py +++ b/sh/gh/bump.py @@ -224,9 +224,21 @@ class Bump: alongside: str | None = None +def baselines(found: list[Package], head: str) -> dict[str, str | None]: + """`released` for each package in `found`.""" + since = {pkg.name: released(pkg, head) for pkg in found} + # A rider's own tag comes later, from the release its host's tag starts, so + # while that run is in flight its version shipped at the host's tag. + for rider in found: + host = since.get(rider.rides) if rider.rides else None + if since[rider.name] is None and host and rider.version(host) == rider.version(head): + since[rider.name] = host + return since + + def plan(head: str) -> list[Bump]: found = packages(head) - since = {pkg.name: released(pkg, head) for pkg in found} + since = baselines(found, head) bumps: dict[str, Bump] = {} for pkg in found: if since[pkg.name] is None: @@ -292,16 +304,17 @@ class Tag: def pending(head: str) -> list[Tag]: """Each release tag HEAD's versions lack, for the packages that push their own.""" found = packages(head) + since = baselines(found, head) tags = [] ffi: str | None = None checked = False for pkg in found: - if not pkg.tag or pkg.rides or released(pkg, head) is not None: + if not pkg.tag or pkg.rides or since[pkg.name] is not None: continue blocker = None # A rider still at a released version needs its bump merged first: the # tag would publish that version again, which its release refuses. - stale = [rider for rider in found if rider.rides == pkg.name and released(rider, head) is not None] + stale = [rider for rider in found if rider.rides == pkg.name and since[rider.name] is not None] if stale: blocker = f"{stale[0].name} {stale[0].version(head)} is already released; merge its bump first" elif pkg.ffi: diff --git a/sh/gh/bump.test.py b/sh/gh/bump.test.py index a0a71536734..25a3418c0d1 100644 --- a/sh/gh/bump.test.py +++ b/sh/gh/bump.test.py @@ -241,6 +241,16 @@ def test_untagged_cpp_bumps_a_released_obs_before_its_tag(self): self.commit("chore: bump") self.assertEqual(self.pending(assets), {"c-v0.8.1": None, "cpp-v0.2.0": None}) + def test_obs_in_flight_from_a_cpp_tag_is_released(self): + self.commit("chore: bump", {"cpp/moq/VERSION": "0.1.1\n", "cpp/obs/VERSION": "0.7.1\n"}) + # Release Cpp has not cut obs-moq-v0.7.1 yet. + self.git("tag", "cpp-v0.1.1") + self.commit("docs: unrelated", {"README.md": ""}) + self.assertEqual(self.planned(), {}) + self.assertEqual(self.pending(), {}) + self.commit("fix(obs): plugin only", {"cpp/obs/src/plugin.cpp": "// fix\n"}) + self.assertEqual(self.planned(), {"moq (C++)": "0.1.2", "obs-moq (OBS)": "0.7.2"}) + def test_obs_never_pushes_its_own_tag(self): self.commit("chore: bump obs", {"cpp/obs/VERSION": "0.8.0\n"}) self.assertEqual(self.planned(), {"moq (C++)": "0.1.1"}) From 099b017090fde323e74d1361c4e47e7f9a9215b8 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 10:04:11 -0700 Subject: [PATCH 06/12] refactor(gh): check release backports bump versions instead of bumping them The maintainer reversed the bump bot: an automated bump is only safe if it understands semver, and nothing here can judge it for the hand-written wrappers. So no workflow picks or writes a version any more. - sh/gh/bump.py becomes sh/gh/versions.py. `check` merges a PR's head into `release` and fails when a hand-versioned package's shipped sources change at a released version, a version goes backwards, or its bun.lock/uv.lock entry is left behind. C++ and OBS must move together. It reuses the bot's detection, never edits, and only reports on the cut. - The Release gate runs it as a new Versions job. - release-bump.yml becomes release-tag.yml, which only pushes the missing moq-dart-v/c-v/cpp-v tags for versions a human set, with the same moq-ffi and OBS guards. - CONTRIBUTING.md and the m2 Dart and C++ release quests describe the manual bump. The quest stays, rewritten to the remaining release-side work. Co-Authored-By: Claude Opus 5.5 --- .github/justfile | 8 +- .github/workflows/alert.yml | 2 +- .github/workflows/release-bump.yml | 65 ------ .github/workflows/release-gate.yml | 30 +++ .github/workflows/release-tag.yml | 52 +++++ CONTRIBUTING.md | 6 +- quest/m1/README.md | 1 + quest/m1/release-versions.md | 61 +++++ quest/m2/release/cpp-release.md | 16 +- quest/m2/release/dart-publish.md | 2 +- sh/gh/bump.test.py | 309 -------------------------- sh/gh/{bump.py => versions.py} | 323 +++++++++++++-------------- sh/gh/versions.test.py | 344 +++++++++++++++++++++++++++++ 13 files changed, 660 insertions(+), 559 deletions(-) delete mode 100644 .github/workflows/release-bump.yml create mode 100644 .github/workflows/release-tag.yml create mode 100644 quest/m1/release-versions.md delete mode 100644 sh/gh/bump.test.py rename sh/gh/{bump.py => versions.py} (60%) create mode 100644 sh/gh/versions.test.py diff --git a/.github/justfile b/.github/justfile index 88170c6366b..9c94a6b6240 100644 --- a/.github/justfile +++ b/.github/justfile @@ -49,7 +49,7 @@ test: bun sh/gh/queue.test.ts python3 sh/gh/install-formula.test.py sh/gh/release.test.sh - python3 sh/gh/bump.test.py + python3 sh/gh/versions.test.py python3 sh/gh/ffi.test.py bun test sh/gh/ffi.test.ts sh/gh/cpp.test.ts python3 sh/rs/package-rename.test.py @@ -71,10 +71,10 @@ test: release *args: sh/gh/release.sh "$@" -# Patch-bump and tag hand-versioned packages with unreleased changes: `just gh bump check|apply|pr|tag`. +# Check a `release` PR bumps the hand-versioned packages it changes, or push their release tags: `just gh versions check|tag`. [positional-arguments] -bump *args: - python3 sh/gh/bump.py "$@" +versions *args: + python3 sh/gh/versions.py "$@" # Post a failed run to Discord, or check cron and publishing alert coverage. [positional-arguments] diff --git a/.github/workflows/alert.yml b/.github/workflows/alert.yml index ac323e62dbc..7dc96dc8661 100644 --- a/.github/workflows/alert.yml +++ b/.github/workflows/alert.yml @@ -22,7 +22,6 @@ on: - moq.sh - Nightly - release-brew - - Release bump - Release C - Release Cpp - Release Dart @@ -39,6 +38,7 @@ on: - Release RS - Release Swift FFI - Release Swift Lib + - Release tag - release-winget - rpm-repo - Smoke diff --git a/.github/workflows/release-bump.yml b/.github/workflows/release-bump.yml deleted file mode 100644 index 82c542570fc..00000000000 --- a/.github/workflows/release-bump.yml +++ /dev/null @@ -1,65 +0,0 @@ -name: Release bump - -# release-plz bumps Rust crates on `release`. This opens one PR that -# patch-bumps every other package whose version a human owns and whose sources -# changed since that version shipped, so a backport publishes once the PR -# merges. For packages a tag publishes (Dart, C, C++), it then pushes each tag -# the versions on `release` lack. See sh/gh/bump.py for which packages it -# covers. -permissions: - contents: read - -on: - # Every push, not only those touching a package: a C or C++ tag waiting on - # its pinned moq-ffi release retries on the next one. - push: - branches: [release] - # Regenerating the PR, or retrying a tag, by hand. Dispatch it on `release`; - # the job refuses any other ref. - workflow_dispatch: - -# Each run rebuilds the PR from the pushed `release`, so a newer push makes an -# in-flight run moot. -concurrency: - group: release-bump - cancel-in-progress: true - -jobs: - bump: - name: Bump - runs-on: ubuntu-latest - if: ${{ github.repository_owner == 'moq-dev' && github.ref == 'refs/heads/release' }} - - steps: - # The app's token, as release-plz uses, so the PR it opens runs the - # Release gate and the tags it pushes run their release workflows. - - name: Generate GitHub token - uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3 - id: generate-token - with: - client-id: ${{ secrets.APP_CLIENT_ID }} - private-key: ${{ secrets.APP_PRIVATE_KEY }} - - - name: Checkout repository - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - fetch-depth: 0 - token: ${{ steps.generate-token.outputs.token }} - - - uses: ./.github/actions/just - - # Before the PR step, which moves HEAD to the bump commit. A version the - # bump PR raises is tagged on the push its merge makes. - - name: Push release tags - env: - GH_TOKEN: ${{ steps.generate-token.outputs.token }} - run: just gh bump tag - - - name: Open or update the bump PR - env: - GH_TOKEN: ${{ steps.generate-token.outputs.token }} - GIT_AUTHOR_NAME: moq-bot - GIT_AUTHOR_EMAIL: moq-bot[bot]@users.noreply.github.com - GIT_COMMITTER_NAME: moq-bot - GIT_COMMITTER_EMAIL: moq-bot[bot]@users.noreply.github.com - run: just gh bump pr diff --git a/.github/workflows/release-gate.yml b/.github/workflows/release-gate.yml index f8d91b7ef38..77822aca274 100644 --- a/.github/workflows/release-gate.yml +++ b/.github/workflows/release-gate.yml @@ -336,6 +336,35 @@ jobs: uses: ./.github/workflows/features.yml secrets: inherit + # A backport must bump each hand-versioned package whose shipped sources it + # changes, or its release workflow skips the fix as already published. Only a + # human picks the version, so this checks and never edits. The cut (a PR from + # `main`) only reports: it carries changes that need no release, which only + # the maintainer judges. + versions: + name: Versions + if: github.event.action != 'closed' + runs-on: ubuntu-24.04-arm + timeout-minutes: 10 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + # `release`'s first-parent history and the release tags say which + # version of each package shipped. + fetch-depth: 0 + - uses: ./.github/actions/just + - name: Check versions + run: | + mode=() + # Only this repository's own `main` is the cut; a fork's `main` is a backport. + [[ "$HEAD_REF" != main || "$HEAD_REPO" != "$GITHUB_REPOSITORY" ]] || mode=(--report) + just gh versions check --base "origin/$BASE_REF" "${mode[@]}" + env: + BASE_REF: ${{ github.base_ref }} + HEAD_REF: ${{ github.head_ref }} + HEAD_REPO: ${{ github.event.pull_request.head.repo.full_name }} + android: name: Android needs: scope @@ -505,6 +534,7 @@ jobs: - scope - platform - features + - versions - android - api-diff - audio-quality diff --git a/.github/workflows/release-tag.yml b/.github/workflows/release-tag.yml new file mode 100644 index 00000000000..d3db8fcf0a5 --- /dev/null +++ b/.github/workflows/release-tag.yml @@ -0,0 +1,52 @@ +name: Release tag + +# Dart, C, and C++ publish from a tag rather than on a push to `release`. This +# pushes each such tag a version on `release` lacks, so a version a human set +# there publishes once it lands; it never picks or writes a version. C and C++ +# tags wait for the moq-ffi release their pin names. See sh/gh/versions.py. +permissions: + contents: read + +on: + # Every push, not only those touching a package: a C or C++ tag waiting on + # its pinned moq-ffi release retries on the next one. + push: + branches: [release] + # Retrying a tag by hand. Dispatch it on `release`; the job refuses any other + # ref. + workflow_dispatch: + +# Each run tags the pushed `release`, so a newer push makes an in-flight run +# moot. +concurrency: + group: release-tag + cancel-in-progress: true + +jobs: + tag: + name: Tag + runs-on: ubuntu-latest + if: ${{ github.repository_owner == 'moq-dev' && github.ref == 'refs/heads/release' }} + + steps: + # The app's token, as release-plz uses, so the tags it pushes run their + # release workflows. + - name: Generate GitHub token + uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3 + id: generate-token + with: + client-id: ${{ secrets.APP_CLIENT_ID }} + private-key: ${{ secrets.APP_PRIVATE_KEY }} + + - name: Checkout repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 0 + token: ${{ steps.generate-token.outputs.token }} + + - uses: ./.github/actions/just + + - name: Push release tags + env: + GH_TOKEN: ${{ steps.generate-token.outputs.token }} + run: just gh versions tag diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4e7194f8696..46bf2ccd154 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -96,13 +96,13 @@ For an advisory against Quinn, follow the advisory triage in [`rs/moq-quic/READM - Smoke clients call only the latest published APIs. A release cut updates `test/smoke/clients` to the APIs being published, then verifies the affected smoke lanes. - A release is cut by hand: a PR merging `main` into `release`, with a merge commit. - A breaking cut first writes, on `main`, each changed package's version-transition section in its `MIGRATE.md` (`## 0.3 -> 0.4`) from the migration audit: the `!` subjects since the `release` merge base, each of those PRs' Impact section, and the API diff. -- Suggest a `release` backport for any serious bug fix (crash, security, data loss, broken interop), whether found on `main` or reported against a release. Land it on `main` first, then port it onto `release` as a separate PR that keeps the published API, so it ships as a patch. The Release gate's API diff (`just api-diff --deny`) fails a backport that changes one; on the cut PR, whose head is `main`, it only reports. Don't bump versions in it: release-plz bumps the crates, and the Release bump workflow opens a `release-bump` PR patch-bumping every other hand-versioned package whose sources changed, then pushes the Dart, C, and C++ tags those bumps need; merging both publishes the fix. +- Suggest a `release` backport for any serious bug fix (crash, security, data loss, broken interop), whether found on `main` or reported against a release. Land it on `main` first, then port it onto `release` as a separate PR that keeps the published API, so it ships as a patch. The Release gate's API diff (`just api-diff --deny`) fails a backport that changes one; on the cut PR, whose head is `main`, it only reports. Leave crate versions to release-plz, but bump every other package whose shipped sources it changes, picking the level by hand (see [Versions](#versions)); the Release gate's version check (`just gh versions check`) fails a backport that leaves one at its released version, since its release workflow would skip the fix as already published. - After any release publishes, patch or breaking, run `just test wire-compat --remeasure` on `main` and land the rewritten `test/interop/compat/planned-breaks.json` with refreshed reasons. Otherwise the nightly wire-compat check fails on the stale pins. - Nothing merges `release` into `main` automatically. Before cutting a minor or breaking release, the maintainer merges `release` into `main` once, so trunk carries the published versions, CHANGELOGs, and any `release`-only commits. It lands as a merge commit with `--admin`, never squashed, so the cut starts from an advanced merge base. # Versions -Releases are cut separately; bump only when asked. On `release`, the Release bump workflow (`just gh bump check` previews it) keeps one `release-bump` PR patch-bumping every hand-versioned package whose sources changed since its current version shipped, with the matching lockfile entries. A JavaScript, Python, Swift, or Kotlin version ships when it first reaches `release`. A Dart, C, or C++ version ships when its tag is pushed, and the workflow pushes the tag for each such version on `release` that lacks one; C and C++ wait until the moq-ffi release `cpp/moq/FFI_VERSION` pins is published, retrying on each later push. Go derives its own patch. Each package's version lives in one place: +Releases are cut separately; bump only when asked, or in a `release` backport. No tool picks a version: a human judges the semver level. On a PR into `release`, the Release gate's version check fails a backport that changes a hand-versioned package's shipped sources while leaving it at a released version, lowers a version, or leaves its lockfile entry behind; on the cut it only reports. A JavaScript, Python, Swift, or Kotlin version ships when it first reaches `release`. A Dart, C, or C++ version ships when its tag is pushed, and the Release tag workflow pushes the tag for each such version on `release` that lacks one; C and C++ wait until the moq-ffi release `cpp/moq/FFI_VERSION` pins is published, retrying on each later push. Go derives its own patch. Each package's version lives in one place: - **Rust**: release-plz owns crate versions and Rust dependency requirements. - **JavaScript**: `js/*/package.json` packages with a `scripts.release` entry (skip private ones like `@moq/wasm`), plus the matching workspace version in `bun.lock`. @@ -111,4 +111,4 @@ Releases are cut separately; bump only when asked. On `release`, the Release bum - **Go**: `go/wrapper/VERSION` holds a human-owned `MAJOR.MINOR` line; CI derives the patch, so only edit it for a breaking API. Leave the placeholder FFI version in `go.mod` alone. - **C**: `c/moq/VERSION`, human-owned; a `c-v` tag releases it. - **C++**: `cpp/moq/VERSION`, human-owned; a `cpp-v` tag releases it. -- **OBS**: `cpp/obs/VERSION`, human-owned; each C++ release also cuts `obs-moq-v`, and refuses one whose version already names an earlier release. So the Release bump workflow bumps the plugin with every C++ release, and C++ with every plugin change. +- **OBS**: `cpp/obs/VERSION`, human-owned; each C++ release also cuts `obs-moq-v`, and refuses one whose version already names an earlier release. So the version check requires a plugin bump with every C++ release, and a C++ bump with every plugin change, and the Release tag workflow holds a `cpp-v` tag while the plugin is at a released version. diff --git a/quest/m1/README.md b/quest/m1/README.md index 6a0a8b43f63..31528c1cf91 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -178,6 +178,7 @@ removed). - [JS SETUP rejection ownership](/quest/m1/js-setup-rejection.md) - reproduce an unhandled version-refusal rejection without weakening the refusal - [moq-clock moves into kio::clock](/quest/m1/kio-clock.md) - moq-clock and its test macro become `kio::clock` and `#[kio::test]` before the cut publishes them - [Poll-first async](/quest/m1/poll-first/README.md) - library crates compile no coroutines by default: async wrappers are generated from `poll_*` and gated behind an `async` feature +- [Every package's release backport ships](/quest/m1/release-versions.md) - the Release gate fails a backport that leaves a changed package at its released version; backport that check to `release` and hand-bump the fixes already there ## Related diff --git a/quest/m1/release-versions.md b/quest/m1/release-versions.md new file mode 100644 index 00000000000..b2c232446d5 --- /dev/null +++ b/quest/m1/release-versions.md @@ -0,0 +1,61 @@ +# [S] Every package's release backport ships + +## Goal + +A fix merged into `release` for any hand-versioned package publishes. Today +release-plz bumps only Rust crates, so a backport touching only a JS, Python, +Dart, Kotlin, Swift, C, or C++ package can land on `release` unchanged in +version, and its release workflow skips it as already published. On `main`, +the Release gate's Versions check (`just gh versions check`) now fails such a +backport, and the Release tag workflow pushes the Dart, C, and C++ tags for +versions a human set on `release`. Done when both run on `release` and the +fixes `release` already carries unreleased are bumped there. + +## Plan + +Found 2026-10-28: the @moq/flate and @moq/net backports +([moq-dev/moq#6961](https://github.com/moq-dev/moq/pull/6961), +[moq-dev/moq#6959](https://github.com/moq-dev/moq/pull/6959), +[moq-dev/moq#6967](https://github.com/moq-dev/moq/pull/6967)) merged and the +release PR merged, but Release JS logged "already published, skipping" and a +separate hand bump was needed. + +Decided 2026-10-28 (maintainer), superseding the earlier bump bot: "we can +only use an automated bump tool like this if it understands semver. If it +doesn't, we need manual bumps." Nothing here can judge semver, since +`just api-diff --deny` doesn't cover the hand-written wrappers. + +- ✅ Bumps on `release` stay manual; a human picks the level. No workflow or + script picks or writes a version. +- ✅ A Release gate check fails a backport that changes a hand-versioned + package's shipped sources without moving its version past the released one, + or without moving its `bun.lock`/`uv.lock` entry. It names each package and + how to bump it, requires an OBS bump with each C++ release and the reverse, + and never edits. +- ✅ A workflow still pushes the missing `moq-dart-v*`, `c-v*`, and `cpp-v*` + tag for a version a human set on `release`: C and C++ only once the pinned + moq-ffi release is published, and C++ only once OBS is off its released + version. + +Remaining, on `release` (neither publish nor tag by hand here): + +1. Backport the tooling as a tooling-only PR: `sh/gh/versions.py`, + `sh/gh/versions.test.py`, the `.github/justfile` recipe and test line, + `.github/workflows/release-tag.yml`, its `alert.yml` entry, and the + Versions job in `release-gate.yml`. Its own Release gate runs the check. + Its merge is a push, so Release tag pushes `moq-dart-v0.1.1`, which + `release` has carried untagged since the drain backport + ([335e0b5f29](https://github.com/moq-dev/moq/commit/335e0b5f29)); check + that Release Dart publishes it ([dart-publish](/quest/m2/release/dart-publish.md)). +2. Bump by hand the packages `release` already carries unreleased changes for + (`just gh versions check --base origin/release --head origin/release` lists + them): + - @moq/hang 0.5.4, from + [moq-dev/moq#6631](https://github.com/moq-dev/moq/pull/6631). + - @moq/publish 0.5.3, from + [moq-dev/moq#6598](https://github.com/moq-dev/moq/pull/6598). + - @moq/boy 0.4.1, only if swapping its esbuild devDependency for rolldown + ([moq-dev/moq#5354](https://github.com/moq-dev/moq/pull/5354)) changed + what it ships. + +Public API: none. Wire: none. diff --git a/quest/m2/release/cpp-release.md b/quest/m2/release/cpp-release.md index 57495d52031..d11abffa8d0 100644 --- a/quest/m2/release/cpp-release.md +++ b/quest/m2/release/cpp-release.md @@ -25,14 +25,14 @@ The nightly C++ and OBS release checks pass: the which failed only in unrelated lanes, checked the installed consumers of all four packaged archives and built both OBS plugins against them. It packages this run's prebuilt moq-ffi input, while a tag with `cpp/moq/FFI_VERSION` unset builds moq-ffi from source, a -path no nightly checks. The Release bump workflow pushes the `cpp-v` tag -itself on the first push to `release` after the pin names a published -moq-ffi release, and never while it is unset. So if the pin is still unset, -first dispatch `release-cpp.yml` on the ref to be tagged without -`prebuilt-test`: it runs the tag's source build and archive check on all four -hosts, then both OBS builds, and publishes nothing. Then tag it by hand. This -quest owns that dry run, any hand tagging, and verifying the published -downloads. +path no nightly checks. The Release tag workflow pushes the `cpp-v` tag +itself on the first push to `release` once the pin names a published moq-ffi +release and OBS sits at an unreleased version, and never while the pin is +unset. So if the pin is still unset, first dispatch `release-cpp.yml` on the +ref to be tagged without `prebuilt-test`: it runs the tag's source build and +archive check on all four hosts, then both OBS builds, and publishes nothing. +Then tag it by hand. This quest owns that dry run, any hand tagging, and +verifying the published downloads. ## Required diff --git a/quest/m2/release/dart-publish.md b/quest/m2/release/dart-publish.md index b7b113b0870..d17f87c6330 100644 --- a/quest/m2/release/dart-publish.md +++ b/quest/m2/release/dart-publish.md @@ -19,7 +19,7 @@ can proceed independently; cutting the tag cannot. - Confirm trusted publishing is configured for `moq` (repository `moq-dev/moq`, tag pattern `moq-dart-v{{version}}`). - Raise `dart/moq/pubspec.yaml`'s `moq_ffi` floor to the release the wrapper now needs (the media namespace and codec types), since `^0.4.3` resolves bindings without them. -- The Release bump workflow pushes each `moq-dart-v*` tag a version on `release` lacks. Its first run on `release` tags `moq-dart-v0.1.1`, the drain backport ([335e0b5f29](https://github.com/moq-dev/moq/commit/335e0b5f29)); confirm Release Dart publishes it unattended, and that the cut's binding-break release publishes the same way. +- The Release tag workflow pushes each `moq-dart-v*` tag a version on `release` lacks. Its first run on `release` tags `moq-dart-v0.1.1`, the drain backport ([335e0b5f29](https://github.com/moq-dev/moq/commit/335e0b5f29)); confirm Release Dart publishes it unattended, and that the cut's binding-break release publishes the same way. - Verify the published `moq_ffi` resolves its native asset from a clean machine with no monorepo checkout, since that download path is the one CI never exercises. diff --git a/sh/gh/bump.test.py b/sh/gh/bump.test.py deleted file mode 100644 index 25a3418c0d1..00000000000 --- a/sh/gh/bump.test.py +++ /dev/null @@ -1,309 +0,0 @@ -"""Find hand-versioned packages with unreleased changes on a first-parent history.""" - -import importlib.util -import json -import os -from pathlib import Path -import subprocess -import tempfile -import unittest -from unittest import mock - -spec = importlib.util.spec_from_file_location("bump", Path(__file__).with_name("bump.py")) -bump = importlib.util.module_from_spec(spec) -spec.loader.exec_module(bump) - - -def package_json(name: str, version: str, private: bool = False) -> str: - manifest = {"name": name, "version": version, "scripts": {"release": "bun ../common/release.ts"}} - if private: - manifest["private"] = True - return json.dumps(manifest, indent="\t") + "\n" - - -BUN_LOCK = """{ - "workspaces": { - "js/a": { - "name": "@moq/a", - "version": "0.1.0", - }, - "js/b": { - "name": "@moq/b", - "version": "0.2.0", - }, - }, -} -""" - -PYPROJECT = """[build-system] -requires = ["hatchling"] - -[project] -name = "moq-rs" -# Bumped by hand. -version = "0.5.0" -""" - -UV_LOCK = """[[package]] -name = "moq-ffi" -version = "0.4.0" - -[[package]] -name = "moq-rs" -version = "0.5.0" -source = { editable = "py/moq-rs" } -""" - - -class BumpTest(unittest.TestCase): - def setUp(self): - self.dir = tempfile.TemporaryDirectory() - self.root = Path(self.dir.name) - self.cwd = os.getcwd() - os.chdir(self.root) - self.git("init", "--quiet", "--initial-branch", "release") - self.git("config", "user.name", "test") - self.git("config", "user.email", "test@example.com") - self.write({ - "js/a/package.json": package_json("@moq/a", "0.1.0"), - "js/a/src/index.ts": "export {};\n", - "js/b/package.json": package_json("@moq/b", "0.2.0"), - "js/b/src/index.ts": "export {};\n", - "js/wasm/package.json": package_json("@moq/wasm", "0.0.0", private=True), - "js/wasm/src/index.ts": "export {};\n", - "bun.lock": BUN_LOCK, - "py/moq-rs/pyproject.toml": PYPROJECT, - "py/moq-rs/moq/__init__.py": "", - "uv.lock": UV_LOCK, - "swift/VERSION": "0.3.0\n", - "swift/Sources/Moq/Moq.swift": "", - "kt/gradle.properties": "moqffi.version=0.0.0-dev\nmoq.version=0.4.0\n", - "kt/moq/src/jvmMain/Moq.kt": "", - "dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.0\n", - "dart/moq/lib/moq.dart": "", - "c/moq/VERSION": "0.8.0\n", - "c/moq/src/moq.c": "", - "cpp/moq/VERSION": "0.1.0\n", - "cpp/moq/src/moq.cpp": "", - "cpp/moq/FFI_VERSION": "# Unset until published.\n", - "cpp/moq/FFI_SHA256.json": "{}\n", - "cpp/obs/VERSION": "0.7.0\n", - "cpp/obs/src/plugin.cpp": "", - }) - self.commit("initial") - # Every tag-published package has released its current version. - for tag in ("moq-dart-v0.1.0", "c-v0.8.0", "cpp-v0.1.0", "obs-moq-v0.7.0"): - self.git("tag", tag) - - def tearDown(self): - os.chdir(self.cwd) - self.dir.cleanup() - - def git(self, *args: str) -> str: - return subprocess.run(["git", *args], capture_output=True, text=True, check=True).stdout.strip() - - def write(self, files: dict[str, str]): - for path, text in files.items(): - (self.root / path).parent.mkdir(parents=True, exist_ok=True) - (self.root / path).write_text(text) - - def commit(self, message: str, files: dict[str, str] | None = None) -> str: - if files: - self.write(files) - self.git("add", "--all") - self.git("commit", "--quiet", "--message", message) - return self.git("rev-parse", "HEAD") - - def planned(self) -> dict[str, str]: - return {b.package.name: b.new for b in bump.plan("HEAD")} - - def pending(self, assets: set[str] = frozenset()) -> dict[str, str | None]: - with mock.patch.object(bump, "release_assets", return_value=assets): - return {t.name: t.blocker for t in bump.pending("HEAD")} - - def pin(self, version: str) -> set[str]: - """Pin the C and C++ packages to moq-ffi `version`, returning its asset names.""" - ffi = bump.load_ffi() - assets = {ffi.name(version, target) + ".tar.gz" for target in ffi.TARGETS} - self.commit("chore: pin moq-ffi", { - "cpp/moq/FFI_VERSION": f"# Pinned.\n{version}\n", - "cpp/moq/FFI_SHA256.json": json.dumps({asset: "0" * 64 for asset in assets}) + "\n", - }) - return assets - - def test_nothing_changed(self): - self.assertEqual(self.planned(), {}) - - def test_backport_bumps_only_its_package(self): - self.commit("fix(a): a backport", {"js/a/src/index.ts": "export const fix = 1;\n"}) - self.assertEqual(self.planned(), {"@moq/a": "0.1.1"}) - - def test_unshipped_files_are_not_a_release(self): - self.commit("test(a): more tests", { - "js/a/src/index.test.ts": "", - "js/a/api/index.api.md": "", - "js/a/MIGRATE.md": "", - "kt/moq/src/jvmTest/MoqTest.kt": "", - "swift/Tests/MoqTests/MoqTests.swift": "", - }) - self.assertEqual(self.planned(), {}) - - def test_private_package_is_skipped(self): - self.commit("fix(wasm): private", {"js/wasm/src/index.ts": "export const fix = 1;\n"}) - self.assertEqual(self.planned(), {}) - - def test_every_ecosystem(self): - self.commit("fix: everywhere", { - "py/moq-rs/moq/__init__.py": "fix = 1\n", - "swift/Sources/Moq/Moq.swift": "// fix\n", - "kt/moq/src/jvmMain/Moq.kt": "// fix\n", - }) - self.assertEqual(self.planned(), { - "moq-rs (Python)": "0.5.1", - "Moq (Swift)": "0.3.1", - "dev.moq:moq (Kotlin)": "0.4.1", - }) - - def test_bump_resets_the_baseline(self): - self.commit("fix(a): a backport", {"js/a/src/index.ts": "export const fix = 1;\n"}) - bump.apply(bump.plan("HEAD"), self.root) - self.commit("chore: bump") - self.assertEqual(self.planned(), {}) - - def test_cut_merge_ships_main_changes_made_after_its_bump(self): - # On main: bump @moq/a, then land another change, then cut into release. - self.git("checkout", "--quiet", "-b", "main") - self.commit("chore: bump a", {"js/a/package.json": package_json("@moq/a", "0.2.0")}) - self.commit("feat(a): after the bump", {"js/a/src/index.ts": "export const feature = 1;\n"}) - self.git("checkout", "--quiet", "release") - self.git("merge", "--quiet", "--no-ff", "--message", "cut", "main") - # The cut's tree is what Release JS published, so nothing is pending. - self.assertEqual(self.planned(), {}) - self.commit("fix(a): a backport", {"js/a/src/index.ts": "export const fix = 1;\n"}) - self.assertEqual(self.planned(), {"@moq/a": "0.2.1"}) - - def test_apply_edits_manifest_and_lockfiles(self): - self.commit("fix: a and python", { - "js/a/src/index.ts": "export const fix = 1;\n", - "py/moq-rs/moq/__init__.py": "fix = 1\n", - }) - bump.apply(bump.plan("HEAD"), self.root) - self.assertEqual(json.loads((self.root / "js/a/package.json").read_text())["version"], "0.1.1") - self.assertEqual( - (self.root / "bun.lock").read_text(), - BUN_LOCK.replace('"version": "0.1.0"', '"version": "0.1.1"'), - ) - self.assertIn('version = "0.5.1"', (self.root / "py/moq-rs/pyproject.toml").read_text()) - self.assertEqual( - (self.root / "uv.lock").read_text(), - UV_LOCK.replace('version = "0.5.0"', 'version = "0.5.1"'), - ) - - def test_apply_refuses_a_stale_lockfile(self): - self.commit("fix(a): a backport", { - "js/a/src/index.ts": "export const fix = 1;\n", - "bun.lock": BUN_LOCK.replace('"version": "0.1.0"', '"version": "0.0.9"'), - }) - with self.assertRaisesRegex(ValueError, "bun.lock does not carry @moq/a 0.1.0"): - bump.apply(bump.plan("HEAD"), self.root) - self.assertEqual(self.git("status", "--porcelain"), "") - - def test_tagged_package_bumps_after_a_change_since_its_tag(self): - self.commit("test(dart): more tests", {"dart/moq/test/moq_test.dart": ""}) - self.assertEqual(self.planned(), {}) - self.commit("fix(dart): a backport", {"dart/moq/lib/moq.dart": "// fix\n"}) - self.assertEqual(self.planned(), {"moq (Dart)": "0.1.1"}) - self.assertEqual(self.pending(), {}) - - def test_untagged_version_awaits_its_tag_instead_of_a_bump(self): - self.commit("chore: bump dart", {"dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.1\n"}) - self.commit("fix(dart): after the bump", {"dart/moq/lib/moq.dart": "// fix\n"}) - self.assertEqual(self.planned(), {}) - self.assertEqual(self.pending(), {"moq-dart-v0.1.1": None}) - - def test_cpp_and_obs_release_together(self): - self.commit("fix(obs): plugin only", {"cpp/obs/src/plugin.cpp": "// fix\n"}) - self.assertEqual(self.planned(), {"moq (C++)": "0.1.1", "obs-moq (OBS)": "0.7.1"}) - alongside = {b.package.name: b.alongside for b in bump.plan("HEAD")} - self.assertEqual(alongside, {"moq (C++)": "obs-moq (OBS)", "obs-moq (OBS)": None}) - - def test_cpp_release_bumps_obs(self): - self.commit("fix(cpp): library only", {"cpp/moq/src/moq.cpp": "// fix\n"}) - self.assertEqual(self.planned(), {"moq (C++)": "0.1.1", "obs-moq (OBS)": "0.7.1"}) - - def test_untagged_cpp_bumps_a_released_obs_before_its_tag(self): - assets = self.pin("0.4.20") - self.commit("chore: bump cpp", {"cpp/moq/VERSION": "0.2.0\n"}) - self.assertEqual(self.planned(), {"moq (C)": "0.8.1", "obs-moq (OBS)": "0.7.1"}) - # Tagging now would publish OBS 0.7.0 again. - self.assertIn("obs-moq (OBS) 0.7.0 is already released", self.pending(assets)["cpp-v0.2.0"]) - bump.apply(bump.plan("HEAD"), self.root) - self.commit("chore: bump") - self.assertEqual(self.pending(assets), {"c-v0.8.1": None, "cpp-v0.2.0": None}) - - def test_obs_in_flight_from_a_cpp_tag_is_released(self): - self.commit("chore: bump", {"cpp/moq/VERSION": "0.1.1\n", "cpp/obs/VERSION": "0.7.1\n"}) - # Release Cpp has not cut obs-moq-v0.7.1 yet. - self.git("tag", "cpp-v0.1.1") - self.commit("docs: unrelated", {"README.md": ""}) - self.assertEqual(self.planned(), {}) - self.assertEqual(self.pending(), {}) - self.commit("fix(obs): plugin only", {"cpp/obs/src/plugin.cpp": "// fix\n"}) - self.assertEqual(self.planned(), {"moq (C++)": "0.1.2", "obs-moq (OBS)": "0.7.2"}) - - def test_obs_never_pushes_its_own_tag(self): - self.commit("chore: bump obs", {"cpp/obs/VERSION": "0.8.0\n"}) - self.assertEqual(self.planned(), {"moq (C++)": "0.1.1"}) - self.assertEqual(self.pending(), {}) - - def test_ffi_pin_change_releases_c_and_cpp(self): - self.pin("0.4.20") - self.assertEqual(self.planned(), {"moq (C)": "0.8.1", "moq (C++)": "0.1.1", "obs-moq (OBS)": "0.7.1"}) - - def test_c_and_cpp_tags_wait_for_the_pinned_ffi_release(self): - self.commit("chore: bump", { - "dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.1\n", - "c/moq/VERSION": "0.8.1\n", - "cpp/moq/VERSION": "0.1.1\n", - "cpp/obs/VERSION": "0.7.1\n", - }) - self.assertEqual(self.pending(), { - "moq-dart-v0.1.1": None, - "c-v0.8.1": "cpp/moq/FFI_VERSION pins no single moq-ffi release", - "cpp-v0.1.1": "cpp/moq/FFI_VERSION pins no single moq-ffi release", - }) - assets = self.pin("0.4.20") - missing = sorted(assets)[-1] - blocked = self.pending(assets - {missing}) - self.assertIn(missing, blocked["c-v0.8.1"]) - self.assertEqual(self.pending(assets), {"moq-dart-v0.1.1": None, "c-v0.8.1": None, "cpp-v0.1.1": None}) - - def test_tag_pushes_only_unblocked_tags(self): - remote_dir = tempfile.TemporaryDirectory() - self.addCleanup(remote_dir.cleanup) - remote = Path(remote_dir.name) - subprocess.run(["git", "init", "--quiet", "--bare", str(remote)], check=True) - self.git("remote", "add", "origin", str(remote)) - head = self.commit("chore: bump", {"dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.1\n", "c/moq/VERSION": "0.8.1\n"}) - with mock.patch.object(bump, "release_assets", return_value=set()): - bump.push_tags() - pushed = subprocess.run(["git", "ls-remote", "--tags", str(remote)], capture_output=True, text=True, check=True) - self.assertEqual(pushed.stdout, f"{head}\trefs/tags/moq-dart-v0.1.1\n") - - def test_missing_tagged_packages_are_skipped(self): - # A `release` before the cut that brings C, C++, and OBS. - self.git("rm", "--quiet", "-r", "c", "cpp") - self.commit("chore: drop c and cpp") - # That `release` lacks sh/cpp/ffi.py too. - with mock.patch.object(bump, "load_ffi", side_effect=AssertionError("loaded ffi.py")): - self.assertEqual(self.planned(), {}) - self.assertEqual(self.pending(), {}) - - def test_patch(self): - self.assertEqual(bump.patch("0.9.9"), "0.9.10") - with self.assertRaises(ValueError): - bump.patch("1.0.0-rc.1") - - -if __name__ == "__main__": - unittest.main() diff --git a/sh/gh/bump.py b/sh/gh/versions.py similarity index 60% rename from sh/gh/bump.py rename to sh/gh/versions.py index 8c1539163d4..422488136e9 100755 --- a/sh/gh/bump.py +++ b/sh/gh/versions.py @@ -1,9 +1,11 @@ #!/usr/bin/env python3 -"""Patch-bump hand-versioned packages whose sources changed since they last shipped. +"""Check that backports bump hand-versioned packages, and push their release tags. -release-plz bumps Rust crates on `release`; this does the same for the packages -whose version a human owns. Without a bump, a backport touching only one of -them lands unchanged in version and never publishes. +release-plz bumps Rust crates on `release`; every other package's version is +bumped by hand, since only a human can judge a change's semver level. A +backport that changes one of them without a bump lands unchanged in version, +and its release workflow skips it as already published. `check` fails that PR; +it never picks or writes a version. Two kinds of package publish differently: @@ -17,30 +19,28 @@ moq-ffi release `cpp/moq/FFI_VERSION` pins is published. Any change to a package's sources since it last shipped is unreleased, so it -needs a patch bump. +needs a bump. Usage: - bump.py check [--ref ] List the packages that need a bump or a tag. - bump.py apply Bump them in the working tree. - bump.py pr Bump them on the `release-bump` branch and open - or update its PR into `release`; close it when - nothing is left. Needs GH_TOKEN. - bump.py tag Push each release tag HEAD's versions lack. - Needs GH_TOKEN. + versions.py check [--base ] [--head ] [--report] + Fail when merging head into base changes a package's shipped sources at a + released version, lowers a version, or leaves a lockfile entry behind. + Also lists the packages base already carries unreleased changes for. + --report lists the failures without failing. + versions.py tag [--dry-run] + Push each release tag HEAD's versions lack. Needs GH_TOKEN. """ import argparse from dataclasses import dataclass, field import importlib.util import json +import os from pathlib import Path import re import subprocess import sys -BASE = "release" -BRANCH = "release-bump" -TITLE = "chore: patch-bump hand-versioned packages" REPO = "moq-dev/moq" # The C and C++ packages' moq-ffi pin. @@ -66,20 +66,17 @@ def show(rev: str, path: str) -> str | None: return result.stdout if result.returncode == 0 else None -def patch(version: str) -> str: - match = re.fullmatch(r"(\d+)\.(\d+)\.(\d+)", version) - if not match: - raise ValueError(f"cannot patch-bump non-release version {version!r}") - major, minor, micro = match.groups() - return f"{major}.{minor}.{int(micro) + 1}" - - -def substitute(pattern: str, text: str, version: str, path: str) -> str: - """Replace group 2 of the first match of `pattern` with `version`.""" - out, count = re.subn(pattern, lambda m: m.group(1) + version + m.group(3), text, count=1, flags=re.M) - if count != 1: - raise ValueError(f"version not found in {path}") - return out +def newer(version: str, than: str) -> bool | None: + """Whether `version` sorts after `than` by semver, or None if either is not one.""" + def key(text: str): + match = re.fullmatch(r"(\d+)\.(\d+)\.(\d+)(?:-([0-9A-Za-z.-]+))?", text) + if not match: + return None + major, minor, micro, pre = match.groups() + # A prerelease sorts before its release; prereleases compare as text. + return (int(major), int(minor), int(micro), pre is None, pre or "") + a, b = key(version), key(than) + return None if a is None or b is None else a > b @dataclass @@ -88,6 +85,8 @@ class Edit: path: str pattern: str + # The command that rewrites it, for a lockfile entry. + fix: str | None = None def read(self, text: str) -> str | None: match = re.search(self.pattern, text, flags=re.M) @@ -142,13 +141,17 @@ def packages(rev: str) -> list[Package]: manifest=Edit(f"{directory}/package.json", r'^(\t"version": ")([^"]+)(")'), # API reports, benches, and examples stay out of the built package. paths=[directory, *(f":(exclude){directory}/{d}" for d in ("api", "bench", "examples"))], - locks=[Edit("bun.lock", rf'^( "{re.escape(directory)}": \{{\n "name": "{re.escape(name)}",\n "version": ")([^"]+)(")')], + locks=[Edit( + "bun.lock", + rf'^( "{re.escape(directory)}": \{{\n "name": "{re.escape(name)}",\n "version": ")([^"]+)(")', + fix="bun install", + )], )) found.append(Package( name="moq-rs (Python)", manifest=Edit("py/moq-rs/pyproject.toml", r'^(\[project\]\n(?:(?!\[).*\n)*?version = ")([^"]+)(")'), paths=["py/moq-rs", ":(exclude)py/moq-rs/benches", ":(exclude)py/moq-rs/examples"], - locks=[Edit("uv.lock", r'^(\[\[package\]\]\nname = "moq-rs"\nversion = ")([^"]+)(")')], + locks=[Edit("uv.lock", r'^(\[\[package\]\]\nname = "moq-rs"\nversion = ")([^"]+)(")', fix="uv lock")], )) found.append(Package( name="Moq (Swift)", @@ -213,17 +216,6 @@ def released(pkg: Package, head: str) -> str | None: return None -@dataclass -class Bump: - package: Package - old: str - new: str - since: str - changed: list[str] - # The package it releases alongside, when its own sources did not change. - alongside: str | None = None - - def baselines(found: list[Package], head: str) -> dict[str, str | None]: """`released` for each package in `found`.""" since = {pkg.name: released(pkg, head) for pkg in found} @@ -236,33 +228,105 @@ def baselines(found: list[Package], head: str) -> dict[str, str | None]: return since -def plan(head: str) -> list[Bump]: +def group(pkg: Package, found: list[Package]) -> set[str]: + """`pkg` and every package one tag publishes along with it.""" + names = {pkg.name, *([pkg.rides] if pkg.rides else [])} + return names | {other.name for other in found if other.rides in names} + + +@dataclass +class Unreleased: + """A package still at a released version that must move to ship.""" + + package: Package + version: str + since: str + changed: list[str] + # The package it releases alongside, when its own sources did not change. + alongside: str | None = None + + +def unreleased(head: str) -> list[Unreleased]: + """Every package at `head` whose released version no longer matches what would ship.""" found = packages(head) since = baselines(found, head) - bumps: dict[str, Bump] = {} + stale: dict[str, Unreleased] = {} for pkg in found: if since[pkg.name] is None: continue changed = git("diff", "--name-only", since[pkg.name], head, "--", *pkg.paths, *UNSHIPPED).stdout.split() if changed: - old = pkg.version(head) - bumps[pkg.name] = Bump(pkg, old, patch(old), since[pkg.name], changed) + stale[pkg.name] = Unreleased(pkg, pkg.version(head), since[pkg.name], changed) # One tag publishes a package and every package riding it, so once any of - # them releases, each other one still at a released version needs a bump. + # them releases, each other one still at a released version must move too. for rider in found: - group = [pkg for pkg in found if pkg.name == rider.rides] - if not group: + members = [pkg for pkg in found if pkg.name == rider.rides] + if not members: continue - group.append(rider) - releasing = [pkg.name for pkg in group if pkg.name in bumps or since[pkg.name] is None] + members.append(rider) + releasing = [pkg.name for pkg in members if pkg.name in stale or since[pkg.name] is None] if not releasing: continue - for pkg in group: - if pkg.name not in bumps and since[pkg.name] is not None: - old = pkg.version(head) - bumps[pkg.name] = Bump(pkg, old, patch(old), since[pkg.name], [], alongside=releasing[0]) - return [bumps[pkg.name] for pkg in found if pkg.name in bumps] + for pkg in members: + if pkg.name not in stale and since[pkg.name] is not None: + stale[pkg.name] = Unreleased(pkg, pkg.version(head), since[pkg.name], [], alongside=releasing[0]) + return [stale[pkg.name] for pkg in found if pkg.name in stale] + + +def merge(base: str, head: str) -> str: + """A commit landing `head` on `base` as a PR's merge does: first parent `base`.""" + result = git("merge-tree", "--write-tree", base, head, check=False) + if result.returncode: + raise RuntimeError(f"{head} does not merge cleanly into {base}") + tree = result.stdout.split()[0] + identity = {f"GIT_{who}_{key}": value for who in ("AUTHOR", "COMMITTER") + for key, value in (("NAME", "versions.py"), ("EMAIL", "versions.py@localhost"))} + return subprocess.run(["git", "commit-tree", tree, "-p", base, "-p", head, "-m", f"merge {head} into {base}"], + capture_output=True, text=True, check=True, env={**os.environ, **identity}).stdout.strip() + + +def how(pkg: Package, version: str) -> str: + """How to move `pkg` past its released `version`.""" + text = f"raise its version in {pkg.manifest.path} past {version} (CONTRIBUTING.md's Versions: patch for a fix)" + fixes = [lock.fix for lock in pkg.locks if lock.fix] + if fixes: + text += ", then run " + " and ".join(f"`{fix}`" for fix in fixes) + return text + + +def check(base: str, head: str) -> tuple[list[str], list[str]]: + """The problems merging `head` into `base` brings, and the packages `base` + already holds unreleased changes for.""" + merged = merge(base, head) + found = packages(merged) + touched = { + pkg.name for pkg in found + if git("diff", "--name-only", base, merged, "--", pkg.manifest.path, *pkg.paths, *UNSHIPPED).stdout.strip() + } + + problems, notes = [], [] + for item in unreleased(merged): + pkg = item.package + if item.alongside: + reason = f"{item.alongside} releases, and the same tag would publish its released {item.version} again" + else: + reason = f"{len(item.changed)} shipped {'file' if len(item.changed) == 1 else 'files'} changed since {item.version} shipped" + line = f"{pkg.name}: {reason}; {how(pkg, item.version)}." + (problems if group(pkg, found) & touched else notes).append(line) + + for pkg in found: + if pkg.name not in touched: + continue + old, new = pkg.version(base), pkg.version(merged) + if old and new != old and newer(new, old) is False: + problems.append(f"{pkg.name}: {new} does not come after {old}, the version on {base}.") + for lock in pkg.locks: + text = show(merged, lock.path) + have = lock.read(text) if text is not None else None + if have != new: + problems.append(f"{pkg.name}: {lock.path} carries {have}, not {new}; run `{lock.fix}`.") + return problems, notes def release_assets(tag: str) -> set[str]: @@ -312,11 +376,11 @@ def pending(head: str) -> list[Tag]: if not pkg.tag or pkg.rides or since[pkg.name] is not None: continue blocker = None - # A rider still at a released version needs its bump merged first: the - # tag would publish that version again, which its release refuses. + # A rider still at a released version must be bumped first: the tag + # would publish that version again, which its release refuses. stale = [rider for rider in found if rider.rides == pkg.name and since[rider.name] is not None] if stale: - blocker = f"{stale[0].name} {stale[0].version(head)} is already released; merge its bump first" + blocker = f"{stale[0].name} {stale[0].version(head)} is already released; bump it on `release` first" elif pkg.ffi: if not checked: ffi, checked = ffi_blocker(head), True @@ -325,91 +389,7 @@ def pending(head: str) -> list[Tag]: return tags -def apply(bumps: list[Bump], root: Path) -> None: - # Every edit is checked before any file is written, so a stale lockfile - # leaves the tree untouched. Packages share lockfiles, so edits chain. - texts: dict[str, str] = {} - for bump in bumps: - for edit in [bump.package.manifest, *bump.package.locks]: - text = texts.get(edit.path) or (root / edit.path).read_text() - if edit.read(text) != bump.old: - raise ValueError(f"{edit.path} does not carry {bump.package.name} {bump.old}") - texts[edit.path] = substitute(edit.pattern, text, bump.new, edit.path) - for path, text in texts.items(): - (root / path).write_text(text) - - -def body(bumps: list[Bump], head: str) -> str: - lines = [ - "Patch-bumps every hand-versioned package whose sources changed on `release` since its current version shipped, " - "so merging this publishes them through their release workflows.", - "", - "| Package | Version | Unreleased changes |", - "| --- | --- | --- |", - ] - for bump in bumps: - files = "file" if len(bump.changed) == 1 else "files" - compare = f"[{len(bump.changed)} {files}](https://github.com/{REPO}/compare/{bump.since[:10]}...{head[:10]})" - if bump.alongside: - compare = f"none; releases with {bump.alongside}" - lines.append(f"| {bump.package.name} | {bump.old} → {bump.new} | {compare} |") - tags = [f"`{b.package.tag}{b.new}`" for b in bumps if b.package.tag and not b.package.rides] - if tags: - lines += [ - "", - f"Once this merges, the Release bump workflow pushes {', '.join(tags)} to publish them; " - "a C or C++ tag waits until the moq-ffi release `cpp/moq/FFI_VERSION` pins is published.", - ] - lines += [ - "", - "Each bump is a patch, as a backport keeping the published API needs. " - "The next push to `release` regenerates this branch from scratch, dropping any edit to it.", - "", - f"Generated by `just gh bump pr` at {head[:10]}.", - ] - return "\n".join(lines) + "\n" - - -def gh(*args: str, stdin: str | None = None) -> str: - return subprocess.run(["gh", *args], input=stdin, capture_output=True, text=True, check=True).stdout.strip() - - -def open_pr() -> str | None: - number = gh("pr", "list", "--repo", REPO, "--base", BASE, "--head", BRANCH, "--state", "open", - "--json", "number", "--jq", ".[0].number // empty") - return number or None - - -def pr(root: Path) -> None: - head = git("rev-parse", "HEAD").stdout.strip() - bumps = plan(head) - existing = open_pr() - if not bumps: - if existing: - gh("pr", "close", existing, "--repo", REPO, "--delete-branch", - "--comment", f"Every hand-versioned package on `release` is bumped as of {head[:10]}.") - print(f"closed #{existing}: nothing left to bump") - else: - print("nothing to bump") - return - - apply(bumps, root) - names = ", ".join(f"{b.package.name} {b.new}" for b in bumps) - git("checkout", "-B", BRANCH) - git("commit", "--all", "--message", f"{TITLE}\n\n{names}") - # The branch is regenerated from `release` on every run, like release-plz's. - git("push", "--force", "origin", f"HEAD:refs/heads/{BRANCH}") - text = body(bumps, head) - if existing: - gh("pr", "edit", existing, "--repo", REPO, "--title", TITLE, "--body-file", "-", stdin=text) - print(f"updated #{existing}: {names}") - else: - url = gh("pr", "create", "--repo", REPO, "--base", BASE, "--head", BRANCH, "--title", TITLE, - "--body-file", "-", stdin=text) - print(f"opened {url}: {names}") - - -def push_tags() -> None: +def push_tags(dry_run: bool = False) -> None: head = git("rev-parse", "HEAD").stdout.strip() tags = pending(head) for tag in tags: @@ -417,6 +397,9 @@ def push_tags() -> None: # The next push to `release` retries it. print(f"waiting to push {tag.name}: {tag.blocker}") continue + if dry_run: + print(f"would push {tag.name} at {head[:10]}") + continue git("push", "origin", f"{head}:refs/tags/{tag.name}") print(f"pushed {tag.name} at {head[:10]}") if not tags: @@ -426,32 +409,36 @@ def push_tags() -> None: def main() -> int: parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) sub = parser.add_subparsers(dest="command", required=True) - check = sub.add_parser("check") - check.add_argument("--ref", default="HEAD") - sub.add_parser("apply") - sub.add_parser("pr") - sub.add_parser("tag") + check_parser = sub.add_parser("check") + check_parser.add_argument("--base", default="origin/release") + check_parser.add_argument("--head", default="HEAD") + check_parser.add_argument("--report", action="store_true") + tag_parser = sub.add_parser("tag") + tag_parser.add_argument("--dry-run", action="store_true") args = parser.parse_args() - root = Path(git("rev-parse", "--show-toplevel").stdout.strip()) - if args.command == "check": - bumps = plan(args.ref) - for bump in bumps: - reason = f"releases with {bump.alongside}" if bump.alongside else f"{len(bump.changed)} files" - print(f"{bump.package.name}: {bump.old} -> {bump.new} ({reason} since {bump.since[:10]})") - for path in bump.changed: - print(f" {path}") - for tag in pending(args.ref): - print(f"{tag.package.name}: tag {tag.name}" + (f" (waiting: {tag.blocker})" if tag.blocker else "")) - if not bumps: - print("nothing to bump") - elif args.command == "apply": - apply(plan("HEAD"), root) - elif args.command == "tag": - push_tags() - else: - pr(root) - return 0 + if args.command == "tag": + push_tags(args.dry_run) + return 0 + + try: + problems, notes = check(args.base, args.head) + except RuntimeError as error: + print(error, file=sys.stderr) + return 2 + for line in notes: + print(f"already unreleased on {args.base}: {line}") + for line in problems: + print(f"::error::{line}" if os.environ.get("GITHUB_ACTIONS") and not args.report else line) + if not problems: + print(f"no version problems merging {args.head} into {args.base}") + return 0 + if args.report: + return 0 + print(f"{len(problems)} version {'problem' if len(problems) == 1 else 'problems'}; " + "a backport bumps each hand-versioned package whose shipped sources it changes, " + "or its release workflow skips the fix as already published.") + return 1 if __name__ == "__main__": diff --git a/sh/gh/versions.test.py b/sh/gh/versions.test.py new file mode 100644 index 00000000000..abd7afadaad --- /dev/null +++ b/sh/gh/versions.test.py @@ -0,0 +1,344 @@ +"""Check backports bump hand-versioned packages, and find the release tags to push.""" + +import importlib.util +import json +import os +from pathlib import Path +import subprocess +import tempfile +import unittest +from unittest import mock + +spec = importlib.util.spec_from_file_location("versions", Path(__file__).with_name("versions.py")) +versions = importlib.util.module_from_spec(spec) +spec.loader.exec_module(versions) + + +def package_json(name: str, version: str, private: bool = False) -> str: + manifest = {"name": name, "version": version, "scripts": {"release": "bun ../common/release.ts"}} + if private: + manifest["private"] = True + return json.dumps(manifest, indent="\t") + "\n" + + +BUN_LOCK = """{ + "workspaces": { + "js/a": { + "name": "@moq/a", + "version": "0.1.0", + }, + "js/b": { + "name": "@moq/b", + "version": "0.2.0", + }, + }, +} +""" + +PYPROJECT = """[build-system] +requires = ["hatchling"] + +[project] +name = "moq-rs" +# Bumped by hand. +version = "0.5.0" +""" + +UV_LOCK = """[[package]] +name = "moq-ffi" +version = "0.4.0" + +[[package]] +name = "moq-rs" +version = "0.5.0" +source = { editable = "py/moq-rs" } +""" + + +class VersionsTest(unittest.TestCase): + def setUp(self): + self.dir = tempfile.TemporaryDirectory() + self.root = Path(self.dir.name) + self.cwd = os.getcwd() + os.chdir(self.root) + self.git("init", "--quiet", "--initial-branch", "release") + self.git("config", "user.name", "test") + self.git("config", "user.email", "test@example.com") + self.write({ + "js/a/package.json": package_json("@moq/a", "0.1.0"), + "js/a/src/index.ts": "export {};\n", + "js/b/package.json": package_json("@moq/b", "0.2.0"), + "js/b/src/index.ts": "export {};\n", + "js/wasm/package.json": package_json("@moq/wasm", "0.0.0", private=True), + "js/wasm/src/index.ts": "export {};\n", + "bun.lock": BUN_LOCK, + "py/moq-rs/pyproject.toml": PYPROJECT, + "py/moq-rs/moq/__init__.py": "", + "uv.lock": UV_LOCK, + "swift/VERSION": "0.3.0\n", + "swift/Sources/Moq/Moq.swift": "", + "kt/gradle.properties": "moqffi.version=0.0.0-dev\nmoq.version=0.4.0\n", + "kt/moq/src/jvmMain/Moq.kt": "", + "dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.0\n", + "dart/moq/lib/moq.dart": "", + "c/moq/VERSION": "0.8.0\n", + "c/moq/src/moq.c": "", + "cpp/moq/VERSION": "0.1.0\n", + "cpp/moq/src/moq.cpp": "", + "cpp/moq/FFI_VERSION": "# Unset until published.\n", + "cpp/moq/FFI_SHA256.json": "{}\n", + "cpp/obs/VERSION": "0.7.0\n", + "cpp/obs/src/plugin.cpp": "", + }) + self.commit("initial") + # Every tag-published package has released its current version. + for tag in ("moq-dart-v0.1.0", "c-v0.8.0", "cpp-v0.1.0", "obs-moq-v0.7.0"): + self.git("tag", tag) + + def tearDown(self): + os.chdir(self.cwd) + self.dir.cleanup() + + def git(self, *args: str) -> str: + return subprocess.run(["git", *args], capture_output=True, text=True, check=True).stdout.strip() + + def write(self, files: dict[str, str]): + for path, text in files.items(): + (self.root / path).parent.mkdir(parents=True, exist_ok=True) + (self.root / path).write_text(text) + + def commit(self, message: str, files: dict[str, str] | None = None) -> str: + if files: + self.write(files) + self.git("add", "--all") + self.git("commit", "--quiet", "--message", message) + return self.git("rev-parse", "HEAD") + + def stale(self) -> dict[str, str | None]: + """The packages at HEAD still at a released version, with what each releases alongside.""" + return {u.package.name: u.alongside for u in versions.unreleased("HEAD")} + + def backport(self, files: dict[str, str]) -> tuple[list[str], list[str]]: + """Check a PR into `release` committing `files`, as names of the packages failing and noted.""" + self.git("checkout", "--quiet", "-B", "backport", "release") + self.commit("fix: a backport", files) + self.git("checkout", "--quiet", "release") + self.problems, self.notes = versions.check("release", "backport") + return names(self.problems), names(self.notes) + + def pending(self, assets: set[str] = frozenset()) -> dict[str, str | None]: + with mock.patch.object(versions, "release_assets", return_value=assets): + return {t.name: t.blocker for t in versions.pending("HEAD")} + + def pin(self, version: str) -> tuple[dict[str, str], set[str]]: + """The files pinning the C and C++ packages to moq-ffi `version`, and its asset names.""" + ffi = versions.load_ffi() + assets = {ffi.name(version, target) + ".tar.gz" for target in ffi.TARGETS} + return { + "cpp/moq/FFI_VERSION": f"# Pinned.\n{version}\n", + "cpp/moq/FFI_SHA256.json": json.dumps({asset: "0" * 64 for asset in assets}) + "\n", + }, assets + + def bump_a(self, version: str) -> dict[str, str]: + return { + "js/a/package.json": package_json("@moq/a", version), + "bun.lock": BUN_LOCK.replace('"version": "0.1.0"', f'"version": "{version}"'), + } + + def test_nothing_changed(self): + self.assertEqual(self.backport({"README.md": "docs\n"}), ([], [])) + + def test_backport_without_a_bump_fails(self): + self.assertEqual(self.backport({"js/a/src/index.ts": FIX}), (["@moq/a"], [])) + self.assertIn("1 shipped file changed since 0.1.0 shipped", self.problems[0]) + self.assertIn("js/a/package.json past 0.1.0", self.problems[0]) + self.assertIn("`bun install`", self.problems[0]) + + def test_backport_with_a_bump_passes(self): + self.assertEqual(self.backport({"js/a/src/index.ts": FIX, **self.bump_a("0.1.1")}), ([], [])) + self.assertEqual(self.backport({"js/a/src/index.ts": FIX, **self.bump_a("0.2.0")}), ([], [])) + + def test_a_bump_moves_its_lockfile_entry(self): + files = {"js/a/src/index.ts": FIX, "js/a/package.json": package_json("@moq/a", "0.1.1")} + self.assertEqual(self.backport(files), (["@moq/a"], [])) + self.assertEqual(self.problems, ["@moq/a: bun.lock carries 0.1.0, not 0.1.1; run `bun install`."]) + + def test_a_lowered_version_fails(self): + self.assertEqual(self.backport({"js/a/src/index.ts": FIX, **self.bump_a("0.0.9")}), (["@moq/a"], [])) + self.assertIn("0.0.9 does not come after 0.1.0", self.problems[0]) + + def test_branch_history_is_merged_first(self): + # The bump and the fix land in separate commits on the PR's branch. + self.git("checkout", "--quiet", "-b", "backport") + self.commit("chore: bump a", self.bump_a("0.1.1")) + self.commit("fix(a): the fix", {"js/a/src/index.ts": FIX}) + self.git("checkout", "--quiet", "release") + self.assertEqual(versions.check("release", "backport"), ([], [])) + + def test_unshipped_files_are_not_a_release(self): + self.assertEqual(self.backport({ + "js/a/src/index.test.ts": "", + "js/a/api/index.api.md": "", + "js/a/MIGRATE.md": "", + "kt/moq/src/jvmTest/MoqTest.kt": "", + "swift/Tests/MoqTests/MoqTests.swift": "", + "dart/moq/test/moq_test.dart": "", + }), ([], [])) + + def test_private_package_is_skipped(self): + self.assertEqual(self.backport({"js/wasm/src/index.ts": FIX}), ([], [])) + + def test_every_ecosystem(self): + failing, _ = self.backport({ + "py/moq-rs/moq/__init__.py": "fix = 1\n", + "swift/Sources/Moq/Moq.swift": "// fix\n", + "kt/moq/src/jvmMain/Moq.kt": "// fix\n", + }) + self.assertEqual(failing, ["moq-rs (Python)", "Moq (Swift)", "dev.moq:moq (Kotlin)"]) + self.assertIn("`uv lock`", self.problems[0]) + self.assertEqual(self.backport({ + "py/moq-rs/moq/__init__.py": "fix = 1\n", + "py/moq-rs/pyproject.toml": PYPROJECT.replace('"0.5.0"', '"0.5.1"'), + "uv.lock": UV_LOCK.replace('version = "0.5.0"', 'version = "0.5.1"'), + "swift/Sources/Moq/Moq.swift": "// fix\n", + "swift/VERSION": "0.3.1\n", + "kt/moq/src/jvmMain/Moq.kt": "// fix\n", + "kt/gradle.properties": "moqffi.version=0.0.0-dev\nmoq.version=0.4.1\n", + }), ([], [])) + + def test_unreleased_changes_already_on_base_are_only_noted(self): + self.commit("fix(b): merged without a bump", {"js/b/src/index.ts": FIX}) + self.assertEqual(self.backport({"js/a/src/index.test.ts": ""}), ([], ["@moq/b"])) + # A backport touching it must bump it, which ships the earlier change too. + self.assertEqual(self.backport({"js/b/src/other.ts": FIX}), (["@moq/b"], [])) + self.assertIn("2 shipped files changed", self.problems[0]) + + def test_cut_merge_ships_main_changes_made_after_its_bump(self): + # On main: bump @moq/a, then land another change, then cut into release. + self.git("checkout", "--quiet", "-b", "main") + self.commit("chore: bump a", self.bump_a("0.2.0")) + self.commit("feat(a): after the bump", {"js/a/src/index.ts": "export const feature = 1;\n"}) + self.git("checkout", "--quiet", "release") + self.git("merge", "--quiet", "--no-ff", "--message", "cut", "main") + # The cut's tree is what Release JS published, so nothing is unreleased. + self.assertEqual(self.stale(), {}) + self.assertEqual(self.backport({"js/a/src/index.ts": FIX}), (["@moq/a"], [])) + self.assertIn("past 0.2.0", self.problems[0]) + + def test_tagged_package_needs_a_bump_after_its_tag(self): + self.assertEqual(self.backport({"dart/moq/lib/moq.dart": "// fix\n"}), (["moq (Dart)"], [])) + self.assertIn("dart/moq/pubspec.yaml past 0.1.0", self.problems[0]) + + def test_untagged_version_needs_no_bump(self): + self.commit("chore: bump dart", {"dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.1\n"}) + self.assertEqual(self.backport({"dart/moq/lib/moq.dart": "// fix\n"}), ([], [])) + + def test_cpp_and_obs_release_together(self): + self.assertEqual(self.backport({"cpp/moq/src/moq.cpp": "// fix\n"}), (["moq (C++)", "obs-moq (OBS)"], [])) + self.assertEqual(self.backport({"cpp/moq/src/moq.cpp": "// fix\n", "cpp/moq/VERSION": "0.1.1\n"}), + (["obs-moq (OBS)"], [])) + self.assertIn("moq (C++) releases, and the same tag would publish its released 0.7.0 again", self.problems[0]) + self.assertEqual(self.backport({"cpp/obs/src/plugin.cpp": "// fix\n", "cpp/obs/VERSION": "0.7.1\n"}), + (["moq (C++)"], [])) + self.assertEqual(self.backport({ + "cpp/obs/src/plugin.cpp": "// fix\n", + "cpp/moq/VERSION": "0.1.1\n", + "cpp/obs/VERSION": "0.7.1\n", + }), ([], [])) + + def test_ffi_pin_change_releases_c_and_cpp(self): + files, _ = self.pin("0.4.20") + self.assertEqual(self.backport(files), (["moq (C)", "moq (C++)", "obs-moq (OBS)"], [])) + + def test_untagged_version_awaits_its_tag(self): + self.commit("chore: bump dart", {"dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.1\n"}) + self.commit("fix(dart): after the bump", {"dart/moq/lib/moq.dart": "// fix\n"}) + self.assertEqual(self.stale(), {}) + self.assertEqual(self.pending(), {"moq-dart-v0.1.1": None}) + + def test_untagged_cpp_waits_for_the_obs_bump(self): + files, assets = self.pin("0.4.20") + self.commit("chore: pin moq-ffi and bump", {**files, "c/moq/VERSION": "0.8.1\n", "cpp/moq/VERSION": "0.2.0\n"}) + self.assertEqual(self.stale(), {"obs-moq (OBS)": "moq (C++)"}) + # Tagging now would publish OBS 0.7.0 again. + self.assertIn("obs-moq (OBS) 0.7.0 is already released", self.pending(assets)["cpp-v0.2.0"]) + self.commit("chore: bump obs", {"cpp/obs/VERSION": "0.7.1\n"}) + self.assertEqual(self.pending(assets), {"c-v0.8.1": None, "cpp-v0.2.0": None}) + + def test_obs_in_flight_from_a_cpp_tag_is_released(self): + self.commit("chore: bump", {"cpp/moq/VERSION": "0.1.1\n", "cpp/obs/VERSION": "0.7.1\n"}) + # Release Cpp has not cut obs-moq-v0.7.1 yet. + self.git("tag", "cpp-v0.1.1") + self.commit("docs: unrelated", {"README.md": ""}) + self.assertEqual(self.stale(), {}) + self.assertEqual(self.pending(), {}) + self.assertEqual(self.backport({"cpp/obs/src/plugin.cpp": "// fix\n"}), (["moq (C++)", "obs-moq (OBS)"], [])) + + def test_obs_never_pushes_its_own_tag(self): + self.commit("chore: bump obs", {"cpp/obs/VERSION": "0.8.0\n"}) + self.assertEqual(self.stale(), {"moq (C++)": "obs-moq (OBS)"}) + self.assertEqual(self.pending(), {}) + + def test_c_and_cpp_tags_wait_for_the_pinned_ffi_release(self): + self.commit("chore: bump", { + "dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.1\n", + "c/moq/VERSION": "0.8.1\n", + "cpp/moq/VERSION": "0.1.1\n", + "cpp/obs/VERSION": "0.7.1\n", + }) + self.assertEqual(self.pending(), { + "moq-dart-v0.1.1": None, + "c-v0.8.1": "cpp/moq/FFI_VERSION pins no single moq-ffi release", + "cpp-v0.1.1": "cpp/moq/FFI_VERSION pins no single moq-ffi release", + }) + files, assets = self.pin("0.4.20") + self.commit("chore: pin moq-ffi", files) + missing = sorted(assets)[-1] + blocked = self.pending(assets - {missing}) + self.assertIn(missing, blocked["c-v0.8.1"]) + self.assertEqual(self.pending(assets), {"moq-dart-v0.1.1": None, "c-v0.8.1": None, "cpp-v0.1.1": None}) + + def test_tag_pushes_only_unblocked_tags(self): + remote_dir = tempfile.TemporaryDirectory() + self.addCleanup(remote_dir.cleanup) + remote = Path(remote_dir.name) + subprocess.run(["git", "init", "--quiet", "--bare", str(remote)], check=True) + self.git("remote", "add", "origin", str(remote)) + head = self.commit("chore: bump", {"dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.1\n", "c/moq/VERSION": "0.8.1\n"}) + + def pushed() -> str: + return subprocess.run(["git", "ls-remote", "--tags", str(remote)], capture_output=True, text=True, check=True).stdout + + with mock.patch.object(versions, "release_assets", return_value=set()): + versions.push_tags(dry_run=True) + self.assertEqual(pushed(), "") + versions.push_tags() + self.assertEqual(pushed(), f"{head}\trefs/tags/moq-dart-v0.1.1\n") + + def test_missing_tagged_packages_are_skipped(self): + # A `release` before the cut that brings C, C++, and OBS. + self.git("rm", "--quiet", "-r", "c", "cpp") + self.commit("chore: drop c and cpp") + # That `release` lacks sh/cpp/ffi.py too. + with mock.patch.object(versions, "load_ffi", side_effect=AssertionError("loaded ffi.py")): + self.assertEqual(self.stale(), {}) + self.assertEqual(self.pending(), {}) + self.assertEqual(self.backport({"js/a/src/index.ts": FIX}), (["@moq/a"], []), self.problems) + + def test_newer(self): + self.assertTrue(versions.newer("0.9.10", "0.9.9")) + self.assertTrue(versions.newer("0.1.0", "0.1.0-rc.1")) + self.assertFalse(versions.newer("0.1.0-rc.1", "0.1.0")) + self.assertFalse(versions.newer("0.1.0", "0.1.0")) + self.assertIsNone(versions.newer("next", "0.1.0")) + + +def names(lines: list[str]) -> list[str]: + return [line.split(": ", 1)[0] for line in lines] + + +FIX = "export const fix = 1;\n" + + +if __name__ == "__main__": + unittest.main() From ad120b987d9ebb28ce96e76ab3a56549951f5449 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 10:11:32 -0700 Subject: [PATCH 07/12] fix(gh): order release versions by semver precedence Prerelease identifiers compared as text, so rc.10 sorted before rc.9, and a version outside the regex (such as one with build metadata) passed the check silently. Compare identifiers numerically where numeric, ignore build metadata, and fail on a version that can't be ordered. Co-Authored-By: Claude Opus 5.5 --- sh/gh/versions.py | 29 +++++++++++++++++++---------- sh/gh/versions.test.py | 12 ++++++++++++ 2 files changed, 31 insertions(+), 10 deletions(-) diff --git a/sh/gh/versions.py b/sh/gh/versions.py index 422488136e9..1fd363a4da1 100755 --- a/sh/gh/versions.py +++ b/sh/gh/versions.py @@ -66,16 +66,21 @@ def show(rev: str, path: str) -> str | None: return result.stdout if result.returncode == 0 else None +def precedence(version: str) -> tuple | None: + """A sort key for semver precedence, or None if `version` is not semver.""" + match = re.fullmatch(r"(\d+)\.(\d+)\.(\d+)(?:-([0-9A-Za-z.-]+))?(?:\+[0-9A-Za-z.-]+)?", version) + if not match: + return None + major, minor, micro, pre = match.groups() + # A prerelease sorts before its release. Its identifiers compare in order, + # numeric ones by value and below alphanumeric ones; build metadata is ignored. + ids = [(0, int(part), "") if part.isdigit() else (1, 0, part) for part in pre.split(".")] if pre else [] + return (int(major), int(minor), int(micro), pre is None, ids) + + def newer(version: str, than: str) -> bool | None: """Whether `version` sorts after `than` by semver, or None if either is not one.""" - def key(text: str): - match = re.fullmatch(r"(\d+)\.(\d+)\.(\d+)(?:-([0-9A-Za-z.-]+))?", text) - if not match: - return None - major, minor, micro, pre = match.groups() - # A prerelease sorts before its release; prereleases compare as text. - return (int(major), int(minor), int(micro), pre is None, pre or "") - a, b = key(version), key(than) + a, b = precedence(version), precedence(than) return None if a is None or b is None else a > b @@ -319,8 +324,12 @@ def check(base: str, head: str) -> tuple[list[str], list[str]]: if pkg.name not in touched: continue old, new = pkg.version(base), pkg.version(merged) - if old and new != old and newer(new, old) is False: - problems.append(f"{pkg.name}: {new} does not come after {old}, the version on {base}.") + if old and new != old: + order = newer(new, old) + if order is None: + problems.append(f"{pkg.name}: cannot order {new} after {old}, the version on {base}, as semver.") + elif not order: + problems.append(f"{pkg.name}: {new} does not come after {old}, the version on {base}.") for lock in pkg.locks: text = show(merged, lock.path) have = lock.read(text) if text is not None else None diff --git a/sh/gh/versions.test.py b/sh/gh/versions.test.py index abd7afadaad..2ee01e367f3 100644 --- a/sh/gh/versions.test.py +++ b/sh/gh/versions.test.py @@ -325,11 +325,23 @@ def test_missing_tagged_packages_are_skipped(self): self.assertEqual(self.pending(), {}) self.assertEqual(self.backport({"js/a/src/index.ts": FIX}), (["@moq/a"], []), self.problems) + def test_a_version_semver_cannot_order_fails(self): + self.assertEqual(self.backport({"js/a/src/index.ts": FIX, **self.bump_a("next")}), (["@moq/a"], [])) + self.assertIn("cannot order next after 0.1.0", self.problems[0]) + def test_newer(self): self.assertTrue(versions.newer("0.9.10", "0.9.9")) self.assertTrue(versions.newer("0.1.0", "0.1.0-rc.1")) self.assertFalse(versions.newer("0.1.0-rc.1", "0.1.0")) self.assertFalse(versions.newer("0.1.0", "0.1.0")) + # Numeric prerelease identifiers compare by value, below alphanumeric ones. + self.assertTrue(versions.newer("1.0.0-rc.10", "1.0.0-rc.9")) + self.assertFalse(versions.newer("1.0.0-rc.9", "1.0.0-rc.10")) + self.assertTrue(versions.newer("1.0.0-alpha.beta", "1.0.0-alpha.1")) + self.assertTrue(versions.newer("1.0.0-alpha.1", "1.0.0-alpha")) + # Build metadata takes no part in precedence. + self.assertFalse(versions.newer("0.4.0+fix", "0.5.0")) + self.assertFalse(versions.newer("0.5.0+fix", "0.5.0")) self.assertIsNone(versions.newer("next", "0.1.0")) From dbc5ee64f35def60013759d92b8b96ba4d8d03aa Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 10:18:12 -0700 Subject: [PATCH 08/12] fix(gh): count C++'s moq-ffi CMake module as C's source, and log a failed tag push c/moq/CMakeLists.txt includes cpp/moq/cmake/moq-ffi.cmake, so a fix there ships in C too. And a rejected tag push now raises with git's stderr, so the job log says why. Co-Authored-By: Claude Opus 5.5 --- sh/gh/versions.py | 10 +++++++--- sh/gh/versions.test.py | 3 +++ 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/sh/gh/versions.py b/sh/gh/versions.py index 1fd363a4da1..60545103af3 100755 --- a/sh/gh/versions.py +++ b/sh/gh/versions.py @@ -46,6 +46,7 @@ # The C and C++ packages' moq-ffi pin. FFI_VERSION = "cpp/moq/FFI_VERSION" FFI_SHA256 = "cpp/moq/FFI_SHA256.json" +FFI_CMAKE = "cpp/moq/cmake/moq-ffi.cmake" def load_ffi(): @@ -176,11 +177,12 @@ def packages(rev: str) -> list[Package]: paths=["dart/moq", ":(exclude)dart/moq/test", ":(exclude)dart/moq/bench"], tag="moq-dart-v", ), - # The C package links the same pinned moq-ffi input as C++. + # The C package links the same pinned moq-ffi input as C++, through C++'s + # CMake module. Package( name="moq (C)", manifest=Edit("c/moq/VERSION", version_file), - paths=["c/moq", ":(exclude)c/moq/test", FFI_VERSION, FFI_SHA256], + paths=["c/moq", ":(exclude)c/moq/test", FFI_VERSION, FFI_SHA256, FFI_CMAKE], tag="c-v", ffi=True, ), @@ -409,7 +411,9 @@ def push_tags(dry_run: bool = False) -> None: if dry_run: print(f"would push {tag.name} at {head[:10]}") continue - git("push", "origin", f"{head}:refs/tags/{tag.name}") + result = git("push", "origin", f"{head}:refs/tags/{tag.name}", check=False) + if result.returncode: + raise RuntimeError(f"pushing {tag.name} failed:\n{result.stderr}") print(f"pushed {tag.name} at {head[:10]}") if not tags: print("nothing to tag") diff --git a/sh/gh/versions.test.py b/sh/gh/versions.test.py index 2ee01e367f3..1249e767352 100644 --- a/sh/gh/versions.test.py +++ b/sh/gh/versions.test.py @@ -249,6 +249,9 @@ def test_cpp_and_obs_release_together(self): def test_ffi_pin_change_releases_c_and_cpp(self): files, _ = self.pin("0.4.20") self.assertEqual(self.backport(files), (["moq (C)", "moq (C++)", "obs-moq (OBS)"], [])) + # C builds through C++'s moq-ffi CMake module too. + self.assertEqual(self.backport({"cpp/moq/cmake/moq-ffi.cmake": "# fix\n"}), + (["moq (C)", "moq (C++)", "obs-moq (OBS)"], [])) def test_untagged_version_awaits_its_tag(self): self.commit("chore: bump dart", {"dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.1\n"}) From 0329eb4468959ad44d11863723fb73807b8ec59d Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 10:28:15 -0700 Subject: [PATCH 09/12] feat(gh): warn when a shared JS build script changes since a package shipped Each JS build runs js/common/package.ts, and watch and publish bundle through js/common/vite-plugin-worklet.ts, so a fix there can change what those packages ship. Whether it did takes a human: release's last change to the packager only added CI checks. So the check tracks the js/common scripts each package's build script or vite config names, and warns, without failing, when only those changed. Co-Authored-By: Claude Opus 5.5 --- sh/gh/versions.py | 45 ++++++++++++++++++++++++++++++------------ sh/gh/versions.test.py | 19 +++++++++++++++++- 2 files changed, 50 insertions(+), 14 deletions(-) diff --git a/sh/gh/versions.py b/sh/gh/versions.py index 60545103af3..133e1a22b84 100755 --- a/sh/gh/versions.py +++ b/sh/gh/versions.py @@ -25,7 +25,8 @@ versions.py check [--base ] [--head ] [--report] Fail when merging head into base changes a package's shipped sources at a released version, lowers a version, or leaves a lockfile entry behind. - Also lists the packages base already carries unreleased changes for. + Warns of the packages base already carries unreleased changes for, and + those only a shared build script changed for. --report lists the failures without failing. versions.py tag [--dry-run] Push each release tag HEAD's versions lack. Needs GH_TOKEN. @@ -114,6 +115,9 @@ class Package: rides: str | None = None # Whether its tag waits for the pinned moq-ffi release. ffi: bool = False + # Build scripts shared with other packages. A change to one may or may not + # change what this ships, which only a human can judge. + shared: list[str] = field(default_factory=list) def version(self, rev: str) -> str | None: text = show(rev, self.manifest.path) @@ -142,6 +146,10 @@ def packages(rev: str) -> list[Package]: if manifest.get("private") or not manifest.get("scripts", {}).get("release"): continue name = manifest["name"] + # The js/common scripts its build runs, such as the packager that + # rewrites its published manifest. + build = manifest["scripts"].get("build", "") + "\n" + (show(rev, f"{directory}/vite.config.ts") or "") + shared = sorted({f"js/common/{script}" for script in re.findall(r"\.\./common/([\w.-]+\.ts)\b", build)}) found.append(Package( name=name, manifest=Edit(f"{directory}/package.json", r'^(\t"version": ")([^"]+)(")'), @@ -152,6 +160,7 @@ def packages(rev: str) -> list[Package]: rf'^( "{re.escape(directory)}": \{{\n "name": "{re.escape(name)}",\n "version": ")([^"]+)(")', fix="bun install", )], + shared=shared, )) found.append(Package( name="moq-rs (Python)", @@ -261,7 +270,7 @@ def unreleased(head: str) -> list[Unreleased]: for pkg in found: if since[pkg.name] is None: continue - changed = git("diff", "--name-only", since[pkg.name], head, "--", *pkg.paths, *UNSHIPPED).stdout.split() + changed = git("diff", "--name-only", since[pkg.name], head, "--", *pkg.paths, *pkg.shared, *UNSHIPPED).stdout.split() if changed: stale[pkg.name] = Unreleased(pkg, pkg.version(head), since[pkg.name], changed) @@ -303,24 +312,33 @@ def how(pkg: Package, version: str) -> str: def check(base: str, head: str) -> tuple[list[str], list[str]]: - """The problems merging `head` into `base` brings, and the packages `base` - already holds unreleased changes for.""" + """The problems merging `head` into `base` brings, and warnings: the packages + `base` already holds unreleased changes for, and those only a shared build + script changed for.""" merged = merge(base, head) found = packages(merged) touched = { pkg.name for pkg in found - if git("diff", "--name-only", base, merged, "--", pkg.manifest.path, *pkg.paths, *UNSHIPPED).stdout.strip() + if git("diff", "--name-only", base, merged, "--", pkg.manifest.path, *pkg.paths, *pkg.shared, *UNSHIPPED).stdout.strip() } - problems, notes = [], [] + problems, warnings = [], [] for item in unreleased(merged): pkg = item.package + shared = [path for path in item.changed if path in pkg.shared] + only_shared = bool(item.changed) and len(shared) == len(item.changed) + fix = how(pkg, item.version) if item.alongside: reason = f"{item.alongside} releases, and the same tag would publish its released {item.version} again" + elif only_shared: + reason = f"only the shared build {'script' if len(shared) == 1 else 'scripts'} {', '.join(shared)} changed since {item.version} shipped" + fix = f"if that changed what it ships, {fix}" else: reason = f"{len(item.changed)} shipped {'file' if len(item.changed) == 1 else 'files'} changed since {item.version} shipped" - line = f"{pkg.name}: {reason}; {how(pkg, item.version)}." - (problems if group(pkg, found) & touched else notes).append(line) + if not group(pkg, found) & touched: + reason += f", already on {base}" + line = f"{pkg.name}: {reason}; {fix}." + (problems if group(pkg, found) & touched and not only_shared else warnings).append(line) for pkg in found: if pkg.name not in touched: @@ -337,7 +355,7 @@ def check(base: str, head: str) -> tuple[list[str], list[str]]: have = lock.read(text) if text is not None else None if have != new: problems.append(f"{pkg.name}: {lock.path} carries {have}, not {new}; run `{lock.fix}`.") - return problems, notes + return problems, warnings def release_assets(tag: str) -> set[str]: @@ -435,14 +453,15 @@ def main() -> int: return 0 try: - problems, notes = check(args.base, args.head) + problems, warnings = check(args.base, args.head) except RuntimeError as error: print(error, file=sys.stderr) return 2 - for line in notes: - print(f"already unreleased on {args.base}: {line}") + actions = os.environ.get("GITHUB_ACTIONS") + for line in warnings: + print(f"::warning::{line}" if actions else f"warning: {line}") for line in problems: - print(f"::error::{line}" if os.environ.get("GITHUB_ACTIONS") and not args.report else line) + print(f"::error::{line}" if actions and not args.report else line) if not problems: print(f"no version problems merging {args.head} into {args.base}") return 0 diff --git a/sh/gh/versions.test.py b/sh/gh/versions.test.py index 1249e767352..81711d90a73 100644 --- a/sh/gh/versions.test.py +++ b/sh/gh/versions.test.py @@ -15,7 +15,8 @@ def package_json(name: str, version: str, private: bool = False) -> str: - manifest = {"name": name, "version": version, "scripts": {"release": "bun ../common/release.ts"}} + scripts = {"build": "rimraf dist && tsc -b && bun ../common/package.ts", "release": "bun ../common/release.ts"} + manifest = {"name": name, "version": version, "scripts": scripts} if private: manifest["private"] = True return json.dumps(manifest, indent="\t") + "\n" @@ -69,6 +70,10 @@ def setUp(self): "js/a/src/index.ts": "export {};\n", "js/b/package.json": package_json("@moq/b", "0.2.0"), "js/b/src/index.ts": "export {};\n", + "js/b/vite.config.ts": 'import { worklet } from "../common/vite-plugin-worklet.ts";\n', + "js/common/package.ts": "", + "js/common/release.ts": "", + "js/common/vite-plugin-worklet.ts": "", "js/wasm/package.json": package_json("@moq/wasm", "0.0.0", private=True), "js/wasm/src/index.ts": "export {};\n", "bun.lock": BUN_LOCK, @@ -185,6 +190,17 @@ def test_unshipped_files_are_not_a_release(self): "dart/moq/test/moq_test.dart": "", }), ([], [])) + def test_shared_build_scripts_warn_the_packages_that_run_them(self): + # Whether a shared build script changed what a package ships takes a human. + self.assertEqual(self.backport({"js/common/package.ts": "// fix\n"}), ([], ["@moq/a", "@moq/b"])) + self.assertIn("only the shared build script js/common/package.ts changed since 0.1.0 shipped; if that", self.notes[0]) + self.assertEqual(self.backport({"js/common/vite-plugin-worklet.ts": "// fix\n"}), ([], ["@moq/b"])) + # Publishing changes nothing a package ships. + self.assertEqual(self.backport({"js/common/release.ts": "// fix\n"}), ([], [])) + # A change of its own still fails. + self.assertEqual(self.backport({"js/common/package.ts": "// fix\n", "js/a/src/index.ts": FIX}), (["@moq/a"], ["@moq/b"])) + self.assertIn("2 shipped files changed", self.problems[0]) + def test_private_package_is_skipped(self): self.assertEqual(self.backport({"js/wasm/src/index.ts": FIX}), ([], [])) @@ -209,6 +225,7 @@ def test_every_ecosystem(self): def test_unreleased_changes_already_on_base_are_only_noted(self): self.commit("fix(b): merged without a bump", {"js/b/src/index.ts": FIX}) self.assertEqual(self.backport({"js/a/src/index.test.ts": ""}), ([], ["@moq/b"])) + self.assertIn("since 0.2.0 shipped, already on release;", self.notes[0]) # A backport touching it must bump it, which ships the earlier change too. self.assertEqual(self.backport({"js/b/src/other.ts": FIX}), (["@moq/b"], [])) self.assertIn("2 shipped files changed", self.problems[0]) From 7540dfcb9a9344e959720639a9a55a76c06d77f9 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 10:29:14 -0700 Subject: [PATCH 10/12] quest: note the CI-only packager warnings on release Co-Authored-By: Claude Opus 5.5 --- quest/m1/release-versions.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/quest/m1/release-versions.md b/quest/m1/release-versions.md index b2c232446d5..561f706d6f4 100644 --- a/quest/m1/release-versions.md +++ b/quest/m1/release-versions.md @@ -58,4 +58,8 @@ Remaining, on `release` (neither publish nor tag by hand here): ([moq-dev/moq#5354](https://github.com/moq-dev/moq/pull/5354)) changed what it ships. + It also warns for a dozen JS packages over `js/common/package.ts`; that + change ([e44d51fecd](https://github.com/moq-dev/moq/commit/e44d51fecd)) + only added CI API-report checks, so they need no release. + Public API: none. Wire: none. From b5432b5d0fa47e450a08845541bcf02ad6d41ba6 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 29 Oct 2026 08:30:59 -0700 Subject: [PATCH 11/12] revert the backport version gate; the quest is being deleted Co-Authored-By: Claude Opus 5.5 --- .github/justfile | 6 - .github/workflows/alert.yml | 1 - .github/workflows/release-gate.yml | 30 -- .github/workflows/release-tag.yml | 52 ---- CONTRIBUTING.md | 7 +- quest/m1/README.md | 2 +- quest/m1/release-versions.md | 65 +--- quest/m2/release/cpp-release.md | 18 +- quest/m2/release/dart-publish.md | 2 +- sh/gh/versions.py | 477 ----------------------------- sh/gh/versions.test.py | 376 ----------------------- 11 files changed, 30 insertions(+), 1006 deletions(-) delete mode 100644 .github/workflows/release-tag.yml delete mode 100755 sh/gh/versions.py delete mode 100644 sh/gh/versions.test.py diff --git a/.github/justfile b/.github/justfile index 9c94a6b6240..1d5c0d883b6 100644 --- a/.github/justfile +++ b/.github/justfile @@ -49,7 +49,6 @@ test: bun sh/gh/queue.test.ts python3 sh/gh/install-formula.test.py sh/gh/release.test.sh - python3 sh/gh/versions.test.py python3 sh/gh/ffi.test.py bun test sh/gh/ffi.test.ts sh/gh/cpp.test.ts python3 sh/rs/package-rename.test.py @@ -71,11 +70,6 @@ test: release *args: sh/gh/release.sh "$@" -# Check a `release` PR bumps the hand-versioned packages it changes, or push their release tags: `just gh versions check|tag`. -[positional-arguments] -versions *args: - python3 sh/gh/versions.py "$@" - # Post a failed run to Discord, or check cron and publishing alert coverage. [positional-arguments] alert *args: diff --git a/.github/workflows/alert.yml b/.github/workflows/alert.yml index ec15e8f7f48..929829f6de0 100644 --- a/.github/workflows/alert.yml +++ b/.github/workflows/alert.yml @@ -39,7 +39,6 @@ on: - Release RS - Release Swift FFI - Release Swift Lib - - Release tag - release-winget - rpm-repo - Smoke diff --git a/.github/workflows/release-gate.yml b/.github/workflows/release-gate.yml index 241bf71668d..ad27d8f8c53 100644 --- a/.github/workflows/release-gate.yml +++ b/.github/workflows/release-gate.yml @@ -342,35 +342,6 @@ jobs: uses: ./.github/workflows/features.yml secrets: inherit - # A backport must bump each hand-versioned package whose shipped sources it - # changes, or its release workflow skips the fix as already published. Only a - # human picks the version, so this checks and never edits. The cut (a PR from - # `main`) only reports: it carries changes that need no release, which only - # the maintainer judges. - versions: - name: Versions - if: github.event.action != 'closed' - runs-on: ubuntu-24.04-arm - timeout-minutes: 10 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - persist-credentials: false - # `release`'s first-parent history and the release tags say which - # version of each package shipped. - fetch-depth: 0 - - uses: ./.github/actions/just - - name: Check versions - run: | - mode=() - # Only this repository's own `main` is the cut; a fork's `main` is a backport. - [[ "$HEAD_REF" != main || "$HEAD_REPO" != "$GITHUB_REPOSITORY" ]] || mode=(--report) - just gh versions check --base "origin/$BASE_REF" "${mode[@]}" - env: - BASE_REF: ${{ github.base_ref }} - HEAD_REF: ${{ github.head_ref }} - HEAD_REPO: ${{ github.event.pull_request.head.repo.full_name }} - android: name: Android needs: scope @@ -540,7 +511,6 @@ jobs: - scope - platform - features - - versions - android - api-diff - audio-quality diff --git a/.github/workflows/release-tag.yml b/.github/workflows/release-tag.yml deleted file mode 100644 index d3db8fcf0a5..00000000000 --- a/.github/workflows/release-tag.yml +++ /dev/null @@ -1,52 +0,0 @@ -name: Release tag - -# Dart, C, and C++ publish from a tag rather than on a push to `release`. This -# pushes each such tag a version on `release` lacks, so a version a human set -# there publishes once it lands; it never picks or writes a version. C and C++ -# tags wait for the moq-ffi release their pin names. See sh/gh/versions.py. -permissions: - contents: read - -on: - # Every push, not only those touching a package: a C or C++ tag waiting on - # its pinned moq-ffi release retries on the next one. - push: - branches: [release] - # Retrying a tag by hand. Dispatch it on `release`; the job refuses any other - # ref. - workflow_dispatch: - -# Each run tags the pushed `release`, so a newer push makes an in-flight run -# moot. -concurrency: - group: release-tag - cancel-in-progress: true - -jobs: - tag: - name: Tag - runs-on: ubuntu-latest - if: ${{ github.repository_owner == 'moq-dev' && github.ref == 'refs/heads/release' }} - - steps: - # The app's token, as release-plz uses, so the tags it pushes run their - # release workflows. - - name: Generate GitHub token - uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3 - id: generate-token - with: - client-id: ${{ secrets.APP_CLIENT_ID }} - private-key: ${{ secrets.APP_PRIVATE_KEY }} - - - name: Checkout repository - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - fetch-depth: 0 - token: ${{ steps.generate-token.outputs.token }} - - - uses: ./.github/actions/just - - - name: Push release tags - env: - GH_TOKEN: ${{ steps.generate-token.outputs.token }} - run: just gh versions tag diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d1d9d9feacb..9fee425a82a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -97,19 +97,18 @@ For an advisory against Quinn, follow the advisory triage in [`rs/moq-quic/READM - Smoke clients call only the latest published APIs. A release cut updates `test/smoke/clients` to the APIs being published, then verifies the affected smoke lanes. - A release is cut by hand: a PR merging `main` into `release`, with a merge commit. - A breaking cut first writes, on `main`, each changed package's version-transition section in its `MIGRATE.md` (`## 0.3 -> 0.4`) from the migration audit: the `!` subjects since the `release` merge base, each of those PRs' Impact section, and the API diff. -- Suggest a `release` backport for any serious bug fix (crash, security, data loss, broken interop), whether found on `main` or reported against a release. Land it on `main` first, then port it onto `release` as a separate PR that keeps the published API, so it ships as a patch. The Release gate's API diff (`just api-diff --deny`) fails a backport that changes one; on the cut PR, whose head is `main`, it only reports. Leave crate versions to release-plz, but bump every other package whose shipped sources it changes, picking the level by hand (see [Versions](#versions)); the Release gate's version check (`just gh versions check`) fails a backport that leaves one at its released version, since its release workflow would skip the fix as already published. +- Suggest a `release` backport for any serious bug fix (crash, security, data loss, broken interop), whether found on `main` or reported against a release. Land it on `main` first, then port it onto `release` as a separate PR that keeps the published API, so it ships as a patch. The Release gate's API diff (`just api-diff --deny`) fails a backport that changes one; on the cut PR, whose head is `main`, it only reports. - After any release publishes, patch or breaking, run `just test wire-compat --remeasure` on `main` and land the rewritten `test/interop/compat/planned-breaks.json` with refreshed reasons. Otherwise the nightly wire-compat check fails on the stale pins. - Nothing merges `release` into `main` automatically. Before cutting a minor or breaking release, the maintainer merges `release` into `main` once, so trunk carries the published versions, CHANGELOGs, and any `release`-only commits. It lands as a merge commit with `--admin`, never squashed, so the cut starts from an advanced merge base. # Versions -Releases are cut separately; bump only when asked, or in a `release` backport. No tool picks a version: a human judges the semver level. On a PR into `release`, the Release gate's version check fails a backport that changes a hand-versioned package's shipped sources while leaving it at a released version, lowers a version, or leaves its lockfile entry behind; on the cut it only reports. A JavaScript, Python, Swift, or Kotlin version ships when it first reaches `release`. A Dart, C, or C++ version ships when its tag is pushed, and the Release tag workflow pushes the tag for each such version on `release` that lacks one; C and C++ wait until the moq-ffi release `cpp/moq/FFI_VERSION` pins is published, retrying on each later push. Go derives its own patch. Each package's version lives in one place: +Releases are cut separately; bump only when asked. Each package's version lives in one place: - **Rust**: release-plz owns crate versions and Rust dependency requirements. - **JavaScript**: `js/*/package.json` packages with a `scripts.release` entry (skip private ones like `@moq/wasm`), plus the matching workspace version in `bun.lock`. - **Python**: `py/moq-rs/pyproject.toml`, plus the matching `moq-rs` entry in the root `uv.lock`; `py/moq-ffi` follows the `moq-ffi-v*` tag and Rust crate. - **Swift**: `swift/VERSION`. **Kotlin**: `moq.version` in `kt/gradle.properties`. **Dart**: `version` in `dart/moq/pubspec.yaml`. Their FFI counterparts track the Rust crate. - **Go**: `go/wrapper/VERSION` holds a human-owned `MAJOR.MINOR` line; CI derives the patch, so only edit it for a breaking API. Leave the placeholder FFI version in `go.mod` alone. -- **C**: `c/moq/VERSION`, human-owned; a `c-v` tag releases it. - **C++**: `cpp/moq/VERSION`, human-owned; a `cpp-v` tag releases it. -- **OBS**: `cpp/obs/VERSION`, human-owned; each C++ release also cuts `obs-moq-v`, and refuses one whose version already names an earlier release. So the version check requires a plugin bump with every C++ release, and a C++ bump with every plugin change, and the Release tag workflow holds a `cpp-v` tag while the plugin is at a released version. +- **OBS**: `cpp/obs/VERSION`, human-owned; each C++ release also cuts `obs-moq-v`, and refuses one whose version already names an earlier release. diff --git a/quest/m1/README.md b/quest/m1/README.md index 68c54c710bf..48bc378104c 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -186,7 +186,7 @@ removed). - [JS SETUP rejection ownership](/quest/m1/js-setup-rejection.md) - reproduce an unhandled version-refusal rejection without weakening the refusal - [moq-clock moves into kio::clock](/quest/m1/kio-clock.md) - moq-clock and its test macro become `kio::clock` and `#[kio::test]` before the cut publishes them - [Poll-first async](/quest/m1/poll-first/README.md) - library crates compile no coroutines by default: async wrappers are generated from `poll_*` and gated behind an `async` feature -- [Every package's release backport ships](/quest/m1/release-versions.md) - the Release gate fails a backport that leaves a changed package at its released version; backport that check to `release` and hand-bump the fixes already there +- [Every package's release backport ships](/quest/m1/release-versions.md) - a bot bumps every hand-versioned package changed on `release`, so backports publish without a manual bump ## Related diff --git a/quest/m1/release-versions.md b/quest/m1/release-versions.md index 561f706d6f4..7888c2d6315 100644 --- a/quest/m1/release-versions.md +++ b/quest/m1/release-versions.md @@ -1,15 +1,12 @@ -# [S] Every package's release backport ships +# [M] Every package's release backport ships ## Goal -A fix merged into `release` for any hand-versioned package publishes. Today -release-plz bumps only Rust crates, so a backport touching only a JS, Python, -Dart, Kotlin, Swift, C, or C++ package can land on `release` unchanged in -version, and its release workflow skips it as already published. On `main`, -the Release gate's Versions check (`just gh versions check`) now fails such a -backport, and the Release tag workflow pushes the Dart, C, and C++ tags for -versions a human set on `release`. Done when both run on `release` and the -fixes `release` already carries unreleased are bumped there. +A fix merged into `release` for any published package publishes without a +hand-made version bump. Today release-plz bumps only Rust crates, so a +backport touching only a JS, Python, Dart, Kotlin, Swift, Go, C, or C++ +package lands on `release` unchanged in version and its release workflow +skips it as already published. ## Plan @@ -20,46 +17,18 @@ Found 2026-10-28: the @moq/flate and @moq/net backports release PR merged, but Release JS logged "already published, skipping" and a separate hand bump was needed. -Decided 2026-10-28 (maintainer), superseding the earlier bump bot: "we can -only use an automated bump tool like this if it understands semver. If it -doesn't, we need manual bumps." Nothing here can judge semver, since -`just api-diff --deny` doesn't cover the hand-written wrappers. +Decided 2026-10-28 (maintainer): -- ✅ Bumps on `release` stay manual; a human picks the level. No workflow or - script picks or writes a version. -- ✅ A Release gate check fails a backport that changes a hand-versioned - package's shipped sources without moving its version past the released one, - or without moving its `bun.lock`/`uv.lock` entry. It names each package and - how to bump it, requires an OBS bump with each C++ release and the reverse, - and never edits. -- ✅ A workflow still pushes the missing `moq-dart-v*`, `c-v*`, and `cpp-v*` - tag for a version a human set on `release`: C and C++ only once the pinned - moq-ffi release is published, and C++ only once OBS is off its released - version. +- Scope: every hand-versioned package listed in CONTRIBUTING.md's Versions + section, not just JS. +- Mechanism: a release-plz-like bot. On each push to `release`, it opens or + updates one PR that patch-bumps every package whose sources changed since + its last published version (with the matching lockfile entries, e.g. + `bun.lock`, `uv.lock`); merging it publishes through the existing release + workflows. Prefer an existing maintained tool if one covers these + ecosystems; extending release-plz's own PR is also fine. +- Until this lands, a backport author bumps by hand. -Remaining, on `release` (neither publish nor tag by hand here): - -1. Backport the tooling as a tooling-only PR: `sh/gh/versions.py`, - `sh/gh/versions.test.py`, the `.github/justfile` recipe and test line, - `.github/workflows/release-tag.yml`, its `alert.yml` entry, and the - Versions job in `release-gate.yml`. Its own Release gate runs the check. - Its merge is a push, so Release tag pushes `moq-dart-v0.1.1`, which - `release` has carried untagged since the drain backport - ([335e0b5f29](https://github.com/moq-dev/moq/commit/335e0b5f29)); check - that Release Dart publishes it ([dart-publish](/quest/m2/release/dart-publish.md)). -2. Bump by hand the packages `release` already carries unreleased changes for - (`just gh versions check --base origin/release --head origin/release` lists - them): - - @moq/hang 0.5.4, from - [moq-dev/moq#6631](https://github.com/moq-dev/moq/pull/6631). - - @moq/publish 0.5.3, from - [moq-dev/moq#6598](https://github.com/moq-dev/moq/pull/6598). - - @moq/boy 0.4.1, only if swapping its esbuild devDependency for rolldown - ([moq-dev/moq#5354](https://github.com/moq-dev/moq/pull/5354)) changed - what it ships. - - It also warns for a dozen JS packages over `js/common/package.ts`; that - change ([e44d51fecd](https://github.com/moq-dev/moq/commit/e44d51fecd)) - only added CI API-report checks, so they need no release. +Update CONTRIBUTING.md's Versions and backport bullets to describe the result. Public API: none. Wire: none. diff --git a/quest/m2/release/cpp-release.md b/quest/m2/release/cpp-release.md index d11abffa8d0..23fe95f1a4a 100644 --- a/quest/m2/release/cpp-release.md +++ b/quest/m2/release/cpp-release.md @@ -5,8 +5,9 @@ The OBS release path is dry-run nightly, then the first `cpp-v` tag is pushed and `release-cpp.yml` publishes one `moq-cpp--` archive per target to its GitHub release, plus the -matching `obs-moq-v` plugin release, once `main` reaches `release`; -check with `gh release list --repo moq-dev/moq | grep cpp-v`. +matching `obs-moq-v` plugin release. A maintainer cuts it by hand +once `main` reaches `release`; check with +`gh release list --repo moq-dev/moq | grep cpp-v`. ## Plan @@ -25,14 +26,11 @@ The nightly C++ and OBS release checks pass: the which failed only in unrelated lanes, checked the installed consumers of all four packaged archives and built both OBS plugins against them. It packages this run's prebuilt moq-ffi input, while a tag with `cpp/moq/FFI_VERSION` unset builds moq-ffi from source, a -path no nightly checks. The Release tag workflow pushes the `cpp-v` tag -itself on the first push to `release` once the pin names a published moq-ffi -release and OBS sits at an unreleased version, and never while the pin is -unset. So if the pin is still unset, first dispatch `release-cpp.yml` on the -ref to be tagged without `prebuilt-test`: it runs the tag's source build and -archive check on all four hosts, then both OBS builds, and publishes nothing. -Then tag it by hand. This quest owns that dry run, any hand tagging, and -verifying the published downloads. +path no nightly checks. So if the pin is still unset, first dispatch +`release-cpp.yml` on the ref to be tagged without `prebuilt-test`: it runs +the tag's source build and archive check on all four hosts, then both OBS +builds, and publishes nothing. This quest owns that dry run, tagging, and verifying the +published downloads. ## Required diff --git a/quest/m2/release/dart-publish.md b/quest/m2/release/dart-publish.md index d17f87c6330..23fb5f21536 100644 --- a/quest/m2/release/dart-publish.md +++ b/quest/m2/release/dart-publish.md @@ -19,7 +19,7 @@ can proceed independently; cutting the tag cannot. - Confirm trusted publishing is configured for `moq` (repository `moq-dev/moq`, tag pattern `moq-dart-v{{version}}`). - Raise `dart/moq/pubspec.yaml`'s `moq_ffi` floor to the release the wrapper now needs (the media namespace and codec types), since `^0.4.3` resolves bindings without them. -- The Release tag workflow pushes each `moq-dart-v*` tag a version on `release` lacks. Its first run on `release` tags `moq-dart-v0.1.1`, the drain backport ([335e0b5f29](https://github.com/moq-dev/moq/commit/335e0b5f29)); confirm Release Dart publishes it unattended, and that the cut's binding-break release publishes the same way. +- Cut the next `moq-dart-v*` release with the binding breaks and confirm the workflow publishes it. - Verify the published `moq_ffi` resolves its native asset from a clean machine with no monorepo checkout, since that download path is the one CI never exercises. diff --git a/sh/gh/versions.py b/sh/gh/versions.py deleted file mode 100755 index 133e1a22b84..00000000000 --- a/sh/gh/versions.py +++ /dev/null @@ -1,477 +0,0 @@ -#!/usr/bin/env python3 -"""Check that backports bump hand-versioned packages, and push their release tags. - -release-plz bumps Rust crates on `release`; every other package's version is -bumped by hand, since only a human can judge a change's semver level. A -backport that changes one of them without a bump lands unchanged in version, -and its release workflow skips it as already published. `check` fails that PR; -it never picks or writes a version. - -Two kinds of package publish differently: - -- JS, Python, Swift, and Kotlin publish on a push to `release` once the registry - lacks their version. Each first-parent commit is a push their release workflow - saw, so one last shipped at the oldest first-parent commit carrying its - current version. -- Dart, C, and C++ publish from a `` tag, and OBS rides each - C++ tag. One last shipped at its current version's tag; a current version - with no tag is unreleased, so `tag` pushes it, while C and C++ wait until the - moq-ffi release `cpp/moq/FFI_VERSION` pins is published. - -Any change to a package's sources since it last shipped is unreleased, so it -needs a bump. - -Usage: - versions.py check [--base ] [--head ] [--report] - Fail when merging head into base changes a package's shipped sources at a - released version, lowers a version, or leaves a lockfile entry behind. - Warns of the packages base already carries unreleased changes for, and - those only a shared build script changed for. - --report lists the failures without failing. - versions.py tag [--dry-run] - Push each release tag HEAD's versions lack. Needs GH_TOKEN. -""" - -import argparse -from dataclasses import dataclass, field -import importlib.util -import json -import os -from pathlib import Path -import re -import subprocess -import sys - -REPO = "moq-dev/moq" - -# The C and C++ packages' moq-ffi pin. -FFI_VERSION = "cpp/moq/FFI_VERSION" -FFI_SHA256 = "cpp/moq/FFI_SHA256.json" -FFI_CMAKE = "cpp/moq/cmake/moq-ffi.cmake" - - -def load_ffi(): - """sh/cpp/ffi.py, which names the pinned assets. Loaded only once C or C++ is - present, since a `release` from before their cut lacks it.""" - spec = importlib.util.spec_from_file_location("ffi", Path(__file__).parents[1] / "cpp/ffi.py") - module = importlib.util.module_from_spec(spec) - spec.loader.exec_module(module) - return module - - -def git(*args: str, check: bool = True) -> subprocess.CompletedProcess: - return subprocess.run(["git", *args], capture_output=True, text=True, check=check) - - -def show(rev: str, path: str) -> str | None: - result = git("show", f"{rev}:{path}", check=False) - return result.stdout if result.returncode == 0 else None - - -def precedence(version: str) -> tuple | None: - """A sort key for semver precedence, or None if `version` is not semver.""" - match = re.fullmatch(r"(\d+)\.(\d+)\.(\d+)(?:-([0-9A-Za-z.-]+))?(?:\+[0-9A-Za-z.-]+)?", version) - if not match: - return None - major, minor, micro, pre = match.groups() - # A prerelease sorts before its release. Its identifiers compare in order, - # numeric ones by value and below alphanumeric ones; build metadata is ignored. - ids = [(0, int(part), "") if part.isdigit() else (1, 0, part) for part in pre.split(".")] if pre else [] - return (int(major), int(minor), int(micro), pre is None, ids) - - -def newer(version: str, than: str) -> bool | None: - """Whether `version` sorts after `than` by semver, or None if either is not one.""" - a, b = precedence(version), precedence(than) - return None if a is None or b is None else a > b - - -@dataclass -class Edit: - """A version string in a file: group 2 of `pattern` is the version.""" - - path: str - pattern: str - # The command that rewrites it, for a lockfile entry. - fix: str | None = None - - def read(self, text: str) -> str | None: - match = re.search(self.pattern, text, flags=re.M) - return match.group(2) if match else None - - -@dataclass -class Package: - name: str - # The version's source of truth; its value at a commit says what shipped. - manifest: Edit - # Shipped sources, as git pathspecs. - paths: list[str] - # Copies of the version that must move with it, such as a lockfile entry. - locks: list[Edit] = field(default_factory=list) - # The prefix of the tags naming its releases, for a package a tag publishes. - tag: str | None = None - # The package whose tag publishes this one, when it is not its own. - rides: str | None = None - # Whether its tag waits for the pinned moq-ffi release. - ffi: bool = False - # Build scripts shared with other packages. A change to one may or may not - # change what this ships, which only a human can judge. - shared: list[str] = field(default_factory=list) - - def version(self, rev: str) -> str | None: - text = show(rev, self.manifest.path) - return None if text is None else self.manifest.read(text) - - -# Files none of these packages ship, so changing one alone is not a release. -UNSHIPPED = [ - ":(exclude,glob)**/*.test.*", - ":(exclude,glob)**/tests/**", - ":(exclude,glob)**/Tests/**", - ":(exclude,glob)**/MIGRATE.md", -] - - -def packages(rev: str) -> list[Package]: - """Every hand-versioned package present at `rev`. Go derives its patch in CI.""" - found = [] - listing = git("ls-tree", "--name-only", rev, "js/").stdout.split() - for directory in sorted(listing): - text = show(rev, f"{directory}/package.json") - if text is None: - continue - manifest = json.loads(text) - # CONTRIBUTING.md's rule: a release script, and not private. - if manifest.get("private") or not manifest.get("scripts", {}).get("release"): - continue - name = manifest["name"] - # The js/common scripts its build runs, such as the packager that - # rewrites its published manifest. - build = manifest["scripts"].get("build", "") + "\n" + (show(rev, f"{directory}/vite.config.ts") or "") - shared = sorted({f"js/common/{script}" for script in re.findall(r"\.\./common/([\w.-]+\.ts)\b", build)}) - found.append(Package( - name=name, - manifest=Edit(f"{directory}/package.json", r'^(\t"version": ")([^"]+)(")'), - # API reports, benches, and examples stay out of the built package. - paths=[directory, *(f":(exclude){directory}/{d}" for d in ("api", "bench", "examples"))], - locks=[Edit( - "bun.lock", - rf'^( "{re.escape(directory)}": \{{\n "name": "{re.escape(name)}",\n "version": ")([^"]+)(")', - fix="bun install", - )], - shared=shared, - )) - found.append(Package( - name="moq-rs (Python)", - manifest=Edit("py/moq-rs/pyproject.toml", r'^(\[project\]\n(?:(?!\[).*\n)*?version = ")([^"]+)(")'), - paths=["py/moq-rs", ":(exclude)py/moq-rs/benches", ":(exclude)py/moq-rs/examples"], - locks=[Edit("uv.lock", r'^(\[\[package\]\]\nname = "moq-rs"\nversion = ")([^"]+)(")', fix="uv lock")], - )) - found.append(Package( - name="Moq (Swift)", - manifest=Edit("swift/VERSION", r"\A()(\S+)(\s*)\Z"), - paths=["swift/Sources/Moq", "swift/Package.swift.template"], - )) - found.append(Package( - name="dev.moq:moq (Kotlin)", - manifest=Edit("kt/gradle.properties", r"^(moq\.version=)(\S+)()$"), - paths=["kt/moq", ":(exclude,glob)kt/moq/src/*Test/**", "kt/build.gradle.kts", "kt/settings.gradle.kts", "kt/gradle.properties"], - )) - version_file = r"\A()(\S+)(\s*)\Z" - tagged = [ - Package( - name="moq (Dart)", - manifest=Edit("dart/moq/pubspec.yaml", r"^(version: )(\S+)()$"), - paths=["dart/moq", ":(exclude)dart/moq/test", ":(exclude)dart/moq/bench"], - tag="moq-dart-v", - ), - # The C package links the same pinned moq-ffi input as C++, through C++'s - # CMake module. - Package( - name="moq (C)", - manifest=Edit("c/moq/VERSION", version_file), - paths=["c/moq", ":(exclude)c/moq/test", FFI_VERSION, FFI_SHA256, FFI_CMAKE], - tag="c-v", - ffi=True, - ), - Package( - name="moq (C++)", - manifest=Edit("cpp/moq/VERSION", version_file), - paths=["cpp/moq", ":(exclude)cpp/moq/test", ":(exclude)cpp/moq/bench"], - tag="cpp-v", - ffi=True, - ), - # Release Cpp builds the plugin on every C++ tag and cuts its obs-moq tag. - Package( - name="obs-moq (OBS)", - manifest=Edit("cpp/obs/VERSION", version_file), - paths=["cpp/obs", ":(exclude)cpp/obs/test"], - tag="obs-moq-v", - rides="moq (C++)", - ), - ] - # `release` gains a package only when a cut brings it. - found += [pkg for pkg in tagged if show(rev, pkg.manifest.path) is not None] - return found - - -def released(pkg: Package, head: str) -> str | None: - """The commit that shipped `pkg`'s current version, or None while it is unreleased.""" - current = pkg.version(head) - if current is None: - raise ValueError(f"no version in {pkg.manifest.path} at {head}") - if pkg.tag: - result = git("rev-parse", "--quiet", "--verify", f"refs/tags/{pkg.tag}{current}^{{commit}}", check=False) - return result.stdout.strip() or None - # The oldest first-parent commit since which it has had its current version. - log = git("log", "--first-parent", "--format=%H", head, "--", pkg.manifest.path).stdout.split() - for rev in log: - if pkg.version(f"{rev}^") != current: - return rev - return None - - -def baselines(found: list[Package], head: str) -> dict[str, str | None]: - """`released` for each package in `found`.""" - since = {pkg.name: released(pkg, head) for pkg in found} - # A rider's own tag comes later, from the release its host's tag starts, so - # while that run is in flight its version shipped at the host's tag. - for rider in found: - host = since.get(rider.rides) if rider.rides else None - if since[rider.name] is None and host and rider.version(host) == rider.version(head): - since[rider.name] = host - return since - - -def group(pkg: Package, found: list[Package]) -> set[str]: - """`pkg` and every package one tag publishes along with it.""" - names = {pkg.name, *([pkg.rides] if pkg.rides else [])} - return names | {other.name for other in found if other.rides in names} - - -@dataclass -class Unreleased: - """A package still at a released version that must move to ship.""" - - package: Package - version: str - since: str - changed: list[str] - # The package it releases alongside, when its own sources did not change. - alongside: str | None = None - - -def unreleased(head: str) -> list[Unreleased]: - """Every package at `head` whose released version no longer matches what would ship.""" - found = packages(head) - since = baselines(found, head) - stale: dict[str, Unreleased] = {} - for pkg in found: - if since[pkg.name] is None: - continue - changed = git("diff", "--name-only", since[pkg.name], head, "--", *pkg.paths, *pkg.shared, *UNSHIPPED).stdout.split() - if changed: - stale[pkg.name] = Unreleased(pkg, pkg.version(head), since[pkg.name], changed) - - # One tag publishes a package and every package riding it, so once any of - # them releases, each other one still at a released version must move too. - for rider in found: - members = [pkg for pkg in found if pkg.name == rider.rides] - if not members: - continue - members.append(rider) - releasing = [pkg.name for pkg in members if pkg.name in stale or since[pkg.name] is None] - if not releasing: - continue - for pkg in members: - if pkg.name not in stale and since[pkg.name] is not None: - stale[pkg.name] = Unreleased(pkg, pkg.version(head), since[pkg.name], [], alongside=releasing[0]) - return [stale[pkg.name] for pkg in found if pkg.name in stale] - - -def merge(base: str, head: str) -> str: - """A commit landing `head` on `base` as a PR's merge does: first parent `base`.""" - result = git("merge-tree", "--write-tree", base, head, check=False) - if result.returncode: - raise RuntimeError(f"{head} does not merge cleanly into {base}") - tree = result.stdout.split()[0] - identity = {f"GIT_{who}_{key}": value for who in ("AUTHOR", "COMMITTER") - for key, value in (("NAME", "versions.py"), ("EMAIL", "versions.py@localhost"))} - return subprocess.run(["git", "commit-tree", tree, "-p", base, "-p", head, "-m", f"merge {head} into {base}"], - capture_output=True, text=True, check=True, env={**os.environ, **identity}).stdout.strip() - - -def how(pkg: Package, version: str) -> str: - """How to move `pkg` past its released `version`.""" - text = f"raise its version in {pkg.manifest.path} past {version} (CONTRIBUTING.md's Versions: patch for a fix)" - fixes = [lock.fix for lock in pkg.locks if lock.fix] - if fixes: - text += ", then run " + " and ".join(f"`{fix}`" for fix in fixes) - return text - - -def check(base: str, head: str) -> tuple[list[str], list[str]]: - """The problems merging `head` into `base` brings, and warnings: the packages - `base` already holds unreleased changes for, and those only a shared build - script changed for.""" - merged = merge(base, head) - found = packages(merged) - touched = { - pkg.name for pkg in found - if git("diff", "--name-only", base, merged, "--", pkg.manifest.path, *pkg.paths, *pkg.shared, *UNSHIPPED).stdout.strip() - } - - problems, warnings = [], [] - for item in unreleased(merged): - pkg = item.package - shared = [path for path in item.changed if path in pkg.shared] - only_shared = bool(item.changed) and len(shared) == len(item.changed) - fix = how(pkg, item.version) - if item.alongside: - reason = f"{item.alongside} releases, and the same tag would publish its released {item.version} again" - elif only_shared: - reason = f"only the shared build {'script' if len(shared) == 1 else 'scripts'} {', '.join(shared)} changed since {item.version} shipped" - fix = f"if that changed what it ships, {fix}" - else: - reason = f"{len(item.changed)} shipped {'file' if len(item.changed) == 1 else 'files'} changed since {item.version} shipped" - if not group(pkg, found) & touched: - reason += f", already on {base}" - line = f"{pkg.name}: {reason}; {fix}." - (problems if group(pkg, found) & touched and not only_shared else warnings).append(line) - - for pkg in found: - if pkg.name not in touched: - continue - old, new = pkg.version(base), pkg.version(merged) - if old and new != old: - order = newer(new, old) - if order is None: - problems.append(f"{pkg.name}: cannot order {new} after {old}, the version on {base}, as semver.") - elif not order: - problems.append(f"{pkg.name}: {new} does not come after {old}, the version on {base}.") - for lock in pkg.locks: - text = show(merged, lock.path) - have = lock.read(text) if text is not None else None - if have != new: - problems.append(f"{pkg.name}: {lock.path} carries {have}, not {new}; run `{lock.fix}`.") - return problems, warnings - - -def release_assets(tag: str) -> set[str]: - """The asset names on the GitHub release for `tag`, empty when it has none.""" - result = subprocess.run(["gh", "release", "view", tag, "--repo", REPO, "--json", "assets", "--jq", ".assets[].name"], - capture_output=True, text=True) - if result.returncode: - if "release not found" in result.stderr.lower(): - return set() - raise RuntimeError(result.stderr) - return set(result.stdout.split()) - - -def ffi_blocker(head: str) -> str | None: - """Why a C or C++ tag on `head` must wait for moq-ffi, or None once its pin is published.""" - text = show(head, FFI_VERSION) or "" - pins = [line.strip() for line in text.splitlines() if line.strip() and not line.strip().startswith("#")] - if len(pins) != 1: - return f"{FFI_VERSION} pins no single moq-ffi release" - release = pins[0] - ffi = load_ffi() - hashes = json.loads(show(head, FFI_SHA256) or "{}") - published = release_assets(f"moq-ffi-v{release}") - missing = [asset for asset in (ffi.name(release, target) + ".tar.gz" for target in ffi.TARGETS) - if asset not in hashes or asset not in published] - if missing: - return f"moq-ffi v{release} lacks the published, pinned {', '.join(missing)}" - return None - - -@dataclass -class Tag: - name: str - package: Package - # Why it cannot be pushed yet, or None. - blocker: str | None - - -def pending(head: str) -> list[Tag]: - """Each release tag HEAD's versions lack, for the packages that push their own.""" - found = packages(head) - since = baselines(found, head) - tags = [] - ffi: str | None = None - checked = False - for pkg in found: - if not pkg.tag or pkg.rides or since[pkg.name] is not None: - continue - blocker = None - # A rider still at a released version must be bumped first: the tag - # would publish that version again, which its release refuses. - stale = [rider for rider in found if rider.rides == pkg.name and since[rider.name] is not None] - if stale: - blocker = f"{stale[0].name} {stale[0].version(head)} is already released; bump it on `release` first" - elif pkg.ffi: - if not checked: - ffi, checked = ffi_blocker(head), True - blocker = ffi - tags.append(Tag(f"{pkg.tag}{pkg.version(head)}", pkg, blocker)) - return tags - - -def push_tags(dry_run: bool = False) -> None: - head = git("rev-parse", "HEAD").stdout.strip() - tags = pending(head) - for tag in tags: - if tag.blocker: - # The next push to `release` retries it. - print(f"waiting to push {tag.name}: {tag.blocker}") - continue - if dry_run: - print(f"would push {tag.name} at {head[:10]}") - continue - result = git("push", "origin", f"{head}:refs/tags/{tag.name}", check=False) - if result.returncode: - raise RuntimeError(f"pushing {tag.name} failed:\n{result.stderr}") - print(f"pushed {tag.name} at {head[:10]}") - if not tags: - print("nothing to tag") - - -def main() -> int: - parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) - sub = parser.add_subparsers(dest="command", required=True) - check_parser = sub.add_parser("check") - check_parser.add_argument("--base", default="origin/release") - check_parser.add_argument("--head", default="HEAD") - check_parser.add_argument("--report", action="store_true") - tag_parser = sub.add_parser("tag") - tag_parser.add_argument("--dry-run", action="store_true") - args = parser.parse_args() - - if args.command == "tag": - push_tags(args.dry_run) - return 0 - - try: - problems, warnings = check(args.base, args.head) - except RuntimeError as error: - print(error, file=sys.stderr) - return 2 - actions = os.environ.get("GITHUB_ACTIONS") - for line in warnings: - print(f"::warning::{line}" if actions else f"warning: {line}") - for line in problems: - print(f"::error::{line}" if actions and not args.report else line) - if not problems: - print(f"no version problems merging {args.head} into {args.base}") - return 0 - if args.report: - return 0 - print(f"{len(problems)} version {'problem' if len(problems) == 1 else 'problems'}; " - "a backport bumps each hand-versioned package whose shipped sources it changes, " - "or its release workflow skips the fix as already published.") - return 1 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/sh/gh/versions.test.py b/sh/gh/versions.test.py deleted file mode 100644 index 81711d90a73..00000000000 --- a/sh/gh/versions.test.py +++ /dev/null @@ -1,376 +0,0 @@ -"""Check backports bump hand-versioned packages, and find the release tags to push.""" - -import importlib.util -import json -import os -from pathlib import Path -import subprocess -import tempfile -import unittest -from unittest import mock - -spec = importlib.util.spec_from_file_location("versions", Path(__file__).with_name("versions.py")) -versions = importlib.util.module_from_spec(spec) -spec.loader.exec_module(versions) - - -def package_json(name: str, version: str, private: bool = False) -> str: - scripts = {"build": "rimraf dist && tsc -b && bun ../common/package.ts", "release": "bun ../common/release.ts"} - manifest = {"name": name, "version": version, "scripts": scripts} - if private: - manifest["private"] = True - return json.dumps(manifest, indent="\t") + "\n" - - -BUN_LOCK = """{ - "workspaces": { - "js/a": { - "name": "@moq/a", - "version": "0.1.0", - }, - "js/b": { - "name": "@moq/b", - "version": "0.2.0", - }, - }, -} -""" - -PYPROJECT = """[build-system] -requires = ["hatchling"] - -[project] -name = "moq-rs" -# Bumped by hand. -version = "0.5.0" -""" - -UV_LOCK = """[[package]] -name = "moq-ffi" -version = "0.4.0" - -[[package]] -name = "moq-rs" -version = "0.5.0" -source = { editable = "py/moq-rs" } -""" - - -class VersionsTest(unittest.TestCase): - def setUp(self): - self.dir = tempfile.TemporaryDirectory() - self.root = Path(self.dir.name) - self.cwd = os.getcwd() - os.chdir(self.root) - self.git("init", "--quiet", "--initial-branch", "release") - self.git("config", "user.name", "test") - self.git("config", "user.email", "test@example.com") - self.write({ - "js/a/package.json": package_json("@moq/a", "0.1.0"), - "js/a/src/index.ts": "export {};\n", - "js/b/package.json": package_json("@moq/b", "0.2.0"), - "js/b/src/index.ts": "export {};\n", - "js/b/vite.config.ts": 'import { worklet } from "../common/vite-plugin-worklet.ts";\n', - "js/common/package.ts": "", - "js/common/release.ts": "", - "js/common/vite-plugin-worklet.ts": "", - "js/wasm/package.json": package_json("@moq/wasm", "0.0.0", private=True), - "js/wasm/src/index.ts": "export {};\n", - "bun.lock": BUN_LOCK, - "py/moq-rs/pyproject.toml": PYPROJECT, - "py/moq-rs/moq/__init__.py": "", - "uv.lock": UV_LOCK, - "swift/VERSION": "0.3.0\n", - "swift/Sources/Moq/Moq.swift": "", - "kt/gradle.properties": "moqffi.version=0.0.0-dev\nmoq.version=0.4.0\n", - "kt/moq/src/jvmMain/Moq.kt": "", - "dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.0\n", - "dart/moq/lib/moq.dart": "", - "c/moq/VERSION": "0.8.0\n", - "c/moq/src/moq.c": "", - "cpp/moq/VERSION": "0.1.0\n", - "cpp/moq/src/moq.cpp": "", - "cpp/moq/FFI_VERSION": "# Unset until published.\n", - "cpp/moq/FFI_SHA256.json": "{}\n", - "cpp/obs/VERSION": "0.7.0\n", - "cpp/obs/src/plugin.cpp": "", - }) - self.commit("initial") - # Every tag-published package has released its current version. - for tag in ("moq-dart-v0.1.0", "c-v0.8.0", "cpp-v0.1.0", "obs-moq-v0.7.0"): - self.git("tag", tag) - - def tearDown(self): - os.chdir(self.cwd) - self.dir.cleanup() - - def git(self, *args: str) -> str: - return subprocess.run(["git", *args], capture_output=True, text=True, check=True).stdout.strip() - - def write(self, files: dict[str, str]): - for path, text in files.items(): - (self.root / path).parent.mkdir(parents=True, exist_ok=True) - (self.root / path).write_text(text) - - def commit(self, message: str, files: dict[str, str] | None = None) -> str: - if files: - self.write(files) - self.git("add", "--all") - self.git("commit", "--quiet", "--message", message) - return self.git("rev-parse", "HEAD") - - def stale(self) -> dict[str, str | None]: - """The packages at HEAD still at a released version, with what each releases alongside.""" - return {u.package.name: u.alongside for u in versions.unreleased("HEAD")} - - def backport(self, files: dict[str, str]) -> tuple[list[str], list[str]]: - """Check a PR into `release` committing `files`, as names of the packages failing and noted.""" - self.git("checkout", "--quiet", "-B", "backport", "release") - self.commit("fix: a backport", files) - self.git("checkout", "--quiet", "release") - self.problems, self.notes = versions.check("release", "backport") - return names(self.problems), names(self.notes) - - def pending(self, assets: set[str] = frozenset()) -> dict[str, str | None]: - with mock.patch.object(versions, "release_assets", return_value=assets): - return {t.name: t.blocker for t in versions.pending("HEAD")} - - def pin(self, version: str) -> tuple[dict[str, str], set[str]]: - """The files pinning the C and C++ packages to moq-ffi `version`, and its asset names.""" - ffi = versions.load_ffi() - assets = {ffi.name(version, target) + ".tar.gz" for target in ffi.TARGETS} - return { - "cpp/moq/FFI_VERSION": f"# Pinned.\n{version}\n", - "cpp/moq/FFI_SHA256.json": json.dumps({asset: "0" * 64 for asset in assets}) + "\n", - }, assets - - def bump_a(self, version: str) -> dict[str, str]: - return { - "js/a/package.json": package_json("@moq/a", version), - "bun.lock": BUN_LOCK.replace('"version": "0.1.0"', f'"version": "{version}"'), - } - - def test_nothing_changed(self): - self.assertEqual(self.backport({"README.md": "docs\n"}), ([], [])) - - def test_backport_without_a_bump_fails(self): - self.assertEqual(self.backport({"js/a/src/index.ts": FIX}), (["@moq/a"], [])) - self.assertIn("1 shipped file changed since 0.1.0 shipped", self.problems[0]) - self.assertIn("js/a/package.json past 0.1.0", self.problems[0]) - self.assertIn("`bun install`", self.problems[0]) - - def test_backport_with_a_bump_passes(self): - self.assertEqual(self.backport({"js/a/src/index.ts": FIX, **self.bump_a("0.1.1")}), ([], [])) - self.assertEqual(self.backport({"js/a/src/index.ts": FIX, **self.bump_a("0.2.0")}), ([], [])) - - def test_a_bump_moves_its_lockfile_entry(self): - files = {"js/a/src/index.ts": FIX, "js/a/package.json": package_json("@moq/a", "0.1.1")} - self.assertEqual(self.backport(files), (["@moq/a"], [])) - self.assertEqual(self.problems, ["@moq/a: bun.lock carries 0.1.0, not 0.1.1; run `bun install`."]) - - def test_a_lowered_version_fails(self): - self.assertEqual(self.backport({"js/a/src/index.ts": FIX, **self.bump_a("0.0.9")}), (["@moq/a"], [])) - self.assertIn("0.0.9 does not come after 0.1.0", self.problems[0]) - - def test_branch_history_is_merged_first(self): - # The bump and the fix land in separate commits on the PR's branch. - self.git("checkout", "--quiet", "-b", "backport") - self.commit("chore: bump a", self.bump_a("0.1.1")) - self.commit("fix(a): the fix", {"js/a/src/index.ts": FIX}) - self.git("checkout", "--quiet", "release") - self.assertEqual(versions.check("release", "backport"), ([], [])) - - def test_unshipped_files_are_not_a_release(self): - self.assertEqual(self.backport({ - "js/a/src/index.test.ts": "", - "js/a/api/index.api.md": "", - "js/a/MIGRATE.md": "", - "kt/moq/src/jvmTest/MoqTest.kt": "", - "swift/Tests/MoqTests/MoqTests.swift": "", - "dart/moq/test/moq_test.dart": "", - }), ([], [])) - - def test_shared_build_scripts_warn_the_packages_that_run_them(self): - # Whether a shared build script changed what a package ships takes a human. - self.assertEqual(self.backport({"js/common/package.ts": "// fix\n"}), ([], ["@moq/a", "@moq/b"])) - self.assertIn("only the shared build script js/common/package.ts changed since 0.1.0 shipped; if that", self.notes[0]) - self.assertEqual(self.backport({"js/common/vite-plugin-worklet.ts": "// fix\n"}), ([], ["@moq/b"])) - # Publishing changes nothing a package ships. - self.assertEqual(self.backport({"js/common/release.ts": "// fix\n"}), ([], [])) - # A change of its own still fails. - self.assertEqual(self.backport({"js/common/package.ts": "// fix\n", "js/a/src/index.ts": FIX}), (["@moq/a"], ["@moq/b"])) - self.assertIn("2 shipped files changed", self.problems[0]) - - def test_private_package_is_skipped(self): - self.assertEqual(self.backport({"js/wasm/src/index.ts": FIX}), ([], [])) - - def test_every_ecosystem(self): - failing, _ = self.backport({ - "py/moq-rs/moq/__init__.py": "fix = 1\n", - "swift/Sources/Moq/Moq.swift": "// fix\n", - "kt/moq/src/jvmMain/Moq.kt": "// fix\n", - }) - self.assertEqual(failing, ["moq-rs (Python)", "Moq (Swift)", "dev.moq:moq (Kotlin)"]) - self.assertIn("`uv lock`", self.problems[0]) - self.assertEqual(self.backport({ - "py/moq-rs/moq/__init__.py": "fix = 1\n", - "py/moq-rs/pyproject.toml": PYPROJECT.replace('"0.5.0"', '"0.5.1"'), - "uv.lock": UV_LOCK.replace('version = "0.5.0"', 'version = "0.5.1"'), - "swift/Sources/Moq/Moq.swift": "// fix\n", - "swift/VERSION": "0.3.1\n", - "kt/moq/src/jvmMain/Moq.kt": "// fix\n", - "kt/gradle.properties": "moqffi.version=0.0.0-dev\nmoq.version=0.4.1\n", - }), ([], [])) - - def test_unreleased_changes_already_on_base_are_only_noted(self): - self.commit("fix(b): merged without a bump", {"js/b/src/index.ts": FIX}) - self.assertEqual(self.backport({"js/a/src/index.test.ts": ""}), ([], ["@moq/b"])) - self.assertIn("since 0.2.0 shipped, already on release;", self.notes[0]) - # A backport touching it must bump it, which ships the earlier change too. - self.assertEqual(self.backport({"js/b/src/other.ts": FIX}), (["@moq/b"], [])) - self.assertIn("2 shipped files changed", self.problems[0]) - - def test_cut_merge_ships_main_changes_made_after_its_bump(self): - # On main: bump @moq/a, then land another change, then cut into release. - self.git("checkout", "--quiet", "-b", "main") - self.commit("chore: bump a", self.bump_a("0.2.0")) - self.commit("feat(a): after the bump", {"js/a/src/index.ts": "export const feature = 1;\n"}) - self.git("checkout", "--quiet", "release") - self.git("merge", "--quiet", "--no-ff", "--message", "cut", "main") - # The cut's tree is what Release JS published, so nothing is unreleased. - self.assertEqual(self.stale(), {}) - self.assertEqual(self.backport({"js/a/src/index.ts": FIX}), (["@moq/a"], [])) - self.assertIn("past 0.2.0", self.problems[0]) - - def test_tagged_package_needs_a_bump_after_its_tag(self): - self.assertEqual(self.backport({"dart/moq/lib/moq.dart": "// fix\n"}), (["moq (Dart)"], [])) - self.assertIn("dart/moq/pubspec.yaml past 0.1.0", self.problems[0]) - - def test_untagged_version_needs_no_bump(self): - self.commit("chore: bump dart", {"dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.1\n"}) - self.assertEqual(self.backport({"dart/moq/lib/moq.dart": "// fix\n"}), ([], [])) - - def test_cpp_and_obs_release_together(self): - self.assertEqual(self.backport({"cpp/moq/src/moq.cpp": "// fix\n"}), (["moq (C++)", "obs-moq (OBS)"], [])) - self.assertEqual(self.backport({"cpp/moq/src/moq.cpp": "// fix\n", "cpp/moq/VERSION": "0.1.1\n"}), - (["obs-moq (OBS)"], [])) - self.assertIn("moq (C++) releases, and the same tag would publish its released 0.7.0 again", self.problems[0]) - self.assertEqual(self.backport({"cpp/obs/src/plugin.cpp": "// fix\n", "cpp/obs/VERSION": "0.7.1\n"}), - (["moq (C++)"], [])) - self.assertEqual(self.backport({ - "cpp/obs/src/plugin.cpp": "// fix\n", - "cpp/moq/VERSION": "0.1.1\n", - "cpp/obs/VERSION": "0.7.1\n", - }), ([], [])) - - def test_ffi_pin_change_releases_c_and_cpp(self): - files, _ = self.pin("0.4.20") - self.assertEqual(self.backport(files), (["moq (C)", "moq (C++)", "obs-moq (OBS)"], [])) - # C builds through C++'s moq-ffi CMake module too. - self.assertEqual(self.backport({"cpp/moq/cmake/moq-ffi.cmake": "# fix\n"}), - (["moq (C)", "moq (C++)", "obs-moq (OBS)"], [])) - - def test_untagged_version_awaits_its_tag(self): - self.commit("chore: bump dart", {"dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.1\n"}) - self.commit("fix(dart): after the bump", {"dart/moq/lib/moq.dart": "// fix\n"}) - self.assertEqual(self.stale(), {}) - self.assertEqual(self.pending(), {"moq-dart-v0.1.1": None}) - - def test_untagged_cpp_waits_for_the_obs_bump(self): - files, assets = self.pin("0.4.20") - self.commit("chore: pin moq-ffi and bump", {**files, "c/moq/VERSION": "0.8.1\n", "cpp/moq/VERSION": "0.2.0\n"}) - self.assertEqual(self.stale(), {"obs-moq (OBS)": "moq (C++)"}) - # Tagging now would publish OBS 0.7.0 again. - self.assertIn("obs-moq (OBS) 0.7.0 is already released", self.pending(assets)["cpp-v0.2.0"]) - self.commit("chore: bump obs", {"cpp/obs/VERSION": "0.7.1\n"}) - self.assertEqual(self.pending(assets), {"c-v0.8.1": None, "cpp-v0.2.0": None}) - - def test_obs_in_flight_from_a_cpp_tag_is_released(self): - self.commit("chore: bump", {"cpp/moq/VERSION": "0.1.1\n", "cpp/obs/VERSION": "0.7.1\n"}) - # Release Cpp has not cut obs-moq-v0.7.1 yet. - self.git("tag", "cpp-v0.1.1") - self.commit("docs: unrelated", {"README.md": ""}) - self.assertEqual(self.stale(), {}) - self.assertEqual(self.pending(), {}) - self.assertEqual(self.backport({"cpp/obs/src/plugin.cpp": "// fix\n"}), (["moq (C++)", "obs-moq (OBS)"], [])) - - def test_obs_never_pushes_its_own_tag(self): - self.commit("chore: bump obs", {"cpp/obs/VERSION": "0.8.0\n"}) - self.assertEqual(self.stale(), {"moq (C++)": "obs-moq (OBS)"}) - self.assertEqual(self.pending(), {}) - - def test_c_and_cpp_tags_wait_for_the_pinned_ffi_release(self): - self.commit("chore: bump", { - "dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.1\n", - "c/moq/VERSION": "0.8.1\n", - "cpp/moq/VERSION": "0.1.1\n", - "cpp/obs/VERSION": "0.7.1\n", - }) - self.assertEqual(self.pending(), { - "moq-dart-v0.1.1": None, - "c-v0.8.1": "cpp/moq/FFI_VERSION pins no single moq-ffi release", - "cpp-v0.1.1": "cpp/moq/FFI_VERSION pins no single moq-ffi release", - }) - files, assets = self.pin("0.4.20") - self.commit("chore: pin moq-ffi", files) - missing = sorted(assets)[-1] - blocked = self.pending(assets - {missing}) - self.assertIn(missing, blocked["c-v0.8.1"]) - self.assertEqual(self.pending(assets), {"moq-dart-v0.1.1": None, "c-v0.8.1": None, "cpp-v0.1.1": None}) - - def test_tag_pushes_only_unblocked_tags(self): - remote_dir = tempfile.TemporaryDirectory() - self.addCleanup(remote_dir.cleanup) - remote = Path(remote_dir.name) - subprocess.run(["git", "init", "--quiet", "--bare", str(remote)], check=True) - self.git("remote", "add", "origin", str(remote)) - head = self.commit("chore: bump", {"dart/moq/pubspec.yaml": "name: moq\nversion: 0.1.1\n", "c/moq/VERSION": "0.8.1\n"}) - - def pushed() -> str: - return subprocess.run(["git", "ls-remote", "--tags", str(remote)], capture_output=True, text=True, check=True).stdout - - with mock.patch.object(versions, "release_assets", return_value=set()): - versions.push_tags(dry_run=True) - self.assertEqual(pushed(), "") - versions.push_tags() - self.assertEqual(pushed(), f"{head}\trefs/tags/moq-dart-v0.1.1\n") - - def test_missing_tagged_packages_are_skipped(self): - # A `release` before the cut that brings C, C++, and OBS. - self.git("rm", "--quiet", "-r", "c", "cpp") - self.commit("chore: drop c and cpp") - # That `release` lacks sh/cpp/ffi.py too. - with mock.patch.object(versions, "load_ffi", side_effect=AssertionError("loaded ffi.py")): - self.assertEqual(self.stale(), {}) - self.assertEqual(self.pending(), {}) - self.assertEqual(self.backport({"js/a/src/index.ts": FIX}), (["@moq/a"], []), self.problems) - - def test_a_version_semver_cannot_order_fails(self): - self.assertEqual(self.backport({"js/a/src/index.ts": FIX, **self.bump_a("next")}), (["@moq/a"], [])) - self.assertIn("cannot order next after 0.1.0", self.problems[0]) - - def test_newer(self): - self.assertTrue(versions.newer("0.9.10", "0.9.9")) - self.assertTrue(versions.newer("0.1.0", "0.1.0-rc.1")) - self.assertFalse(versions.newer("0.1.0-rc.1", "0.1.0")) - self.assertFalse(versions.newer("0.1.0", "0.1.0")) - # Numeric prerelease identifiers compare by value, below alphanumeric ones. - self.assertTrue(versions.newer("1.0.0-rc.10", "1.0.0-rc.9")) - self.assertFalse(versions.newer("1.0.0-rc.9", "1.0.0-rc.10")) - self.assertTrue(versions.newer("1.0.0-alpha.beta", "1.0.0-alpha.1")) - self.assertTrue(versions.newer("1.0.0-alpha.1", "1.0.0-alpha")) - # Build metadata takes no part in precedence. - self.assertFalse(versions.newer("0.4.0+fix", "0.5.0")) - self.assertFalse(versions.newer("0.5.0+fix", "0.5.0")) - self.assertIsNone(versions.newer("next", "0.1.0")) - - -def names(lines: list[str]) -> list[str]: - return [line.split(": ", 1)[0] for line in lines] - - -FIX = "export const fix = 1;\n" - - -if __name__ == "__main__": - unittest.main() From 19fdedefa1653fbc1ac2544d4e9e8901019dc396 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 29 Oct 2026 08:31:09 -0700 Subject: [PATCH 12/12] quest: delete release-versions Backports should be free to batch into one version; a gate requiring each one to bump hand-versioned packages prevents that for no gain. Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 1 - quest/m1/release-versions.md | 34 ---------------------------------- 2 files changed, 35 deletions(-) delete mode 100644 quest/m1/release-versions.md diff --git a/quest/m1/README.md b/quest/m1/README.md index 48bc378104c..a86e825a417 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -186,7 +186,6 @@ removed). - [JS SETUP rejection ownership](/quest/m1/js-setup-rejection.md) - reproduce an unhandled version-refusal rejection without weakening the refusal - [moq-clock moves into kio::clock](/quest/m1/kio-clock.md) - moq-clock and its test macro become `kio::clock` and `#[kio::test]` before the cut publishes them - [Poll-first async](/quest/m1/poll-first/README.md) - library crates compile no coroutines by default: async wrappers are generated from `poll_*` and gated behind an `async` feature -- [Every package's release backport ships](/quest/m1/release-versions.md) - a bot bumps every hand-versioned package changed on `release`, so backports publish without a manual bump ## Related diff --git a/quest/m1/release-versions.md b/quest/m1/release-versions.md deleted file mode 100644 index 7888c2d6315..00000000000 --- a/quest/m1/release-versions.md +++ /dev/null @@ -1,34 +0,0 @@ -# [M] Every package's release backport ships - -## Goal - -A fix merged into `release` for any published package publishes without a -hand-made version bump. Today release-plz bumps only Rust crates, so a -backport touching only a JS, Python, Dart, Kotlin, Swift, Go, C, or C++ -package lands on `release` unchanged in version and its release workflow -skips it as already published. - -## Plan - -Found 2026-10-28: the @moq/flate and @moq/net backports -([moq-dev/moq#6961](https://github.com/moq-dev/moq/pull/6961), -[moq-dev/moq#6959](https://github.com/moq-dev/moq/pull/6959), -[moq-dev/moq#6967](https://github.com/moq-dev/moq/pull/6967)) merged and the -release PR merged, but Release JS logged "already published, skipping" and a -separate hand bump was needed. - -Decided 2026-10-28 (maintainer): - -- Scope: every hand-versioned package listed in CONTRIBUTING.md's Versions - section, not just JS. -- Mechanism: a release-plz-like bot. On each push to `release`, it opens or - updates one PR that patch-bumps every package whose sources changed since - its last published version (with the matching lockfile entries, e.g. - `bun.lock`, `uv.lock`); merging it publishes through the existing release - workflows. Prefer an existing maintained tool if one covers these - ecosystems; extending release-plz's own PR is also fine. -- Until this lands, a backport author bumps by hand. - -Update CONTRIBUTING.md's Versions and backport bullets to describe the result. - -Public API: none. Wire: none.