Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1580,6 +1580,7 @@ jobs:
run: |
set -euo pipefail
[[ "$RUNNER_OS/$RUNNER_ARCH" == macOS/X64 ]]
[[ "$(scripts/package-release-binaries.sh --print-platform)" == darwin-amd64 ]]
df -Pk .
df -Pk . | awk 'NR == 2 { if ($4 < 8 * 1024 * 1024) { print "Intel runner has under 8 GiB free"; exit 1 } }'
# source-gate runs these tests on Linux. Run them here on macOS too,
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ journeys tested on that device.

## Install from the publisher

The public installer looks up signed releases for Linux x86_64/aarch64 and macOS Apple silicon. Intel Mac and other OS families fail closed.
The public installer looks up signed releases for Linux x86_64/aarch64 and macOS Apple silicon. Other OS families fail closed. For an Intel Mac, see [Intel Mac](docs/MAC.md#intel-mac).

The maintainer release DID is
`did:key:z6MkgwHd2BCWe1jHMXPiR6H1q1RFPcv1YzhMbK5G1kBarbfe`. Compare the
Expand Down
2 changes: 1 addition & 1 deletion docs/GETTING_STARTED.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Install from the publisher

The public installer looks up signed releases for Linux x86_64/aarch64 and macOS Apple silicon. Intel Mac and other OS families fail closed. See the [Mac runbook](MAC.md) for source-home staging.
The public installer looks up signed releases for Linux x86_64/aarch64 and macOS Apple silicon. Other OS families fail closed. For an Intel Mac, see [Intel Mac](MAC.md#intel-mac). See the [Mac runbook](MAC.md) for source-home staging.

```bash
curl -fsSL https://elastos.elacitylabs.com/install.sh | bash
Expand Down
2 changes: 1 addition & 1 deletion docs/INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ journeys tested on that device.

## Install from the publisher

The installer looks up signed releases for Linux x86_64/aarch64 and macOS Apple silicon. Intel Mac and other OS families fail closed. The signed release determines which binaries exist for a given platform.
The installer looks up signed releases for Linux x86_64/aarch64 and macOS Apple silicon. Other OS families fail closed. For an Intel Mac, see [Intel Mac](MAC.md#intel-mac). The signed release determines which binaries exist for a given platform.

This checkout's installer combines installation, Home setup, and launch into one
command:
Expand Down
4 changes: 2 additions & 2 deletions docs/LAUNCHER_CONTRACT.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,5 +148,5 @@ A supervisor uses these exit codes. It does not parse English sentences from
stdout. Exit 0 covers more than one successful update outcome. A later issue
owns a machine-readable update status.

Intel Mac's release id is `x86_64-darwin`. This contract does not publish a
download for that id.
This contract publishes no download for an Intel Mac. [Intel Mac](MAC.md#intel-mac)
records the current state.
18 changes: 18 additions & 0 deletions docs/MAC.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ artifacts, and `.dmg` packaging work.

## Intel Mac

This section is the one record of Intel Mac support. Other documents link
here instead of repeating it.

The public release id for an Intel Mac is `x86_64-darwin`. `scripts/install.sh`
stops on that host before a download lookup. The message is
`macOS release lookup requires Apple silicon`.
Expand All @@ -15,6 +18,21 @@ installer on an Apple silicon Mac. The source-home commands later in this
runbook are a developer build. They are not the installer, and this note does
not claim that build has passed on Intel.

These files hold the rules:

- Signing: the release platform allowlist in `scripts/release-signer.py`.
`x86_64-darwin` is not in it. `scripts/release-signer-test.py` runs the
signer with `python -O` on an `x86_64-darwin` input and requires the
refusal.
- Package name: `scripts/release-platforms.json` maps the package key
`darwin-amd64` to `x86_64-darwin`.
- Packaging: `scripts/package-release-binaries.sh`. `--print-platform` prints
`darwin-amd64` only when `sysctl` shows Intel hardware. It stops in a
Rosetta shell on Apple silicon and when `sysctl` is missing or unclear.
Packaging writes no `darwin-amd64` tarball. For the other keys, it stops
on a binary that does not match the key, which includes a universal binary.
- Install: the platform check in `scripts/install.sh`.

CI job `gateway-smoke-macos-intel` builds the native `elastos` binary on
`macos-15-intel`. It runs `scripts/gateway-healthz-smoke.sh`, which starts the
command in [the launcher contract](LAUNCHER_CONTRACT.md) and requires the
Expand Down
26 changes: 22 additions & 4 deletions scripts/ci-release-policy-test.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,16 @@


WORKFLOW = Path(__file__).resolve().parents[1] / ".github/workflows/ci.yml"


def native_header(package_key):
"""A minimal executable header that packaging accepts for this package key."""
import struct
if package_key.startswith("linux-"):
machine = {"linux-amd64": 62, "linux-arm64": 183}[package_key]
return (b"\x7fELF\x02\x01\x01" + bytes(9) + struct.pack("<HH", 3, machine)).ljust(64, b"\0")
machine = {"darwin-arm64": 0x100000C}[package_key]
return (b"\xcf\xfa\xed\xfe" + struct.pack("<III", machine, 0, 2)).ljust(64, b"\0")
SOURCE = WORKFLOW.read_text()
# CI source-gate steps run sections of this script; it owns the command list.
SOURCE_GATE = (WORKFLOW.parents[2] / "scripts/test-source-gate.sh").read_text()
Expand Down Expand Up @@ -1145,7 +1155,15 @@ def test_packages_select_shared_outputs_and_preserve_binary_hashes(self):
root = Path(directory)
script = root / "scripts/package-release-binaries.sh"
script.parent.mkdir()
script.write_bytes((WORKFLOW.parents[2] / "scripts/package-release-binaries.sh").read_bytes())
source_scripts = WORKFLOW.parents[2] / "scripts"
script.write_bytes((source_scripts / "package-release-binaries.sh").read_bytes())
# Packaging reads the native header rule and the package key map.
for name in ("release-platform-input.py", "release-platforms.json", "release-signer.py",
"components-release-integrity-check.py", "browser-host-release.py"):
(script.parent / name).write_bytes((source_scripts / name).read_bytes())
package_key = subprocess.run(["bash", str(script), "--print-platform"], capture_output=True,
text=True, check=True).stdout.strip()
header = native_header(package_key)
shims = root / "shims"
shims.mkdir()
cargo = shims / "cargo"
Expand Down Expand Up @@ -1175,12 +1193,12 @@ def test_packages_select_shared_outputs_and_preserve_binary_hashes(self):
for name in ("elastos", "unused.rlib", "dependency-tool", "browser-local-exit",
"browser-engine-supervisor", "browser-native-proxy-engine", "browser-stream-bridge"):
path = outputs / name
path.write_bytes(name.encode())
path.write_bytes(header + name.encode())
path.chmod(0o755)
capsule_outputs = outputs if shared else capsule / "target/release"
capsule_outputs.mkdir(parents=True, exist_ok=True)
provider = capsule_outputs / "custody-provider"
provider.write_bytes(b"custody-provider")
provider.write_bytes(header + b"custody-provider")
provider.chmod(0o755)
if shared:
stale = root / "elastos/target/release/stale"
Expand All @@ -1194,7 +1212,7 @@ def test_packages_select_shared_outputs_and_preserve_binary_hashes(self):
with tarfile.open(package) as archive:
binaries = {Path(item.name).name: hashlib.sha256(archive.extractfile(item).read()).hexdigest()
for item in archive if item.isfile()}
self.assertEqual(binaries, {name: hashlib.sha256(name.encode()).hexdigest()
self.assertEqual(binaries, {name: hashlib.sha256(header + name.encode()).hexdigest()
for name in ("elastos", "custody-provider")})

def test_carrier_binary_paths_use_the_shared_target_or_workspace_default(self):
Expand Down
225 changes: 225 additions & 0 deletions scripts/package-release-binaries-test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,225 @@
#!/usr/bin/env python3
"""Package names and binary checks for scripts/package-release-binaries.sh."""

import json
import os
import shutil
import stat
import struct
import subprocess
import tarfile
import tempfile
import textwrap
import unittest
from pathlib import Path


ROOT = Path(__file__).resolve().parents[1]
SCRIPT = ROOT / "scripts/package-release-binaries.sh"
BASH = shutil.which("bash")
# Native Intel Mac: sysctl -i prints 0 or nothing for these two OIDs.
NATIVE_INTEL = ("0", "0")


def executable(path, text):
path.write_text(text)
path.chmod(path.stat().st_mode | stat.S_IEXEC)


def write_shims(directory, system, machine, sysctl=NATIVE_INTEL):
"""uname reports system/machine. sysctl is a pair of answers, "fail", or None for missing."""
directory = Path(directory)
executable(directory / "uname",
"#!/bin/sh\n"
f"case \"$1\" in -s) printf '%s\\n' {system!r} ;; *) printf '%s\\n' {machine!r} ;; esac\n")
if sysctl == "fail":
executable(directory / "sysctl", "#!/bin/sh\necho 'sysctl: broken' >&2\nexit 1\n")
elif sysctl is not None:
translated, arm64 = sysctl
executable(directory / "sysctl", textwrap.dedent(f"""\
#!/bin/sh
[ "$1" = -in ] || exit 2
case "$2" in
sysctl.proc_translated) printf '%s' {translated!r} ;;
hw.optional.arm64) printf '%s' {arm64!r} ;;
*) exit 2 ;;
esac
"""))


def run_script(directory, *args, cwd=ROOT, script=SCRIPT, extra_env=None, only_shims=False):
env = os.environ.copy()
if only_shims:
# Only the shims and dirname: a tool that is not shimmed is missing.
os.symlink(shutil.which("dirname"), Path(directory) / "dirname")
env["PATH"] = str(directory)
else:
env["PATH"] = str(directory) + os.pathsep + env["PATH"]
env.update(extra_env or {})
return subprocess.run([BASH, str(script), *args], cwd=cwd, env=env, text=True,
capture_output=True, timeout=60)


def package_name(system, machine, sysctl=NATIVE_INTEL, only_shims=False):
with tempfile.TemporaryDirectory() as directory:
write_shims(directory, system, machine, sysctl)
return run_script(directory, "--print-platform", only_shims=only_shims)


def native_header(kind, machine):
if kind == "elf":
return (b"\x7fELF\x02\x01\x01" + bytes(9) + struct.pack("<HH", 3, machine)).ljust(64, b"\0")
if kind == "macho":
return (b"\xcf\xfa\xed\xfe" + struct.pack("<III", machine, 0, 2)).ljust(64, b"\0")
# A universal (fat) Mach-O with arm64 and x86_64 slices.
return (b"\xca\xfe\xba\xbe" + struct.pack(">IIIIII", 2, 0x100000C, 0, 0, 0, 0)).ljust(64, b"\0")


ELF_AMD64 = native_header("elf", 62)
ELF_ARM64 = native_header("elf", 183)
MACHO_ARM64 = native_header("macho", 0x100000C)
MACHO_AMD64 = native_header("macho", 0x1000007)
UNIVERSAL = native_header("fat", 0)


class PackageReleaseNameTest(unittest.TestCase):
def test_native_hosts_use_the_release_platform_names(self):
expected = {
("Linux", "x86_64"): "linux-amd64",
("Linux", "aarch64"): "linux-arm64",
("Linux", "arm64"): "linux-arm64",
("Darwin", "arm64"): "darwin-arm64",
("Darwin", "x86_64"): "darwin-amd64",
}
for host, name in expected.items():
with self.subTest(host=host):
result = package_name(*host)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(result.stdout.strip(), name)

def test_native_intel_answers_name_darwin_amd64(self):
# sysctl -i prints nothing for an OID that an Intel macOS lacks.
for answers in (("0", "0"), ("", ""), ("0", ""), ("", "0")):
with self.subTest(answers=answers):
result = package_name("Darwin", "x86_64", answers)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(result.stdout.strip(), "darwin-amd64")

def test_rosetta_shell_on_apple_silicon_is_not_named_darwin_amd64(self):
for answers in (("1", "1"), ("1", ""), ("0", "1"), ("", "1")):
with self.subTest(answers=answers):
result = package_name("Darwin", "x86_64", answers)
self.assertEqual(result.returncode, 1, result.stdout)
self.assertEqual(result.stdout, "")
self.assertIn("Rosetta on an Apple silicon Mac", result.stderr)

def test_intel_name_fails_closed_without_a_clear_sysctl_answer(self):
cases = {"missing": None, "fails": "fail", "unknown value": ("2", "0"), "text": ("yes", "")}
for case, sysctl in cases.items():
with self.subTest(case=case):
result = package_name("Darwin", "x86_64", sysctl, only_shims=True)
self.assertEqual(result.returncode, 1, result.stdout)
self.assertEqual(result.stdout, "")
self.assertIn("sysctl", result.stderr)

def test_intel_package_key_prints_and_packaging_refuses_the_tarball(self):
platforms = json.loads((ROOT / "scripts/release-platforms.json").read_text())
self.assertEqual(platforms["darwin-amd64"], "x86_64-darwin")
printed = package_name("Darwin", "x86_64")
self.assertEqual(printed.returncode, 0, printed.stderr)
self.assertEqual(printed.stdout.strip(), "darwin-amd64")
with tempfile.TemporaryDirectory() as directory:
write_shims(directory, "Darwin", "x86_64")
before = set(ROOT.glob("elastos-runtime-*.tar.gz"))
result = run_script(directory, extra_env={"TMPDIR": directory})
self.assertEqual(result.returncode, 1, result.stdout)
self.assertIn("darwin-amd64 is not a signed release platform", result.stderr)
self.assertEqual(set(ROOT.glob("elastos-runtime-*.tar.gz")), before)
self.assertEqual(list(Path(directory).glob("elastos-runtime-*")), [])

def test_print_platform_writes_nothing(self):
with tempfile.TemporaryDirectory() as directory:
write_shims(directory, "Darwin", "x86_64")
scratch = Path(directory) / "tmp"
scratch.mkdir()
before = {path.name for path in ROOT.iterdir()}
result = run_script(directory, "--print-platform", extra_env={"TMPDIR": str(scratch)})
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(result.stdout.strip(), "darwin-amd64")
self.assertEqual({path.name for path in ROOT.iterdir()}, before)
self.assertEqual(list(scratch.iterdir()), [])

def test_other_darwin_hosts_stop(self):
result = package_name("Darwin", "i386")
self.assertNotEqual(result.returncode, 0)
self.assertIn("Unsupported release platform: Darwin-i386", result.stderr)


class PackageBinaryArchitectureTest(unittest.TestCase):
"""Packaging writes a tarball only when every binary matches its name."""

def package(self, system, machine, headers):
directory = tempfile.TemporaryDirectory()
self.addCleanup(directory.cleanup)
root = Path(directory.name)
scripts = root / "scripts"
scripts.mkdir()
for name in ("package-release-binaries.sh", "release-platform-input.py", "release-platforms.json",
"release-signer.py", "components-release-integrity-check.py", "browser-host-release.py"):
shutil.copy2(ROOT / "scripts" / name, scripts / name)
shims = root / "shims"
shims.mkdir()
write_shims(shims, system, machine)
names = sorted(headers)
executable(shims / "cargo", "#!/bin/sh\nprintf '%s' " + repr(json.dumps({
"workspace_members": ["member"],
"packages": [{"id": "member", "targets": [{"name": name, "kind": ["bin"]} for name in names]}],
})) + "\n")
(root / "elastos").mkdir()
(root / "elastos/Cargo.toml").touch()
release = root / "elastos/target/release"
release.mkdir(parents=True)
for name, header in headers.items():
(release / name).write_bytes(header + name.encode())
(release / name).chmod(0o755)
env = {"TMPDIR": str(root), "COPYFILE_DISABLE": "1", "CARGO_TARGET_DIR": str(root / "elastos/target")}
result = run_script(shims, cwd=root, script=scripts / "package-release-binaries.sh", extra_env=env)
return result, sorted(path.name for path in root.glob("elastos-runtime-*"))

def test_matching_binaries_are_packaged(self):
cases = {("Linux", "x86_64"): ("linux-amd64", ELF_AMD64),
("Linux", "aarch64"): ("linux-arm64", ELF_ARM64),
("Darwin", "arm64"): ("darwin-arm64", MACHO_ARM64)}
for host, (key, header) in cases.items():
with self.subTest(host=host):
result, written = self.package(*host, {"elastos": header, "custody-provider": header})
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(written, [f"elastos-runtime-{key}.tar.gz", f"elastos-runtime-{key}.tar.gz.sha256"])
tarball = Path(result.stdout.strip().splitlines()[-1].removeprefix("Wrote "))
with tarfile.open(tarball) as archive:
self.assertEqual(sorted(Path(item.name).name for item in archive if item.isfile()),
["custody-provider", "elastos"])

def test_binary_for_another_processor_or_universal_is_refused(self):
cases = {
"arm64 ELF named linux-amd64": (("Linux", "x86_64"), ELF_ARM64),
"x86_64 ELF named linux-arm64": (("Linux", "aarch64"), ELF_AMD64),
"x86_64 Mach-O named darwin-arm64": (("Darwin", "arm64"), MACHO_AMD64),
"universal Mach-O named darwin-arm64": (("Darwin", "arm64"), UNIVERSAL),
"Linux ELF named darwin-arm64": (("Darwin", "arm64"), ELF_ARM64),
"shell script named linux-amd64": (("Linux", "x86_64"), b"#!/bin/sh\n".ljust(64, b"#")),
"short file named linux-amd64": (("Linux", "x86_64"), b"\x7fELF"),
}
for case, (host, header) in cases.items():
with self.subTest(case=case):
good = {"Linux-x86_64": ELF_AMD64, "Linux-aarch64": ELF_ARM64, "Darwin-arm64": MACHO_ARM64}["-".join(host)]
result, written = self.package(*host, {"elastos": good, "wrong-tool": header})
self.assertEqual(result.returncode, 1, result.stdout)
self.assertIn("Refusing to package", result.stderr)
self.assertIn("wrong-tool", result.stderr)
self.assertNotIn("elastos:", result.stderr)
self.assertEqual(written, [])


if __name__ == "__main__":
unittest.main()
Loading
Loading