Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@ jobs:
- name: Build the image
run: docker build -t runner:test .

- name: GitHub App JWT test
run: docker run --rm -v "$PWD:/src:ro" --entrypoint /src/tests/app-jwt.sh runner:test

- name: Compose smoke test
run: IMAGE=runner:test tests/compose-smoke.sh

Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
.env
access-token
github_access-token
*.pem
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ USER root
# dockerd runs INSIDE the container. This needs the Sysbox runtime on the host.
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ca-certificates curl jq git git-lfs iptables \
ca-certificates curl jq git git-lfs iptables openssl \
&& curl -fsSL https://get.docker.com | sh \
&& usermod -aG docker runner \
&& rm -rf /var/lib/apt/lists/*
Expand Down
48 changes: 45 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,9 @@ base image.

- Linux host with [Sysbox CE](https://github.com/nestybox/sysbox) installed
(the `sysbox-runc` runtime must show in `docker info`).
- A GitHub personal access token that can manage organization self-hosted
runners (fine-grained: *Self-hosted runners: read and write*).
- A way to manage organization self-hosted runners, one of:
- a GitHub App (recommended, see [GitHub App](#github-app)), or
- a personal access token (fine-grained: *Self-hosted runners: read and write*).

## Configuration

Expand All @@ -40,6 +41,9 @@ cp .env.example .env # set ORGANIZATION (and RUNNER_LABELS, if needed)
| `ORGANIZATION` | GitHub organization that owns the runners | required |
| `RUNNER_LABELS` | Comma-separated runner labels | `sysbox` |
| `ACCESS_TOKEN_FILE` | Path to the file that holds the PAT | set in the compose files |
| `APP_ID` | GitHub App ID (with `APP_PRIVATE_KEY_FILE`, replaces the PAT) | none |
| `APP_PRIVATE_KEY_FILE` | Path to the GitHub App private key (`.pem`) | none |
| `APP_INSTALLATION_ID` | GitHub App installation ID. If not set, the runner gets it from the API. | none |

## Docker Compose

Expand All @@ -55,6 +59,42 @@ Compose restarts the same container after each job. `start.sh` deletes the
inner Docker data (containers, volumes, images) at each start, so jobs do not
share Docker state. Images are not cached between jobs.

## GitHub App

A GitHub App belongs to the organization, not to a user, and has no
expiration date. Use it instead of a PAT.

1. In the organization: **Settings → Developer settings → GitHub Apps →
New GitHub App**.
- Webhook: off.
- Permissions: **Organization → Self-hosted runners: Read and write**. No
other permissions.
- Where can this GitHub App be installed: **Only on this account**.
2. Note the **App ID**. Generate a **private key** (a `.pem` file).
3. **Install App** on the organization.
4. Docker Compose: save the key as `app-key.pem` (`chmod 600`), and create
`docker-compose.override.yml`:

```yaml
secrets:
github_app-key:
file: ./app-key.pem # Never commit this file.

services:
runner:
secrets: !override [github_app-key]
environment:
APP_ID: "123456"
APP_PRIVATE_KEY_FILE: /run/secrets/github_app-key
```

Then: `docker compose up -d --force-recreate`. You can then delete
`access-token` and revoke the PAT.

Docker Swarm: create the secret with
`docker secret create github_app-key app-key.pem`, and make the same changes
in `docker-stack.yml`.

## Docker Swarm

> **Important:** Docker Swarm does **not** support the `runtime` option for
Expand Down Expand Up @@ -115,6 +155,7 @@ The `Test` workflow installs Sysbox on a GitHub-hosted runner and runs:

- `tests/compose-smoke.sh`: inner `dockerd`, `docker run`, `docker build`,
isolation from the host, and a clean state after a restart.
- `tests/app-jwt.sh`: the GitHub App JWT (claims and RS256 signature).
- `tests/swarm-smoke.sh`: `docker-stack.yml` on a one-node Swarm, the node
label constraint, and the inner `dockerd` under the default runtime.

Expand All @@ -130,5 +171,6 @@ IMAGE=runner:test tests/compose-smoke.sh

## Security notes

- Keep the PAT in a Docker secret or in `access-token`. Never put it in the image or in Git.
- Keep the PAT or the GitHub App private key in a Docker secret or in an untracked
file. Never put it in the image or in Git.
- Self-hosted runners must not run workflows from untrusted forks.
48 changes: 44 additions & 4 deletions start.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,12 @@
set -euo pipefail

: "${ORGANIZATION:?ORGANIZATION is required}"
: "${ACCESS_TOKEN_FILE:?ACCESS_TOKEN_FILE is required}"

# Authentication: a GitHub App (APP_ID and APP_PRIVATE_KEY_FILE) or a PAT
# (ACCESS_TOKEN_FILE). The GitHub App has priority.
if [[ -z "${APP_ID:-}" || -z "${APP_PRIVATE_KEY_FILE:-}" ]]; then
: "${ACCESS_TOKEN_FILE:?set ACCESS_TOKEN_FILE, or APP_ID and APP_PRIVATE_KEY_FILE}"
fi

# A restarted container (Compose) keeps the old runtime state of dockerd.
# A stale PID file stops dockerd if a new process gets the same PID.
Expand Down Expand Up @@ -32,19 +37,54 @@ if [[ "${SKIP_REGISTRATION:-false}" == "true" ]]; then
exec sleep infinity
fi

# Get a short-lived registration or removal token with the PAT.
# sudo: a Compose file secret keeps its host owner and mode (0600).

base64url() { openssl base64 -A | tr '+/' '-_' | tr -d '='; }

# A JWT for the GitHub App, valid for 9 minutes (the maximum is 10).
app_jwt() {
local now header payload signature
now=$(date +%s)
header=$(printf '{"alg":"RS256","typ":"JWT"}' | base64url)
payload=$(printf '{"iat":%d,"exp":%d,"iss":"%s"}' \
$((now - 60)) $((now + 540)) "${APP_ID}" | base64url)
signature=$(printf '%s.%s' "${header}" "${payload}" \
| sudo openssl dgst -sha256 -sign "${APP_PRIVATE_KEY_FILE}" | base64url)
printf '%s.%s.%s' "${header}" "${payload}" "${signature}"
}

# The token for the GitHub API: a GitHub App installation token (valid for
# 1 hour), or the PAT.
api_token() {
if [[ -n "${APP_ID:-}" && -n "${APP_PRIVATE_KEY_FILE:-}" ]]; then
local jwt installation_id
jwt=$(app_jwt)
installation_id="${APP_INSTALLATION_ID:-$(curl -fsS \
-H "Authorization: Bearer ${jwt}" \
-H "Accept: application/vnd.github+json" \
"https://api.github.com/orgs/${ORGANIZATION}/installation" | jq -r .id)}"
curl -fsS -X POST \
-H "Authorization: Bearer ${jwt}" \
-H "Accept: application/vnd.github+json" \
"https://api.github.com/app/installations/${installation_id}/access_tokens" \
| jq -r .token
else
sudo cat "${ACCESS_TOKEN_FILE}"
fi
}

# Get a short-lived registration or removal token.
github_token() {
curl -fsS -X POST \
-H "Authorization: Bearer $(sudo cat "${ACCESS_TOKEN_FILE}")" \
-H "Authorization: Bearer $(api_token)" \
-H "Accept: application/vnd.github+json" \
"https://api.github.com/orgs/${ORGANIZATION}/actions/runners/$1-token" \
| jq -r .token
}

REG_TOKEN=$(github_token registration || true)
if [[ -z "${REG_TOKEN}" || "${REG_TOKEN}" == "null" ]]; then
echo "Could not get a registration token. Check the PAT and ORGANIZATION." >&2
echo "Could not get a registration token. Check the PAT or GitHub App, and ORGANIZATION." >&2
exit 1
fi

Expand Down
47 changes: 47 additions & 0 deletions tests/app-jwt.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
#!/usr/bin/env bash
# Test of the GitHub App JWT in start.sh: signature and claims.
# Runs in the runner image (needs openssl, jq and sudo):
#
# docker run --rm -v "$PWD:/src:ro" --entrypoint /src/tests/app-jwt.sh runner:test
set -euo pipefail

SRC="$(dirname "$0")/.."
WORK="$(mktemp -d)"
trap 'rm -rf "${WORK}"' EXIT

fail() { echo "FAIL: $*" >&2; exit 1; }

# Load only the JWT functions from start.sh.
eval "$(sed -n '/^base64url() /p; /^app_jwt() {/,/^}/p' "${SRC}/start.sh")"

openssl genrsa -out "${WORK}/key.pem" 2048 2>/dev/null
openssl rsa -in "${WORK}/key.pem" -pubout -out "${WORK}/pub.pem" 2>/dev/null
export APP_ID=12345
export APP_PRIVATE_KEY_FILE="${WORK}/key.pem"

jwt="$(app_jwt)"
IFS=. read -r header payload signature <<<"${jwt}"

decode() {
local s="$1"
s="${s//-/+}"; s="${s//_//}"
while (( ${#s} % 4 )); do s="${s}="; done
printf '%s' "${s}" | openssl base64 -d -A
}

decode "${header}" | jq -e '.alg == "RS256" and .typ == "JWT"' >/dev/null || fail "header"
echo "ok: header"

now=$(date +%s)
decode "${payload}" | jq -e --argjson now "${now}" \
'.iss == "12345" and .iat <= $now and .exp > $now and (.exp - .iat) <= 600' >/dev/null \
|| fail "claims"
echo "ok: claims (iss, iat, exp, max. 10 minutes)"

decode "${signature}" > "${WORK}/sig"
printf '%s.%s' "${header}" "${payload}" \
| openssl dgst -sha256 -verify "${WORK}/pub.pem" -signature "${WORK}/sig" >/dev/null \
|| fail "signature"
echo "ok: RS256 signature"

echo "PASS: GitHub App JWT"
Loading