Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions .github/actions/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# Shared actions

Composite actions every megaeth-labs repository consumes at `@main`. Two
families and one standalone action; each action directory has a README whose
input, output, step and error tables are generated from its `action.yml`.
families plus standalone building blocks (git credentials, dotenv loading,
Lark notifications, merge-queue skipping, PR lint); each action directory has
a README whose input, output, step and error tables are generated from its
`action.yml`.

<!-- generated: catalogue -->
| Action | Family | Does |
Expand All @@ -11,6 +13,10 @@ input, output, step and error tables are generated from its `action.yml`.
| [`claude-issue-triage`](claude-issue-triage/README.md) | [Claude CI](CLAUDE-CI.md) | Run the centralized MegaETH Claude issue triage. |
| [`claude-label-check`](claude-label-check/README.md) | [Claude CI](CLAUDE-CI.md) | Run the centralized MegaETH Claude pull request label check. |
| [`claude-pr-review`](claude-pr-review/README.md) | [Claude CI](CLAUDE-CI.md) | Run the staged, incremental MegaETH Claude pull request review. |
| [`export-env`](export-env/README.md) | Standalone | Load a dotenv file into the job: every `KEY=value` line (an `export ` prefix, surrounding quotes, blank lines and `#` comments are handled) becomes an environment variable for the following steps, or an entry in the `json` output, or both. Only the keys are logged, never the values — a `.env` that carries a credential stays out of the log — and `mask: true` also registers each value with `::add-mask::`. `prefix` limits the load to one namespace and `strip_prefix` drops it from the exported names. |
| [`git-credentials`](git-credentials/README.md) | Standalone | Let git — and so cargo, go, pip and anything else that shells out to git — fetch private repositories over HTTPS without prompting, by adding a global `url.<https-with-token>.insteadOf <https>` rewrite for one host. The token never appears in the script: it is passed through the environment and masked in the log. Any token works: a GitHub App installation token (the recommended kind — mint one with `actions/create-github-app-token` for the repositories the build needs), a fine-grained or classic PAT. Optional extras cover the two things the old per-repository copies also did — mark every directory safe for container runners whose workspace belongs to another user, and disable git's low-speed abort for slow mirrors — plus an SSH rewrite for lockfiles that pin `git@host:` URLs. `mode: unset` removes the rewrite again. Nothing is organisation-specific; the host is an input. |
| [`merge-queue-skipper`](merge-queue-skipper/README.md) | Standalone | Decide whether a merge-queue run may skip checks that already passed on the pull request. GitHub always re-runs required checks on the temporary merge branch; this outputs `skip-check: true` only when that branch's tree is provably the one the PR's own checks covered: the entry was enqueued at the head of the target branch, the PR branch still contains that commit, and the PR branch and the queue branch have no diff. Any other situation — not a merge-queue ref, a queue entry behind another, a PR updated after enqueueing — yields `false`. The caller gates its expensive jobs on the output. Needs the repository history: the action checks it out itself (`fetch-depth: 0`) unless `checkout: false`, in which case the caller must already have a full checkout with `origin` remote. |
| [`notify-lark`](notify-lark/README.md) | Standalone | Post a message to a Lark (Feishu) group through a custom-bot webhook. The default is a plain text message; `msg_type: post` sends rich text with a title, and `payload` sends any JSON you built yourself (interactive cards, mentions), untouched. If the bot has signature verification enabled, pass its signing `secret` and the request is signed with the documented timestamp + HMAC-SHA256 scheme. The webhook URL and secret go through the environment and are masked. A non-2xx response or a Lark error code fails the step unless `fail_on_error` is `false`; the response body is an output either way. Nothing is organisation-specific: the webhook is an input. |
| [`pr-lint`](pr-lint/README.md) | Standalone | Lint a pull request. Currently validates that the PR title follows Conventional Commits, posting a sticky comment on failure and removing it once fixed; further PR-level lint steps can be added here over time. Run as a step inside a job the consumer names, so the resulting status-check context is that job name. |
| [`release-assets`](release-assets/README.md) | [Release pipeline](RELEASE.md) | Attach files, plus a generated `SHA256SUMS`, to the GitHub Release for a tag. Re-runs replace assets of the same name (`--clobber`), so the step is idempotent. `dry_run` writes and prints `SHA256SUMS` but attaches nothing. Needs a token with `contents: write` on the repository (the job token is enough). Guide: .github/actions/RELEASE.md in megaeth-labs/.github. |
| [`release-candidate`](release-candidate/README.md) | [Release pipeline](RELEASE.md) | Start a release (trunk-first). `stage: propose` bumps the version file on the default branch, drafts this release's changelog entry (dated at settle) from the commits since the previous tag, syncs the previous release's entry from its tag, and opens a `chore/release-candidate-vX.Y.Z` PR; `stage: cut`, run when that PR merges, creates `release-vX.Y.Z` at the merge commit. No tag is created at either stage — tags come from release-publish, once, at settlement. Run as a step in a job the consumer owns; the consumer checks the repository out first (`fetch-depth: 0`, `persist-credentials: false`). Guide: .github/actions/RELEASE.md in megaeth-labs/.github. |
Expand Down Expand Up @@ -48,7 +54,9 @@ actions share ([README](release-tools/README.md)).

- `actions-test.yml` is the only gate: the unit tests of
`claude-pr-review/review_pipeline.py` and `release-tools/*.py`, the
end-to-end drive of `release-verify-version`, and the documentation check.
end-to-end drives of `release-verify-version`, `git-credentials`,
`export-env`, `notify-lark` (against a local stand-in for the webhook) and
`merge-queue-skipper` (outside a queue), and the documentation check.
- After editing an `action.yml`, run `.github/scripts/action_docs.py`: it
rewrites the generated blocks in that action's README and the catalogue
above. CI runs it with `--check` and fails if the docs are stale. Prose
Expand Down
86 changes: 86 additions & 0 deletions .github/actions/export-env/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Export Env

`uses: megaeth-labs/.github/.github/actions/export-env@main`

<!-- generated: description -->
Load a dotenv file into the job: every `KEY=value` line (an `export ` prefix, surrounding quotes, blank lines and `#` comments are handled) becomes an environment variable for the following steps, or an entry in the `json` output, or both. Only the keys are logged, never the values — a `.env` that carries a credential stays out of the log — and `mask: true` also registers each value with `::add-mask::`. `prefix` limits the load to one namespace and `strip_prefix` drops it from the exported names.
<!-- /generated -->

Standalone action; no family guide.

## Inputs

<!-- generated: inputs -->
| Input | Required | Default | Description |
|---|---|---|---|
| `file` | yes | | Path of the dotenv file, relative to the workspace. |
| `target` | no | `env` | `env` (the next steps' environment), `json` (only the `json` output), or `both`. |
| `prefix` | no | | Only load keys that start with this prefix. |
| `strip_prefix` | no | `false` | `true`: remove `prefix` from the exported names. |
| `override` | no | `true` | `false`: keep a variable that is already set in the job environment. |
| `mask` | no | `false` | `true`: `::add-mask::` every loaded value of 8 characters or more, so it is redacted wherever it later shows up in the log. Shorter values are never masked (masking `1` or `true` would redact every log line containing them). |
| `required` | no | `true` | `true`: fail if the file does not exist; `false`: exit quietly with nothing loaded. |
<!-- /generated -->

## Outputs

<!-- generated: outputs -->
| Output | Description |
|---|---|
| `keys` | Comma-separated list of the names that were exported. |
| `count` | How many variables were exported. |
| `json` | The loaded pairs as a JSON object, for `fromJSON(steps.<id>.outputs.json).<NAME>` in later steps or jobs. |
<!-- /generated -->

## What it runs

<!-- generated: steps -->
1. Load the dotenv file
<!-- /generated -->

## Errors it reports

<!-- generated: errors -->
- `target must be env, json or both, got`
- `jq is not on this runner`
- `dotenv file not found: $FILE`
<!-- /generated -->

## Example

```yaml
- uses: megaeth-labs/.github/.github/actions/export-env@main
with:
file: ci/topologies/cluster.env

- run: echo "$OP_NODE_TAG" # every KEY in the file is now set
```

Load one namespace as data instead of environment, and read it back with
`fromJSON`:

```yaml
- uses: megaeth-labs/.github/.github/actions/export-env@main
id: cfg
with:
file: deploy.env
target: json
prefix: DEPLOY_
strip_prefix: "true"

- run: echo "${{ fromJSON(steps.cfg.outputs.json).REGION }}"
```

## Notes

- Values are never printed; the log lists the keys only. `mask: "true"`
additionally registers values of 8+ characters with the runner so they
are redacted wherever they appear later (shorter values are left alone —
masking `1` or `true` would redact every log line that contains them).
- Parsing is dotenv-style: `export KEY=value` lines, blank lines, `#`
comments, and one pair of surrounding single or double quotes are
handled; no variable expansion, no multi-line values.
- `override: "false"` keeps a variable the job already has, so a workflow
can let its own `env:` win over the file.
- With `target: json` nothing enters the environment; the `json` output
holds every loaded pair (the values included, so treat it like the file).
114 changes: 114 additions & 0 deletions .github/actions/export-env/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
name: Export Env
description: >-
Load a dotenv file into the job: every `KEY=value` line (an `export `
prefix, surrounding quotes, blank lines and `#` comments are handled)
becomes an environment variable for the following steps, or an entry in
the `json` output, or both. Only the keys are logged, never the values — a `.env`
that carries a credential stays out of the log — and `mask: true` also
registers each value with `::add-mask::`. `prefix` limits the load to one
namespace and `strip_prefix` drops it from the exported names.

inputs:
file:
description: "Path of the dotenv file, relative to the workspace."
required: true
target:
description: "`env` (the next steps' environment), `json` (only the `json` output), or `both`."
required: false
default: env
prefix:
description: "Only load keys that start with this prefix."
required: false
default: ""
strip_prefix:
description: "`true`: remove `prefix` from the exported names."
required: false
default: "false"
override:
description: "`false`: keep a variable that is already set in the job environment."
required: false
default: "true"
mask:
description: >-
`true`: `::add-mask::` every loaded value of 8 characters or more, so
it is redacted wherever it later shows up in the log. Shorter values
are never masked (masking `1` or `true` would redact every log line
containing them).
required: false
default: "false"
required:
description: "`true`: fail if the file does not exist; `false`: exit quietly with nothing loaded."
required: false
default: "true"

outputs:
keys:
description: Comma-separated list of the names that were exported.
value: ${{ steps.load.outputs.keys }}
count:
description: How many variables were exported.
value: ${{ steps.load.outputs.count }}
json:
description: "The loaded pairs as a JSON object, for `fromJSON(steps.<id>.outputs.json).<NAME>` in later steps or jobs."
value: ${{ steps.load.outputs.json }}

runs:
using: composite
steps:
- name: Load the dotenv file
id: load
shell: bash
env:
FILE: ${{ inputs.file }}
TARGET: ${{ inputs.target }}
PREFIX: ${{ inputs.prefix }}
STRIP_PREFIX: ${{ inputs.strip_prefix }}
OVERRIDE: ${{ inputs.override }}
MASK: ${{ inputs.mask }}
REQUIRED: ${{ inputs.required }}
run: |
set -euo pipefail
case "$TARGET" in env|json|both) ;; *) echo "::error::target must be env, json or both, got '$TARGET'"; exit 1 ;; esac
command -v jq >/dev/null || { echo "::error::jq is not on this runner"; exit 1; }
if [[ ! -f "$FILE" ]]; then
if [[ "$REQUIRED" == "true" ]]; then echo "::error::dotenv file not found: $FILE"; exit 1; fi
echo "no $FILE; nothing loaded"; { echo "keys="; echo "count=0"; echo "json={}"; } >> "$GITHUB_OUTPUT"; exit 0
fi
names=()
json='{}'
while IFS= read -r line || [[ -n "$line" ]]; do
line="${line%$'\r'}"
# Skip blanks and comments; drop a leading `export `.
[[ "$line" =~ ^[[:space:]]*(#|$) ]] && continue
line="${line#"${line%%[![:space:]]*}"}"
line="${line#export }"
[[ "$line" == *=* ]] || { echo "::warning::ignoring line without '=': ${line%%=*}"; continue; }
key="${line%%=*}"; value="${line#*=}"
key="${key%"${key##*[![:space:]]}"}"
[[ "$key" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]] || { echo "::warning::ignoring invalid name '$key'"; continue; }
if [[ -n "$PREFIX" && "$key" != "$PREFIX"* ]]; then continue; fi
# Trim, then strip one pair of matching quotes.
value="${value#"${value%%[![:space:]]*}"}"; value="${value%"${value##*[![:space:]]}"}"
if [[ ${#value} -ge 2 ]]; then
case "$value" in
\"*\") value="${value:1:${#value}-2}" ;;
\'*\') value="${value:1:${#value}-2}" ;;
esac
fi
name="$key"
if [[ -n "$PREFIX" && "$STRIP_PREFIX" == "true" ]]; then name="${key#"$PREFIX"}"; fi
[[ "$name" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]] || { echo "::warning::ignoring '$key': stripping the prefix leaves an invalid name"; continue; }
if [[ "$OVERRIDE" != "true" && -n "${!name+x}" ]]; then echo "keeping existing $name"; continue; fi
if [[ "$MASK" == "true" && ${#value} -ge 8 ]]; then echo "::add-mask::$value"; fi
if [[ "$TARGET" == "env" || "$TARGET" == "both" ]]; then
delim="megaeth_export_env_$RANDOM$RANDOM"
printf '%s<<%s\n%s\n%s\n' "$name" "$delim" "$value" "$delim" >> "$GITHUB_ENV"
fi
if [[ "$TARGET" == "json" || "$TARGET" == "both" ]]; then
json="$(jq -c --arg k "$name" --arg v "$value" '. + {($k): $v}' <<<"$json")"
fi
names+=("$name")
done < "$FILE"
joined="$(IFS=,; echo "${names[*]:-}")"
{ echo "keys=$joined"; echo "count=${#names[@]}"; echo "json=$json"; } >> "$GITHUB_OUTPUT"
echo "loaded ${#names[@]} variable(s) from $FILE into $TARGET: ${joined:-<none>}"
91 changes: 91 additions & 0 deletions .github/actions/git-credentials/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Git Credentials

`uses: megaeth-labs/.github/.github/actions/git-credentials@main`

<!-- generated: description -->
Let git — and so cargo, go, pip and anything else that shells out to git — fetch private repositories over HTTPS without prompting, by adding a global `url.<https-with-token>.insteadOf <https>` rewrite for one host. The token never appears in the script: it is passed through the environment and masked in the log. Any token works: a GitHub App installation token (the recommended kind — mint one with `actions/create-github-app-token` for the repositories the build needs), a fine-grained or classic PAT. Optional extras cover the two things the old per-repository copies also did — mark every directory safe for container runners whose workspace belongs to another user, and disable git's low-speed abort for slow mirrors — plus an SSH rewrite for lockfiles that pin `git@host:` URLs. `mode: unset` removes the rewrite again. Nothing is organisation-specific; the host is an input.
<!-- /generated -->

Standalone action; no family guide.

## Inputs

<!-- generated: inputs -->
| Input | Required | Default | Description |
|---|---|---|---|
| `token` | no | | The token to embed in the rewritten URL. Required unless `mode` is `unset`. |
| `host` | no | `github.com` | Git host to rewrite, without scheme. |
| `username` | no | `x-access-token` | Username part of the rewritten URL. `x-access-token` is what GitHub expects for App installation tokens and also accepts for PATs. |
| `mode` | no | `set` | `set` adds the rewrite, `unset` removes it (and the SSH rewrite, if any). |
| `rewrite_ssh` | no | `false` | `true`: also rewrite `ssh://git@<host>/` and `git@<host>:` to the token URL, for lockfiles or manifests that pin SSH remotes. |
| `safe_directory` | no | `false` | `true`: `git config --global --add safe.directory '*'`, needed on container runners whose workspace is owned by a different user than the one running git. |
| `disable_low_speed_abort` | no | `false` | `true`: set `http.lowSpeedLimit 0` and `http.lowSpeedTime 999999` so git never aborts a slow fetch. The old per-repository copies did this. |
| `show_config` | no | `false` | `true`: print `git config --global --list` afterwards (the token is masked). |
<!-- /generated -->

## Outputs

<!-- generated: outputs -->
| Output | Description |
|---|---|
| `rewrite` | The `url.<…>.insteadOf` key that was set or unset, token elided. |
<!-- /generated -->

## What it runs

<!-- generated: steps -->
1. Configure git
<!-- /generated -->

## Errors it reports

<!-- generated: errors -->
- `mode must be`
- `host must be a bare host name, got`
- `username must be a plain word, got`
- `token is required when mode is`
- `token contains characters that cannot go in a URL`
<!-- /generated -->

## Example

Mint an App installation token for the private repositories the build pulls,
then let git use it. Cargo, go and pip all go through git for
`https://github.com/…` sources:

```yaml
- uses: actions/create-github-app-token@v3
id: app-token
with:
client-id: ${{ vars.MEGA_MAXWELL_CLIENT_ID }}
private-key: ${{ secrets.MEGA_MAXWELL_PK }}
owner: megaeth-labs
repositories: private-dep-a,private-dep-b # what the build fetches

- uses: megaeth-labs/.github/.github/actions/git-credentials@main
with:
token: ${{ steps.app-token.outputs.token }}

- run: cargo build --release
```

A PAT works the same way (`token: ${{ secrets.SOME_PAT }}`). On a container
runner whose workspace is owned by another user add `safe_directory: "true"`;
for lockfiles that pin `git@github.com:` remotes add `rewrite_ssh: "true"`;
to take the rewrite out again before a step that must not see it,
`mode: unset`.

## Notes

- The rewrite lives in the runner's global git config for the rest of the
job. Ephemeral runners discard it; on a long-lived self-hosted runner use
`mode: unset` at the end of the job, or the next job inherits it.
- Running it twice replaces the token, it never stacks two rewrites for the
same host. Only one token per host at a time.
- App installation tokens expire after one hour. A job that fetches late in
a long build needs the token minted just before the fetch.
- The `x-access-token` username is what GitHub expects for App tokens and
also accepts for PATs; the old per-repository copies embedded the PAT as
the username instead, which works for PATs only.
- The old copies also set `credential.helper store`; the URL rewrite makes
it redundant and it is not set here.
Loading
Loading