feat: carry a failing command's stderr snippet in the failed:<step> tag#65
Merged
Conversation
The phone-home failure tag only ever named the step that failed, giving no clue why. gh_runner_phone_home_failed() now tees each shell's stderr into a scratch file and appends a sanitized, single-line, ~150-char tail of it to the tag as failed:<step>:<detail>, capped well under the 256-char EC2 tag-value limit. Applies to buildUserData(), buildReusableUserData(), and its registerScript. src/wait.js splits on the first ':' after the failed: prefix to recover step vs. detail, staying backward compatible with the old no-detail shape. Signed-off-by: kurok <22548029+kurok@users.noreply.github.com>
kurok
added a commit
to namecheap/terraform-provider-namecheap
that referenced
this pull request
Jul 6, 2026
The warm-restart path (unlike the already-verified cold-launch path) still fails at the configuring step with zero detail beyond the step name. Since buildReuseUserData() regenerates the identical bootstrap script buildUserData() uses (already confirmed sed-fix-correct), the warm-restart-specific failure cause is otherwise invisible without live instance access. namecheap/ec2-github-runner#65 adds a captured stderr snippet to the failed:<step> phone-home tag (now readable via this repo's existing debug:true diagnostic flag), merged as 453ef2e, v4.0.0 moved to match. Re-pinning this PR's 4 usages so the next warm-restart run surfaces the real config.sh error instead of another opaque 'bootstrap failed during configuring'. Signed-off-by: kurok <22548029+kurok@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The bootstrap phone-home tag (
ec2-github-runner:bootstrap) only ever namedthe step that failed (
failed:<step>) — zero detail on why. This has costreal debugging time (a #63-class bug, and an unexplained warm-restart-only
failure at the
configuringstep where the realconfig.sherror wasinvisible without live SSH/SSM).
gh_runner_phone_home_failed()(new, inPHONE_HOME_HELPERS) tees eachshell's stderr into a scratch file and, on failure, appends a sanitized,
single-line, ~150-char tail of it to the tag:
failed:<step>→failed:<step>:<detail>.step's detail only reflects that step's own commands (not noise left by an
earlier
|| true-tolerated one, e.g.mount).of how much garbage a failing command emits, and control bytes/newlines are
sanitized to a single printable line.
buildUserData()'souter + inner shells,
buildReusableUserData()'s outer + inner shells, andits
registerScript(per-boot re-registration forreuse: stop).src/wait.jssplits on the first:after thefailed:prefix torecover step vs. detail (step names never contain
:, so this isunambiguous even when the detail itself has colons, e.g. a
config.sh: error: ...line). Fully backward compatible with the oldno-detail
failed:<step>shape —error.bootstrapDetailis simply absent.Before / after
A note on the tee race
tee's copy into the scratch file runs via a background processsubstitution, so reading the file immediately in the ERR trap can race and
see it still empty.
gh_runner_phone_home_failed()closes this shell's endof that pipe (fd 2 — the script is terminating anyway) and polls (bounded,
~200ms max) for the tee process to exit via
kill -0, which only happensonce it has drained everything written to the pipe. Verified empirically
against both bash 3.2 and bash 5.3 — deliberately not a plain
waiton thetee PID, since that needs bash >= 4.4 and could hang indefinitely if
tee'swrite end ever blocked.
Test plan
tests/phone-home-detail.test.js(new): extracts the actual linesbuildUserData()/buildReusableUserData()produce (outer rootshells, both inner heredocs, and
registerScript), splices in afabricated failing command, and runs the result as a genuine bash
script file against a fake
curl(IMDS) + fakeawsCLI — provingthe real generated code truncates/sanitizes correctly, including one
case with ~3.6KB of multi-line, control-character/ANSI-laden stderr,
and a pathological ~50KB newline-filled case that must stay under 256
chars with no embedded newlines. Also covers the backward-compatible
no-detail case.
tests/wait.test.js: detail surfaced in the thrown error,core.error,and
error.bootstrapDetail; a detail containing colons splitscorrectly; backward compat with old-style
failed:<step>(nobootstrapDetail, unchanged message).npm test— 237 passed, 18 suites.npm run lint— clean.npm run packagerebuilt twice — byte-identical (sha256summatch),diff scoped to the
src/aws.js/src/wait.jschanges only.