Skip to content

ci(tools): parse the run steps of composite actions in the run-syntax gate - #894

Open
gHashTag wants to merge 3 commits into
mainfrom
ci/run-syntax-composite-actions
Open

gHashTag wants to merge 3 commits into
mainfrom
ci/run-syntax-composite-actions

Conversation

@gHashTag

@gHashTag gHashTag commented Oct 5, 2026 •

Copy link
Copy Markdown
Owner

Closes #854. Stacked on #893 — merge #893 first; until then this diff also shows #893's commit (54020cd).

What was wrong

tools/check_run_syntax.py globbed .github/workflows/*.yml|yaml and nothing else. #852 adds the repo's first composite action (.github/actions/openxc7-chipdb/action.yml), called by 53 jobs, and a run step in it that does not parse would have passed the gate.

What changes

  • Composite actions under .github/actions/** (action.yml or action.yaml, any depth) are read like workflows: bash -n per shell, plus the single-quoted bash -c '...' scripts inside each step, through one check_run() shared with workflow steps.
  • A composite run: step must name shell: (GitHub requires it there; there are no job or workflow defaults in an action). One that does not is cannot tell, rc 2.
  • The runner is the caller's: every job that uses: the action directly or through other local actions; a cycle of actions ends. A macOS-possible caller under a gate whose bash is not 3.x is cannot tell, the same rule as for workflow steps.
  • An action no workflow calls is still parsed. Node and Docker actions have no run: steps and are skipped.
  • Output line: workflows read: X of Y; actions read: A of B (U with no caller here); ....

Measured

result
self-test, laptop (bash 3.2) 98 of 98 (23 new: 19 one-tree action cases, 4 direct)
mutants of the new code 17 of 18 killed; survivor has_actions = True is equivalent (rglob of a missing directory is empty)
this branch's tree 122 of 122 workflows, 0 of 0 actions, rc 0
#852's tree (30b2919), as is 120 of 120 workflows, 1 of 1 actions, 20 inner scripts, 0 bad, rc 0
#852's tree with an unclosed " planted in the action's last step new gate rc 1, gate at 54020cd rc 0
callers of openxc7-chipdb on #852's tree 53 jobs, ubuntu-22.04 / ubuntu-latest, 0 macOS-possible

Mutants killed: no action glob · one-level glob · action.yml only · no macOS caller check · macOS callers never found · no transitive callers · no cycle guard (RecursionError) · no shell requirement · uncalled actions skipped · trailing / kept in uses: · every action treated as composite · four cannot-tell returns turned into skips · action findings dropped · action cannot-tells dropped.

The laptop's bash is 3.2, so the macOS-caller cases in the one-tree list expect rc 1 there and rc 2 on CI (bash 5); four direct cases pin the bash 5 answer on any machine.

Not changed here

Not measured

  • The tee-pipefail gate (tools/check_tee_pipefail.py) still reads workflows only. In a composite action shell: bash already carries -o pipefail, so the gap is shell: sh or a custom bash {0} with | tee; no action in any tree here has one. Not fixed in this PR.
  • A Windows caller's Git Bash is not modelled; an action step is checked with the gate's own bash.

🤖 Generated with Claude Code

gHashTag and others added 3 commits October 5, 2026 09:23
check_run_syntax ran bash -n on each run: script, and its own docstring
said what that misses: a `docker run ... bash -c '...'` script is one
string to the outer shell, so an error inside it was not seen unless it
broke the outer quoting. (The apostrophe that broke run 37253683063 did
break the outer quoting, and main's gate already reports it; an unclosed
`if` or a stray `fi` inside the string it does not.)

Once the outer script parses, each single-quoted `bash -c '...'` and
`sh -c '...'` argument is now read out the way the outer shell reads it
-- '\'' is a quote, a splice of an outer value ('"$PART"', '$X') is one
word -- and parsed with bash -n / sh -n. A closing quote glued to
anything else, and a quote that never closes, are "cannot tell" (exit
2), not clean. Double-quoted bash -c "..." is not read: the outer shell
expands it first.

Measured on main's 122 workflows: 24 such scripts, 8 of them with
'"$VAR"' splices, all 24 parse. The self-test plants 12 new cases
(75 of 75 pass); main's gate run against the new cases fails the 7 that
expect a non-zero exit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… gate

check_run_syntax.py read .github/workflows/*.yml only, so a run step inside
a composite action (.github/actions/**/action.yml, as #852 adds) could fail
to parse and the gate would pass. Closes #854.

- Composite actions are read like workflows: bash -n per shell, and the
  single-quoted bash -c scripts inside a step, via a shared check_run().
- A composite run step must name its shell; one that does not is cannot
  tell (rc 2), as GitHub itself rejects it.
- The runner is the caller's: every job that uses the action, directly or
  through other local actions (cycles end). A macOS-possible caller under
  a gate whose bash is not 3.x is cannot tell, as for workflow steps.
- An action no workflow calls is still parsed. Node and Docker actions
  have no run steps and are skipped.

Self-test 98 of 98 (23 new cases); 17 of 18 mutants of the new code are
killed, the survivor is equivalent. On #852's tree the gate reads 1 of 1
actions and catches an unclosed quote planted in its last step, which the
gate at 54020cd passes.

.github/actions/** is not added to the gate's paths: here: no tracked file
matches it on main yet, and the workflow-path gate rejects that entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The note said .github/actions/ is not watched by this workflow's paths:
until a tracked file exists there. #852 creates the directory and is the
PR that adds the entry, so the note would be wrong on whichever of the two
merges second. The reason stays in #894's description.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gHashTag added a commit that referenced this pull request Oct 5, 2026
The note said .github/actions/ is not watched by this workflow's paths:
until a tracked file exists there. #852 creates the directory and is the
PR that adds the entry, so the note would be wrong on whichever of the two
merges second. The reason stays in #894's description.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gHashTag added a commit that referenced this pull request Oct 5, 2026
The openxc7-chipdb action is called by 46 workflows, and every one of them
filters push on its own design files only, with no pull_request trigger. A
change to .github/actions/openxc7-chipdb/ alone therefore ran nothing, on the
PR or on main. Measured on this branch with a paths-coverage script: 46 of 46
triggers did not cover the action.

- ax7203-led-onehot.yml (the smallest design, one seed) now runs on a PR that
  touches the action, and on a push to main that does, so main also warms the
  cache for a new key.
- run-syntax-gate.yml watches .github/actions/**, which this PR creates; the
  gate reads composite actions once #894 lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown

X Brain Health Check

Score: 100.0/100
Status: 🟢 HEALTHY
Threshold: 80/100

X Brain is above merge threshold

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown

🧪 Brain Stress Test Results

Score: not measured (no stress test prints a Score: line)
Status: 🟢 PASS
Threshold: 270/300

X Stress test passed

@dmitrii-f-t27
dmitrii-f-t27 force-pushed the ci/run-syntax-composite-actions branch from ef19f75 to b335f27 Compare October 10, 2026 00:42
dmitrii-f-t27 pushed a commit that referenced this pull request Oct 10, 2026
The openxc7-chipdb action is called by 46 workflows, and every one of them
filters push on its own design files only, with no pull_request trigger. A
change to .github/actions/openxc7-chipdb/ alone therefore ran nothing, on the
PR or on main. Measured on this branch with a paths-coverage script: 46 of 46
triggers did not cover the action.

- ax7203-led-onehot.yml (the smallest design, one seed) now runs on a PR that
  touches the action, and on a push to main that does, so main also warms the
  cache for a new key.
- run-syntax-gate.yml watches .github/actions/**, which this PR creates; the
  gate reads composite actions once #894 lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tri-claw

tri-claw Bot commented Oct 10, 2026

Copy link
Copy Markdown

Summary by t27-bees

The PR extends the run-syntax gate to also parse run: steps in composite actions under .github/actions/**. It extracts and checks single-quoted bash -c '...'/sh -c '...' scripts inside those steps (and workflow steps), resolves transitive callers to determine the runner (macOS vs Linux), and requires composite run: steps to declare shell:.

Worth a look

  • tools/check_run_syntax.py:85-130 (inner_scripts) — The SPLICE regex only matches "...", ${...}, and $VAR. It does not match $(cmd), `cmd`, $((arith)), or parameter expansions like ${var:-default}. A script like bash -c 'echo '$(date)'' will be reported as “glued to $” → cannot tell (rc 2), even though it is valid bash.

  • tools/check_run_syntax.py:195-210 (action_callers) — jobs_of recurses with frozenset for cycle detection, but direct.get(rel, []) returns the same list object for every caller. If a job calls multiple actions, that job dict appears in multiple action’s caller lists. Not a bug now, but mutating those dicts later would cross-contaminate.

  • tools/check_run_syntax.py:212-245 (check_action) — Calls _shell_named(step, {}, {}) with empty job/workflow dicts. _shell_named currently ignores them, but shell_of (used for workflows) does read job/workflow defaults. If _shell_named is ever updated to support defaults, composite actions will silently lose that behavior.

  • tools/check_run_syntax.py:260-270 (main) — actions_root.rglob("action.yml") + rglob("action.yaml") will process both files if they coexist in the same directory. GitHub Actions permits only one; the tool would check both and report duplicate errors.

  • tools/test_check_run_syntax.py:326 — run_case mutates module globals crs.ROOT and crs.WORKFLOWS. Tests run sequentially so it works today, but any future parallel test runner or reordering would cause cross-test pollution.


Read: 3 files, +326 -12, the whole diff. Head b335f27. Model: openai-compatible/nvidia/nemotron-3-ultra-550b-a55b. Commands: @t27-bees review | summary | help | pause | resume. Free for every repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: run-syntax gate does not read composite actions (.github/actions)

1 participant