Skip to content

ci(tools): hold the run-syntax self-test's e6eac090 inputs as pinned files - #911

Merged
dmitrii-f-t27 merged 2 commits into
mainfrom
fix/run-syntax-fixtures-after-rewrite
Oct 11, 2026
Merged

dmitrii-f-t27 merged 2 commits into
mainfrom
fix/run-syntax-fixtures-after-rewrite

Conversation

@dmitrii-f-t27

Copy link
Copy Markdown
Collaborator

Closes #910

run-syntax fails on every PR that touches a workflow, whatever that PR changes. This PR fixes the cause.

Why it fails

tools/test_check_run_syntax.py reads its two regression inputs with git show e6eac090:<path>. main was rewritten after that commit. e6eac090 is no longer an ancestor of main; the same tree is 78606c5d there. A fresh clone does not have the commit, so the read returns nothing. The empty input gives rc 2 instead of 1, and both cases go BAD: 61 of 63 cases as expected (#907, run 38010467066).

What changes

  • tools/run_syntax_fixtures/build-matrix.e6eac090.yml.txt and iddr-golden-diff.e6eac090.yml.txt are byte-identical to the blobs at e6eac090, made with git show e6eac090:<path>. Their git hash-object values are 9d07d679… and 034f4e96…, the blob ids git rev-parse e6eac090:<path> gives.
  • The test reads those files and checks each one against its pinned blob id. A missing or edited fixture becomes an empty input. Its case then goes BAD, and a BAD fixture … line names it.
  • The .txt suffix keeps the files away from YAML tooling. check_run_syntax.py reads only .github/workflows/*.yml.
  • The workflow no longer fetches full history and now also triggers on tools/run_syntax_fixtures/**.

No expected rc changes, no case is dropped, no step is disabled.

Checks run locally (Python 3, macOS)

run result
python3 tools/test_check_run_syntax.py 63 of 63 cases as expected
python3 tools/check_run_syntax.py workflows read: 122 of 122; steps that do not parse: 0
one byte appended to the build-matrix fixture BAD fixture build-matrix.e6eac090.yml.txt: blob 74ae8f62…, pinned 9d07d679…, 62 of 63
build-matrix fixture removed BAD fixture build-matrix.e6eac090.yml.txt: missing, 62 of 63

Related: #893 and #894 change the same gate. They do not touch these lines.

Dmitrii Fedorov (@dmitrii-f-t27)

…ed files

The self-test built its two regression cases with `git show e6eac09:<path>`.
main was rewritten after that commit, so a fresh clone no longer has it, the
read returned nothing, and both cases went BAD (rc 2 instead of 1) on every PR
that touches a workflow.

The two inputs are now files in tools/run_syntax_fixtures/, byte-identical to
the blobs at e6eac09, and the test checks each against its pinned blob id
(9d07d67, 034f4e9). A missing or edited fixture is an empty input and a
named BAD line. Expected rcs and the case count (63) are unchanged. The
checkout no longer needs full history, and the gate now also runs when a
fixture changes.

Closes #910
@tri-claw

tri-claw Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Summary by t27-bees

The PR fixes a flaky self-test by replacing git show e6eac090:<path> reads (which fail on shallow clones because that commit is no longer in history) with pinned fixture files verified by blob ID. The workflow no longer needs full history and now triggers on the fixtures directory.

Worth a look

  • tools/test_check_run_syntax.py:105–135 — The pinned() function returns "" on mismatch or missing file, which makes the test case receive empty input and produce rc=2 (while the case expects rc=1). This correctly surfaces as a BAD case, but the case still runs and counts toward the total; the output will show both the explicit BAD fixture … line and the case failure.
  • tools/test_check_run_syntax.py:137–146 — The two blob IDs (9d07d679… and 034f4e96…) are hardcoded. If the fixture files are ever regenerated, these must be updated to match git hash-object of the new content, otherwise every run will report BAD fixture … blob mismatch.
  • .github/workflows/run-syntax-gate.yml:15, 21 — Added tools/run_syntax_fixtures/** to both push.paths and pull_request.paths. This ensures the gate runs when fixtures change, but also means any edit to those files (even whitespace) triggers the workflow.
  • .github/workflows/run-syntax-gate.yml:30–32 — Removed fetch-depth: 0 with a comment explaining fixtures are now local. If any other step in this job implicitly relied on full history, it would now break (though the job only runs the self-test).
  • tools/run_syntax_fixtures/iddr-golden-diff.e6eac090.yml.txt — The fixture contains a workflow that fetches and builds prjxray from source at runtime (lines 60–80). This is not executed by the syntax test, but if the fixture were ever used as a real workflow it would depend on external network access and specific commit availability.

Read: 4 files, +379 -13, the whole diff. Head 8bbf451. Model: openai-compatible/nvidia/nemotron-3-ultra-550b-a55b. Commands: @t27-bees review | summary | help | pause | resume. Free for every repository.

@github-actions

Copy link
Copy Markdown

X Brain Health Check

Score: 100.0/100
Status: 🟢 HEALTHY
Threshold: 80/100

X Brain is above merge threshold

@github-actions

Copy link
Copy Markdown

🧪 Brain Stress Test Results

Score: not measured (no stress test prints a Score: line)
Status: 🟢 PASS
Threshold: 270/300

X Stress test passed

@dmitrii-f-t27
dmitrii-f-t27 merged commit 21990e0 into main Oct 11, 2026
65 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci(tools): run-syntax self-test reads e6eac090, which the rewritten main no longer has

1 participant