Skip to content

ci(brain): let the merge gate fail, and fail on a failed detect-build - #884

Open
gHashTag wants to merge 2 commits into
mainfrom
ci/brain-gate-can-fail
Open

gHashTag wants to merge 2 commits into
mainfrom
ci/brain-gate-can-fail

Conversation

@gHashTag

@gHashTag gHashTag commented Oct 4, 2026

Copy link
Copy Markdown
Owner

Closes #883. Part of #851 (W15).

What changes

brain-ci.yml, two edits:

  1. 🔒 Merge Gate: if: github.event_name == 'pull_request' becomes if: ${{ !cancelled() && github.event_name == 'pull_request' }}. Without a status function GitHub adds success(), so the gate ran only when everything upstream had passed, and its test "the report did not pass" could never be true. On 3 of 3 pull_request runs where the report failed, the gate was skipped.
  2. 📋 Brain Health Report: before trusting needs.detect-build.outputs.present, require needs.detect-build.result == 'success'. A failed detect-build never writes present, and [ "" != "true" ] printed "Brain CI skipped — no root build.zig … Non-blocking" and exited 0. main has a root build.zig.

Measured (dispatch A/B, both arms with a forced exit 1 in detect-build)

arm run report Merge Gate run
control: main (exp/gate-defect-ab) 37240659458 success: if [ "" != "true" ] → "no root build.zig … Non-blocking" skipped, 0 steps failure
fix (exp/gate-fix-ab) 37240661475 failure: [ "failure" == "success" ] → "detect-build was failure" failure: [ "failure" != "success" ] → "Merge gate blocked" failure

On both throwaway branches, merge-gate's pull_request filter is dropped so a dispatch reaches it. The control keeps the implicit success(), and the fix arm keeps !cancelled(). Apart from that, each arm differs from its base by the forced step alone.

The control also corrected me. I had expected the whole brain CI to go green. It does not, because detect-build's own check stays red. The false part is the report alone: a green check that gives a wrong reason. The issue body records the correction.

Order and overlap

Not measured

  • Whether detect-build has ever failed on a real run.
  • This PR's own pull_request run, nothing forced: the report and the Merge Gate should both succeed. Pending.

🤖 Generated with Claude Code

merge-gate had `if: github.event_name == 'pull_request'` and no status
function, so GitHub added success(): it ran only when the report and
every job upstream of it had passed, and its one test ("the report did
not pass") could never be true. On 3 of 3 pull_request runs where the
report failed, the gate was skipped. It now runs on !cancelled().

The report read an empty `present` output as "no root build.zig" and
exited 0. A failed detect-build never writes that output, so a broken
detector gave a green report check claiming the repo has no root
build.zig (detect-build's own check stays red). The report now requires
detect-build to have succeeded before it trusts `present`.

Closes #883. Part of #851.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown

X Brain Health Check

Score: 100.0/100
Status: 🟢 HEALTHY
Threshold: 80/100

X Brain is above merge threshold

@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown

🧪 Brain Stress Test Results

Score: not measured (no stress test prints a Score: line)
Status: 🟢 PASS
Threshold: 270/300

X Stress test passed

The verdict step tested `[ "$detect_ok" != true ]`, a negated comparison;
every added bash if here tests a named bool with `= true`. Renamed to
detect_failed so the if reads what it checks. Same exit for every
detect-build result under bash -e: success 0; failure, cancelled, skipped
and empty 1 (5/5, old and new side by side).

Found by running speed wflint on a trial merge of the 18 open loop PRs:
the only lint finding the merged tree adds over main 57de4a1.

Refs #883, #851.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gHashTag

gHashTag commented Oct 5, 2026

Copy link
Copy Markdown
Owner Author

7fbeb9b renames the verdict bool and changes nothing else. detect_ok / != true becomes detect_failed / = true.

How it was found. I trial-merged the 18 open PRs that merge cleanly (speed mergeorder, using merge-tree and commit-tree only, no refs). Then I ran my workflow lint (speed wflint) on the merged tree. One of its rules is that every if must test a named bool in the form [ "$x" = true ]. Main has 175 findings. The merged tree had 176, and the extra one was this line in brain-ci.yml, which this PR added. The lint accepts only = true, so it flagged != true. The behaviour was already correct.

Check. I ran the old and new snippets under bash -e for each needs.detect-build.result value. Both give the same exit code for all five:

result old new want
success 0 0 0
failure 1 1 1
cancelled 1 1 1
skipped 1 1 1
(empty) 1 1 1

After the fix the merged tree has 175 findings again, and comm -13 against main shows 0 new.

Not measured. I have not re-run the merge gate end to end on 7fbeb9b. This PR's checks are running now.

Refs #883, #851

🤖 Generated with Claude Code

@dmitrii-f-t27
dmitrii-f-t27 force-pushed the ci/brain-gate-can-fail branch from 7fbeb9b to 98da33e Compare October 10, 2026 00:42
@tri-claw

tri-claw Bot commented Oct 10, 2026

Copy link
Copy Markdown

Summary by t27-bees

The PR fixes two related CI logic bugs: the Merge Gate was implicitly guarded by success() so it only ran after a passing report (making its failure check unreachable), and the Brain Health Report treated a failed detect-build as "no root build.zig" instead of an error.

Worth a look

  • .github/workflows/brain-ci.yml:517–527 – The new detect_failed block exits 1 when detect-build result is not success. If detect-build is cancelled (not failed), ${{ needs.detect-build.result }} equals "cancelled", so this will also fail the report. That may be intentional, but it differs from the prior "non-blocking" behavior for any non-success state.
  • .github/workflows/brain-ci.yml:633 – The gate now runs when !cancelled() && github.event_name == 'pull_request'. If the workflow is cancelled mid-run, the gate is skipped (correct). However, if detect-build fails and the report fails (exit 1), the gate will run because the workflow isn't cancelled—this is the desired fix. Just confirm that needs: [brain-health-report] without an explicit result filter means the gate runs regardless of the report's conclusion (which is the point).
  • .github/workflows/brain-ci.yml:521 – The summary message uses ${{ needs.detect-build.result }} directly in the string. If the result contains characters that break Markdown (unlikely for GitHub's fixed result strings), it could render oddly. Low risk.

Read: 1 files, +12 -1, the whole diff. Head 98da33e. Model: openai-compatible/nvidia/nemotron-3-ultra-550b-a55b. Commands: @t27-bees review | summary | help | pause | resume. Free for every repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

brain-ci: Merge Gate can never fail, and a failed detect-build passes as 'not applicable'

1 participant