Skip to content

qa-pro.yml drifted from the reusable workflow's cost-avoidance design (user-registration-pro) - what should we do to stop it recurring? #5

Closed

Description

@asheshmagar

What we found

pro-suite.yml's own header comment documents the intended trigger design: "It runs when a PR opens, then ON REQUEST: @claudegrill suite on the PR... The shipped caller drops synchronize, so a push spends no metered minutes."

themegrill/user-registration-pro/.github/workflows/qa-pro.yml doesn't follow that. It includes synchronize in its trigger types, so every push to an open PR re-runs the full ~9 minute QA suite (pro job ~3m52s + free-with-pro job ~7m55s). themegrill/colormag-pro's copy of the same file is configured correctly, no synchronize. It's currently the only other repo using this workflow.

Repeated pushes to one release PR (#1572) triggered 15+ full QA-pro runs in two days, and that lines up directly with themegrill org Actions billing going from $0/day (inside included minutes) to roughly $5.4/day billed starting Sep 9, 2026.

Why this isn't just a one-off fix

qa-pro.yml is, per its own header, "Written by /claudegrill:setup... a plain caller", a file copied into each pro repo individually, not centrally enforced or version-locked to a template. Two repos already disagree on this exact setting. As more pro repos get qa-pro.yml set up, the same drift can reappear repo by repo with the same cost impact, and nothing today would catch it before it shows up on a bill.

Open question: what else could we do here, beyond just editing this one file?

Some options worth discussing, not proposing any one as decided.

Option A: have /claudegrill:setup, or a separate command, re-sync or diff existing callers against the canonical template on request, so drift can be found deliberately instead of discovered via a bill.

Option B: a lightweight CI check, in claudegrill itself or org-wide, that flags any qa-pro.yml whose "on:" block includes synchronize, so it surfaces on the next PR rather than silently.

Option C: add a concurrency group to the caller template so even a future reintroduction of synchronize, or rapid manual dispatches, cancels in-flight runs instead of stacking them.

Option D: something else entirely, e.g. surfacing runner-minutes per PR somewhere more visible than the org billing page, so this kind of drift is noticed in days, not weeks.

Opening this as a discussion rather than a fix-this-one-line issue, since the interesting part is preventing recurrence, not just patching user-registration-pro.

Activity

  1. changed the title [-]————————qa-pro.yml drifted from the reusable workflow's cost-avoidance design (user-registration-pro) - what should we do to stop it recurring?[/-] [+]qa-pro.yml drifted from the reusable workflow's cost-avoidance design (user-registration-pro) - what should we do to stop it recurring?[/+] on Sep 11, 2026
  2. iamprazol commented on Sep 11, 2026

    @iamprazol
    Collaborator

    Structural fix is up: #6.

    The fix has already been pushed to the base branch of each product, What #6 adds is the recurrence prevention this issue asked for, going with a mix of Option A and Option C's intent, done centrally instead of per-caller:

    suite.yml and pro-suite.yml now allow-list pull_request actions to opened/reopened/ready_for_review in their own job-level if:. If any caller's on: block ever includes synchronize again — this repo's or a future one's — the job is skipped outright: $0, no runner starts, regardless of what the caller says. This was chosen over a caller-only fix specifically because the callers have no central version lock and have already disagreed on this exact line once.
    setup-product.mjs check-workflow — a new read-only command that reads an existing qa-pro.yml/qa-suite.yml and flags synchronize or pull_request_target, so this can be checked on request rather than discovered on a bill. Ran it against all three live pro-repo callers (colormag-pro, user-registration-pro, zakra-pro) — all clean today.
    Option D (surfacing runner-minutes more visibly than the billing page) is not addressed — each job already reports its own elapsed minutes to the run summary, but nothing aggregates across runs/repos.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions