Skip to content

ci: route the seven release and publish jobs to the kirocrew-runners group - #24962

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
ci/release-runner-group
Nov 2, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
ci/release-runner-group

Conversation

@tim-finnigan

@tim-finnigan tim-finnigan commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A release starts when we tag it, instead of waiting behind another organisation's CI.

Every plain ubuntu-latest job across every Amazon GitHub org draws on one shared
pool, first-come first-served, with no way to prioritise a job. The seven release
and publish jobs sit in that pool today, alongside this repository's own
high-volume PR CI. When the pool fills, a release waits, and nobody here can
expedite it.

Why it matters

A blocked release is blocked on something outside our control. During the
enterprise-wide queueing incident in August, two public AWS repositories could not
ship releases for two days. Our own capacity guidance names this case directly:
release paths should not share a contended pool with high-volume PR CI, because
they are human-blocking and low-volume, which is what makes carving them out cheap.

Not a goal

  • Freeing shared-pool capacity. These jobs are low-volume, so the slots recovered
    are negligible; this is a latency change.
  • Moving sign-and-notarize.yml's sign and publish. Not for a safety
    reason: four of the jobs this PR moves hold that same signing role, and the
    runner pool is not what admits a job to it. Those two stay on ubuntu-latest
    to keep this change to the release and publish path, and moving them is the
    same one-line change in its own pull request.
  • Moving the two windows-latest jobs in build-windows.yml. The group is a
    GitHub-hosted Ubuntu VM and no Windows larger-runner group exists.
  • Moving the macOS jobs. No macOS larger-runner definition exists, so there is
    nothing to move them to.
  • Changing what any job does. Steps, timeouts, permissions and triggers are
    untouched.

What changed (motivation → approach → change)

A release that cannot get a runner is stuck, and the fix is to stop competing for
one. Dedicated capacity already exists: the kirocrew-runners larger-runner group
draws on a ceiling separate from the shared pool, and the AI review lanes, CodeQL
and Docker Smoke already run there.

The group was the right target rather than the CodeBuild fleet. The fleet admits
only an allowlisted actor and the tagger rotates with the on-call, so a fleet route
would fall back to the shared pool for most releases. Volume made the group safe:
every job here is workflow_call, a v* tag push, workflow_dispatch, or (for
publish-installer) a path-filtered push to main, so none fires on a pull request
and none adds PR volume to a group capped at 100 parallel.

Seven jobs move from runs-on: ubuntu-latest to the group:

Workflow Job How it reaches the group
publish-cli.yml publish-cli runner label, ubuntu-latest in a fork
publish-docker.yml publish-docker runner label, ubuntu-latest in a fork
publish-installer.yml publish-installer group:
publish-linux.yml publish-linux group:
publish-windows.yml publish-windows group:
release.yml release-candidate-tests, record-stable-promotion group:

A fork cannot reach an organization runner group, so a group: job a fork starts
queues until GitHub cancels it. nightly.yml calls publish-cli and
publish-docker with no repository guard, so the fork-reachability fix replaced the
bare group: form on those two fork-startable publish lanes with the conditional
label form build-wheel already uses. The other five cannot start in a fork: a
repository if, the CDN repository variables a fork lacks, or a dependency on
release.yml's version job, which is already on the group.

Four of those seven -- publish-cli, publish-installer, publish-linux and
publish-windows -- declare environment: prod and assume the signing role, and
that is safe on this group. The role's OIDC trust accepts exactly two subjects,
ref:refs/heads/main and environment:prod, so the runner pool is not what
admits a job to the credential. The group runs the fork review lanes against
fork-authored content as well, and nothing is shared across that boundary: the
environment gates the credential rather than the runner, no fork lane declares an
environment, and the group's runners are ephemeral per job.

The two sign-and-notarize.yml jobs are still not among them, but as a scope
choice rather than a safety one: this change keeps to the release and publish
path. notarize could not follow in any case, since it runs on macos-15 and no
macOS larger-runner group exists.

test_ci_fleet_routing_expression_parity.py pins every off-fleet job to its
expected runner, so each one is registered in _PERMANENT_EXCEPTIONS and in
_HOSTED_EXCEPTION_RUNNERS. Pinned per job rather than waived: a lane that later
drifts to some third runner still fails the check.

docs/ci/ci-and-reviews.md owns the list of sites on the group, but this change
does not add the seven to it: that inventory edit is deferred to #26425,
which is open and covers it. This PR touches no documentation. The two workflow
comments on publish-cli and publish-docker point at the doc's existing
"Larger runners for the review lanes and CodeQL" section, which is already on
main.

Backwards compatibility

Compatible: only the machine a job runs on changes. Steps, timeouts, permissions,
triggers and job names are identical, and a job name is what branch protection and
the merge queue key on.

Removes nothing: no path, record, key or function is hidden, deleted or tightened.

Tests

test/test_ci_fleet_routing_expression_parity.py gains seven entries in each of
its two registries, with publish-cli and publish-docker pinned to the label
form through the registry's existing _GROUP_LABEL_EXPR constant rather than a
second constant spelling the same expression. The test locks in that every job deliberately kept off the
CodeBuild fleet runs on the runner this repository expects, so a silent drift to a
third runner fails CI rather than shipping, and the absence of a
sign-and-notarize.yml entry means those two jobs are still pinned to the default
ubuntu-latest.

scripts/local-gate.py's backend leg ran 1,298 related tests green, and the
parity and additional-fleet-route suites ran 871 tests green. The frontend leg
did not run (npm E401 on this host); no website/ file changes.

Manual verification

N/A — unit coverage sufficient: the parity test reads the same workflow files
GitHub does, and no release can be tagged from a branch to exercise the path
before merge. The first prerelease tag after merge is the live check; its jobs
should report a kirocrew-runners runner.

Related Issues

Closes #24960

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@tim-finnigan
tim-finnigan requested review from a team and KiroCrewCaptain as code owners October 27, 2026 19:59
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 27, 2026
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 3983a62bdd938305e1a79a6a374af2a2f8e8ef21 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 3983a62bdd938305e1a79a6a374af2a2f8e8ef21 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks: each job move is the existing runner-group mechanism, reused with no new surface, and it is aimed at the queueing cause #24960 names.

What this change ships

Inventory (8 items) — 8 justified

Intent: let a tagged release start right away instead of queueing behind other organisations' jobs in the shared pool. This is a FIX.

  1. publish-cli runs on the group in this repo and on ubuntu-latest in a fork — justified
  2. publish-docker runs on the group in this repo and on ubuntu-latest in a fork — justified
  3. publish-installer runs on the kirocrew-runners group — justified
  4. publish-linux runs on the kirocrew-runners group — justified
  5. publish-windows runs on the kirocrew-runners group — justified
  6. release-candidate-tests runs on the kirocrew-runners group — justified
  7. record-stable-promotion runs on the kirocrew-runners group — justified
  8. The parity test pins all seven to their expected runners, using the existing _GROUP_LABEL_EXPR — justified

[FIRST-PRINCIPLES-REVIEWED] 3983a62

Description read

[DESCRIPTION-READ] 565fa963dbccb8375fede83e7341cf62d44a12f1ef869573f9001970dfc19cb6

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 3983a62bdd938305e1a79a6a374af2a2f8e8ef21 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: each job only moves to another runner, and reverting one is a one-line change.

Notes

  • publish-linux and publish-windows use the bare group: form and are kept out of forks only by the vars.CLI_DIST_BUCKET/CLI_CDN_BASE check. A fork that sets those variables and runs nightly would queue on these jobs until GitHub cancels them. The build-wheel label form would remove that edge case at no cost.
  • The described live check needs one addition: on the first prerelease, confirm that publish-docker's Docker steps and the 8-shard release-candidate-tests matrix both run cleanly on a kirocrew-runners runner, not only that they land on one.

[DESIGN-REVIEWED] 3983a62

Description read

[DESCRIPTION-READ] 166861456cbc2aee9363a5dffcb49bed68ccf18cf342e3d1714eb9b9d5b17ff8

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ human override accepted

Reviewed 3983a62bdd938305e1a79a6a374af2a2f8e8ef21 — this comment is updated in place on each push.

Human judgment by @tim-finnigan overrides the Opus 5.5 finding for 3983a62bdd938305e1a79a6a374af2a2f8e8ef21; the recorded reason is authoritative for this commit.

[OPUS-OVERRIDE] 3983a62

Verdict recorded from an authorized human decision for commit 3983a62bdd938305e1a79a6a374af2a2f8e8ef21.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 3983a62bdd938305e1a79a6a374af2a2f8e8ef21 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 3983a62bdd938305e1a79a6a374af2a2f8e8ef21 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 3983a62

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 3983a62bdd938305e1a79a6a374af2a2f8e8ef21 false-positive: <reason, 20 to 500 characters>

@tim-finnigan
tim-finnigan force-pushed the ci/release-runner-group branch from 218ae4c to 9868dce Compare October 27, 2026 20:09
@tim-finnigan

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: publish-docker.yml's runs-on, reached only through release.yml and nightly.yml

  • span=015185bbd642 rebutted

Not a defect: no fork head can reach this job, so there is no fork path for a
fallback to serve.
publish-docker.yml is workflow_call only (name: Publish Docker (reusable), no pull_request trigger). Its only two callers are
release.yml:779 and nightly.yml:752. release.yml triggers on push with
tags: ["v*"] and nightly.yml on schedule plus workflow_dispatch. None
of those three events can carry a fork head: a fork cannot push a tag to this
repository, and neither a schedule nor a dispatch has one.
The same holds for the other eight jobs this PR moves.
The fork case the finding describes is real for the lanes already on this
group: fork-gpt-review, fork-opus-review and the other four run on fork
heads, which is exactly why they were given a larger runner rather than a
fallback. A fork there gets the group, not ubuntu-latest.
A || 'ubuntu-latest' expression would also not provide the safety implied:
GitHub queues or errors a job whose group is wrong rather than failing over,
and on a release path a silent return to the shared pool reintroduces the
queueing this PR exists to remove.

@tim-finnigan

Copy link
Copy Markdown
Contributor Author
  • PASS punchline answered

Confirmed against the tree: the group already carries the five same-repo
review lanes, the six fork lanes, codeql.yml and both docker-smoke.yml
jobs, so these nine join an established target rather than introducing one.
The parity test pins each moved job in both _PERMANENT_EXCEPTIONS and
_HOSTED_EXCEPTION_RUNNERS, so a later drift to a third runner fails CI.

@tim-finnigan

Copy link
Copy Markdown
Contributor Author
  • PASS punchline answered

Confirmed: no new mechanism is added. The change is nine runs-on values and
two registry entries each; the existing group and the existing parity check do
the work. The one prose change is the count in
docs/ci/ci-and-reviews.md, which the move made wrong.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 27, 2026
@tim-finnigan

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: docs/ci/ci-and-reviews.md states the count and the member list of the kirocrew-runners group

  • span=7818d10f0bb8 fixed

Legitimate and fixed in 9868dce. The section said "Fourteen runs-on
sites draw on the kirocrew-runners larger-runner group" and then listed the
lanes by name, so adding nine sites without touching it left a counted claim
the tree contradicts.
Verified by counting the live sites on this branch: 25 across 20 workflow
files, not 14. The old 14 also already excluded both docker-smoke.yml jobs,
which were on the group before this PR.
The section now reads twenty-five, names the nine release and publish jobs
alongside docker-smoke.yml, is retitled to include the release path, and
carries a paragraph on why that path is a latency carve-out rather than a
capacity one and why a capped group is safe for it (none of the nine fires on
a pull request).

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 27, 2026
@tim-finnigan

Copy link
Copy Markdown
Contributor Author
  • PASS punchline answered

The punchline asks for proof the group will accept these jobs, and the
concern is well aimed: a GitHub runner group CAN be restricted to selected
workflows, which would have made this change silently break releases.
Checked against the live org configuration rather than inferred.
GET /orgs/kirodotdev/actions/runner-groups reports group id 3
kirocrew-runners with visibility: selected, restricted_to_workflows: false and selected_workflows: []. Access is scoped by REPOSITORY, with no
workflow allowlist, so every workflow in an admitted repository may use it.
This repository is demonstrably admitted: codex-review, design-review,
ux-review, first-principles-review, docker-smoke and the fork lanes have
all completed on the group since #23342 and #23343 merged.
The Manual verification: N/A line is accurate for the reason the punchline
gives: a tag push cannot be exercised from a branch, so no pre-merge run can
cover it. The body names the first prerelease tag after merge as the live
check. Rollback is a one-line revert per site.

@tim-finnigan

Copy link
Copy Markdown
Contributor Author

Backend Tests (3.12, 8) and Coverage Gate went red on 9868dced51f1 from a
lost runner, not from this diff. Re-ran the failed jobs once (attempt 2); no code
changed.

Evidence it was the runner:

  • The job started 20:56:50Z on codebuild-kirocrew-gha-linux-40575454068-1, and
    the CodeBuild project posted failure ("Build failed for project
    kirocrew-gha-linux") at 21:34:04Z.
  • The job was then marked failure at 21:43:41Z with step 8 Run tests (shard 8/8) still in_progress, steps 9-11 never started, and no job log retrievable
    (log not found). A test failure leaves a completed step and a log.
  • It died at 46m51s against timeout-minutes: 60, so it was not a timeout either.
  • Coverage Gate is downstream: it needs backend-test and runs under
    !cancelled(), so it executed and found no shard-8 coverage artifact.

Nothing recorded in the flake ledger: there is no failing test node id to record,
and a lost runner is not a flaky test.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 27, 2026
@tim-finnigan
tim-finnigan force-pushed the ci/release-runner-group branch from 9868dce to a70e41d Compare October 28, 2026 05:50
@tim-finnigan

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in docs/ci/ci-and-reviews.md and
test/test_ci_fleet_routing_expression_parity.py since this head went green
(ab57fba -> 10e53a9); CI re-running.

#24738 touched both files in regions this PR does not: the parity test's import re move and its merge-queue budget test, and the doc's lines 132 and 883. The
rebase was clean and the registry entries and doc section this PR adds are
unchanged. Scoped tests for the overlapping files pass (708), as does the
related backend set (1293).

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Oct 28, 2026
@tim-finnigan

Copy link
Copy Markdown
Contributor Author
  • PASS punchline answered

Confirmed on this head. Each of the nine jobs is workflow_call, a v* tag
push, or workflow_dispatch; none carries a pull_request trigger, so no
pull request puts load on the capped group.
On whether the group will schedule them: GET /orgs/kirodotdev/actions/runner-groups reports group id 3 kirocrew-runners
with restricted_to_workflows: false and selected_workflows: [], so access
is scoped by repository and carries no workflow allowlist.
Rollback is per site and mechanical: restore runs-on: ubuntu-latest and drop
that job's two registry entries.

@tim-finnigan

Copy link
Copy Markdown
Contributor Author
  • PASS punchline answered

Agreed and confirmed. The diff is nine runs-on values, nine pairs of
registry entries, and the doc count the move made wrong; no runtime code, no
user-facing surface and no new mechanism.
Each moved job is pinned per job in _HOSTED_EXCEPTION_RUNNERS rather than
waived, so a later drift to a third runner fails the parity check.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 28, 2026
@tim-finnigan
tim-finnigan force-pushed the ci/release-runner-group branch from 7a85713 to 4f9cd2c Compare October 30, 2026 20:34
@tim-finnigan

Copy link
Copy Markdown
Contributor Author

Rebased onto main and dropped this PR's docs/ci/ci-and-reviews.md hunk, which was the only conflicting file. That hunk bumped the same hand-maintained larger-runner running totals as four sibling routing PRs, which is what made them all conflict with each other; #26303 removes those running totals from the doc and has merged, so there is nothing left here to re-count. The seven workflow routes and the parity-test registrations are unchanged.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Oct 30, 2026
@tim-finnigan
tim-finnigan force-pushed the ci/release-runner-group branch from 4f9cd2c to 10aaa85 Compare October 30, 2026 21:29
@tim-finnigan

tim-finnigan commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: bare group: runner on publish callees a fork's nightly can start

  • Nightly CLI and Docker publish queue forever in a fork on the bare group: form (span=8e1bce626d7b)

fixed in 10aaa85: publish-cli and publish-docker now use the label form github.repository == 'kirodotdev/KiroCrew' && 'kirocrew-linux-4-core' || 'ubuntu-latest', and both _HOSTED_EXCEPTION_RUNNERS entries pin it.
Class swept (a group: site this PR adds that a fork can start). The other five stay on group: because none can start in a fork:
publish-installer has job if: github.repository == 'kirodotdev/KiroCrew'; publish-linux and publish-windows need vars.CLI_DIST_BUCKET and vars.CLI_CDN_BASE, which a fork does not inherit;
release.yml's release-candidate-tests and record-stable-promotion run only on a v* tag and need version, which is already group: on main, so in a fork they never start.
The Notes item is fixed in the same commit: the parity-test comment names publish-installer's path-filtered push to main.

@tim-finnigan

tim-finnigan commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: routing-only diff; the doc inventory edit is tracked in #26425
resolution: deferred to #26425

  • Doc inventory omits the seven new kirocrew-runners sites (span=c2662526f729)

accepted-and-deferred to #26425: the finding is correct and is not being rebutted. docs/ci/ci-and-reviews.md:1090 does require the owning entry and subsection to be edited by a routing change.
It is out of scope HERE because all five open routing PRs must append to the same inventories. Carrying the edit per PR puts the five back in conflict with each other and serializes the wave behind repeated rebases.
#26425 makes that edit once for every landed wave, including this PR's seven sites and release-candidate-tests' matrix capacity. It carries the deferred-finding label, an assignee and a Due date.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 30, 2026
@tim-finnigan

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: routing-only diff; the doc inventory edit is tracked in #26425
resolution: deferred to #26425

  • Doc inventory omits this PR's seven kirocrew-runners sites (span=7818d10f0bb8)

accepted-and-deferred to #26425: the finding is correct and is not being
rebutted. docs/ci/ci-and-reviews.md:1090 does require a routing change to edit
the owning entry and subsection.
It is out of scope HERE because all five open routing PRs append to the same
inventories (the group: list near 1075-1089 and the label-route lists near
1363-1403). Carrying the edit per PR puts the five back in conflict with each
other, which is the state these PRs were just cleaned up to escape.
#26425 makes that edit once for every landed wave. It is open, carries the
deferred-finding label, an assignee and a Due date, and names this PR's sites.

@tim-finnigan

Copy link
Copy Markdown
Contributor Author

/ai-review override fable 10aaa85 deferred #26425: agent: owner-directed deferral. The doc inventory edit for this PR's seven kirocrew-runners sites goes to #26425, which makes it once for all five routing PRs so they stop conflicting on the same lists. The finding is correct and is not rebutted.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@tim-finnigan recorded a deferred release (class deferred) for the fable AI finding on 10aaa85207e2e8c9e04f8c2b78edd5a3a94620b1.

agent: owner-directed deferral. The doc inventory edit for this PR's seven kirocrew-runners sites goes to #26425, which makes it once for all five routing PRs so they stop conflicting on the same lists. The finding is correct and is not rebutted.

This decision applies only to this commit. A new push requires a new judgment.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: checking Automated validation is still running labels Oct 30, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

HOLD at 10aaa85 for a human reviewer, and the branch needs a rebase.

The routing change itself reads as sound: seven runs-on values move to kirocrew-runners, the two fork-startable nightly callees use the label-or-ubuntu-latest form, steps/permissions/triggers are unchanged, and the parity test pins each job. CI is green and the one Opus finding is deferred to #26425, which is open and covers it.

Why it is held:

  1. Prod credential on a different runner pool. Thank you for taking sign-and-notarize.yml out after the last review. Four of the seven moved jobs still declare environment: prod and assume the signing role, as the description says: publish-linux.yml L109-L116, publish-cli.yml L69-L77, publish-windows.yml L101-L104, publish-installer.yml L56-L65. Your argument (OIDC gated by environment:prod, no fork lane declares an environment, ephemeral runners) is reasonable, but choosing which pool runs a prod signing credential is reserved for a maintainer by our review policy, so I am not the one to sign it off.
  2. Merge conflict. GitHub reports the PR as conflicting with main; the conflict is in test/test_ci_fleet_routing_expression_parity.py.

To clear: rebase onto main, then a maintainer approves the runner choice for the four environment: prod jobs (or they are split out like the signing jobs were).

— crew-pr-reviewer (r137)

@tim-finnigan
tim-finnigan force-pushed the ci/release-runner-group branch from 10aaa85 to c369dbb Compare October 31, 2026 00:51
@iamwhatever

Copy link
Copy Markdown
Contributor

HOLD at 9bbad55 for a human reviewer.

Thank you for the rebase; the branch now merges cleanly with main. The routing change still reads as sound: seven runs-on values move to kirocrew-runners, publish-cli and publish-docker keep the fork-safe label form, steps/permissions/triggers are unchanged, and the parity test pins every job. The ci-and-reviews inventory edit is tracked in #26425.

Why it is held:

  1. Prod credential on a different runner pool. Four moved jobs declare environment: prod and assume the signing role (publish-linux.yml L109-L116, publish-cli.yml L69-L77, publish-windows.yml L101-L104, publish-installer.yml L56-L65). Which pool runs a prod signing credential is a maintainer's call under our review policy.
  2. PR Readiness is red on two lanes with no reply on this head.
    • First Principles BLOCK: _GROUP_BY_LABEL_OR_HOSTED spells the same expression as the existing _GROUP_LABEL_EXPR (test L185-L204). Mapping both publish entries to _GROUP_LABEL_EXPR clears it. Its note is also right that the description still says docs/ci/ci-and-reviews.md carries the seven sites, which the head commit defers to docs(ci): list the current wave's larger-runner sites in the ci-and-reviews inventories #26425.
    • GPT 6.1 BLOCK cites docs/decisions/2026-10-31-the-share-file-is-a-crewjam-file.md, which is on main and not in this diff. It looks like a false positive for this PR; a re-run or a maintainer override at this head would clear it.

To clear: a maintainer approves the runner choice for the four environment: prod jobs, and the two lanes above are answered.

— crew-pr-reviewer (r138)

@tim-finnigan

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: routing-only diff; the doc inventory edit is tracked in #26425
resolution: deferred to #26425

  • Doc inventory omits this PR's seven kirocrew-runners sites (span=7818d10f0bb8)

accepted-and-deferred to #26425: the finding is correct and is not being
rebutted. docs/ci/ci-and-reviews.md:1090 does require a routing change to
edit the owning entry and subsection.
It is out of scope HERE because the open routing PRs all append to the same
two inventories (the group: list and the label-route lists), so carrying
the edit per PR puts them back in conflict with each other, which is the
state this wave was cleaned up to escape.
#26425 makes that edit once for every landed wave. Verified on this head:
it is OPEN, carries the deferred-finding label, assignee tim-finnigan
and Due: 2026-11-07, and its body names this PR's sites explicitly.
Re-stamped for head 6831d1a because the ruling above was recorded against
10aaa85; the diff's doc-facing shape is unchanged between the two.

…group

Seven runs-on values move off the shared standard pool: release-candidate-tests
and record-stable-promotion in release.yml, and the publish-cli, publish-docker,
publish-installer, publish-linux and publish-windows jobs. A release path is
human-blocking and low-volume, so carving it out of a pool nobody here can
expedite is cheap.

publish-cli and publish-docker, which a fork's nightly can start, reach the
group by its runner label and fall back to ubuntu-latest in a fork; the other
five use `group:` directly, since no fork can start them.

test_ci_fleet_routing_expression_parity.py registers all seven in
_PERMANENT_EXCEPTIONS and in _HOSTED_EXCEPTION_RUNNERS, so a lane that later
drifts to a third runner still fails the check. The two label-form entries reuse
the registry's existing _GROUP_LABEL_EXPR constant rather than a second constant
spelling the same expression.

No documentation changes: every PR in this routing wave would append to the same
inventories in docs/ci/ci-and-reviews.md and conflict with the others, so #26425
is the single pass that updates them once the waves have landed.

Closes #24960
@tim-finnigan

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in .github/workflows/publish-docker.yml, test/test_ci_fleet_routing_expression_parity.py since this head went green (048bbc1 -> e666f7a); CI re-running.

@tim-finnigan

Copy link
Copy Markdown
Contributor Author

/ai-review override fable 3983a62 deferred #26425: accepted-and-deferred, re-stamped for head 3983a62 after tonight's rebase. The finding is correct and is not rebutted: docs/ci/ci-and-reviews.md:1090 does require a routing change to edit the owning entry and subsection. It is out of scope here because the open routing PRs all append to the same two inventories, so carrying the edit per PR puts them back in conflict. #26425 makes it once for the whole wave; verified open, labelled deferred-finding, assigned, Due: 2026-11-07.

@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Human judgment recorded

@tim-finnigan recorded a deferred release (class deferred) for the fable AI finding on 3983a62bdd938305e1a79a6a374af2a2f8e8ef21.

accepted-and-deferred, re-stamped for head 3983a62 after tonight's rebase. The finding is correct and is not rebutted: docs/ci/ci-and-reviews.md:1090 does require a routing change to edit the owning entry and subsection. It is out of scope here because the open routing PRs all append to the same two inventories, so carrying the edit per PR puts them back in conflict. #26425 makes it once for the whole wave; verified open, labelled deferred-finding, assigned, Due: 2026-11-07.

This decision applies only to this commit. A new push requires a new judgment.

@iamwhatever

Copy link
Copy Markdown
Contributor

HOLD at 3983a62 for a human reviewer.

The items from r138 are cleared: publish-cli and publish-docker now use the existing _GROUP_LABEL_EXPR (test L209-L212), GPT 6.1 has no findings, the description points the doc inventory at #26425, and all 94 checks are success or skipped.

What still holds it: four moved jobs declare environment: prod and assume the signing role (publish-linux.yml L109-L116, publish-cli.yml L69-L77, publish-windows.yml L101-L104, publish-installer.yml L56-L65). Your OIDC and ephemeral-runner argument is reasonable, but which runner pool may execute a prod signing credential is reserved for a maintainer under our review policy. This clears when a maintainer approves that runner choice, or the four jobs are split out.

AI findings judged non-blocking: Design Review's fork edge case for publish-linux/publish-windows affects only a fork that sets the CDN variables, and its live-check suggestion is post-merge verification advice.

— crew-pr-reviewer (r140)

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved on behalf of the watch holder (bolichen). Lanes pass on 3983a62, no conflict, no open change request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: route the release and publish jobs to the kirocrew-runners group

3 participants