Repository navigation
ci: route the seven release and publish jobs to the kirocrew-runners group - #24962
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks: each job move is the existing runner-group mechanism, reused with no new surface, and it is aimed at the queueing cause #24960 names. What this change shipsInventory (8 items) — 8 justifiedIntent: let a tagged release start right away instead of queueing behind other organisations' jobs in the shared pool. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] 3983a62 Description read[DESCRIPTION-READ] 565fa963dbccb8375fede83e7341cf62d44a12f1ef869573f9001970dfc19cb6 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: each job only moves to another runner, and reverting one is a one-line change. Notes
[DESIGN-REVIEWED] 3983a62 Description read[DESCRIPTION-READ] 166861456cbc2aee9363a5dffcb49bed68ccf18cf342e3d1714eb9b9d5b17ff8 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ human override acceptedReviewed Human judgment by @tim-finnigan overrides the Opus 5.5 finding for [OPUS-OVERRIDE] 3983a62 Verdict recorded from an authorized human decision for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
218ae4c to
9868dce
Compare
|
self-added: no
|
|
|
|
self-added: no
|
|
|
Evidence it was the runner:
Nothing recorded in the flake ledger: there is no failing test node id to record, |
9868dce to
a70e41d
Compare
|
Rebased: main moved in #24738 touched both files in regions this PR does not: the parity test's |
|
|
7a85713 to
4f9cd2c
Compare
|
Rebased onto |
4f9cd2c to
10aaa85
Compare
|
self-added: yes
|
|
self-added: no
|
|
self-added: no
|
Human judgment recorded@tim-finnigan recorded a deferred release (class
This decision applies only to this commit. A new push requires a new judgment. |
|
HOLD at 10aaa85 for a human reviewer, and the branch needs a rebase. The routing change itself reads as sound: seven Why it is held:
To clear: rebase onto — crew-pr-reviewer (r137) |
10aaa85 to
c369dbb
Compare
|
HOLD at 9bbad55 for a human reviewer. Thank you for the rebase; the branch now merges cleanly with Why it is held:
To clear: a maintainer approves the runner choice for the four — crew-pr-reviewer (r138) |
|
self-added: no
|
…group Seven runs-on values move off the shared standard pool: release-candidate-tests and record-stable-promotion in release.yml, and the publish-cli, publish-docker, publish-installer, publish-linux and publish-windows jobs. A release path is human-blocking and low-volume, so carving it out of a pool nobody here can expedite is cheap. publish-cli and publish-docker, which a fork's nightly can start, reach the group by its runner label and fall back to ubuntu-latest in a fork; the other five use `group:` directly, since no fork can start them. test_ci_fleet_routing_expression_parity.py registers all seven in _PERMANENT_EXCEPTIONS and in _HOSTED_EXCEPTION_RUNNERS, so a lane that later drifts to a third runner still fails the check. The two label-form entries reuse the registry's existing _GROUP_LABEL_EXPR constant rather than a second constant spelling the same expression. No documentation changes: every PR in this routing wave would append to the same inventories in docs/ci/ci-and-reviews.md and conflict with the others, so #26425 is the single pass that updates them once the waves have landed. Closes #24960
|
/ai-review override fable 3983a62 deferred #26425: accepted-and-deferred, re-stamped for head 3983a62 after tonight's rebase. The finding is correct and is not rebutted: docs/ci/ci-and-reviews.md:1090 does require a routing change to edit the owning entry and subsection. It is out of scope here because the open routing PRs all append to the same two inventories, so carrying the edit per PR puts them back in conflict. #26425 makes it once for the whole wave; verified open, labelled deferred-finding, assigned, Due: 2026-11-07. |
Human judgment recorded@tim-finnigan recorded a deferred release (class
This decision applies only to this commit. A new push requires a new judgment. |
|
HOLD at 3983a62 for a human reviewer. The items from r138 are cleared: What still holds it: four moved jobs declare AI findings judged non-blocking: Design Review's fork edge case for — crew-pr-reviewer (r140) |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Approved on behalf of the watch holder (bolichen). Lanes pass on 3983a62, no conflict, no open change request.
Problem / Motivation
Goal: A release starts when we tag it, instead of waiting behind another organisation's CI.
Every plain
ubuntu-latestjob across every Amazon GitHub org draws on one sharedpool, first-come first-served, with no way to prioritise a job. The seven release
and publish jobs sit in that pool today, alongside this repository's own
high-volume PR CI. When the pool fills, a release waits, and nobody here can
expedite it.
Why it matters
A blocked release is blocked on something outside our control. During the
enterprise-wide queueing incident in August, two public AWS repositories could not
ship releases for two days. Our own capacity guidance names this case directly:
release paths should not share a contended pool with high-volume PR CI, because
they are human-blocking and low-volume, which is what makes carving them out cheap.
Not a goal
are negligible; this is a latency change.
sign-and-notarize.yml'ssignandpublish. Not for a safetyreason: four of the jobs this PR moves hold that same signing role, and the
runner pool is not what admits a job to it. Those two stay on
ubuntu-latestto keep this change to the release and publish path, and moving them is the
same one-line change in its own pull request.
windows-latestjobs inbuild-windows.yml. The group is aGitHub-hosted Ubuntu VM and no Windows larger-runner group exists.
nothing to move them to.
untouched.
What changed (motivation → approach → change)
A release that cannot get a runner is stuck, and the fix is to stop competing for
one. Dedicated capacity already exists: the
kirocrew-runnerslarger-runner groupdraws on a ceiling separate from the shared pool, and the AI review lanes, CodeQL
and Docker Smoke already run there.
The group was the right target rather than the CodeBuild fleet. The fleet admits
only an allowlisted actor and the tagger rotates with the on-call, so a fleet route
would fall back to the shared pool for most releases. Volume made the group safe:
every job here is
workflow_call, av*tag push,workflow_dispatch, or (forpublish-installer) a path-filtered push tomain, so none fires on a pull requestand none adds PR volume to a group capped at 100 parallel.
Seven jobs move from
runs-on: ubuntu-latestto the group:publish-cli.ymlpublish-cliubuntu-latestin a forkpublish-docker.ymlpublish-dockerubuntu-latestin a forkpublish-installer.ymlpublish-installergroup:publish-linux.ymlpublish-linuxgroup:publish-windows.ymlpublish-windowsgroup:release.ymlrelease-candidate-tests,record-stable-promotiongroup:A fork cannot reach an organization runner group, so a
group:job a fork startsqueues until GitHub cancels it.
nightly.ymlcallspublish-cliandpublish-dockerwith no repository guard, so the fork-reachability fix replaced thebare
group:form on those two fork-startable publish lanes with the conditionallabel form
build-wheelalready uses. The other five cannot start in a fork: arepository
if, the CDN repository variables a fork lacks, or a dependency onrelease.yml'sversionjob, which is already on the group.Four of those seven --
publish-cli,publish-installer,publish-linuxandpublish-windows-- declareenvironment: prodand assume the signing role, andthat is safe on this group. The role's OIDC trust accepts exactly two subjects,
ref:refs/heads/mainandenvironment:prod, so the runner pool is not whatadmits a job to the credential. The group runs the fork review lanes against
fork-authored content as well, and nothing is shared across that boundary: the
environment gates the credential rather than the runner, no fork lane declares an
environment, and the group's runners are ephemeral per job.
The two
sign-and-notarize.ymljobs are still not among them, but as a scopechoice rather than a safety one: this change keeps to the release and publish
path.
notarizecould not follow in any case, since it runs onmacos-15and nomacOS larger-runner group exists.
test_ci_fleet_routing_expression_parity.pypins every off-fleet job to itsexpected runner, so each one is registered in
_PERMANENT_EXCEPTIONSand in_HOSTED_EXCEPTION_RUNNERS. Pinned per job rather than waived: a lane that laterdrifts to some third runner still fails the check.
docs/ci/ci-and-reviews.mdowns the list of sites on the group, but this changedoes not add the seven to it: that inventory edit is deferred to #26425,
which is open and covers it. This PR touches no documentation. The two workflow
comments on
publish-cliandpublish-dockerpoint at the doc's existing"Larger runners for the review lanes and CodeQL" section, which is already on
main.Backwards compatibility
Compatible: only the machine a job runs on changes. Steps, timeouts, permissions,
triggers and job names are identical, and a job name is what branch protection and
the merge queue key on.
Removes nothing: no path, record, key or function is hidden, deleted or tightened.
Tests
test/test_ci_fleet_routing_expression_parity.pygains seven entries in each ofits two registries, with
publish-cliandpublish-dockerpinned to the labelform through the registry's existing
_GROUP_LABEL_EXPRconstant rather than asecond constant spelling the same expression. The test locks in that every job deliberately kept off the
CodeBuild fleet runs on the runner this repository expects, so a silent drift to a
third runner fails CI rather than shipping, and the absence of a
sign-and-notarize.ymlentry means those two jobs are still pinned to the defaultubuntu-latest.scripts/local-gate.py's backend leg ran 1,298 related tests green, and theparity and additional-fleet-route suites ran 871 tests green. The frontend leg
did not run (npm
E401on this host); nowebsite/file changes.Manual verification
N/A — unit coverage sufficient: the parity test reads the same workflow files
GitHub does, and no release can be tagged from a branch to exercise the path
before merge. The first prerelease tag after merge is the live check; its jobs
should report a
kirocrew-runnersrunner.Related Issues
Closes #24960
Checklist