Repository navigation
test(ci): add backend-test canary runs-on to resolver-consumer census - #11876
Conversation
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Confirm the canary [DESIGN-REVIEWED] fbc0cb0 |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
|
@helenastafford -- thanks for the quick allowlist heal. Main has been red on @bolichen97 -- flagging as #11699's owner in case you would rather land the |
…census #11699 routed backend-test shard 1 to the CodeBuild fleet via a canary-conditional runs-on that reads needs.changes.outputs.linux_runner_large, but did not add the job to _EXPECTED_RESOLVER_CONSUMER_JOBS in test_ci_fleet_routing_expression_parity.py. The census test's added-consumer assertion now fails on main for every PR (and main itself). Add the entry with its exact canary expression so the byte-match check also pins the canary form against silent drift.
65fe8db to
fbc0cb0
Compare
…census (kirodotdev#11876) kirodotdev#11699 routed backend-test shard 1 to the CodeBuild fleet via a canary-conditional runs-on that reads needs.changes.outputs.linux_runner_large, but did not add the job to _EXPECTED_RESOLVER_CONSUMER_JOBS in test_ci_fleet_routing_expression_parity.py. The census test's added-consumer assertion now fails on main for every PR (and main itself). Add the entry with its exact canary expression so the byte-match check also pins the canary form against silent drift. Co-authored-by: Helena Stafford <helenars@amazon.com>
Problem / Motivation
Main's own CI is red.
test/test_ci_fleet_routing_expression_parity.pyfails on every run. The census test finds a newruns-on:that readsneeds.changes.outputs.linux_runner_largeinci.yml'sbackend-testjob (the shard-1 CodeBuild canary), but that job is not in_EXPECTED_RESOLVER_CONSUMER_JOBS.Why it matters
Every open PR inherits the red through its merge ref. One backend shard fails for all of them, no matter what they change. Nobody can tell a real failure from this one.
What changed (motivation → approach → change)
The census test locks every resolver read to its exact expression text. The canary read is real and correct, so the fix is to list it, not to loosen the test.
One entry is added to
_EXPECTED_RESOLVER_CONSUMER_JOBSintest/test_ci_fleet_routing_expression_parity.py:("ci.yml", "backend-test"), bound to the canary's exact conditional expression as a named constant_CANARY_SHARD1_CONSUMER_EXPR. The byte-match branch of the test now pins the canary form too. A hardcoded fleet label or a dropped resolver read in that job still fails the test.Backwards compatibility
Compatible: test-only change. No runtime code or workflow file is touched.
Tests
test/test_ci_fleet_routing_expression_parity.py— 3/3 pass with this change. Without it,test_every_ci_yml_resolver_consumer_reads_the_resolver_not_a_literalfails on main tip.Manual verification
N/A — unit coverage sufficient; the change is one entry in a test's expected set.
Related Issues
Fixes #11883
Pattern harvest
Rule candidate: review-prompt — a PR that adds or changes a
runs-on:reading the fleet resolver must update_EXPECTED_RESOLVER_CONSUMER_JOBSin the same commit.Pattern: a guard test owned by one file and the behavior it guards owned by another, merged separately, each green alone and red together.
Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)