Skip to content

fix(test): census the backend-test canary as a resolver consumer - #11889

Closed
helenastafford wants to merge 1 commit into
kirodotdev:mainfrom
helenastafford:fix/ci-census-backend-test
Closed

helenastafford wants to merge 1 commit into
kirodotdev:mainfrom
helenastafford:fix/ci-census-backend-test

Conversation

@helenastafford

@helenastafford helenastafford commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Main's own CI is red. test_every_ci_yml_resolver_consumer_reads_the_resolver_not_a_literal in test/test_ci_fleet_routing_expression_parity.py fails on every run. ci.yml's backend-test job reads needs.changes.outputs.linux_runner_large (the shard-1 CodeBuild canary, conditional on matrix.group), but the job is not in _EXPECTED_RESOLVER_CONSUMER_JOBS. The census sees a new resolver read and fails.

Why it matters

Every open PR inherits the red through its merge ref. One backend shard fails for all of them, no matter what they change.

What changed (motivation → approach → change)

The census locks every resolver read to its exact expression text. The canary read is real and correct, so the fix lists it instead of loosening the test.

One entry is added to _EXPECTED_RESOLVER_CONSUMER_JOBS in test/test_ci_fleet_routing_expression_parity.py: ("ci.yml", "backend-test") with the canary's exact conditional expression. The byte-form of the canary read is locked the same way the canonical reads are. Silent drift in the conditional still trips the test.

Backwards compatibility

Compatible: test-only change. No runtime code or workflow file is touched.

Tests

  • test/test_ci_fleet_routing_expression_parity.py — 3/3 pass with the fix. Without it, the census test fails on pristine main.

Manual verification

N/A — unit coverage sufficient; the change is one entry in a test's expected set.

Related Issues

no linked issue: main breakage found by CI, no issue was filed.

Pattern harvest

Rule candidate: review-prompt — when a workflow-census test locks jobs by exact expression text, a PR that adds or changes a runs-on: resolver read must update the census in the same commit.
Pattern: a CI-routing change adds a new consumer of a censused output without updating the census's expected set.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@helenastafford
helenastafford requested review from a team and dwu96 September 18, 2026 20:33
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 18, 2026
PR kirodotdev#11699 made ci.yml's backend-test job read
needs.changes.outputs.linux_runner_large (shard-1 CodeBuild canary,
conditional on matrix.group) without adding the job to
_EXPECTED_RESOLVER_CONSUMER_JOBS, so
test_every_ci_yml_resolver_consumer_reads_the_resolver_not_a_literal
fails on main tip and reds Backend Tests shard 2 for every open PR.

Add the job with its exact conditional expression so the census locks
the canary's byte-form the same way it locks the canonical reads.
@iamwhatever
iamwhatever force-pushed the fix/ci-census-backend-test branch from f14c2a9 to f73f8b1 Compare September 18, 2026 21:57
@iamwhatever

Copy link
Copy Markdown
Contributor

Closing as a duplicate of #11876, which carries the same one-line census fix and is hosted on a same-repo branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants