Repository navigation
Code review can read CI before Codecov reports, so require_ci_green misses coverage #162
Description
What happens
The code review bot starts when Tests or Lint finishes, and require_ci_green reads every check run on the head commit at that moment. Codecov posts codecov/patch and codecov/project only after every other check has finished, 35 to 50 seconds after Tests, and nothing starts a review when they finish. Measured on #159 in MarketData-App/code-review-bot#25, as the time from the review reading CI to Codecov completing:
| head | run | margin | outcome |
|---|---|---|---|
| 7759248 | 36462367405 | -24 s | reviewed, Codecov not yet reported |
| 9f65a39 | 36445268644 | -15 s | reviewed, Codecov not yet reported |
| b8035b2 | 36436007907 | -3 s | reviewed, Codecov not yet reported |
| bbeea3c | 36452710753 | +2 s | reviewed |
| 7f929dc | 36460028984 | +8 s | reviewed |
| 21efa02 | 36465597062 | -5 s | skipped: CI has not finished on the head commit |
So a review that reads CI before Codecov reports does not see coverage, and a red codecov/project does not stop it, although main is at 100%. When the read lands while Codecov is running, the review skips and nothing brings it back: 21efa02 was reviewed only after a manual @marketdata-code-review re-review.
Why
.github/code-review/policy.yml sets require_ci_green: true and lists no check to wait for. MarketData-App/code-review-bot#25 (merged 2026-10-01) added wait_for_checks: the review waits up to wait_for_checks_seconds (300 by default) until every listed check run exists and nothing on the head is still running, and stops at once when CI is already red. A repository that does not set the key behaves as before.
Codecov reports here as check runs, which is what the wait reads: on da96b09, the head of #157, codecov/patch and codecov/project are check runs of the codecov app, and the commit has no statuses.
Proposal
Add wait_for_checks: ["codecov/patch", "codecov/project"] to .github/code-review/policy.yml, next to require_ci_green. The bot reads the policy from the base branch, so the wait applies to pull requests once it is on main.
Cost: a documentation-only pull request skips the suite, so no coverage is uploaded and the two checks never appear. Its review waits the 300 s cap on the runner (not on the shared lease) and then skips, since ignore_paths covers every file it changes.
Activity