Skip to content

feat: wait for Codecov by default, up to two minutes - #26

Merged
MarketDataDev02 merged 1 commit into
mainfrom
feat/wait-for-codecov-by-default
Oct 2, 2026
Merged

MarketDataDev02 merged 1 commit into
mainfrom
feat/wait-for-codecov-by-default

Conversation

@MarketDataDev02

Copy link
Copy Markdown
Collaborator

Why

After #25 each SDK had to list codecov/patch and codecov/project in its own policy, as sdk-py #163 did. Codecov reports on sdk-py, sdk-csharp, sdk-php, sdk-go and sdk-java, so that line repeated one decision five times. It belongs in the bot.

The change

  • reviewbot/defaults/policy.yml: wait_for_checks: ["codecov/patch", "codecov/project"] and wait_for_checks_seconds: 120, where they were [] and 300.
  • reviewbot wait-for-checks does not wait when every changed file matches ignore_paths, since the review skips that pull request anyway. It reads the changed files once to tell.
  • docs/setup.md: the default; wait_for_checks: [] to turn it off; and that a failing check, listed or not, skips the review, so informational: true in a repository's Codecov configuration keeps coverage from holding a review back.

Why the bot does not detect Codecov per repository

Checked on the six SDKs:

  • On the default branch Codecov posts commit statuses (sdk-py) or nothing, because CI does not run there (sdk-csharp). The App has no statuses permission, so it cannot read them.
  • GitHub creates a codecov check suite on every head, sdk-js included, and it stays queued where Codecov never reports.
  • Codecov check runs on recent merged pull request heads: sdk-py 4 of 6, sdk-csharp 3 of 6, sdk-php 3 of 4, sdk-go 5 of 6, sdk-java 2 of 5, sdk-js 0 of 6. Most without are dependabot's, which the gate refuses before the wait, and documentation-only changes.

So where Codecov does not report, a review waits the cap, which is why the default is two minutes rather than five. Codecov reported 35 to 50 seconds after Tests on sdk-py, well inside it.

Rollout

  • sdk-js adds wait_for_checks: [] to its policy. The key exists since feat: wait for the checks another app reports after CI, before reading it #25, so this can land before or after this pull request.
  • sdk-py's explicit list from #163 now equals the default; it can stay.
  • _lease_minutes already reserves the wait in the job budget, now two minutes for every repository that waits. That shortens a lease wait only from timeout_minutes 18 up.
  • On every recent pull request in the five repositories, codecov/patch and codecov/project passed, so the default skips no review today.

Tests

  • tests/test_wait_for_checks.py: Codecov is waited for by default; the default gives up after two minutes; a pull request whose files all match ignore_paths neither waits nor pins its head; one file outside them keeps the wait; the real client reads the changed files; an explicit wait_for_checks: [] reads no check run.
  • tests/test_config.py: the new defaults, and an empty list turning the wait off.

uv run pytest: 760 passed on Windows. The other 54 fail on an unmodified main on Windows too, because their fake claude and codex executables are POSIX scripts; ubuntu CI runs them. ruff check . and ruff format --check . are clean.

Codecov reports on five of the six SDKs, so listing its checks in each
repository's policy repeated one decision per repository. Its checks are now
the default, with a two-minute cap instead of five, and the wait is skipped
for a pull request whose every changed file matches ignore_paths, which the
review skips anyway. A repository whose CI does not report to Codecov, sdk-js
today, sets `wait_for_checks: []`.

The bot cannot tell ahead of time whether Codecov will report on a head: on
the default branch it posts commit statuses, which the App cannot read, and
GitHub creates a queued Codecov check suite on every head whether or not it
ever reports. So where it does not report, a review waits the cap.
@marketdata-code-review

Copy link
Copy Markdown

✅ Ready — Code review

This changes the default policy to wait up to 120 seconds for the two Codecov checks and skips that wait when all changed paths are ignored. It updates the setup documentation and extends coverage for defaults, opt-out, timeout, path filtering, and real-client requests.

Rating patch 5/6 (solid) · proof 5/6 (reproducible) · overall 5/6


Reviewed 289e8f7 · revision 1 · codex (gpt-5.6-sol) · claude unavailable · rules: default

@MarketDataDev02
MarketDataDev02 merged commit 7326625 into main Oct 2, 2026
2 checks passed
@MarketDataDev02
MarketDataDev02 deleted the feat/wait-for-codecov-by-default branch October 2, 2026 18:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant