Skip to content

perf: batch the dev fleet prune preview's pr lookups - #31165

Merged
bolichen97 merged 1 commit into
mainfrom
fix/11529-batch-prune-pr-lookups
Nov 6, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/11529-batch-prune-pr-lookups

Conversation

@bolichen97

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The Dev Fleet prune preview asks GitHub a fixed few times per 50 worktrees instead of once or twice for every worktree.

The prune preview (prune-candidates) looked up each worktree's pull request on its own: a gh pr list status lookup on a cache miss, then a fresh gh pr list head-OID check for every merged or closed head. #11450 made those calls concurrent, but a fleet of 100+ worktrees still costs 100+ GitHub round trips per preview.

Why it matters

Every preview, and every auto-prune pass, spends GitHub rate limit and time in proportion to the fleet, on exactly the large fleets the preview exists to clean up, inside a 30 s proxy budget.

Not a goal

  • Changing any prune verdict, or what auto-prune removes.
  • Changing the removal path: its head-OID re-check stays a fresh per-head query.
  • Batching the fleet view's background PR-status refresh.
  • A repo-wide gh pr list index: measured on the issue, it is always truncated on a large repository.

What changed (motivation → approach → change)

Each prune verdict made its own GitHub calls, so the preview's cost grew with the fleet. A capped repo-wide PR list cannot replace them: on a repository with thousands of PRs the page is always full, and a truncated index can mis-rule a reused branch name.

So the batch is keyed on the fleet's own heads. _prune_candidates (dev_fleet/worktree_ops.py) first calls the new _fetch_pr_rows beside it: one gh api graphql request per 50 heads, each head an aliased pullRequests(headRefName:) connection with the same states, newest-first order and saturation-sentinel row cap that gh pr list --head <b> --state all queries. Branch names travel as GraphQL variables, never inside the query text.

Each verdict hands its head's rows to the lookups it already makes in dev_fleet/fleet_state.py. _pr_status_cached and _fetch_pr_status read the newest row instead of querying the upstream repo; _fetch_pr_head_oid applies its unchanged saturation, OPEN-refusal and MERGED rules to the rows. A head whose batch request failed, and a PR found only in a legacy repo, still take the per-head query, and no removal path passes rows. The dev-fleet spec's scan paragraph describes the batch.

Backwards compatibility

Compatible: the preview's response and every verdict code are unchanged, and each new parameter defaults to the per-head behaviour every other caller keeps.
Removes nothing: no route, field, verdict or check goes away; the per-head query stays for every caller that passes no rows.

Tests

  • test_prune_candidates_batches_pr_lookups_and_keeps_per_head_verdicts (new): five worktrees (merged, closed, open, a reused head with a newer MERGED over an older OPEN PR, and no PR) with the batch size set to 2 cost exactly three gh api graphql calls and no gh pr list, and every verdict equals the one the per-head gh pr list path gives for the same PRs.
  • The existing _prune_candidates fakes and the full-head-OID test take the new rows argument, and the five candidate-split tests stub the batch so nothing reaches gh; the TTL test's status-fetch fake in test_dev_fleet_git_facts_cache.py takes it too.
  • Determinism: the new test passed 20 runs at -n0 and 10 at -n 4 --dist loadgroup --max-worker-restart=2 beside test_dev_fleet_server_coverage.py and test_dev_fleet_git_facts_cache.py, and both changed files' 50 tests passed in a shuffled order (seed 8829). With the production change reverted it fails: the preview makes per-head gh pr list calls instead of three gh api graphql calls.

Manual verification

Ran the batch query against this repository with gh api graphql for three heads (main, a merged PR's branch, a branch with no PR) and compared each head's rows with gh pr list --head <b> --state all: the same PRs in the same newest-first order, and no rows for the missing head. gh api graphql exits 1 on a GraphQL error, which the batch treats as a failed request.

Screenshots / video

N/A — backend only, no user-visible change.

Related Issues

Closes #11529

Sibling sweep

N/A — a perf change, not a fix or revert.

Pattern harvest

N/A — a perf change, not a fix or revert.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

The prune preview asked GitHub once or twice per worktree: a PR-status
lookup on a cache miss, then a fresh head-OID check for every merged or
closed head. Its network cost grew with the fleet.

The preview now fetches every head's upstream PRs up front, one
`gh api graphql` request per 50 heads, built from the same
pullRequests(headRefName:) connection `gh pr list --head` queries.
Each verdict rules on its head's rows exactly as on its own query.
A failed batch and a PR found only in a legacy repo still take the
per-head path, and the removal re-check never reads the batch.

Closes #11529
@bolichen97
bolichen97 requested review from a team and KiroCrewCaptain as code owners November 6, 2026 07:01
@bolichen97
bolichen97 requested a review from Zedmor November 6, 2026 07:01
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 6, 2026
@bolichen97
bolichen97 enabled auto-merge (squash) November 6, 2026 07:13
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 4e0497a81aa947f0b2d0d8dffad09deea71a1101 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 4e0497a81aa947f0b2d0d8dffad09deea71a1101 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the batch only replaces the upstream lookups in the preview. Legacy-repo lookups, heads whose batch failed, and the removal re-check all still query GitHub per head, so the destructive path's freshness holds.

Notes

  • _fetch_pr_rows sends its chunks one after another, each with timeout=15. With three or more chunks (101+ heads), a slow GitHub could run the batch alone past the 30 s _PROXY_TIMEOUT before any verdict starts; running the chunks concurrently under _PRUNE_CONCURRENCY would fix that.

[DESIGN-REVIEWED] 4e0497a

Description read

[DESCRIPTION-READ] 66f8e35a93f6756ddc6df95247fdf58ef54b18599100573e8a3b44fc361878cd

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 4e0497a81aa947f0b2d0d8dffad09deea71a1101 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing to check.

What this change ships

Inventory (6 items) — 6 justified

Intent: let operators of large Dev Fleets preview a prune without spending one or two GitHub round trips per worktree; an ADDITION (perf), tied to #11529.

  1. The prune preview now asks GitHub once per 50 heads, through one batched gh api graphql call, instead of one or two gh pr list calls per worktree. — justified
  2. During the preview, a PR-status cache miss reads the head's batched rows instead of querying upstream. — justified
  3. During the preview, the merged/closed head-OID check rules on the batched rows; the removal re-check stays a fresh per-head query. — justified
  4. A head whose batch request failed, and a PR found only in a legacy repo, still get the per-head lookup. — justified
  5. The PR-payload shaping moved into a shared _pr_payload helper that builds on a copy. — justified
  6. The dev-fleet spec's scan-feedback paragraph now describes the batch. — justified

[FIRST-PRINCIPLES-REVIEWED] 4e0497a

Description read

[DESCRIPTION-READ] 1b7863ed263166b795bc732e3db06fd258b5b5457f157bd8adfef1fe1db35806

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 4e0497a81aa947f0b2d0d8dffad09deea71a1101 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 4e0497a

Verdict parsed from the review's SHA-scoped output markers for commit 4e0497a81aa947f0b2d0d8dffad09deea71a1101.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 4e0497a81aa947f0b2d0d8dffad09deea71a1101 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 4e0497a81aa947f0b2d0d8dffad09deea71a1101 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 4e0497a

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 4e0497a81aa947f0b2d0d8dffad09deea71a1101 false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention labels Nov 6, 2026

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Kiro Crew Auto-Pipeline [operator: buluoray]:

APPROVE. No behaviour defect found on 4e0497a.

What it does for users: the Dev Fleet prune preview now asks GitHub once per 50 worktrees instead of once or twice per worktree. A large fleet's preview spends far less rate limit and has more room inside the 30 s proxy budget. Every verdict code and the response shape stay the same.

Same per-PR result as the old per-PR calls:

  • The batch connection asks for the same fields _pr_query_one requests (number, state, url, isDraft, title, body, headRefOid). It uses the same states, newest-first CREATED_AT order, and a row cap of _PR_HEAD_LOOKUP_LIMIT + 1 (worktree_ops.py:255). So rows[0] is the row gh pr list --limit 1 returned (fleet_state.py:157), and the cached payload is built by the same _pr_payload on a copy.
  • _fetch_pr_head_oid applies its unchanged saturation, OPEN-refusal and MERGED rules to the batch rows (fleet_state.py:235, 269). Because the cap is 11 rows, a saturated head still fails closed.
  • The rows are only used when the lookup repo is the upstream repo (fleet_state.py:235). A PR found in a legacy repo still gets its own query.
  • No removal path passes rows. The per-target re-verdict in the prune workers (worktree_ops.py:446) and every _fetch_pr_head_oid call in worktree_ops_removal.py still query fresh, which matches the spec paragraph.
  • New test test_prune_candidates_batches_pr_lookups_and_keeps_per_head_verdicts compares batched and per-head verdicts on five heads. One of them is a reused head with a newer MERGED PR over an older OPEN one.
  • Mutation: reading rows[-1] instead of rows[0] at fleet_state.py:157 reddens that test.

Partial failure and pagination:

  • A chunk whose gh api graphql exits non-zero is skipped (worktree_ops.py:293). So is a chunk whose JSON is missing repository or any alias, or where either is null (worktree_ops.py:298).
  • The found.update dict is built in full before it is applied. A malformed chunk therefore contributes no heads at all, and each of its heads falls back to its own per-head query.
  • Pagination: 50 heads per request, chunked in order. The test pins three calls for five heads at batch size 2.

Tests: 50 passed across test_dev_fleet_server_coverage_prune.py and test_dev_fleet_git_facts_cache.py. CI shows 77 success and 16 skipped, PR Readiness passes, and the head merges cleanly into main. Every bot lane passes; the UX lane skipped because no UI changed.

Source-only:

  • That gh pr list --head <b> --state all queries the same pullRequests(headRefName:) connection. I took this from the author's manual comparison and from gh's documented list path, and did not run it against the live API.
  • That gh api graphql exits non-zero on a GraphQL errors payload. If it exited 0, the missing or null alias would still drop the chunk through the KeyError/TypeError branch.

Follow-ups (non-blocking):

  • Coverage gap: the failed-batch fallback is not pinned. Turning a failed chunk into "no PRs for these heads" at worktree_ops.py:293 leaves all 20 prune tests green. In production that would rule every head in the chunk fresh instead of querying it alone. A test with one failing chunk, asserting per-head gh pr list calls for exactly that chunk's heads, would pin it.
  • The chunks run one after another, each with a 15 s timeout. A fleet of 101 or more heads against a hanging GitHub could spend 45 s in the batch before falling back. Running the chunks under the existing _PRUNE_CONCURRENCY bound would keep the worst case at one timeout.

@bolichen97
bolichen97 merged commit e4ab9d2 into main Nov 6, 2026
97 of 102 checks passed
@bolichen97
bolichen97 deleted the fix/11529-batch-prune-pr-lookups branch November 6, 2026 17:34
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dev-fleet: batch one repo-wide gh pr list for prune preview instead of O(N) per-worktree calls

2 participants