Skip to content

fix(github): cap listing pagination per call - #8219

Closed
iscekic wants to merge 5 commits into
mainfrom
drain/github-pagination-bounds
Closed

iscekic wants to merge 5 commits into
mainfrom
drain/github-pagination-bounds

Conversation

@iscekic

@iscekic iscekic commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Change

Branch, repository, maintenance repository, and inline review-comment reads in the GitHub adapter share one bounded page collector. Each call reads at most the page count its listing configures (10 for branches, 50 for repositories, 5 for inline review comments) and returns the entries it read; when the provider still advertises a following page at that point, it logs a warning. Return types are unchanged, so existing callers keep working.

Review-thread enrichment keeps a per-thread page limit of 5. The request's 500-comment follow-up allowance is split evenly across the page's threads before any request runs, each request asks only for the comments its allowance keeps, and at most 5 requests are in flight, so identical responses produce identical results. Each thread reports commentsTruncated so a client can present a long thread as partial.

Tests

  • adapter.pagination.test.ts: with provider pages of 100 entries, a branch listing makes 10 page requests, a repository listing makes 50, and an inline review-comment listing makes 5; a short page ends a listing early; the warning appears only when a full page still advertises a following page; archived repositories stay filtered; the inline review-comment mapping keeps line, author, and outdated state.
  • review-thread-comments.test.ts: the per-thread page limit, a request that asks only for the comments its allowance keeps, the 5-request concurrency limit, the even 500-comment split repeating identically across runs, and the truncated result for complete and partial threads.
  • github-pr-review-router.test.ts: one listReviewThreads call over 50 threads returns 500 follow-up comments — 10 per thread — after one request per thread, each marked commentsTruncated.

Verification

  • pnpm exec jest over src/lib/integrations, src/lib/github-pr-review, src/lib/bot/platforms/github.test.ts, src/lib/code-reviews/triggers/prepare-review-payload.test.ts, src/lib/cloud-agent/provider-branch-listing.test.ts, and the GitHub Apps and PR-review routers: 89 suites, 1342 tests passed, 1 pre-existing skip.
  • Local dev stack on a work VM (real Next.js, real Postgres, real NextAuth session, real tRPC routers and DTO schemas), with GitHub served by a local stub at the network boundary: githubApps.listRepositories returned 4,950 repositories from 50 page requests, githubApps.listBranches returned 1,000 branches from 10, githubPrReview.listReviewThreads returned 60 comments per thread for a page of 50 threads with one follow-up request per thread and at most 5 in flight, and a single thread returned 300 comments from 5 follow-up requests.
  • Archived rows stayed filtered out, the default branch stayed marked, and every partial thread carried commentsTruncated: true.
  • tsgo --noEmit exit 0; oxlint on the changed files: 0 warnings, 0 errors; oxfmt --list-different: no differences.
  • No UI surface changed, so no screenshot applies.

Proxy answers

The owner's proxy gave these answers in the owner's place.

  1. Question: At the cap: When a listing reaches its page ceiling, what should fetchGitHubBranches/fetchGitHubRepositories do? (fetchGitHubRepositories returns GitHubRepository[] to an excluded-feature caller, so a return-shape change is not available for it.)
    Answer: Truncate + warn log
    Confidence: 0.70
  2. Question: Adapter caps: Which page ceilings for the adapter listings?
    Answer: 10 pages branches / 50 pages repos
    Confidence: 0.60
  3. Question: Thread comment caps: Ceilings for listReviewThreads enrichment?
    Answer: 5 pages/thread, 500/request, concurrency 5
    Confidence: 0.60
  4. Question: Maintenance listing: should fetchGitHubRepositoriesForMaintenance carry the same page ceiling?
    Answer: Yes, same ceiling
    Confidence: 0.85

Branch and repository inventories share one page collector with a fixed page
ceiling, and it logs a warning when it stops at that ceiling. Review-thread
comment follow-ups are capped per thread and per request, with a small
concurrency limit, so one review-threads call performs a fixed amount of
provider work and keeps a bounded response.
@iscekic iscekic added drain Work item from the findings drain janitor:trust Janitor finding: trust boundary hardening labels Oct 20, 2026
Comment thread apps/web/src/routers/github-pr-review-router.ts
Comment thread apps/web/src/routers/github-pr-review-router.ts
Comment thread apps/web/src/lib/integrations/platforms/github/adapter.ts Outdated
Comment thread apps/web/src/lib/integrations/platforms/github/adapter.ts
@kilo-code-bot

kilo-code-bot Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

The incremental commit routes fetchPRInlineComments through the shared bounded collector with a 5-page/500-comment ceiling, and no defects were found in the changed code.

Assumptions: [INFERENCE] inline comments beyond 500 are dropped by design (proxy answer #3); the return type carries no truncation flag, so the only signal is the existing ceiling warning. The ceiling matches the bot's MAX_REVIEW_COMMENT_PAGES = 5 scan, so the comment's cross-reference is accurate.

Files Reviewed (2 files)
  • apps/web/src/lib/integrations/platforms/github/adapter.ts - 0 issues
  • apps/web/src/lib/integrations/platforms/github/adapter.pagination.test.ts - 0 issues

Notes

  • The converted path preserves prior semantics: same 100-per-page size, short-page termination, line/position-based outdated detection, and unknown user fallback.
  • The Link-header stop and ceiling warning are covered by the three new tests, including the exact-500-comment boundary and the no-warning case when the final full page has no rel="next".
  • No listeners, timers, or unbounded collections are introduced; accumulation stays bounded by MAX_INLINE_COMMENT_PAGES × LISTING_PAGE_SIZE.
  • The pre-existing hard-cap and truncation-signal comments remain valid design tradeoffs answered in the PR; no new duplicates were raised.
Previous Review Summaries (4 snapshots, latest commit be63fbf)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit be63fbf)

Status: No Issues Found | Recommendation: Merge

Executive Summary

The incremental commit adds a commentsTruncated signal end-to-end for review threads and correctly detects truncation at both the per-thread allowance and page ceilings; no new defects found in the changed lines.

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 0
Files Reviewed (5 files)
  • apps/web/src/lib/github-pr-review/dtos.ts - 0 issues (new commentsTruncated field passes the strict schema)
  • apps/web/src/lib/github-pr-review/mappers.ts - 0 issues (field is propagated with a false default)
  • apps/web/src/routers/github-pr-review-router.ts - 0 issues (truncated is derived from the final hasNextPage/cursor after the bounded loop, covering budget exhaustion, page ceiling, and short pages)
  • apps/web/src/lib/github-pr-review/review-thread-comments.test.ts - 0 issues
  • apps/web/src/routers/github-pr-review-router.test.ts - 0 issues

Notes

  • Previous WARNING re-verified as resolved: the truncation signal that was missing is now published as commentsTruncated on every thread, so a client can present a long thread as partial instead of inferring completeness from comments.length.
  • The earlier budget-documentation SUGGESTION remains fixed in 874b067a0, and the shared-budget determinism concern remains fixed in c178cceff.
  • The flag is computed from the thread's own allowance, so it stays deterministic for identical responses and does not depend on concurrent completion order.
  • The incremental diff introduces no listeners, timers, subscriptions, or unbounded collections, so it adds no memory-leak vectors; the pagination loop remains bounded by REVIEW_THREAD_COMMENTS_MAX_PAGES and the per-thread allowance.

Previous review (commit 874b067)

Status: 1 Issue Found | Recommendation: Address before merge

Executive Summary

The incremental commit corrects the budget documentation, but review-thread truncation still reaches clients with no signal, so the mobile thread header understates long threads.

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/src/routers/github-pr-review-router.ts 676 Thread comments are still truncated with no DTO signal; apps/mobile/src/components/pr-review/discussion/discussion-thread.tsx:154 derives the count from thread.comments.length, so a long thread shows an understated count and hides replies with no affordance.
Files Reviewed (2 files)
  • apps/web/src/lib/github-pr-review/dtos.ts - 0 issues
  • apps/web/src/routers/github-pr-review-router.ts - 1 issue

Notes

  • Verified as fixed in 874b067a0: the REVIEW_THREAD_COMMENTS_MAX_PER_REQUEST comment now matches enrichReviewThreads, which computes Math.floor(500 / nodes.length) once before dispatch and never mutates a shared budget.
  • Targeted re-check confirmed the remaining truncation-signal warning is unchanged at current HEAD; the author's reply defers the DTO change.
  • The hard-cap suggestion on adapter.ts:189 is dropped: the truncate-and-warn behavior is deliberate, documented in the PR body, and covered by tests.
  • The incremental diff is documentation-only, so it adds no listeners, timers, unbounded collections, or other memory-leak vectors.

Fix these issues in Kilo Cloud

Previous review (commit c178cce)

Status: 3 Issues Found | Recommendation: Address before merge

Executive Summary

The incremental commit resolves the nondeterministic budget race and the false ceiling warning, but review-thread truncation still has no client-visible signal and the follow-up budget constant's documentation now describes behavior the code no longer has.

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 2
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/src/routers/github-pr-review-router.ts 676 Thread comments are truncated with no DTO signal; apps/mobile/.../discussion-thread.tsx:154 derives the comment count from thread.comments.length, so long threads show an understated count and lose replies with no affordance. Author deferred the DTO change; the count is still derived from the truncated array.

SUGGESTION

File Line Issue
apps/web/src/lib/github-pr-review/dtos.ts 238 The REVIEW_THREAD_COMMENTS_MAX_PER_REQUEST comment still says the budget is shared across threads and that threads reached after it is spent keep only their initial page; enrichReviewThreads now splits the budget per thread before dispatch
apps/web/src/lib/integrations/platforms/github/adapter.ts 189 Hard caps still hide repositories/branches beyond the limit from the pickers and cached inventory with only a server-side log (author confirms this is deliberate)
Files Reviewed (7 files)
  • apps/web/src/lib/github-pr-review/dtos.ts - 1 issue
  • apps/web/src/lib/github-pr-review/review-thread-comments.test.ts - 0 issues
  • apps/web/src/lib/integrations/platforms/github/adapter.pagination.test.ts - 0 issues
  • apps/web/src/lib/integrations/platforms/github/adapter.ts - 1 issue
  • apps/web/src/lib/integrations/platforms/github/review-shared-repositories.test.ts - 0 issues
  • apps/web/src/routers/github-pr-review-router.test.ts - 0 issues
  • apps/web/src/routers/github-pr-review-router.ts - 1 issue

Notes

  • Verified as fixed in c178cceff: the follow-up budget is now computed per thread before dispatch (Math.floor(500 / nodes.length)), so identical inputs yield identical per-thread results, and the Link-header check stops the ceiling warning on a last full page without rel="next".
  • The per-thread allowance is reserved before the request and refunded for a short page or null node; first: reserved keeps the provider from returning more than the allowance, and slice(0, reserved) guards the stored result. With REVIEW_THREADS_PAGE_SIZE = 50 a thread gets 10 follow-ups, so the total retained across a page stays within 500.
  • collectListingPages still cannot make more than maxPages requests or keep more than maxPages × 100 rows.
  • No memory-leak vectors: collected arrays are bounded, and the change adds no listeners, timers, or module-level state.

Fix these issues in Kilo Cloud

Previous review (commit 5eed81d)

Status: 4 Issues Found | Recommendation: Address before merge

Executive Summary

The page/budget ceilings are implemented correctly and covered by focused tests, but review-thread truncation is invisible to clients and silently understates the comment count shown by the mobile discussion UI.

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 3
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/src/routers/github-pr-review-router.ts 677 Thread comments are truncated with no DTO signal; apps/mobile/.../discussion-thread.tsx:154 derives the comment count from thread.comments.length, so long threads show a wrong count and lose replies with no affordance

SUGGESTION

File Line Issue
apps/web/src/routers/github-pr-review-router.ts 718 The shared follow-up budget is reserved inside concurrent pLimit tasks, so which threads are truncated varies between identical calls
apps/web/src/lib/integrations/platforms/github/adapter.ts 203 The ceiling warning fires for exact-multiple totals (5,000 repos / 1,000 branches) even when nothing was dropped
apps/web/src/lib/integrations/platforms/github/adapter.ts 189 Hard caps hide repositories/branches beyond the limit from the pickers and cached inventory, with only a server-side log
Files Reviewed (6 files)
  • apps/web/src/lib/github-pr-review/dtos.ts - 0 issues
  • apps/web/src/lib/github-pr-review/review-thread-comments.test.ts - 0 issues
  • apps/web/src/lib/integrations/platforms/github/adapter.pagination.test.ts - 0 issues
  • apps/web/src/lib/integrations/platforms/github/adapter.ts - 2 issues
  • apps/web/src/routers/github-pr-review-router.test.ts - 0 issues
  • apps/web/src/routers/github-pr-review-router.ts - 2 issues

Notes

  • Budget accounting (reserve before request, refund on short page / null node) and the pLimit(5) ceiling check out; the fetchAllThreadComments and collectListingPages loops cannot exceed their caps.
  • Mocks in the new adapter test cover the required auth/app-selector/utils modules and the relative ./adapter import bypasses the Jest manual mock, so the suite exercises real adapter code as intended.
  • No memory-leak vectors were introduced: all collected arrays are bounded and no listeners, timers, or module-level state were added.

Fix these issues in Kilo Cloud


Reviewed by deepseek-flash-latest · Input: 36.4K · Output: 6.2K · Cached: 540.4K

Review guidance: REVIEW.md from base branch main

Review-thread follow-up comments are now divided evenly across the page's
threads before any request runs, so identical responses give identical
results, and each request asks only for the comments it keeps. The listing
collector reads GitHub's Link header and warns at its page ceiling only when
the provider still reports another page.
Comment thread apps/web/src/lib/github-pr-review/dtos.ts Outdated
The budget comment now states how the request limits are divided.
Each review thread now reports whether the response carries all of its
comments, so a client can present a long thread as partial instead of as
the whole list.
@iscekic
iscekic enabled auto-merge October 20, 2026 02:14
@iscekic iscekic added the merge:human Substantive functional risk requires human review. label Oct 20, 2026
@iscekic
iscekic disabled auto-merge October 20, 2026 19:08
Route the inline review comment listing through the shared bounded page collector, with the same page ceiling the bot review-comment scan already uses, and cover the ceiling with mocked full pages.
@iscekic iscekic added merge:blocked:insufficient-e2e Verification does not cover the change merge:human Substantive functional risk requires human review. merge:bot Bot-owned review; merge when ready once readiness passes and a review approves. and removed merge:human Substantive functional risk requires human review. merge:blocked:insufficient-e2e Verification does not cover the change labels Oct 21, 2026
@iscekic
iscekic enabled auto-merge October 22, 2026 13:41
@iscekic

iscekic commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator Author

bot: Review-thread reads must keep following every page until the provider reports no next page, because the thread contract promises the complete comment list and a test pins it. Address provider cost at the call site with a visible partial-result signal, never with a silent ceiling.

@iscekic iscekic closed this Oct 23, 2026
auto-merge was automatically disabled October 23, 2026 14:05

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

drain Work item from the findings drain janitor:trust Janitor finding: trust boundary hardening merge:bot Bot-owned review; merge when ready once readiness passes and a review approves.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant