Repository navigation
fix(refresh-index): re-read a timed-out discovery page instead of aborting - #11
polylane[bot] wants to merge 1 commit into
Conversation
…rting Co-authored-by: polylane[bot] <277585245+polylane[bot]@users.noreply.github.com>
|
This pull request is waiting on a decision: merge it if the change is still wanted, or close it if the fix is no longer needed. Either one settles it. It was opened on 2026-10-27 and has had no new activity for a week. If nothing happens on it within a week, Polylane closes it and keeps the branch, so reopening it brings the change back exactly as it stands. @justinhelmer and @boristane, the change touches what you own, so the review is with you. |
|
This pull request is closing without a merge, and nothing is lost: the branch It was opened on 2026-10-27, Polylane reminded the maintainers about it on 2026-11-07, and nothing has happened on it since. @justinhelmer and @boristane, you were asked to review this pull request; reopening it puts it back on your list. |
The scheduled daily refresh sometimes stopped before publishing because GitHub's repository search occasionally answers a discovery request with a partial result. On 2026-10-27 the whole run failed and the site kept the previous day's index. The refresh now reads a partial page again before giving up, and still stops when the page stays incomplete.
flowchart TD A["Scheduled refresh at 10:09 UTC"] --> B["Restore published checkpoints"] B --> C["Read /search/repositories page"] C --> D{"incomplete_results?"} D -- "false" --> E["Keep page, read next page"] D -- "true: search timed out" --> F["Observed before fix: throw Incomplete discovery"] F --> G["Run aborts, no daily index published"] D -- "true: search timed out" --> H["Fix: wait 5s, 10s, 15s and read the page again"] H --> D E --> I["Publish the daily index"]Why this fix
GitHub documents a partial search response when a query times out server-side, so that response cannot be removed from here. Runs 2 through 20 succeeded on the identical commit, which rules out a code change as the trigger, and the same query returned complete pages when re-read. The refresh now re-reads a partial discovery page a few times before aborting, so a single transient no longer stops the daily run. The completeness guard stays, so a genuinely incomplete page is still not published. If GitHub times out across every re-read the run still stops and keeps the last published index.
This change is a declared stopgap: it contains the symptom while the producer's fix is tracked as a follow-up on the issue timeline.
Production impact: Daily-index script change leaves the repolore Worker unchanged
The repository deploys to the Cloudflare Worker repo-lore. The changed paths are a CI-only script, its offline test, and the Check workflow. The Worker bundle builds only from
srcandserver(viatsconfig.build.jsonandtsconfig.server.json), so merging does not change the deployed Worker's runtime behavior. The added test runs offline throughnode --test.Checked and not affected:
repo-lore.Observability
Capture and publish the full daily poolstill logsRestored N published checkpoints.before discovery andError: Incomplete discovery.when a page stays incomplete, unchanged by this change.Polylane assessed this pull request at
6756058.Fix walkthrough
Watch the fix explained (1:19). Recorded for commit
6756058ef8ef68298a48415d992cfbb2dff22d42.Causal chain
4 files changed (+120/-19)
.github/workflows/check.yml: modified, +1/-1scripts/index-discovery.mjs: added, +32/-0scripts/refresh-index.mjs: modified, +3/-18test/index-discovery.test.mjs: added, +84/-0Generated by Polylane. You can ask follow-ups by mentioning @polylane in a comment.