fix: keep fetch when shadow-DOM prerender retry is 4xx - #277
Merged
Conversation
#276 only cancelled a parallel fetch when prerender itself was usable. The auto-mode shadow-DOM upgrade still treated WAF/challenge HTML from page.content() as success and replaced a working 2xx fetch. Co-authored-by: kikohumanbeatbox <kikohumanbeatbox@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Kikobeats
marked this pull request as ready for review
August 18, 2026 08:26
Kikobeats
pushed a commit
that referenced
this pull request
Aug 20, 2026
#276 returns the parallel fetch (mode: fetch, 2xx) when page.content() is a 4xx challenge. The #277 upgrade guard only checked status, so that later fetch replaced the already-good first snapshot — including a 200 soft-block or interstitial body. Only accept a successful prerender result for the auto-mode upgrade. Co-authored-by: Cursor Agent <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#276 stopped a 4xx prerender from cancelling a parallel 2xx fetch. The auto-mode shadow-DOM upgrade still treated
page.content()on a WAF/challenge document as success and replaced the working fetch.Bug
A fetch-mode page with custom elements is upgraded to prerender. If the browser hits CloudFront/WAF (403 HTML) and the retry's inner fetch also fails (timeout/2, browser UA, or a later 4xx), the 2xx fetch is overwritten with the challenge page. Microlink then extracts metadata from the error document.
Fix
Only accept the prerender retry when it is actually usable (
htmlandstatusCode < 400), the same predicate as #276.Validation
npx ava test/shadow-dom.js test/mode.js— 11 passed, including the new regression test.Note
Low Risk
Narrow guard on an existing auto-upgrade path; behavior change only when prerender retry is 4xx/empty, with a dedicated regression test.
Overview
Fixes auto-mode shadow DOM upgrades replacing a good fetch result with WAF/challenge HTML when the prerender retry still returns
page.content()but with a 4xx status.The shadow-DOM retry now only swaps in prerender when the result is usable (
htmlpresent andstatusCode < 400), matching the existing prerender-vs-parallel-fetch fallback in #276. Tests add a 403 prerender regression case and shareSHADOW_DOM_HTMLacross shadow-DOM specs.Reviewed by Cursor Bugbot for commit 6b2ab1e. Bugbot is set up for automated code reviews on this repo. Configure here.