fix(webapp): survive asset hash rotation across rolling deploys#4260
fix(webapp): survive asset hash rotation across rolling deploys#4260kathiekiwi wants to merge 11 commits into
Conversation
|
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThe deployment workflow resolves a prior image and passes it to Docker. Docker preserves previous hashed web assets, overlays current assets, prunes files older than 14 days, and enforces a size limit. Document responses default to 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Documents are content-hash-coupled to the build baked into the image that rendered them, so during a rolling deploy a stale document requests /build asset URLs that 404 on the new image: unstyled pages and dead buttons from partial hydration. Three layers: - Carry the previous published image's /build assets into each new image (PREV_IMAGE build arg, resolved in the publish workflow, no-op default for forks/local builds), pruned after 14 days. - Default document responses to Cache-Control: no-cache so browsers always revalidate HTML. - Inline recovery script that force-reloads (at most twice, 30s apart) when a /build stylesheet/script fails to load or a chunk import rejects - covers rollbacks and tabs older than the retention window.
kathiekiwi
force-pushed
the
fix/css-loading
branch
from
c872c9c to
5f61600
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Pruning before the current build lands makes the current build's assets unreachable by the prune, whatever mtimes the builder produces - a broken prune can only mean slow image growth, never a broken image.
… semver fallback The 14-day prune depends on the builder preserving mtimes through COPY. Assert the outcome instead of the mechanism: fail the image build if public/build exceeds 1GB, so silent retention failure becomes a loud CI error. Also try :latest before :main when resolving the previous image, so semver releases carry forward their true predecessor's assets.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
nicktrn
left a comment
nicktrn
left a comment
There was a problem hiding this comment.
We need to find a different solution for this that doesn't involve creating Docker images with assets from two different builds. Lots of additional complexity to debug when things go wrong.
Mixing assets from two builds in one image adds too much debugging complexity. Stale clients during a deploy window are covered by the client-side recovery reload; a longer-term fix (serving /build from object storage with retention) can be designed separately.
Instead of blindly reloading on a /build asset failure, the recovery script now polls a new /build-version endpoint (no-store) with backoff and reloads only once the server reports a different build than the one the page was rendered with (window.__remixManifest.version). If the versions never diverge - the asset failed for some other reason - or the reload budget is spent, it shows a manual-reload banner instead of leaving a dead page. Turns a speculative reload into a deterministic wait for a compatible deployment state.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…ate restore Final recovery policy: - All responses carry X-Build-Id. A working page is left alone when a new build exists; when a Remix loader fetch during navigation reveals a different build, the navigation becomes a full document load - the user lands on the new build without seeing an incompatible state. - Only real breakage (a /build asset 404 or failed chunk import) starts recovery: an overlay goes up immediately, the client polls /build-version and reloads once the server reports a different build. If versions never diverge or the reload budget is spent, the overlay offers a manual reload instead of a dead page. - Form fields and scroll position are snapshotted to sessionStorage before a recovery reload and restored after (native value setters + input events so React state stays in sync). history.state is deliberately not restored - the Remix router owns it.
…ton only on final state
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
| type: fix | ||
| --- | ||
|
|
||
| Fix intermittent unstyled/broken pages during rolling deploys, caused by stale HTML requesting `/build` asset hashes that 404 on the new image. Documents now default to `Cache-Control: no-cache`, and all responses carry an `X-Build-Id` header. A working page is never touched: when a Remix navigation reveals a new build, it becomes a full document load. Only real breakage (a `/build` asset 404 or failed chunk import) triggers recovery: an overlay goes up, the client polls the new `/build-version` endpoint and reloads once the server reports a different build (form fields and scroll are snapshotted and restored across the reload); if versions never diverge or the reload budget is spent, the overlay offers a manual reload. |
There was a problem hiding this comment.
| Fix intermittent unstyled/broken pages during rolling deploys, caused by stale HTML requesting `/build` asset hashes that 404 on the new image. Documents now default to `Cache-Control: no-cache`, and all responses carry an `X-Build-Id` header. A working page is never touched: when a Remix navigation reveals a new build, it becomes a full document load. Only real breakage (a `/build` asset 404 or failed chunk import) triggers recovery: an overlay goes up, the client polls the new `/build-version` endpoint and reloads once the server reports a different build (form fields and scroll are snapshotted and restored across the reload); if versions never diverge or the reload budget is spent, the overlay offers a manual reload. | |
| Fix pages occasionally loading unstyled or failing to load during deploys. The dashboard now detects this and reloads to recover automatically, or prompts you to reload if it can't. |
There was a problem hiding this comment.
It's a bit painful to get these right. We need to force the existing instructions into context maybe. Current locations:
- .server-changes/README.md
- AGENTS.md - section "Changesets and Server Changes", points at above
Claude makes these far too long, they should be short, largely non-technical, user-facing.
|
Solid bug fix! I like that you addressed the root cause. One thought: adding a comment explaining WHY this approach is needed would help future contributors. |
|
Good fix for asset hash rotation! This is the kind of subtle deployment issue that can be really hard to debug. Rolling deploys with asset hash changes can cause 404s for briefly coexisting versions. |
4 participants
Problem
Webapp HTML references content-hashed /build assets, and each Docker image contains exactly one build with a hard 404 for unknown hashes. During a rolling deploy, a client holding HTML from the old build may request old asset hashes from a replica running the new image, causing missing styles or failed chunk loads.
The page should recover automatically once a compatible build becomes available, without reload loops or unnecessary interruptions during normal deployments.
What changed