Repository navigation
Serve the site with a per-request file middleware instead of MapStaticAssets - #2178
Sébastien Ros (sebastienros) wants to merge 3 commits into
Conversation
…cAssets MapStaticAssets needs a manifest of every file of the web root. With the site's ~70,500 files (8.4 GB), building it dominates every StaticHost build and publish, and each instance loads it at startup into one route per file. StaticHost now turns static web assets off and serves the web root with SiteFileMiddleware: - It looks the file up per request, prefers the brotli copy next to it when the client accepts br, and answers single byte ranges. - It computes the ETag (base64 SHA-256, the same value MapStaticAssets sent) the first time a file is served and caches it in memory. - It keeps the existing caching: immutable for /_astro/, the SDK's one-hour default for images, video and fonts, no-cache otherwise. - It steps aside when an endpoint matched, so API, health and download endpoints keep precedence. Site.targets publishes the frontend's output to wwwroot, as the static web assets pipeline did: it asks frontend.esproj for JavaScriptOutputGroup (which runs its BuildCommand) and writes a brotli copy (Optimal, the level used before) of each text file, reusing copies newer than their source. It replaces Caching.targets. The Frontend Build job no longer copies dist into wwwroot and rebuilds StaticHost to regenerate the manifest. Measured on a 4-core VM against the real production build of the site, current setup vs this change: - dotnet publish (Astro build excluded): 213 s -> 55 s (22 s when the brotli copies are reused) - CI-style build with dist in wwwroot: 86 s -> 9 s - Endpoints manifest: 142 MB -> none - First response after start: 13-18 s -> 2.3-5.8 s (the rest is startup both share) - Memory after warm load: 2.1-2.2 GB -> 280 MB RSS - Each of the 33k pages once: 1,450 -> 3,900 req/s, p99 56 -> 9 ms - Warm load: 5,700-5,850 -> 10,100-10,800 req/s, p99 21 -> 9 ms Comparing both servers on every page and file, with and without br (207,786 responses), Content-Type, Cache-Control, ETag, Vary and Content-Length match. The differences: Last-Modified on brotli responses is now the file's own, 4 tiny files no longer get a brotli copy that is not smaller, and StaticHost's own wwwroot (.gitignore, the dev-only Scalar theme) is no longer published. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01871HTuV3jb825jyAUQ3L7d
Frontend HTML artifact readyThe latest frontend build uploaded the This comment updates automatically when a new frontend build artifact is uploaded. |
Krish Khinchi (krishkhinchi)
left a comment
There was a problem hiding this comment.
The middleware approach looks promising, especially the reduction in startup memory and build time. However, there are a few compatibility concerns worth addressing before merging, particularly around dotnet run no longer serving src/frontend/dist, the changed POST behavior for static file paths, and the Last-Modified difference for Brotli responses. The inline MSBuild Brotli task also adds some complexity and would benefit from a more maintainable implementation. Additionally, the container build, App Service deployment, and GitHub Actions workflow still need validation. Recommendation: Request Changes until these deployment and compatibility edge cases are verified.
- Write the brotli copies with src/statichost/compress-site.cs, a file-based app that Site.targets runs, instead of an inline MSBuild task that loaded BrotliStream by reflection (which fails under .NET Framework MSBuild). - Reuse a brotli copy only while it carries its source's exact timestamp. A source replaced by new content with an older timestamp (a cache restore or an archive extraction) kept the copy of the old content. - Answer If-Match and If-Unmodified-Since with 412, as MapStaticAssets did. - Apply a date in If-Range only to uncompressed responses: Last-Modified is the original file's, so it does not identify the bytes of its brotli copy. - Key cached ETags by the file's path in the web root, not its physical path or bare name, and parse the ETag once per request.
- Compute ETags with XxHash128 instead of SHA-256: an ETag only has to tell versions of a file apart, which needs no cryptographic hash. It still hashes the content, so unchanged files keep their ETag across deployments. - Say why the brotli copy is looked up even for clients that do not accept brotli (the Vary header both versions need), and why cached ETags are keyed by ordinal path.
Summary
Draft for feedback: StaticHost serves the site with a small per-request file middleware instead of
MapStaticAssets. Eric Erhardt (@eerhardt) David Pine (@IEvangelist), I'd value your take on the approach before I take it further.Why.
MapStaticAssetsneeds a manifest of every file of the web root. With the site's ~70,500 files (8.4 GB), building it dominates every StaticHost build and publish (the SDK's static web assets targets), and every instance loads it at startup and builds one route per file. Front Door doesn't help: the route has no caching (x-cache: CONFIG_NOCACHEon every response), so the origin handles all compression, ETags and 304s.What changes.
SiteFileMiddlewarereplacesMapStaticAssets. Per request, it looks the file up in the web root (asUseStaticFilesdoes), serves the.brcopy next to it when the client accepts brotli, and answers single byte ranges (Safari needs them for the.mp4on the AI coding agents page). It runs afterUseRoutingand steps aside when an endpoint matched, so the API, health and download endpoints keep precedence.SiteFileETags). The value is the same base64 SHA-256MapStaticAssetssent, so switching doesn't invalidate browser caches.immutablefor/_astro/, the SDK'smax-age=3600, must-revalidatefor images, video and fonts,no-cacheotherwise.StaticWebAssetsEnabled=false.Site.targets(replacingCaching.targets) publishes the site towwwrootas the static web assets pipeline did: it asksfrontend.esprojforJavaScriptOutputGroup(which runs itsBuildCommand), then writes a brotli copy of each text file atOptimal, the level used before, reusing copies newer than their source.frontend-build.yml: the Frontend Build job no longer copiesdistintowwwrootand rebuilds StaticHost just to regenerate the manifest.Measured on a 4-core VM against a real
build:productionof the site, current setup vs this PR:dotnet publish(Astro build excluded)distinwwwrootBehavior compared by running both servers side by side over every page and file, with and without
Accept-Encoding: br(207,786 responses): Content-Type, Cache-Control, ETag, Vary and Content-Length match. The only differences:.brfile's.wwwrootis no longer published. Production currently serveshttps://aspire.dev/.gitignoreand the dev-only Scalar theme.Open questions.
dotnet runno longer includessrc/frontend/distin what it serves (the static web assets runtime manifest did that); it serves only its ownwwwroot, whichbuild:statichostfills. Does anything rely on the old behavior?BrotliStreamby reflection (inline tasks compile against .NET Standard). A small tool or a frontend script would also work; happy to switch.Third-party links and affiliations
None.
Validation
tests/StaticHost.Tests: 45 new tests inSiteFileTests(brotli negotiation, ETags and 304s, If-Modified-Since, ranges and If-Range, cache headers, unknown types, HEAD, endpoint precedence). The existing test server now uses the new middleware. All 377 tests pass; the Redis integration tests were not run (no Docker daemon in my environment).distand compared the output: the same 70,531 site files and 1,604 MB of brotli copies.🤖 Generated with Claude Code
https://claude.ai/code/session_01871HTuV3jb825jyAUQ3L7d