Fetch the UI's libraries in the background, so hosts need not run a command after each upgrade by Flix6x · Pull Request #2746 · FlexMeasures/flexmeasures

Flix6x · 2026-10-19T09:06:40Z

Description

Stacked on #2711: its base branch, feat/vendor-ui-assets, mirrors #2711's head, so this diff only shows the fetching.

With #2711, a pip-installed FlexMeasures serves the UI's libraries itself only after the host runs a fetch command, and again after each upgrade that bumps a library. This PR has FlexMeasures fetch them itself:

The first request which a web process serves starts a background thread, which downloads the files that are missing, checking each against its hash. The request doesn't wait for it, and browsers keep loading the files from CDNs until they are stored.
Only web processes serve requests, so CLI commands and job workers never fetch.
Of several web processes on one host, only the one that gets a lock file in static/vendor/ fetches. The lock is released when the process ends, also when it dies. When nothing is missing, it isn't even taken.
This also covers upgrades that bump a library, and versions set with hashes through FLEXMEASURES_JS_VERSIONS, in Docker too.
When fetching fails, for example on a server without internet access or without write access to its installation, FlexMeasures logs a warning with the command to fetch by hand, and keeps serving pages.
A new setting, FLEXMEASURES_FETCH_UI_LIBRARIES, switches it off. It's on by default, and off in testing.
With fetching switched off, the startup log now warns, instead of informing, when files are missing although some were fetched before, which most likely means an upgrade bumped a library.
Temporary files are named after the process, so a fetch command run at the same time can't write into the same file.
Added changelog item in documentation/changelog.rst

Look & Feel

From gunicorn with 3 workers and no --preload, starting with no library files, 12 requests at once:

200 0.35s … 200 0.43s   (all 12, while the download ran)
INFO: 49 file(s) of the UI's libraries were not fetched yet, so browsers load them from CDNs. FlexMeasures fetches them in the background once it serves its first request.   (once per worker)
INFO: Fetched 70 file(s) of the UI's libraries, which FlexMeasures serves itself from now on.   (once, about 10 s later)

Later pages load the files from /ui/static/vendor/. After removing Vega's files and restarting, as an upgrade that bumps Vega would leave things, one worker fetched just those 2 files.

How to test

pytest flexmeasures/ui/tests/test_vendor_assets.py. Each new test fails when the behaviour it covers is broken.
Run FlexMeasures from a pip install or a checkout without flexmeasures/ui/static/vendor/, open any page, and watch the log.

Related Items

#2711, #2718

🤖 Generated with Claude Code

https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1

Loading them from CDNs breaks the UI on poor or no internet connections. FlexMeasures moved to CDNs in #21 as a small speed tweak, to be revisited if CDNs ever failed. BREAKING CHANGE: FLEXMEASURES_JS_VERSIONS no longer has a default and is ignored, so plugins must read the js_versions template variable or flexmeasures.ui.utils.vendor_versions.JS_VERSIONS instead. Refs #18 Signed-off-by: Stijn van Houwelingen <teadrinkingprogrammer@proton.me>

Signed-off-by: Stijn van Houwelingen <teadrinkingprogrammer@proton.me>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

…ir licences The icons have no fixed release to fetch from, as Iconify renders them on request, so they stay committed under static/icons/. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

…their files and hashes Instead of committing ~5 MB of third-party files, record each library's version, files and SHA-256 hashes in flexmeasures/ui/utils/vendor_assets.py. Running python -m flexmeasures.ui.utils.fetch_vendor_assets downloads them, with their licences, and checks each file against its hash. Fetched files are served by FlexMeasures, and files that were not fetched load from their CDN, with the recorded hash as the tag's integrity attribute. Fonts and images which flexmeasures.css referred to move into the vendor.html macros, since a static stylesheet cannot tell whether they were fetched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

… them Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

… on one line Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

…ource copy in the Docker build Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

…rsions of the UI's libraries, optionally with the hashes of their files Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

…cess serves its first request Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

read-the-docs-community · 2026-10-19T09:19:09Z

Documentation build overview

📚 flexmeasures | 🛠️ Build #35243471 | 📁 Comparing 9720aea against latest (0144f89)

🔍 Preview build

11 files changed · + 2 added · ± 9 modified

+ Added

± Modified

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

Flix6x · 2026-10-19T14:36:10Z

Tested this manually, too.

…2746) The background fetch's lock test now also counts the fake library's notice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

TeaDrinkingProgrammer and others added 13 commits October 15, 2026 17:25

feat: serve asset type icons locally

9d93a2c

Signed-off-by: Stijn van Houwelingen <teadrinkingprogrammer@proton.me>

docs: link PR #2711 in the changelog entry

783093d

Signed-off-by: Stijn van Houwelingen <teadrinkingprogrammer@proton.me>

Merge main into feat/vendor-ui-assets

45fdb7d

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

Fetch the UI's libraries into the Docker image, and check that it has…

e48d6ce

… them Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

Document how to serve the UI's libraries without internet access

af1d2be

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

Put the comment on VendorLibrary.name above it, where black leaves it…

6cfa981

… on one line Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

Fetch the UI's libraries into the installed package rather than the s…

57e6794

…ource copy in the Docker build Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

Fetch the UI's missing library files in the background once a web pro…

02cbab1

…cess serves its first request Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

Add a changelog entry for fetching the UI's libraries in the background

21ff8fc

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

Flix6x mentioned this pull request Oct 19, 2026

Serve UI assets from FlexMeasures instead of CDNs #2711

Merged

6 of 8 tasks

Flix6x self-assigned this Oct 19, 2026

Flix6x added the UI label Oct 19, 2026

Flix6x added this to the 1.1.0 milestone Oct 19, 2026

Merge main, now that #2711 was squash-merged

9720aea

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

Flix6x changed the base branch from feat/vendor-ui-assets to main October 19, 2026 13:54

Merge main, now that the Vega and DataTables upgrades were merged

c427e5e

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bawYg67N5T5csiFdwWgZ1 Signed-off-by: F.N. Claessen <felix@seita.nl>

Flix6x merged commit f6f93a9 into main Oct 19, 2026
12 of 13 checks passed

Flix6x deleted the ui-fetch-libraries-on-start branch October 19, 2026 14:38

Flix6x mentioned this pull request Nov 4, 2026

Tidy up the v1.1.0 changelogs #2818

Merged

7 tasks done

Navigation Menu

Repository navigation

Fetch the UI's libraries in the background, so hosts need not run a command after each upgrade - #2746

Fetch the UI's libraries in the background, so hosts need not run a command after each upgrade#2746
Flix6x merged 15 commits into
mainfrom
ui-fetch-libraries-on-start

Flix6x commented Oct 19, 2026 •

edited

Loading

Uh oh!

read-the-docs-community Bot commented Oct 19, 2026 •

edited

Loading

Uh oh!

Flix6x commented Oct 19, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Flix6x commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Look & Feel

How to test

Related Items

Uh oh!

read-the-docs-community Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation build overview

Uh oh!

Flix6x commented Oct 19, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Flix6x commented Oct 19, 2026 •

edited

Loading

read-the-docs-community Bot commented Oct 19, 2026 •

edited

Loading