High tab memory (~600 MB) and slow startup: full 103 MB co-occurrence CSV is loaded into the frontend at startup regardless of settings

[NateScarlet]

Summary

Autocomplete-Plus v1.11.0 loads the entire danbooru co-occurrence dataset (3,236,960 pairs, ~103 MB CSV) into browser memory at startup as nested bidirectional Maps, plus a ~126 MB FlexSearch index. Measured impact on my machine (Firefox, Windows):

• With extension enabled: tab JS heap grows to ~1 GB shortly after the workspace loads
• With the extension disabled: ~400 MB

The README documents this under Known Issues → Performance ("It consumes memory to operate quickly in the browser. This should not be an issue on machines with specs capable of running ComfyUI."), so I understand it is a deliberate trade-off. This report is meant to show why that assumption does not hold in practice, with measurements.

Measurements

Method: Firefox DevTools heap snapshot taken after the default workflow finished loading; heap graph parsed offline and attributed by retainer chains.

window.autoCompleteData.danbooru subtree: 413.9 MB shallow / ~1.66M heap nodes, of which:

Structure	Shallow size	Notes
cooccurrenceMap	273.8 MB	31,060 inner Maps, avg 8.9 KB each; each pair inserted in both directions (~6.5M entries total)
flexSearchDocument	126.2 MB	Full-text index over ~121k tags
sortedTags + tagMap + aliasMap	~14 MB	From danbooru_tags.csv

On top of the resident structures, loading produces a large transient peak: response.text() materializes the whole 103 MB string and split('\n') creates ~3.2M line strings before chunked parsing starts. This pushes the GC heap ceiling up, which is why the tab stays around 1 GB even after the garbage is collected.

Why "a capable machine" does not help

Two assumptions behind the known-issue note do not hold:

1. Browser tabs have hard memory caps that are independent of machine RAM.
   Firefox's content process hits practical limits at roughly the 2 GB scale (SpiderMonkey OOM behavior); past that point the tab freezes or crashes instead of degrading gracefully. It makes no difference whether the machine has 16 GB or 128 GB. Starting at ~1 GB leaves little headroom for normal ComfyUI usage (image previews, node outputs, history), so reaching the freeze threshold during ordinary work is realistic.

2. The web frontend and the ComfyUI backend often run on different machines.
   In a typical LAN setup, the GPU host runs the server while the browser runs on a client device that may be much weaker. The memory cost lands entirely on the client, so "specs capable of running ComfyUI" (GPU/RAM of the server) says nothing about what the browser can spend.

The trade-off itself can be improved without giving up speed

Most of the 273.8 MB is not data but structure overhead: 31k small Maps pay hash-table capacity and per-object headers, and every pair is stored twice. Equally fast or faster alternatives:

• Serve related tags from the backend (the CSV already lives there): /related/<tag>-style endpoint queried on demand → zero resident memory, and the client never downloads 103 MB.
• Or lazy-load only the rows for tags being edited + a small LRU cache.
• If keeping the full dataset client-side is preferred: intern tag strings into ids and store counts in typed arrays (CSR-style). ~6.5M entries fit in tens of MB with O(1) lookup and better cache locality than nested Maps.

Minor related points:

• cache: "no-store" re-downloads all CSVs (including the 103 MB one) on every page reload.
• Settings like enableRelatedTags are not consulted before loadDataAsync() runs, so data is loaded even when the feature is disabled.

Environment

• Autocomplete-Plus v1.11.0
• ComfyUI frontend served over LAN, backend on another host
• Browser: Firefox 154.0, Windows
• Measurement artifacts available on request (heap snapshot attribution tables)