Skip to content

fix(embeddings): persist model cache outside node_modules - #179

Open
HengYangDS wants to merge 5 commits into
obra:mainfrom
HengYangDS:fix/persistent-model-cache
Open

HengYangDS wants to merge 5 commits into
obra:mainfrom
HengYangDS:fix/persistent-model-cache

Conversation

@HengYangDS

@HengYangDS HengYangDS commented Sep 26, 2026 •

Copy link
Copy Markdown

Summary

  • Fix Embedding model cache lives inside node_modules/@huggingface/transformers/.cache/ — every version bump or repair install orphans the 34 MB model and silently requires a fresh Hugging Face download #173 by keeping embedding model files in the durable Episodic Memory configuration directory, outside npm-owned node_modules.
  • Support an absolute EPISODIC_MEMORY_MODEL_CACHE_DIR for a shared or pre-seeded cache and explicit EPISODIC_MEMORY_OFFLINE=1 for remote-download denial. Forward both through the existing Codex STDIO MCP allowlist.
  • Preserve embedding hosts' native local/remote restrictions: initialization never enables a previously disabled permission. Explicit offline mode only restricts remote loading.
  • Retain the native feature-extraction pipeline, original BGE model, q8 dtype, query-prefix/output dimensions, lazy backend failure handling and thread controls. No dependency patch, replacement loader, new controller, model migration or host-specific path is introduced.
  • Report sync errors without a misleading green headline; retain existing exit semantics.

Product boundary

Cache and offline controls use documented process-global Transformers.js configuration. They affect all loaders sharing that module instance; an embedding host coordinates the shared cache and permissions. This contribution does not offer per-model or concurrent configuration isolation. Fresh normal processes remain network-enabled, while pre-existing host denial and explicit offline remain authoritative.

The dependency remains exactly Transformers.js 4.2.0. This repository ignores package-lock.json; its former caret range selected an unqualified 4.3.x/ONNX line. The exact pin is a bounded compatibility choice, not a claim that 4.2.0 is the latest stable or that later releases are defective. Upgrading that constraint requires separate native model/ABI qualification.

Current revision qualification (fedc911)

On macOS arm64 using Node 24.19.0 (ABI 137), native npm and manifest-compatible Vitest 3.2.7:

  • A prior empty-dependency native npm install completed with the unchanged declared manifest and native allowScripts; SQLite and sqlite-vec instantiated. The package cache was warmed; this is not a cold package-network claim.
  • Original initialization failed six focused permission regressions. The minimal correction passed the 22-test focused gate and the complete native npm test: 371/371 tests across 66 files, zero skips, including the actual packaged STDIO MCP tests.
  • Seven actual-model cache regressions cover seeded offline/host-denied loads, empty/partial offline refusal with zero fetch attempts, both-disabled native refusal, and deterministic fixture-transport online recovery. The fixture transport is not represented as live Hub availability.
  • Separate built-library witnesses used the unchanged real public Hub transport. An empty cache completed the first download (eight native fetch calls); deleting only its ONNX file then recovered a partial cache (two calls). Both produced finite normalized 384-dimensional embeddings and the same four model-file hashes. No original model/cache, conversation or credential store changed.
  • npm run build completed. The final MCP bundle differs from exact upstream base 7e065193 only at the intended getModelCacheDir/initEmbeddings product-source sections. The earlier approximately 6,300-line transitive dependency churn has been removed using native build inputs, not compiled-source hand edits. SDK, Zod, Marked and URI parser bytes match the upstream bundle; package constraints are unchanged.
  • An independent read-only review approved the exact source/test/artifact candidate and its product/default/host-permission boundaries.

The first native full-suite prerequisite failed because its isolated Unix-socket pathname was too long for macOS. A shorter caller-owned temporary directory allowed the same unchanged native pretest/full-suite to execute. The initial actual-model recovery assertion also exhausted heap by deeply enumerating a 34 MB Buffer; native Buffer.equals now checks the same bytes without expanding the test runner. Failed receipts are retained, not hidden or used as product-defect claims.

Prior evidence and remaining limits

The earlier parent-head clean Node 22/26 359-test runs remain historical; they are not rerun or represented as acceptance of this revision. The earlier Node 24.21 cleanup/signing failures do not prove an unchanged failure on 24.19, and the current successful 24.19 gate does not prove those 24.21 upstream issues resolved. Ubuntu/Windows, hosted CI and the later Transformers line remain unqualified.

Fork workflow execution requires maintainer approval; submission, local tests and mergeability are not hosted-green evidence. I will address actual review/CI feedback and follow through to merge, stable-release inclusion and installed adoption. Maintainer merge authority is preserved.

AndrewSpell added a commit to AndrewSpell/episodic-memory that referenced this pull request Oct 10, 2026
@HengYangDS

Copy link
Copy Markdown
Author

Author follow-up for #179: the exact PR head is still mergeable, but its CI run remains action_required and has not executed. The consolidated current request on #180 covers this contribution as well: please approve the fork workflow and review its narrow source behavior for merge. I will address resulting failures and keep generated output, portability and generality qualified; no hosted pass or installation update is claimed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant