Repository navigation
fix: Gate model stages on current process memory - #819
Draft
buildingjoshbetter wants to merge 1 commit into
Draft
buildingjoshbetter wants to merge 1 commit into
buildingjoshbetter wants to merge 1 commit into
Conversation
This was referenced Oct 9, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change
The shared daemon can begin another model stage while current process memory is already above its configured budget. This adds opt-in current-RSS admission before model construction, after construction, before every embedding/reranking slice, and before CPU transfer or retry. Stacked on #816; partial progress toward #297, which remains open.
TRUEMEMORY_MODEL_SERVER_MAX_RSS_MBdefaults to zero, disabled pending calibration. For enabled budget B = configured MiB * 1,048,576, current RSS U < B admits work; U >= B or an unavailable/invalid measurement refuses it withserver_busy. A synthetic 1 MiB budget admits 1,048,575 bytes and refuses 1,048,576 and 1,048,577 bytes. This uses currentpsutilRSS, separately from historical peak reporting.Refusal preserves inference ownership, complete-output semantics and sticky CPU recovery. Failed accelerator caches cannot evade CPU selection through a cache hit. Fast-lane refusal cannot fall through to main inference. RSS measurement consumes the same request deadline, so a slow sensor cannot start construction or recovery after expiry. Obsolete model references remain released before distinct replacements.
Models, rerankers, dtype, vector dimensions, result ordering and existing request, batch and retrieval limits are unchanged. Existing synthetic fixture loaders/doubles accept the new private helper/signatures without changing any of their 793 original assertions. The new regression fixture retains all 100 v1 assertions and adds 35. The publication branch also carries the already reviewed config fixture correction: actual delegated config directory/file/lock paths are isolated, all 20 prior assertions remain and eight checks are added. No production config or throttler change.
Validation
abdeff27839ee0c2b58d70b57dba291a48f65b78: 3,063 tests and 2,612 subtests passed, 17 skipped, 3 deselected. Unprivileged isolated run with actual sqlite-vec; exit 0, no timeout/OOM. Log SHA256:a473773393247e37a59bd3d6e0e801c23187b35857035245baa3642de09cce4e.594a02a08ac71a07a6ebc665e7df5a5dfe9ac910ffa15d65ca9074ac60cc5191.f923e15dcb4e6c5cf26ed91e26905c771d111c1c412050f0b58d6cfd9531c272.Rejected v1 source and failed receipts are retained. Independent review caught a constructor-after-deadline defect despite initial focused tests. Full v1 also exposed an unrelated-throttler dependency in the new synthetic fixture, corrected by separating independent sensor cases. The first publication gate retained two inherited config-fixture path failures; the known fixture correction passes without weakening assertions. All 34 hosted checks pass at exact draft head
4bc096095d834095db962a20ee1547b1426a7cd0.Remaining acceptance
This is sampled, opt-in admission, not a hard memory bound or reservation. Concurrent lanes or one native call can overshoot; current RSS does not establish unified-memory footprint or native allocator reclamation. A calibrated finite default, predicted workspace reservations, system headroom, bounded recovery and real Mac/MPS/thermal validation remain required.
All data is synthetic. This draft does not prove the original 26.24 GB Mac symptom fixed, close #297, merge main or release anything.