Repository navigation
Conversation
fetch blocks, so GUI apps often call it from a thread. With one new thread per request, resident memory grows ~300 kB per request on Linux (Nim 2.2.4, default ORC): the memory a thread used is not returned to the OS when the thread exits, see nim-lang/Nim#23361. Puppy itself does not leak per request: the same fetch loop without any threads is flat. Documents the pattern and workarounds in the README and adds examples/threaded_fetch.nim, which reuses one worker thread and hand results back through channels. Measured with the worker pattern, RSS plateaus after a one-time warm-up (flat through 100 requests, vs ~29 MB growth for the thread-per-request loop). Refs treeform#123
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While looking at #123 I reproduced the reported memory growth and traced it. This PR documents what I found and adds a runnable example that avoids the pattern.
What is actually happening (measured on Linux, Nim 2.2.4, ORC default):
getOccupiedMem()stays 0, matching the report).fetchloop entirely in the main thread warms up once and then stays flat (23052 kB, +0 over 125 requests).system/threadimpl.nimonly callsdeallocOsPages()on the old refc path, not on the ARC/ORC path).This PR
-d:useMallocas a workaround for the app itself (it routed allocations through the system allocator; RSS plateaus in my tests, matching the observation in Memory leak when Puppy is used inside a thread #123).examples/threaded_fetch.nim: one long-lived worker thread, jobs/results handed back throughChannel(which deep-copies, so the hand-off is safe). Reusing the worker, RSS plateaus after a one-time warm-up and stays flat through 100 requests, while the thread-per-request loop grows ~29 MB over the same 100 requests.Happy to adjust wording or drop parts of this if the README section is too much — the example is the part I think is most useful for anyone hitting #123.