Conversation
CompileCacheHandler::Persist() writes each compile cache entry to a freshly created temporary file (via uv_fs_mkstemp()) and then renames it to the final cache filename. When the write, close, or rename step fails, the loop body ran `continue` without closing the still-open file descriptor or removing the already-created temporary file. Under a persistent failure condition (ENOSPC, EDQUOT, EFBIG, or any other error once the temporary file exists), this leaked a zero-byte temporary file and an open file descriptor for every module Node tried to cache, for the remaining lifetime of the process. In one reported EDQUOT incident, a single process left 3,424 zero-byte compile-cache temporary files behind in one minute, turning a block quota failure into an inode exhaustion problem. Add a scope guard, armed right after uv_fs_mkstemp() succeeds, that closes the descriptor if it is still open and unlinks the temporary file unless the rename to the final name has already succeeded. The guard runs on every `continue` in the loop body (including the ones already present for the write/close/rename failure paths), so no extra failure handling is needed at each call site beyond marking the descriptor closed after the close attempt and marking the file renamed after a successful rename. The original persistence error is preserved: only the leaked resources are cleaned up, and Persist() continues to `continue` past the failed entry exactly as before. A regression test spawns a child process under `ulimit -f 0`, which lets uv_fs_mkstemp() create an empty (0-byte) temporary file but makes the subsequent uv_fs_write() fail with EFBIG. This reproduces a failure partway through persistence deterministically, without needing to fill up a real filesystem, and the test asserts that no file is left behind in the compile cache directory afterward. Fixes: nodejs#65473 Signed-off-by: hmache <hmache@outlook.com> Assisted-by: Claude
|
Welcome to Node.js, and thank you for your first contribution! Before review, please take a moment to read:
Please make sure every commit is signed off. For a first pull request, GitHub Actions require collaborator approval and Jenkins CI must be started by a collaborator or triager, so an initial wait is normal. |
There are no sign that this has happened, closing |
|
Was drafting this too:
|
Title
src: unlink temporary compile cache file on persist failure
Fixes/Refs #N
Fixes #65473
What the issue was
When Node's module compile cache (
NODE_COMPILE_CACHE/module.enableCompileCache()) fails to persist a cache entry to disk partway through — after the temporary file has already been created but before it is fully written, closed, and renamed into place — the temporary file and its open file descriptor are leaked instead of being cleaned up.Under a persistent failure condition (
ENOSPC,EDQUOT,EFBIG, or any other error that shows up once the temp file exists), this leaks one zero-byte temporary file and one open file descriptor for every module Node attempts to cache, for the remaining lifetime of the process. The reporter measured a single process leaving 3,424 zero-byte temp files behind in one minute during a realEDQUOTincident, turning a disk-quota failure into inode exhaustion.Why it happened
The bug is in
CompileCacheHandler::Persist()insrc/compile_cache.cc. For each compile cache entry,Persist():uv_fs_mkstemp()to create a temporary file (entry->cache_filename + ".XXXXXX"), leaving an open file descriptor inmkstemp_req.result.uv_fs_write()to write the cache header and content to that descriptor.uv_fs_close()to close the descriptor.uv_fs_rename()to rename the temporary file to its final name.Before this fix, steps 2–4 each checked their own
errand, on failure, executedcontinueto move on to the next cache entry — without closing the descriptor from step 1 if it was still open, and without removing the temporary file created in step 1 if the rename in step 4 never happened or itself failed. The only scope guards present (cleanup_mkstemp,cleanup_write,cleanup_close,cleanup_rename) only calluv_fs_req_cleanup()on the correspondinguv_fs_trequest structs; none of them close the file descriptor or unlink the temporary file itself.How this PR fixes it
Right after
uv_fs_mkstemp()succeeds, the patch installs an additionalOnScopeLeaveguard (cleanup_tmp_file) that:mkstemp_req.resultif it is still open (tracked with afd_openbool, cleared right after the existinguv_fs_close()call is attempted, since a descriptor is not retried onclose()failure — matching standard POSIX practice).mkstemp_pathunless the rename to the final cache filename has already succeeded (tracked with arenamedbool, set right afteruv_fs_rename()succeeds).Because the guard is a normal C++ scope guard declared inside the
forloop body, it runs automatically on every existingcontinuein that iteration (write failure, close failure, rename failure) without needing separate cleanup code at each call site. The original error handling andDebug()logging are unchanged — only the leaked resources are cleaned up; the loop still moves on to the next cache entry exactly as before, andentry->persistedis only set totrueon full success as before.What isn't covered
This fix addresses only the resource leak (open descriptor + temporary file) on a failed persist. It does not change:
uv_fs_*APIs already used throughout this function, so it applies uniformly across platforms, but the regression test itself only exercises the POSIX (ulimit/RLIMIT_FSIZE) failure path and is skipped on Windows (see below).This is a full fix for the leak described in #65473, not a partial one —
Fixes #65473is used rather thanRefs #65473.How it was tested
Environment: cloud Linux sandbox (Ubuntu, gcc 13.3.0, Python 3.11.15, 2 CPU cores), built from a fresh shallow clone (
--depth 50) ofnodejs/nodemainat commita3bb551ea70c7e0b1377c5b97d5d186cd66095cd, configured with./configure --ninja(Release build).Baseline (unfixed) result — the new regression test run against the unmodified source (fix stashed via
git stash push -- src/compile_cache.cc, rebuilt incrementally, ~27s):This confirms the test reproduces the real bug: a temp file matching the
uv_fs_mkstemp().XXXXXXpattern (bc140ef7.dEC70X) is left behind under the compile cache directory.Fixed result — same test, fix restored, rebuilt incrementally (~27s):
Broader test results — full
test/parallel/test-compile-cache-*.jssuite (24 files, including the new one), with the fix in place:(24/24 passed: the 23 pre-existing compile-cache tests plus the new regression test.)
./out/Release/cctest --gtest_filter="*CompileCache*"matched no tests (there is no existing cctest coverage for this class), so no C++ unit test regressions apply here.Formatting/lint/build results:
python3 tools/cpplint.py --quiet src/compile_cache.cc— exit 0, no findings.python3 tools/checkimports.py src/compile_cache.cc— exit 0, no findings.clang-format-18 -style=file src/compile_cache.cc— byte-for-byte identical to the committed file (empty diff).node tools/eslint/node_modules/eslint/bin/eslint.js --max-warnings=0 --report-unused-disable-directives --concurrency auto test/parallel/test-compile-cache-persist-write-failure.js— exit 0, no findings../configure --ninja && make -j2) completed with no errors or new warnings attributable to this change; incremental rebuilds after each stash/restore step completed in ~27s each with onlycompile_cache.orecompiled andnode/embedtest/cctestrelinked.Tests that could not be run, and why:
make -j2 testsuite (all oftest/parallel+test/sequential+test/message, etc.) was not run in full — the sandbox has only 2 CPU cores and running the entire suite would take substantially longer than the targeted subsystem run above. Only the compile-cache subsystem tests (directly relevant to the changed file) and the new regression test were run, per the "smallest relevant test file first, then the appropriate broader suite" guidance — the broader suite here is the compile-cache subsystem, since the change is isolated toCompileCacheHandler::Persist().uv_fs_*calls used elsewhere in the same function), but the regression test is POSIX-only (common.isWindowsskip) because it relies onulimit -f/RLIMIT_FSIZE, which has no Windows equivalent.test-fs-write-sigxfsz.js-style shared-library-mode behavior: the regression test also skips whenprocess.config.variables.node_sharedis true, matching the existing convention intest-fs-write-sigxfsz.js, since Node's SIGXFSZ-to-EFBIG translation is not installed in that configuration. This sandbox's build is not shared-library mode, so the primary path was exercised; the shared-library-mode path was not.Steps to test
Note on process: this patch was prepared with AI assistance (an autonomous Claude agent run) per
AGENTS.mdin this repository — disclosed via theAssisted-by: Claudecommit trailer, alongside aSigned-off-byline for the human author. Per that same policy, this draft should be reviewed, tested independently, and actively maintained by a human contributor before (and after) it is opened as a real pull request; it has not been pushed or opened as a PR by the agent.