zlib: add ZIP archive support to zlib (ZipFile,ZipBuffer,ZipEntry)

[pipobscure]

Summary

Add ZIP archive support to the node:zlib module through three classes
and a set of helpers:

• ZipEntry: a single archive member, with buffered reads (content()),
  bounded-memory streaming reads (contentIterator()), and
  create()/createStream() for building members.
• ZipFile: random access to an archive backed by a file descriptor,
  reading members lazily without retaining their content and writing
  new members in place; opened with open()/openSync().
• ZipBuffer: a zero-copy, in-memory view over an archive already held
  in a Buffer.

createZipArchive() serializes a sequence of entries into an archive
byte stream, and setMaxZipContentSize() bounds the default in-memory
decompression size. Every operation has both an asynchronous and a
synchronous form.

P.S.: my CLA should be on file and I wrote this myself so COO is declared

[codecov]

Codecov Report

❌ Patch coverage is 98.24856% with 73 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.31%. Comparing base (396aad0) to head (fb49aca).
⚠️ Report is 8 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/zip/file.js	96.41%	27 Missing and 1 partial ⚠️
lib/internal/zip/entry.js	98.44%	13 Missing and 2 partials ⚠️
lib/internal/zip/headers.js	97.26%	14 Missing ⚠️
lib/internal/zip/fs-util.js	92.00%	6 Missing and 6 partials ⚠️
lib/internal/zip/archive.js	99.37%	2 Missing ⚠️
lib/internal/zip/compression.js	99.33%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #64339      +/-   ##
==========================================
+ Coverage   90.17%   90.31%   +0.13%     
==========================================
  Files         746      760      +14     
  Lines      242776   246928    +4152     
  Branches    45741    46578     +837     
==========================================
+ Hits       218929   223003    +4074     
- Misses      15347    15411      +64     
- Partials     8500     8514      +14     

Files with missing lines	Coverage Δ
lib/internal/errors.js	97.91% <100.00%> (+<0.01%)	⬆️
lib/internal/zip.js	100.00% <100.00%> (ø)
lib/internal/zip/binary.js	100.00% <100.00%> (ø)
lib/internal/zip/buffer.js	100.00% <100.00%> (ø)
lib/internal/zip/constants.js	100.00% <100.00%> (ø)
lib/internal/zip/content-size.js	100.00% <100.00%> (ø)
lib/internal/zip/dos.js	100.00% <100.00%> (ø)
lib/internal/zip/extra-fields.js	100.00% <100.00%> (ø)
lib/internal/zip/header-builders.js	100.00% <100.00%> (ø)
lib/zlib.js	98.20% <100.00%> (+0.12%)	⬆️
... and 6 more

... and 40 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[bakkot]

See also #45651

[pipobscure]

See also #45651

Thanks @bakkot !!!

I think the time has come for it on the one hand, and on the other I added some „motivation“ links earlier. Here some more detail:

Based on this, we can modify the loader to directly load from an archive. If we do that, we get application bundles. pipobscure#5 & pipobscure#6

Which can then in turn be used to easily create application bundles: https://github.com/pipobscure/experimental-sea

So the world had changed enough that it‘s worth proposing again.

[mcollina]

I like this a lot. It's likely better to split this into 2 PRs, one for Zip support and one for VFS-Zip, so the Zip support could theoretically be backportable on its own.

[pipobscure]

As per @mcollina I split this into two PRs.

I have the vfs-provider ready to go as follow up one (as it depends on this being merged)

I also made sure that the streaming side of things was actually as clean as I intended, and added a few more tests.

[bakkot]

Might be worth borrowing some tests from other projects: e.g. python, go, info-zip. There's a lot of edge cases with zip files. (Python's tests are larger than this whole PR put together.)

Not all directly applicable because this doesn't include the ability to extract to disk, fortunately.

[pipobscure]

Might be worth borrowing some tests from other projects

I've added a bunch more tests and compared to what go/python/info-zip do. (info-zip is cli exercise, so it's not entirely clear what gets tested).

I also gave zip.js an once over and concluded that it was too large a file (it originated from separate files that I've had for ages combined into one).

So I split it back out so it would be easier to review. And gave that another look.

[pipobscure]

Rebased on latest main to resolve the conflict

[pipobscure]

The failing mac tests seem to be entirely unrelated

Failed tests:
out/Release/node /Users/runner/work/node/node/node/test/parallel/test-debugger-extract-function-name.mjs
out/Release/node --test-reporter=./test/common/test-error-reporter.js --test-reporter-destination=stdout /Users/runner/work/node/node/node/test/parallel/test-repl-user-error-handler.js

can someone verify my deduction please.

[mcollina]

lgtm

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/75288/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/75295/

[mcollina]

Everything is green, CI is ok, we are missing another @nodejs/tsc approval as this a large PR.

[marco-ippolito]

LGTM

[targos]

I am reviewing. You can dismiss my request for changes if I don't answer by tomorrow.

[panva]

import { createGzip } from 'node:zlib'

This stable import emits ZIP experimental warning because of how our ESM facade evaluates the getters.

[pipobscure]

I rebased on main. Then I addressed the docs issue from @marco-ippolito and the import issue by @panva . I also go a security review from @mcollina and dealt with the issues that raised.

Here are the security-review issues we addressed:

Numbered findings

• Finding 1 (High): After ZipFile.close(), retained entries/operations fell through to a raw fd that the OS could reuse, enabling cross-file read/write/close.
• Finding 2 (High): add() awaited compression before queuing the mutation, so a concurrent close()/closeSync() could tear down the fd mid-add.
• Finding 3 (High): zipFiles() re-opened files by path (and never fstat-checked type), allowing a symlink-swap TOCTOU and accepting FIFOs/devices.
• Finding 4 (Medium): The open-time overlap check used a 30-byte lower bound instead of the exact local-header length, so a quoted-overlap archive passed ZipFile.open though ZipBuffer rejected it.

Additional concerns

• A (in-place corruption): A failed central-directory rewrite can leave the on-disk archive corrupt and diverged from memory — documented as non-atomic (recovery preserved).
• B (streaming ceiling): contentIterator()/stream() ignored the default decompression ceiling, allowing a zip bomb through the streaming API.
• C (EOCD/ZIP64 mismatch): A ZIP64 record silently overrode contradictory classic-EOCD values with no consistency check, a parser-differential risk.
• D (provisional streamed bytes): Streamed chunks are yielded before CRC verification runs at stream end — inherent to streaming, so documented.
• E (FIFOs/devices): Special files were accepted as stream sources — resolved by the same fstat/type-check as Finding 3.

[panva]

I rebased on main.

So we can't easily see and track the requested diff now. Unfortunate as otherwise we could review increments instead.

stale, need to start over now.

[pipobscure]

I rebased on main.

So we can't easily see and track the requested diff now. Unfortunate as otherwise we could review increments instead.

Apologies. I thought the “requested modus operandi “ was to keep it to a single commit. If that’s wrong then I’ll change the approach for the future.

[panva]

Apologies. I thought the “requested modus operandi “ was to keep it to a single commit. If that’s wrong then I’ll change the approach for the future.

No worries. FYI the commit-queue can land fixup commits (autosquash), or we can apply a label to squash, or if the PR has multiple things that can/should be backported independently (e.g. part is semver-major, part isn't) we can also land with rebase.

[pipobscure]

Loving tests right now. It caught me breaking streaming when doing the sec-fixes earlier.

I don't have enough time

[pipobscure]

Can someone trigger CI, so that we can get this over the line?