fix: Download of binary DICOM and study index writes wrong files (rebase of #127 onto current master)

[awatson1978]

This PR is #127, rebased onto the current master. The branch now also merges origin/master (v1.7.8) and has fixes from a review.

User-facing changes

These changes alter what a user or a client sees. Read them before you merge this PR.

• Instance retrieval (WADO-RS, Part 10): the webserver now serves the stored Part 10 file instances/<sop>/index.mht.gz when the file exists. The webserver generates Part 10 data from metadata and bulkdata only when no usable stored file exists.
  • The Content-Type of a multipart/related response now holds the boundary of the stored file.
  • A client that accepts gzip gets the stored file with Content-Encoding: gzip. Another client gets the decompressed multipart body.
  • application/dicom and application/zip responses get the payload from the stored file.
• STOW and ingest: create-dicomweb writes instances/<sop>/index.mht.gz for SEG, SR, and Parametric Map instances. Create-dicomweb generates this file from the stored dataset after the commit of the instance. A failure of this write logs a warning, and the instance stays stored.
• S3 retrieve (deploydicomweb): files get the local names that the other tools read.
  • Multipart objects get .mht or .mht.gz.
  • A key that has children in the listing maps to <key>/index.json.gz, or to index.mht.gz for a multipart object.
  • Retrieve renames an old multipart file at a guessed <key>.gz name to the correct name. Retrieve never renames bulkdata, because the BulkDataURI sets the bulkdata name.
  • Retrieve now downloads a file again when the size or the ETag is different. Before, retrieve kept every existing file.
• S3 upload content types: frames that are raw codestreams (.jxl, .jll, .jls, .jhc) now get the image type of the transfer syntax, not multipart/related.
• CLI: deploydicomweb studies <studyUIDs...> now requires one or more study UIDs, or *. The command fails with an error when you give no UID. Before, the command did nothing.

Implementation changes

• S3Ops.findExistingLocal is the one place that finds an existing local file and renames an old multipart file. DeployGroup calls it.
• DeployGroup now awaits shouldSkip. Before, a Promise that was not awaited made every existing file skip.
• multipartFile.mjs (static-wado-util) checks the content of a file with async I/O.
• readBulkData now finds the closing multipart boundary. Before, readBulkData removed 2 bytes from files that static-wado-creator writes, because those files have no \r\n after the closing boundary.
• readBulkData now adds the byteOffset of the Node Buffer. Before, files smaller than 4 KB could return bytes from the shared buffer pool.
• part10Controller streams the stored file. A read error before the first byte does not close the response, so generation can still send the data.
• writeRawPart10 now reports a failed close of the stream.
• STOW parsing uses clearBuffers: true, so the parser frees the bytes that it has read.

Known limits

• If a SEG or SR is stored again and the new Part 10 file fails, the old index.mht.gz stays. The webserver then serves the old file.
• Create-dicomweb holds one full copy of the instance in memory while it writes the Part 10 file. This applies only to SEG, SR, and Parametric Map instances.
• No tests cover assignFileNames (the directory-index names) or the change to the studies CLI arguments.

Tests

• static-wado-util: 22 tests pass.
• s3-deploy: 12 tests pass.
• static-wado-deploy: 2 tests pass.
• static-wado-webserver: 12 tests pass, and 1 test is skipped.
• create-dicomweb: 69 tests pass.
• static-wado-creator: 4 tests pass.

🤖 Generated with Claude Code