[wittjeff]

Fixes #424.

Problem

Pages that share one /Resources dictionary (as LibreOffice writes them) list every image of the document. pdf_resource<PAGE_XOBJECTS>::set() parses every key, and pdf_resource<PAGE_XOBJECT_IMAGE>::parse() defiltered the samples and the soft mask at once. The xobject_parse_cache works only inside one page decoder, and the threaded parser opens a QPDF per page. So each page result held the decoded pixels of every image in the shared dictionary, whether the page draws it or not.

Change

• parse() reads only the image dictionary: size, colour space, filters, /Decode, palette.
• A new load_pixels() const runs init_stream_data() and init_soft_mask_data() on the first call to a pixel getter (get_raw_stream_data(), get_codec_stream_data(), get_decoded_stream_data(), get_soft_mask_data(), the has_* forms and the soft-mask size getters). Only Do_image and save_to_file() call these, so a page decodes only the images it draws.
• The pixel members and the two QPDFObjectHandle members are mutable. One page decoder runs on one thread, and its QPDF document stays alive until Do runs.

extract_bitmap_pixels=false keeps its meaning: load_pixels() does nothing.

Measurement

20 pages, 8 RGB 1600×1200 images in one /Resources, drawn only on page 1. RSS growth per kept PageParseResult, DoclingThreadedPdfParser:

file	7.23.0 no-render	this PR no-render	7.23.0 render	this PR render
shared, Flate	50.7 MB	7.9 MB	52.4 MB	10.1 MB
unshared, Flate	7.5 MB	7.6 MB	9.8 MB	9.6 MB
shared, DCT	7.8 MB	6.9 MB	10.2 MB	9.2 MB
unshared, DCT	5.5 MB	5.5 MB	7.8 MB	7.8 MB

Tests

• New tests/test_unit_shared_resource_images.py: an 8-page in-memory PDF (built with tests/pdf_builder.py) with one shared 34 MB Flate image drawn on page 1. It checks that only page 1 has the bitmap and limits the RSS growth of the kept results to 3× one decoded copy. On 7.23.0 it fails with 311 MB; with this PR it passes. It reads /proc/self/status and skips where that does not exist.
• Unit suite: 284 passed, 1 skipped.
• Regression suite on the pinned dataset revision 6d3458e2: 106 passed. No groundtruth changes.

🤖 Generated with Claude Code

[github-actions]

✅ DCO Check Passed

Thanks @wittjeff, all your commits are properly signed off. 🎉

[mergify]

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[lazariv]

Tested on the production PDFs behind #424, built from ac2fe96 in the docling-serve 1.38.1 image (local engine, H100 MIG 1g.12gb, OCR + tables + formula/code enrichment + picture classification). Peak RSS of the conversion:

document	7.23.0	this PR	this PR + docling-project/docling#5023
100-page report	14.8 GB	5.7 GB	5.2 GB
109-page manual	14.6 GB	5.7 GB	5.0 GB
160-page manual	33.1 GB	7.3 GB	6.0 GB

Conversion time is unchanged (within 2%). The Markdown is identical to 7.23.0 for the first two documents; the third differs by 0.04% of words, and docling#5023 alone shows the same difference, so it looks like run-to-run variation.

One cost remains. On the synthetic repro from #424 this PR still holds about 55 MB per kept page (5.7 GB for 100 pages, against 1.2 GB unshared); its images compress poorly (a 36 MB file). As far as I can read pdf_decoder<DOCUMENT>, every page is written into a standalone one-page PDF with qpdf_s_preserve, which copies the compressed stream of every XObject in the shared /Resources into each page buffer. Calling QPDFPageObjectHelper::removeUnreferencedResources() on the copied page before QPDFWriter::write() might drop those as well. I haven't tested that.

[PeterStaar-IBM]

Tested on the production PDFs behind #424, built from ac2fe96 in the docling-serve 1.38.1 image (local engine, H100 MIG 1g.12gb, OCR + tables + formula/code enrichment + picture classification). Peak RSS of the conversion:

document 7.23.0 this PR this PR + docling-project/docling#5023
100-page report 14.8 GB 5.7 GB 5.2 GB
109-page manual 14.6 GB 5.7 GB 5.0 GB
160-page manual 33.1 GB 7.3 GB 6.0 GB
Conversion time is unchanged (within 2%). The Markdown is identical to 7.23.0 for the first two documents; the third differs by 0.04% of words, and docling#5023 alone shows the same difference, so it looks like run-to-run variation.

One cost remains. On the synthetic repro from #424 this PR still holds about 55 MB per kept page (5.7 GB for 100 pages, against 1.2 GB unshared); its images compress poorly (a 36 MB file). As far as I can read pdf_decoder<DOCUMENT>, every page is written into a standalone one-page PDF with qpdf_s_preserve, which copies the compressed stream of every XObject in the shared /Resources into each page buffer. Calling QPDFPageObjectHelper::removeUnreferencedResources() on the copied page before QPDFWriter::write() might drop those as well. I haven't tested that.

yes, we write every page so we ca parallelize, so this is a feature we do need.

[wittjeff]

yes, we write every page so we ca parallelize, so this is a feature we do need.

@PeterStaar-IBM Incoming.

[lazariv]

Thanks for merging! To clarify my last paragraph: I wasn't suggesting dropping the per-page write. The idea was to keep it and prune the copied page before it is written, in pdf_decoder<DOCUMENT> (document.h, after fixCopiedAnnotations()):

out_page.removeUnreferencedResources();

QPDFWriter writer(out_pdf);

out_page belongs to the one-page out_pdf, so the source document and the other pages are left alone. Each page buffer would then carry only the XObjects that its own content uses, instead of the compressed stream of every image in the shared /Resources. I haven't tested it: it costs a pass over the page's content stream, and on the #424 repro it should bring the ~55 MB per kept page down toward the unshared case.

If that's what "Incoming" is, I'm happy to run the next PR on the same production PDFs.