[wittjeff]

Follow-up to #425, as requested in #425 (comment) and #425 (comment).

Problem

The threaded parser writes every page to a standalone one-page PDF, so that pages can be decoded in parallel, and each page result keeps that buffer. The copy of the page takes along everything its /Resources names. When all pages share one /Resources (LibreOffice output, #424), every page buffer carries the compressed stream of every image in the document, whether the page draws it or not. On the #424 repro this was about 55 MB per kept page.

Change

get_thread_safe_page_buffer() now removes, from the copied page, the /Font and /XObject entries that no content stream of that page can use, before QPDFWriter::write(). The page belongs to the one-page out_pdf, so the source document and the other pages do not change. QPDFWriter does not write streams that are no longer reachable.

The new src/parse/qpdf/page_resources.h does the walk. It does not call QPDFPageObjectHelper::removeUnreferencedResources(), for two reasons:

1. Size and run time. qpdf's helper gives every form its own copy of a shared /Resources, and the copies write the other resource categories (/ExtGState, ...) inline again. On page 2 of 2508.13113v2.pdf from the regression set, 2298 figure forms share one /Resources: qpdf 12.4.2 --pages . 2 -- --remove-unreferenced-resources=yes takes 11.9 s instead of 0.4 s and writes 94 MB instead of 1.5 MB, which is the opposite of what the page buffer needs. The walk here removes entries in place, so shared dictionaries stay shared. It also visits each stream and each dictionary once: a walk with forEachFormXObject() over the unchanged dictionaries calls its action 5.27 million times on that page, and a first version of this PR that did so took the file from 31 s to over an hour.
2. Names that qpdf does not see. The stream decoder also resolves names against the page resources from Type3 glyph procedures without /Resources (9.6.5), tiling pattern cells, soft-mask groups and annotation appearance streams. qpdf scans only the page content and form XObjects, so it can remove an entry that docling-parse still uses.

The walk collects every name operand from all of these streams. An entry whose name occurs in none of them cannot be resolved, so it is removed. The page is written as before when a stream cannot be parsed (qpdf throws or warns), when qpdf cannot resolve an object, or when streams nest deeper than 32 levels. The walk also reaches forms that are never drawn, so the depth cap keeps a crafted chain of them from exhausting the stack.

Results

• Synthetic: 8 pages share one 11 MB image (random pixels, so Flate cannot compress it) that no page draws. Keeping all page results grew RSS by 115.6 MB on main and by 1.1 MB with this change.
• Time: the 40 largest PDFs of the regression set plus the Every page decodes all Flate images of a shared /Resources dictionary (~150 MB per page result for LibreOffice PDFs) #424 repro (41 files, 1825 pages), threaded parser with 4 threads, run interleaved with main: 71.0 s on main, 70.4 s with this change. 2508.13113v2.pdf: 30.8 s on main, 30.2 s with this change.

Tests

New in tests/test_unit_shared_resource_images.py:

• test_pages_that_do_not_draw_a_shared_image_do_not_hold_its_stream: the RSS bound above (fails on main).
• test_type3_glyph_keeps_an_image_it_takes_from_the_page_resources: a Type3 glyph draws an image that only the page resources name. The image must stay.
• test_forms_that_share_one_xobject_dictionary_are_visited_once: 1000 forms share one /XObject dictionary; the page must parse in a few seconds.

Unit tests and the regression suite pass on HF revision 6d3458e2.

@lazariv, thank you for the offer: a run on the production PDFs behind #424 would be very welcome.

🤖 Generated with Claude Code

[github-actions]

✅ DCO Check Passed

Thanks @wittjeff, all your commits are properly signed off. 🎉

[mergify]

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?: