[wittjeff]

Fixes #399.

Problem

load() on a signed, tagged PDF spent minutes and gigabytes before any page was decoded, holding the GIL. Removing /AcroForm from the catalog made it instant, and page_range made no difference.

Cause

• process_document_from_bytesio() built the document annotations eagerly, which runs to_json() on /AcroForm.
• A certification or FieldMDP signature carries /Reference[*]/Data, which Acrobat and iText point at the catalog. to_json() followed that reference into the page tree and the structure tree.
• to_json() detected cycles with a set of unparse() texts of the objects on the current path, copied into every recursive call. A shared object was expanded once per path, so every /Pg in the structure tree expanded its page again, fonts included. max_level was also not passed down the recursion.

Measured on a 33-page tagged PDF from the regression corpus (11586287754644715199-16.pdf: 15.6k structure elements, 6.4k /Pg references) with one synthetic DocMDP signature added with pikepdf:

variant	before	after
untouched	0.02 s, 125 MB	0.02 s, 85 MB
signature without /Reference	0.02 s	0.02 s
signature with /Data → catalog	48.6 s, 2.0 GB	0.06 s, 85 MB
same, /StructTreeRoot removed	0.12 s	–

(DoclingThreadedPdfParser().load(path, page_range=(1, 1)), peak RSS of the process.) A 91-page untagged PDF with the same signature loaded in 0.02 s before and after: the structure tree is the multiplier.

Change

1. to_json() stops at nested dictionaries whose /Type is /Catalog, /Pages, /Page, /StructTreeRoot or /StructElem and writes "[skipping /Catalog object '1 0 R']", the way it already skips the /Parent and /P keys. A page or the catalog is never form or annotation content.
2. The cycle guard keys indirect objects by QPDFObjGen on one set passed by reference and unwound on return: no unparse(), no copies. Direct objects are not tracked, since they cannot close a cycle. The output for every existing case is unchanged; a cycle still yields "[ref to previous object: 'N 0 R']".
3. max_level is passed down the recursion. The callers that passed 16 now use the default 32, which was the effective depth before; the two font-program callers keep their explicit 2 (descriptor_json and stream_dict_json are not read anywhere).
4. The document annotations (form, metadata, language, outline) are built on the first get_annotations(), get_meta_xml() or get_table_of_contents() call, under thread_safe_buffer_mutex, instead of during load. A conversion that only reads pages no longer pays for them. The extract_doc_annotations timing is recorded when they are first requested.

Points 1 to 3 are one commit, point 4 a second one, so it can be dropped on its own if you prefer to keep the eager build.

Not included

Releasing the GIL during the native load, as suggested in the issue. After this change the load is qpdf's parse only. Releasing the GIL there needs a mutex around the per-key decoder maps against concurrent load() calls on one parser, which is a separate change.

Tests

• tests/test_unit_signed_form_json.py: an in-memory signed, tagged fixture. Asserts that the form export keeps the signature fields and never reaches the page tree or the structure tree, with both parsers.
• The annotation, outline and widget tests pass. Full tests/test_unit_*.py + tests/test_regression_parse.py against the pinned dataset: 269 passed, 6 skipped (pre-existing), 0 failed.

🤖 Generated with Claude Code

[github-actions]

✅ DCO Check Passed

Thanks @wittjeff, all your commits are properly signed off. 🎉

[mergify]

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[PeterStaar-IBM]

@wittjeff can you fix the merge conflict?

[PeterStaar-IBM]

@wittjeff We seem to have another merge conflict (just merged another of your PR's).

[wittjeff]

Merged main into the branch (bccca15). The only conflict was in document.h: #414 added the load-error fields at the same spot where this PR documents the lazily built annotations. I kept both. Full test suite passes locally with the merged build (408 passed, 1 skipped).