Skip to content

fix(pdf): release page backends after caching enrichment images - #5023

Open
Yun-0000 wants to merge 1 commit into
docling-project:mainfrom
Yun-0000:fix/pdf-release-enrichment-backends
Open

Yun-0000 wants to merge 1 commit into
docling-project:mainfrom
Yun-0000:fix/pdf-release-enrichment-backends

Conversation

@Yun-0000

Copy link
Copy Markdown

Checklist:

  • Documentation has been updated, if necessary.
  • Examples have been added, if necessary.
  • Tests have been added, if necessary.

Resolves #1886.

Formula, code, picture, and chart enrichment set keep_backend, so every page backend stayed alive until the conversion ended and memory grew with the page count. Those stages only crop a page image. The pipeline now caches the scales they use and unloads the page backend. A caller that sets keep_backend itself still keeps the live backend. Page.get_image() can crop a cached page image after the backend is gone.

Review evidence:

Before: tests/test_pdf_enrichment_backend_release.py on main: 2 failed (test_cached_page_image_is_cropped_without_a_backend, test_enrichment_releases_the_page_backend_after_caching_its_scale), 1 passed (test_explicit_keep_backend_is_left_alive).
After: the same file passes, and a page whose enrichment scale is cached no longer keeps its backend. The crop of that cached image stays available after _integrate_results.

Passed:

  • uv run pytest tests/test_pdf_enrichment_backend_release.py on main: 2 failed, 1 passed.
  • Same file after the change: 3 passed.
  • uv run pytest tests/test_pdf_enrichment_backend_release.py tests/test_pdf_streaming_foundation.py tests/test_extraction.py::test_standard_pipeline_integrate_preserves_failed_page_category: 11 passed.
  • make validate: passed.

Failed:

  • none after the change.

Not run:

  • make test. No reference files were regenerated.

Formula, code, picture, and chart enrichment kept every page backend
until the conversion ended, so memory grew with the page count. Cache
the page images those stages crop, then unload the backend.

Signed-off-by: Yun. <fangyun1008@gmail.com>
@github-actions

Copy link
Copy Markdown
Contributor

✅ DCO Check Passed

Thanks @Yun-0000, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@github-actions github-actions Bot added ai:ci-safe AI triage found no CI-safety concern (advisory) bug Something isn't working pdf PDF issue (except docling-parse) labels Oct 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

AI triage (advisory)

Checked commit 0aebf1a502. The analysis read the diff only. It did not run code from this pull request.

The PR makes the standard PDF pipeline cache the page images that enrichment stages need. It then unloads the page backend, so memory no longer grows with page count. Page.get_image() can now crop a cached page image when no backend exists.

Topic labels added: bug, pdf.

CI safety: no concern found. A maintainer must still approve CI.

1 related pull request(s) in the same code
PR State Signal Assessment
#4275 open changes the same code related: Both PRs edit standard_pdf_pipeline.py. PR 4275 adds require_full_batches to the stage queues. This PR changes page backend release. The two changes do not conflict in purpose.

The ai:* labels are advisory. They do not approve CI and they do not add tests:full. A new push runs this check again.

@lazariv

lazariv commented Oct 26, 2026

Copy link
Copy Markdown

Tested on the PDFs from docling-project/docling-parse#424 with this PR's two files on top of docling 2.142.1 (docling-serve 1.38.1, H100 MIG 1g.12gb, OCR + tables + formula/code enrichment + picture classification). Peak / RSS after the conversion:

document 2.142.1 this PR this PR + docling-parse#425
100-page report 14.8 / 14.6 GB 14.2 / 7.6 GB 5.2 / 4.6 GB
109-page manual 14.6 / 14.6 GB 13.3 / 6.2 GB 5.0 / 4.9 GB
160-page manual 33.1 / 33.1 GB 26.8 / 10.0 GB 6.0 / 5.8 GB

Conversion time and Markdown output are unchanged. On these PDFs the peak comes mostly from pages still in the parser read-ahead and the stage queues, each holding the shared images from docling-parse#424. So this PR alone lowers the peak by 4–19% and the memory left after the conversion by 48–70%. Together with docling-parse#425 both drop to 5–6 GB.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai:ci-safe AI triage found no CI-safety concern (advisory) bug Something isn't working pdf PDF issue (except docling-parse)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Memory leak occurs when formula enrichment is enabled

2 participants