performance_benchmarks.md

Performance Benchmarks

Overview of other open-source, low-level PDF parsers

At the time of writing (date: 3 Aug 2026), we have the following packages (ranked by download numbers),

Rank	Python package	Primary use	GitHub stars	PyPI downloads/month
1	pypdf	Read, merge, split, crop, encrypt and modify PDFs in pure Python	~10.1k	122,033,329 (GitHub)
2	PyMuPDF	High-performance rendering, extraction, search and modification	~10.4k	106,838,492 (GitHub)
3	pypdfium2	Fast PDFium bindings for rendering, inspection and manipulation	803	74,384,565 (GitHub)
4	pdfminer.six	Detailed text extraction, layout analysis and low-level parsing	~7.0k	69,090,692 (GitHub)
5	pdfplumber	Text and table extraction with coordinates and visual debugging	~10.6k	57,151,282 (GitHub)
6	WeasyPrint	Generate PDFs from HTML and CSS	~9.5k	33,945,746 (GitHub)
7	fpdf2	Lightweight programmatic PDF generation	~1.5k	18,548,247 (GitHub)
8	pikepdf	Robust PDF repair and low-level manipulation using QPDF	~2.8k	9,393,114 (GitHub)
9	OCRmyPDF	Add searchable OCR layers to scanned PDFs and produce PDF/A	~34.3k	1,107,206 (GitHub)
10	camelot-py	Extract structured tables into pandas DataFrames	~3.8k	771,757 (GitHub)

Performance comparison on parsing and rendering

Here we evaluate the top-5 python packages that provide the parsing (i.e. text-extraction from PDF with location) and page-rendering.

What is measured

Two tasks are timed:

• parse — extract the text of every page together with its location. Every package therefore uses its position-bearing API, not a plain-text dump.
• parse+render — the same, plus rasterising the page at --scale (scale=1.0 is 72 dpi) and materialising it as a PIL image.

Python package	parse	parse+render
docling-parse	DoclingThreadedPdfParser	+ RenderConfig
PyMuPDF	page.get_text("rawdict")	+ page.get_pixmap() → pil_image()
pypdfium2	text-page rects + get_text_bounded()	+ page.render() → to_pil()
pdfplumber	page.chars	+ page.to_image()
pdfminer.six	extract_pages() + LTChar walk	not supported
pypdf	extract_text(visitor_text=...)	not supported

Two notes on fairness. PyMuPDF is timed on get_text("rawdict") rather than the much faster get_text("text"), because the latter returns a bare string with no geometry and would not be the same task. pdfplumber has no rasteriser of its own — to_image() delegates to pypdfium2 — so its render row is pdfminer extraction plus a PDFium raster, which is how a pdfplumber user would in practice obtain both.

pdfminer.six and pypdf have no rendering path at all, so they appear only in the parse task.

Threading. docling-parse decodes pages in parallel, and to our knowledge none of the other packages expose a thread-safe multi-page pipeline. docling-parse is therefore measured once per thread count, and every other package is measured single-threaded and reported with threads = 1.

Results

Parse without bitmap bytes (decode only)

Backend	Threads	Wall time (s)	vs threaded (1)	Efficiency	vs PyMuPDF (1t)	vs pypdfium2 (1t)	vs pdfplumber (1t)	vs pdfminer.six (1t)	vs pypdf (1t)	Pages/sec	ms/page
pymupdf (1t)	–	1370.2	0.22×		1.00×	0.19×	1.63×	1.57×	0.55×	39.8	25.1
pypdfium2 (1t)	–	256.229	1.17×		5.35×	1.00×	8.74×	8.38×	2.95×	213	4.69
pdfplumber (1t)	–	2239.82	0.13×		0.61×	0.11×	1.00×	0.96×	0.34×	24.4	41.03
pdfminer.six (1t)	–	2146.92	0.14×		0.64×	0.12×	1.04×	1.00×	0.35×	25.4	39.33
pypdf (1t)	–	755.656	0.40×		1.81×	0.34×	2.96×	2.84×	1.00×	72.2	13.84
docling threaded	1	298.603	1.00×	100%	4.59×	0.86×	7.50×	7.19×	2.53×	182.8	5.47
docling threaded	2	156.293	1.91×	96%	8.77×	1.64×	14.33×	13.74×	4.83×	349.2	2.86
docling threaded	4	86.643	3.45×	86%	15.81×	2.96×	25.85×	24.78×	8.72×	630	1.59
docling threaded	8	52.539	5.68×	71%	26.08×	4.88×	42.63×	40.86×	14.38×	1038.9	0.96
docling threaded	12	44.856	6.66×	55%	30.55×	5.71×	49.93×	47.86×	16.85×	1216.9	0.82
docling threaded	16	44.154	6.76×	42%	31.03×	5.80×	50.73×	48.62×	17.11×	1236.2	0.81

Render (decode + rasterise at scale 2)

M3 max

Backend	Threads	Wall time (s)	vs threaded (1)	Efficiency	vs PyMuPDF (1t)	vs pypdfium2 (1t)	vs pdfplumber (1t)	Pages/sec	ms/page
pymupdf (1t)	–	2299.18	0.81×		1.00×	0.45×	1.43×	23.7	42.12
pypdfium2 (1t)	–	1031.87	1.80×		2.23×	1.00×	3.19×	52.9	18.9
pdfplumber (1t)	–	3293.9	0.56×		0.70×	0.31×	1.00×	16.6	60.35
docling threaded	1	1858.78	1.00×	100%	1.24×	0.56×	1.77×	29.4	34.05
docling threaded	2	931.04	2.00×	100%	2.47×	1.11×	3.54×	58.6	17.06
docling threaded	4	492.556	3.77×	94%	4.67×	2.09×	6.69×	110.8	9.02
docling threaded	8	270.882	6.86×	86%	8.49×	3.81×	12.16×	201.5	4.96
docling threaded	12	205.269	9.06×	75%	11.20×	5.03×	16.05×	265.9	3.76
docling threaded	16	193.763	9.59×	60%	11.87×	5.33×	17.00×	281.7	3.55

AMD Ryzen 7-9800x3d-8-core processor

backend threads wall_time (s) vs threaded(1) efficiency vs pypdfium2 (1t) pages/sec ms/page

---

pypdfium2 (1t) - 1136.58 1.87x 1.00x 48 20.82 docling threaded 1 2126.69 1.00x 100% 0.53x 25.7 38.96 docling threaded 2 1070.48 1.99x 99% 1.06x 51 19.61 docling threaded 4 550.254 3.86x 97% 2.07x 99.2 10.08 docling threaded 8 306.593 6.94x 87% 3.71x 178 5.62 docling threaded 12 286.67 7.42x 62% 3.96x 190.4 5.25 docling threaded 16 267.282 7.96x 50% 4.25x 204.2 4.9

Linear thread scaling

Wall-clock runtime and throughput by docling-parse thread count. Apple M3 Max; parse+render; scale = 2 (144 dpi).

Parallel efficiency

Parallel efficiency relative to the one-thread docling-parse run. Apple M3 Max; parse+render; scale = 2 (144 dpi).

Per-page time distributions

Per-page timing distributions for every measured configuration. Apple M3 Max; parse+render; scale = 2 (144 dpi).

Per-page correlation against other packages

Per-page timing comparison of docling-parse and pypdfium2; the red diagonal marks equal time. Apple M3 Max; parse+render; scale = 2 (144 dpi).

Reproducing these numbers

Install the benchmark dependencies (this pulls in every third-party package in the table):

uv sync --group perf

Then run the comparison suite. It downloads the dataset from Hugging Face on first use, runs the requested tasks, prints the tables, and writes a full report:

uv run python ./scripts/benchmarking/run_performance_benchmarking.py --3rd-party-backends all --only-threaded

You only name a directory. The report and the per-page CSV are named <cpu>_<dataset>_<mode> — for example apple_m3_max_performance-dataset-bo767_render.md — so results from different machines never overwrite each other. Numbers are only comparable within a single row of the System hardware column, so the tables in this file are aggregated from those per-machine reports rather than produced in one run.

Each report is self-contained: it records the exact command it was produced by, the dataset name/revision/size, the machine and the version of every benchmarked package, the decode/content/render configs the run was driven with, and all result tables. That is what makes a number here traceable without the terminal scrollback it came from.

Useful variations:

• --compare on its own compares only docling-parse against pypdfium2, which is the cheapest meaningful run.
• --max-pages 5000 caps the total page count for a quick check; the cap is applied in input order and the last document is truncated, so the subset is reproducible.
• --mode both runs parse and parse+render in one go.
• Omitting --compare gives the thread-scaling tables instead; that mode writes the per-page CSV only.
• The default --output-dir is ./scratch, so an exploratory run does not touch the docs.

For a stable measurement, run on an otherwise idle machine, and note that the first run of a dataset pays for a cold file cache.

The companion CSV holds one row per (backend, task, threads, document, page), including the rendered image_width and image_height. It is the input for the render size check and, via scripts/benchmarking/run_performance_eval.py, for every plot shown above.

Overview of Regression and Performance datasets

docling-project/performance-dataset-bo767 — revision c684df5

• number of documents: 753
• number of pages: 54,584
• median document: ~30 pages; longest: ~1,600 pages

The distribution is broad and roughly log-uniform between 4 and 200 pages, with a spike of single- and two-page documents (110 of the 753) and a thin tail past 1,000 pages. That spread is why per-page quantiles are computed over pages rather than over documents — a per-document average would be dominated by the short documents, which are the majority by count but a small fraction of the pages.