[PeterStaar-IBM]

Summary

Improve the LaTeX backend for arXiv source files. The backend now keeps inline formulas as formulas in text, headings, captions, table cells, and front matter.

Added the script to do large scale scans of latex-backend on entire arxiv,

uv run pytest tests/test_latex/test_large_scale.py --large-scale --max-items 1000000

Changes

• Preserve source commands such as \newcommand, \newenvironment, and \newtheorem as invisible document items.
• Parse authors, affiliations, and related front matter in a separate handler. Store each source role in meta.latex__classification, and keep \inst{...} as an inline superscript formula.
• Replace source line breaks and tabs with spaces in document text and table cells.
• Improve display math and bibliography parsing. Update reference outputs for the intended changes.
• Add an opt-in arXiv corpus test. It checks DocLang against the XSD, reports parse and consistency failures, and can save source and serialized output for comparison.

Validation

• LaTeX tests: 165 passed, 1 failed. test_e2e_latex_conversions differs from the committed Markdown reference for 1706.03762/main.tex.
• The current Markdown output also includes literal title and author lines from the new classification metadata. This needs a fix and a reference-data review before merge.
• make check did not pass locally. Its maximum-lines check scanned ignored corpus artifacts and reported tests/test_latex/test_macros.py at 1,031 lines.

@DanielNg0729 I added a new large scale test, that allows us to iterate via streaming over many many latex files

[github-actions]

✅ DCO Check Passed

Thanks @PeterStaar-IBM, all your commits are properly signed off. 🎉

[mergify]

Merge Protections

🟢 All 2 merge protections satisfied — ready to merge.

Show 2 satisfied protections

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

🟢 Require two reviewer for test updates

When test data is updated, we require two reviewers

• #approved-reviews-by >= 2

[github-actions]

AI triage (advisory)

Checked commit fd2cd36695. The analysis read the diff only. It did not run code from this pull request.

Warning

The AI analysis did not finish. Only the deterministic checks are shown.

CI safety: needs maintainer care before you approve CI.

• pyproject.toml: changes build, dependency, or tool configuration
• tests/test_latex/conftest.py: pytest runs this file at collection
• uv.lock: changes locked dependencies

Possibly related pull requests

PR	State	Signal	Assessment
#4891	open	changes the same code	not assessed

Reference data changes

Change	Files
structure	8
tables	4
text	13
added	16
removed	39

• structure tests/data/latex/groundtruth/1706.03762_main.tex.json: caption: 8 → 9; footnote: 2 → 5; formula: 5 → 138; list_item: 44 → 43; text: 73 → 276; table 0: 12 cell(s) changed; table 1: 14 cell(s) changed; table 2: 9 cell(s) changed
• structure tests/data/latex/groundtruth/2305.03393_main.tex.json: caption: 6 → 8; text: 37 → 50
• structure tests/data/latex/groundtruth/2310.06825_main.tex.json: caption: 7 → 12; formula: 0 → 41; text: 47 → 121; title: 0 → 1; table 0: 9 cell(s) changed; table 1: 2 cell(s) changed; table 3: 4 cell(s) changed; table 4: 1 cell(s) changed
• structure tests/data/latex/groundtruth/2412.19437_main.tex.json: caption: 15 → 20; formula: 15 → 164; text: 172 → 394; table 1: 9 cell(s) changed
• structure tests/data/latex/groundtruth/2501.00089_main.tex.json: caption: 7 → 4; formula: 0 → 87; text: 62 → 182
• structure tests/data/latex/groundtruth/arXiv-2501.01300v2_main.tex.json: caption: 6 → 3; formula: 6 → 131; text: 40 → 188
• structure tests/data/latex/groundtruth/example_01.tex.json: text: 4 → 5
• structure tests/data/latex/groundtruth/example_02.tex.json: caption: 0 → 1; formula: 2 → 3; text: 4 → 5
• tables tests/data/html/groundtruth/wiki_duck.html.md: +4 / -4 lines; Markdown table rows changed
• tables tests/data/latex/groundtruth/1706.03762_main.tex.md: +136 / -338 lines; Markdown table rows changed
• tables tests/data/latex/groundtruth/2305.03393_main.tex.md: +31 / -54 lines; Markdown table rows changed
• tables tests/data/latex/groundtruth/2310.06825_main.tex.md: +48 / -114 lines; Markdown table rows changed
• text tests/data/html/groundtruth/example_08.html.json: item attributes changed
• text tests/data/latex/groundtruth/1706.03762_main.tex.itxt: +607 / -247 lines
• text tests/data/latex/groundtruth/2305.03393_main.tex.itxt: +104 / -107 lines
• text tests/data/latex/groundtruth/2310.06825_main.tex.itxt: +220 / -105 lines
• text tests/data/latex/groundtruth/2412.19437_main.tex.itxt: +738 / -388 lines
• text tests/data/latex/groundtruth/2412.19437_main.tex.md: +295 / -986 lines
• text tests/data/latex/groundtruth/2501.00089_main.tex.itxt: +338 / -119 lines
• text tests/data/latex/groundtruth/2501.00089_main.tex.md: +50 / -128 lines
• text tests/data/latex/groundtruth/arXiv-2501.01300v2_main.tex.itxt: +374 / -87 lines
• text tests/data/latex/groundtruth/arXiv-2501.01300v2_main.tex.md: +48 / -95 lines
• text tests/data/latex/groundtruth/example_01.tex.itxt: +16 / -14 lines
• text tests/data/latex/groundtruth/example_02.tex.itxt: +15 / -9 lines
• text tests/data/latex/groundtruth/example_02.tex.md: +6 / -4 lines
• added tests/data/latex/groundtruth/1706.03762_main.tex.dclg
• added tests/data/latex/groundtruth/1706.03762_main.tex.latex
• added tests/data/latex/groundtruth/2305.03393_main.tex.dclg
• added tests/data/latex/groundtruth/2305.03393_main.tex.latex
• added tests/data/latex/groundtruth/2310.06825_main.tex.dclg
• added tests/data/latex/groundtruth/2310.06825_main.tex.latex
• added tests/data/latex/groundtruth/2412.19437_main.tex.dclg
• added tests/data/latex/groundtruth/2412.19437_main.tex.latex
• added tests/data/latex/groundtruth/2501.00089_main.tex.dclg
• added tests/data/latex/groundtruth/2501.00089_main.tex.latex
• added tests/data/latex/groundtruth/arXiv-2501.01300v2_main.tex.dclg
• added tests/data/latex/groundtruth/arXiv-2501.01300v2_main.tex.latex
• added tests/data/latex/groundtruth/example_01.tex.dclg
• added tests/data/latex/groundtruth/example_01.tex.latex
• added tests/data/latex/groundtruth/example_02.tex.dclg
• added tests/data/latex/groundtruth/example_02.tex.latex
• removed tests/data/ocr/groundtruth/easyocr/full_page/ocr_test.easyocr.full_page.doctags.txt
• removed tests/data/ocr/groundtruth/easyocr/layout_regions/ocr_test.easyocr.layout_regions.doctags.txt
• removed tests/data/ocr/groundtruth/easyocr/pdf_aware_layout_regions/ocr_test.easyocr.pdf_aware_layout_regions.doctags.txt
• removed tests/data/ocr/groundtruth/nemotron_ocr/full_page/nemotron_multipage.nemotron_ocr.full_page.doctags.txt
• removed tests/data/ocr/groundtruth/nemotron_ocr/full_page/ocr_test.nemotron_ocr.full_page.doctags.txt
• removed tests/data/ocr/groundtruth/nemotron_ocr/full_page/ocr_test_rotated_180.nemotron_ocr.full_page.doctags.txt
• removed tests/data/ocr/groundtruth/nemotron_ocr/full_page/ocr_test_rotated_270.nemotron_ocr.full_page.doctags.txt
• removed tests/data/ocr/groundtruth/nemotron_ocr/full_page/ocr_test_rotated_90.nemotron_ocr.full_page.doctags.txt
• removed tests/data/ocr/groundtruth/nemotron_ocr/layout_regions/ocr_test.nemotron_ocr.layout_regions.doctags.txt
• removed tests/data/ocr/groundtruth/nemotron_ocr/layout_regions/ocr_test_rotated_180.nemotron_ocr.layout_regions.doctags.txt
• removed tests/data/ocr/groundtruth/nemotron_ocr/layout_regions/ocr_test_rotated_270.nemotron_ocr.layout_regions.doctags.txt
• removed tests/data/ocr/groundtruth/nemotron_ocr/layout_regions/ocr_test_rotated_90.nemotron_ocr.layout_regions.doctags.txt
• removed tests/data/ocr/groundtruth/nemotron_ocr/pdf_aware_layout_regions/nemotron_multipage.nemotron_ocr.pdf_aware_layout_regions.doctags.txt
• removed tests/data/ocr/groundtruth/nemotron_ocr/pdf_aware_layout_regions/ocr_test.nemotron_ocr.pdf_aware_layout_regions.doctags.txt
• removed tests/data/ocr/groundtruth/nemotron_ocr/pdf_aware_layout_regions/ocr_test_rotated_180.nemotron_ocr.pdf_aware_layout_regions.doctags.txt
• removed tests/data/ocr/groundtruth/nemotron_ocr/pdf_aware_layout_regions/ocr_test_rotated_270.nemotron_ocr.pdf_aware_layout_regions.doctags.txt
• removed tests/data/ocr/groundtruth/nemotron_ocr/pdf_aware_layout_regions/ocr_test_rotated_90.nemotron_ocr.pdf_aware_layout_regions.doctags.txt
• removed tests/data/ocr/groundtruth/ocrmac/full_page/ocr_test.ocrmac.full_page.doctags.txt
• removed tests/data/ocr/groundtruth/ocrmac/full_page/ocr_test_rotated_180.ocrmac.full_page.doctags.txt
• … 20 more file(s) not listed

Only the first 80 of 116 reference files were compared.

_{The ai:* labels are advisory. They do not approve CI and they do not add tests:full. A new push runs this check again.}

[codecov]

Codecov Report

❌ Patch coverage is 88.39779% with 63 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
docling/backend/latex/handlers/macros.py	83.77%	16 Missing and 21 partials ⚠️
docling/backend/latex/handlers/metadata.py	88.77%	4 Missing and 7 partials ⚠️
docling/backend/latex/utils/text.py	87.03%	3 Missing and 4 partials ⚠️
docling/backend/latex/backend.py	95.50%	3 Missing and 1 partial ⚠️
docling/backend/latex/handlers/environments.py	88.46%	0 Missing and 3 partials ⚠️
docling/backend/latex/handlers/math.py	92.30%	0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

[github-actions]

AI first review (advisory): no blocker found

This review covers the full diff at 8d4f1f97d9. It read the code and did not run it.

The code changes are logically sound. The main risk is that the PR description is empty and does not explain why arXiv-2501.01300v2_main.tex.md loses 3 lines of content and one text item is dropped. The reviewer must confirm those removals are intentional improvements, not regressions.

_{A maintainer makes the merge decision. This review never approves the pull request.}

[github-actions]

AI first review (advisory): no blocker found

This review covers the changes since 8d4f1f97d9 at 17f5c19d36. It read the code and did not run it.

The bibliography change now builds each entry from _nodes_to_text, which handles \href differently from the old path. The reference data changed in 14 files and the description gives no before/after evidence. The reviewer must check the 2412.19437 and arXiv-2501.01300v2 diffs, and a \bibitem that contains \href.

Earlier findings

• open (question): 3 lines of Markdown output removed without explanation (arXiv-2501.01300v2_main.tex.md)

_{A maintainer makes the merge decision. This review never approves the pull request.}

[geoHeil]

Thanks for the update, @PeterStaar-IBM. I reviewed a375b3f2 against base 0e542942. I found five blockers below. Six small regression checks pass on the base and fail on this head. Please fix these before merge.

Validation:

• Python 3.10.19 with the locked dependencies, including docling-core 2.100.0.
• uv run --no-sync pytest -q -p no:tach tests/test_latex: 165 passed, 1 failed, 1 skipped. The failure is test_e2e_latex_conversions for 1706.03762/main.tex. The skipped test is the opt-in arXiv corpus test.
• Hosted core CI fails on the same test. The full local core suite and the streamed corpus were not run.
• Ruff, ty (with the current warning policy), Tach, skill routes, dprint, and the lock check pass. make validate reports no changed files.
• make check fails on 17 files over the line limit. The base fails on 16 of them. This PR adds tests/test_latex/test_macros.py at 1,031 lines; it has 804 lines on the base. Please split that test file too.

I also checked the earlier advisory questions. In arXiv-2501.01300v2, the removed 1, .001, and cas-model2-names come from layout definitions and the bibliography style. The removed [mode = title] is a title option. Those removals are expected. The tested bibliography link already exports the URL and title together on the base, so I did not report it as a new regression.

The reference outputs must be checked again after the fixes. The current end-to-end test stops at the first Markdown mismatch.

[DanielNg0729]

Thanks a lot Dr Peter, Looks good to me 🎉

[PeterStaar-IBM]

Thanks a lot Dr Peter, Looks good to me 🎉

Peter is just fine ;)

PS: you might want to look into https://github.com/docling-project/docling/blob/6509f8aea79a4e065647d85413dc3ecd911823c2/tests/test_latex/test_large_scale.py

I found already many source files that threw errors with it (it iterates over the dataset "secemp9/arxiv-complete"). However, it did not complete the entire dataset (only 07 and 08). I am sure many more bugs can be found this way.

[geoHeil]

Thanks for the update. I rechecked 6509f8aea79a4e065647d85413dc3ecd911823c2 against base 2f13b3d7f2217b457f646450844a9543233dce2f.

The two cases from my last comment now pass. Their new regression tests both fail on the previous head, 5e797f3d, for the expected reasons. The earlier comment/verbatim, inline-math, macro-argument, metadata, and caption-placement cases also pass. Two blockers remain.

Blocker (P2): preserve command boundaries when removing comments.

_strip_latex_comments() removes the comment and newline, then joins the surrounding source. This changes separate TeX tokens into one command:

\documentclass{article}
\newcommand{\brand}{Docling}
\newcommand{\term}[1]{\brand% comment
rocks #1}
\begin{document}
Important content. \term{now}
\end{document}

The base exports Important content. Doclingrocks now. This head exports Important content. now. The definition becomes \brandrocks #1; the next parse drops that unknown command and its text. The same step changes \alpha% comment followed by beta on the next line into \alphabeta, which changes the formula. Please preserve the command boundary and add a regression test. Adding a space after every comment would also change ordinary text such as First% comment followed by Second, so please keep that case too.

Blocker (P2): make the default DocLang export pass the schema.

Source-command retention adds invisible text items even for this input:

\documentclass{article}\begin{document}Body.\end{document}

The base's doc.export_to_doclang() passes XSD validation. This head emits <layer value="invisible"/>. The locked doclang 0.7.3 validator rejects it: The value 'invisible' is not an element of the set {'body', 'background', 'furniture'}. I used validate(path, allow_empty_namespace=True, xsd_only=True) on both outputs with the same environment.

The corpus test already filters out this layer. The default SDK export and CLI DocLang export do not use that filter. Caption placement is fixed, and its public body-layer export validates. Please also make the default public export schema-valid, with a regression test for this simple input.

Suggestion: update docs/usage/supported_formats.md and the PR description. They still describe meta.latex__classification; the code now stores roles in groups.

Validation:

• Python 3.10.19 with frozen dependencies, including docling-core 2.102.0.
• uv run --no-sync pytest -q -p no:tach tests/test_latex: 183 passed, 1 skipped. The skipped test is the opt-in corpus run. Public HTML caption and body-layer DocLang checks pass.
• Hosted code CI passes on this exact head, including the core tests on Python 3.10–3.14.
• Ruff, ty under the current warning policy, Tach, module coverage, skill routes, dprint, and the lock check pass. make validate reports no changed files.
• make check stops at 17 files over the line limit. The base reports the exact same files and line counts. This PR adds no line-limit failure.
• I did not run the full local core suite or the streamed arXiv corpus.

I still recommend changes before merge.

[dolfim-ibm]

lgtm

[geoHeil]

I rechecked 74f565eccbbe9debf4e76fb9792307236f127109 against base 2f13b3d7f2217b457f646450844a9543233dce2f.

The two cases from my last comment now pass. The four new regression cases fail on the previous head, 6509f8ae, for the expected reasons. The default public DocLang export now passes XSD validation. The earlier regression cases also pass.

Blocker (P2): preserve command arguments when removing comments.

_strip_latex_comments() inserts {} to keep a command boundary. For a command that takes an argument, this empty group becomes that argument and changes the formula.

This valid input shows the defect:

\documentclass{article}
\newcommand{\expression}{\sqrt% comment
x}
\begin{document}
$\expression$
\end{document}

The formula item and Markdown export contain \sqrt{}x. The root is empty, and x is outside it. The expected formula is \sqrt x.

The same step changes \frac followed by a comment and ab on the next line into \frac{}ab. The numerator becomes empty, a becomes the denominator, and b moves outside the fraction. The expected formula is \frac ab.

I checked both formulas with the installed MathML converter. Two focused tests compare each commented definition with its equivalent space-separated definition. Both tests fail on this head. Please preserve the command boundary without adding an argument. Add regression tests for both cases. Keep the existing zero-argument command and First% comment / Second cases too.

Validation on Python 3.10.19 with frozen dependencies, including docling-core 2.102.0:

• uv run --no-sync pytest -q -p no:tach tests/test_latex: 187 passed, 1 skipped. The skipped test is the opt-in arXiv corpus test.
• tests/test_verify_utils.py: 15 passed. Public HTML caption and default DocLang checks pass.
• Hosted code CI passes on this head.
• Ruff, ty under the current warning policy, Tach, module coverage, skill routes, dprint, and the lock check pass. make validate reports no changed files.
• make check stops at the same 17 files and line counts as the base. This PR adds no line-limit failure.
• I did not run the full local test suite or the streamed arXiv corpus.

I recommend changes before merge.