Repository navigation
feat(pdf): mark superscript and subscript text from glyph size and baseline - #4815
yashikam19 wants to merge 7 commits into
Conversation
…seline
A PDF draws superscript and subscript text as smaller glyphs that are moved
off the baseline. The PDF pipeline keeps them as plain text, so "E = mc²"
becomes "E = mc 2" and "H₂O" becomes "H 2 O".
Add ScriptPositionOptions, off by default. When it is on, a new stage runs
after the heading-hierarchy stage. It reads the baseline and the drawn size
of each character with pypdfium2, for each PDF backend. A character is a
superscript or subscript when it is smaller than the body text of its line
and raised or lowered against it, and it touches that text. A text item
with such characters becomes an inline group of runs with formatting.script,
as the Word backend produces. The stage removes the space that the text-line
builder adds before a script ("mc 2") when the PDF draws the two characters
side by side.
Body text, list items and footnotes are handled. Headings, captions, tables,
code, formulas and OCR text do not change. With the option off, the output
does not change.
Signed-off-by: Yashika Malhotra <106444881+yashikam19@users.noreply.github.com>
…aracters Pick the body of a line from the groups set in the largest letters, so a large symbol, such as a sum sign, does not make the text around it a superscript. A line now ends where the baseline moves by most of a line height or where the pen returns to the start of the line. A superscript stacked over a wide subscript stays on its line. A script must be moved by less than its own size. This rule replaces the drop-cap rule, so a citation on the short last line of a paragraph is marked. Measure only the characters that match the item's text, so characters from a nearby line do not change the result. Join a surrogate pair from PDFium into one character. Compute every rewrite before the stage changes the document, so a failure leaves the document unchanged. Read the source and the password from typed attributes instead of with getattr. Signed-off-by: Yashika Malhotra <106444881+yashikam19@users.noreply.github.com>
A run has no whitespace at its edges, as in the Word backend. The export puts one space between runs, so a converted PDF and a Word document give the same text, and no double space appears Signed-off-by: Yashika Malhotra <106444881+yashikam19@users.noreply.github.com>
Signed-off-by: Yashika Malhotra <106444881+yashikam19@users.noreply.github.com>
|
✅ DCO Check Passed Thanks @yashikam19, all your commits are properly signed off. 🎉 |
Merge Protections🔴 1 of 2 protections blocking · waiting on 👀 reviews
🔴 Require two reviewer for test updatesWaiting for
This rule is failing.When test data is updated, we require two reviewers
Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
github-actions
Bot
added
ai:ci-needs-care
AI triage (advisory)Checked commit Warning The AI analysis did not finish. Only the deterministic checks are shown. CI safety: needs maintainer care before you approve CI.
Possibly related pull requests
The |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Signed-off-by: Yashika Malhotra <106444881+yashikam19@users.noreply.github.com>
…subscript Signed-off-by: Yashika Malhotra <106444881+yashikam19@users.noreply.github.com> # Conflicts: # docs/usage/advanced_options.md
github-actions
Bot
added
ai:ci-needs-care
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
AI first review (advisory): no blocker found
This review covers the full diff at 4ca9071696. It read the code and did not run it.
New opt-in stage that marks superscript and subscript text in PDF documents. The feature is off by default. The algorithm, locking, and document-mutation order are correct. No regressions to existing behavior. The reviewer should confirm that standalone page footnotes being rewritten into inline groups (not just footnotes referenced by tables/pictures) is the intended scope.
A maintainer makes the merge decision. This review never approves the pull request.
github-actions
Bot
added
the
ai:review-lgtm
|
@dolfim-ibm @PeterStaar-IBM @cau-git, this PR is ready for review. The branch is up to date with Analysis on the current head (
|
| Labels on PDFs with a text layer (263 labels, 56 PDFs) | Option off | Option on |
|---|---|---|
| Superscript (257) | 0 | 239 |
| Subscript (6) | 1 | 6 |
| Total | 1 | 245 (93%) |
- 10 more labels are marked, but the text inside the mark differs a little from the label (a space or a different dash from the PDF text layer), so the rule does not match. With these, 255 of 263 (97%) are marked.
- 8 labels are not marked. 4 are in headings or table cells, which the option does not change. 2 are not in the PDF text layer. 1 is a footnote marker on a line with no other letters (the text before it is redacted). For 1 label, I could not find the character it means.
- 68 more labels are on scans, image-only pages, or broken text layers. The option does not change OCR text, so they are out of scope.
False marks: the option never marked normal text
| Set | Marks | Wrong |
|---|---|---|
| The 87 labelled PDFs | 348 | 0 |
| 100 random PDFs without labels | 2 | 0 |
- There are more marks than labels, because a document often has the same superscript more than once. For example, a footnote number is marked in the sentence and again at the footnote.
- Most marks match a label. I read the others by hand, and all of them are real superscripts.
- On the 100 PDFs without labels, the only 2 marks are
m², which is correct.
Cost: for the 1-page PDFs, there is no measurable difference between off and on (about 0.25 s per document). On multi-page PDFs from tests/data: +0.26 s on 9 pages and +0.12 s on 18 pages.
Method and limits
- The scores use ParseBench's
FormattingRule, which passes a label when the output has<sup>text</sup>or<sub>text</sub>. - OCR was off, so scans have no text, and their labels cannot pass. The option does not change OCR text in any case.
- The 100 PDFs without labels were sampled at random from the ParseBench text documents.
- The timings come from single runs on one machine.
A note on the export
The option sets formatting.script on the runs in the DoclingDocument. The exporters show the runs as follows:
| Export | Option off | Option on |
|---|---|---|
| HTML | <p>Water is H2O …</p> |
<span class='inline-group'>Water is H <sub>2</sub> O …</span> |
| Markdown | Water is H2O … |
Water is H 2 O … |
docling-core's inline-group serializer puts a space between runs, and it writes an inline group as a <span> and not as a <p>. The Word backend shows the same output. So with the option on, Markdown gets a space around each subscript that had none in the PDF. Should I fix the spacing in docling-core first, or is the option acceptable as it is, since it is off by default?
Possible follow-ups
These are out of scope for this PR. I can open issues or PRs for them if they are useful.
- Headings and table cells. Ordinals in dates (
22nd) and units in tables (m²) are common there. This PR excludes them to keep the change small. - Spacing between runs in the HTML and Markdown export, so that
H₂OstaysH2Oin Markdown and becomesH<sub>2</sub>Oin HTML. This change belongs in docling-core. - Superscript and subscript in the Markdown export, for example as an option to write
<sup>and<sub>tags. This change belongs in docling-core. - OCR text. OCR engines give word boxes but no character baselines, so scans need a different method.
geoHeil
left a comment
geoHeil
left a comment
There was a problem hiding this comment.
Reviewed 4ca90716963b154a9da48dc1869a4fff025c55ff. Two output defects remain.
- _run_prov keeps only the first page of a matched run. In a valid constructed DoclingDocument with H2O on page 1 and a continuation on page 2, the rewrite drops all page-2 provenance. Keep the provenance for each page. This reproduction checks the document model and stage; it is not a full cross-page PDF conversion.
- _apply replaces paragraph blocks with top-level InlineGroup nodes. HTML changes two source paragraphs from separate
<p>blocks to adjacent<span>elements. The browser loses their paragraph separation. Keep the paragraph blocks.
Local validation: 35 tests pass, and both defects reproduce. Current CI and Docs CI pass. Codecov patch coverage is 88.27930%, but the tests do not detect these defects. The spacing and export question in the author's comment also remains open.
Local make check stops on file-length violations that already exist on base. The other read-only checks pass. I made no source or reference-data changes.
Signed-off-by: Yashika Malhotra <106444881+yashikam19@users.noreply.github.com>
github-actions
Bot
added
ai:ci-needs-care
|
Thanks @geoHeil, both fixed in
Both new tests fail on In the same commit, the step now skips image input. Before, the PDF pipeline logged a false "recovery failed" warning for every image with the option on. Two limits that come from docling-core's inline-group serializer, so I left them to the maintainers: the space between runs ( Could you re-review when you have time? |
|
@yashikam19 I really like this output, but looking at your table,
I would like to make sure that we do not have spaces (eg |
|
Thanks @PeterStaar-IBM, agreed: there should be no space between a script and the text it touches. The PDFs confirm it: in the test fixture, The space comes from docling-core: its HTML and Markdown serializers join the runs of an inline group with How would you like to handle this? Should this PR wait for docling-project/docling-core#843 and then require that docling-core version, or do you prefer another approach? |
OK, then I would first like to
|
geoHeil
left a comment
geoHeil
left a comment
There was a problem hiding this comment.
Reviewed 2aaedddd49f9eb35bec634c068d16780de2ea801. The two defects from my last review are fixed. One blocker remains: the spacing that @PeterStaar-IBM raised. It also changes Markdown, plain text, and chunks, not only HTML.
Fixed
- Each run now has one provenance item for each page, with its
charspanon that page.test_run_crossing_a_page_break_keeps_both_pagesfails with the stage code at4ca90716and passes at this head. - A rewritten paragraph keeps its place and holds the inline group. HTML keeps two
<p>blocks.test_paragraphs_stay_separate_blocks_in_htmlfails at4ca90716and passes at this head. - Image input is skipped.
test_image_input_is_skipped_without_a_warningalso fails at4ca90716and passes at this head.
Blocker: with the option on, text with no space in the PDF gets spaces in every text export
I converted tests/data/pdf/script_position.pdf with the default backend, OCR off, and the option off and on:
| export | option off | option on |
|---|---|---|
| Markdown | Water is H2O and carbon dioxide is CO2. |
Water is H 2 O and carbon dioxide is CO 2 . |
| plain text | Water is H2O and carbon dioxide is CO2. |
Water is H 2 O and carbon dioxide is CO 2 . |
| HTML | Water is H2O and carbon dioxide is CO2. |
Water is H <sub>2</sub> O and carbon dioxide is CO <sub>2</sub> . |
The Markdown and plain-text exports do not show the script, so in these formats the option only adds spaces. HierarchicalChunker chunks change in the same way. This does not agree with the statement that "Markdown, plain text and chunking keep the same words". Also, the PR description says that main converts H₂O to H 2 O. For this fixture, main gives H2O.
The cause is in docling-core: its serializers join the runs of an inline group with " ", and the runs drop their edge whitespace. As @PeterStaar-IBM decided, wait until docling-project/docling-core#693 is released. Then let each run keep its own whitespace, require that docling-core version, and recreate the ground truth. Please add a test that asserts the exact Markdown and HTML output for the H2O line, so that the space cannot come back.
Non-blocking
- When a paragraph has a hyperlink, each run becomes a separate link. In
2206.01062.pdf, the footnote[1 https://developer.ibm.com/…](https://developer.ibm.com/…)becomes[1](https://developer.ibm.com/…) [https://developer.ibm.com/…](https://developer.ibm.com/…). You already reported this limit. - Question: in DocTags, each rewritten item becomes a nested
<inline>block with one<text>element for each run, and the runs have no script mark. The words do not change. Please confirm that this DocTags structure is intended.
Validation
tests/test_script_position.pyandtests/test_script_position_pdf.py: 38 passed at this head. The 3 new tests fail with the stage code from4ca90716.- I compared the exports with the option off and on for the fixture,
2206.01062.pdf,elsevier-00.pdf, andnormal_4pages.pdf. For the three real PDFs, plain text, HTML, and DocTags keep the same words. Only the link split above changes the Markdown of2206.01062.pdf. A JSON round trip keeps the Markdown export for all four files. ruff format --check,ruff check,ty check,tach check, the tach module coverage check, the skill-route check,dprint check, anduv lock --lockedpass.check_max_linesfails only on files that already exceed the limit on base.- CI and Docs CI pass at this head on Python 3.10 to 3.14. Mergify needs two approvals because the PR adds test data.
- Not run: the export comparison with OCR, with table structure, or with the pypdfium2 backend. The PR tests cover both PDF backends.
|
Thanks @geoHeil for the careful re-review. You are right about the exports, and I need to correct my earlier statement. I compared the words without spaces. With spaces, the option changes As @PeterStaar-IBM decided, I will wait for the docling-core release with docling-project/docling-core#693. Then I will let each run keep its own whitespace from the PDF, require that docling-core version, recreate the ground truth, and add a test that asserts the exact Markdown and HTML output of the On DocTags: yes, this structure is expected. It is docling-core's DocTags output for an inline group, and the Word backend gives the same |
|
Tested 2aaeddd with docling-core 2.104.0 on a real document: Finanstilsynet's guidelines for recovery plans (Norwegian FSA, 22 pages, 31 numbered footnotes). With The Markdown is the same as on |
Summary
Add an option that marks superscript and subscript text in PDFs. It is off by default.
Today,
mainconvertsE = mc²toE = mc 2and21stto21 st, andH₂Oloses its subscript and becomesH2O. With the option on, these characters become runs withformatting.scriptset tosuperorsub, as Docling already does for Word documents.A PDF has no label for superscript text. So the new step measures each character: a superscript is smaller than its line and drawn higher, a subscript is smaller and drawn lower.
Changes
docling/utils/pdf_char_geometry.py: reads the size and the baseline of each character with pypdfium2, for both PDF backends. Invisible text (render modes 3 and 7) is skipped.docling/utils/script_position.py: finds the characters that are smaller than their line, raised or lowered, and next to the text they mark.docling/models/stages/script_position/: a new stage after the heading-hierarchy stage. A text item that has such characters keeps its place and holds an inline group of its runs. A list item keeps its marker. Each run has one provenance item for each page it is on.ScriptPositionOptionsinpipeline_options.py, a docs section indocs/usage/advanced_options.md, and the new modules intach.toml.Body text, list items and footnotes are handled. Headings, captions, tables, code and formulas do not change. OCR text, invisible text, and image input are left unchanged.
Issue resolved by this Pull Request:
Resolves #520
Checklist:
Review evidence:
Before and after, with the default backend.
mainis the Markdown export; this branch is the HTML export with the option on.mainmc 2,H2O,21 stmc <sup>2</sup>,H <sub>2</sub> O,21 <sup>st</sup>elsevier-00.pdf, authorsDuarte a,b,c,*Duarte <sup>a,b,c,*</sup>2206.01062.pdf, footnote1 https://developer.ibm.com/…<sup>1</sup> https://developer.ibm.com/…normal_4pages.pdf, Korean note markers상해 1) 나상해 <sup>1)</sup> 나Known limit until docling-core changes: docling-core's serializers join the runs of an inline group with
" ". So with the option on, a script inside a word gets spaces in every text export:H2ObecomesH 2 Oin Markdown, plain text and chunks, andH <sub>2</sub> Oin HTML. As agreed with @PeterStaar-IBM, this PR will update to the docling-core release with feat: make inline serializers concatenate runs faithfully docling-core#693, let the runs keep only the real spaces from the PDF, and recreate the ground truth.Screenshots: PDF page,
main, this branch (rendered)Reference data: no change. The option is off by default.
Tests:
tests/test_script_position.py: text that must be marked, text that must not be marked (such as%,°, small caps and the LaTeX logo), the stage on aDoclingDocument, a run that crosses a page break, and two paragraphs in a row in HTML.tests/test_script_position_pdf.py: a PDF exported from Word, rotated and cropped pages, invisible text, and image input, with both backends.Commands and results:
uv run pytest tests/test_script_position.py tests/test_script_position_pdf.py: 38 passed.uv run pytest tests/test_options.py tests/test_heading_hierarchy.py tests/test_heading_hierarchy_pdf.py tests/test_backend_docling_parse.py tests/test_backend_docling_parse_legacy.py tests/test_backend_pdfium.py tests/test_backend_pdfium_rotation.py tests/test_pdf_outline.py tests/test_font_style.py: 168 passed.make validate: passed.