fix(jats): preserve styling in abstracts and footnotes by Kylinny · Pull Request #4361 · docling-project/docling

Kylinny · 2026-09-24T03:32:10Z

Part of #3583 — covers the remaining Styling (bold, italic, superscript, ...) item.

Problem

The JATS backend flattened abstract paragraphs and footnotes to plain text via _get_text() (which even carries a TODO: apply styling according to child.tag), so <italic>, <bold>, /, <underline> and <strike> were silently dropped there — while body paragraphs already preserved them since #3726.

Changes

Abstract  elements are now walked with the paragraph implementation (_walk_linear, from feat(jats/inline-formula): Support emphasis elements and Inline formulas #3726) at emit time, so abstracts share the body-paragraph styling path instead of a parallel one. This covers plain and structured abstracts, including mixed content.
_add_footnote_group() parses each <fn> with the shared inline walker (_walk_inline_formula + _strip_segments, from feat(jats/inline-formula): Support emphasis elements and Inline formulas #3726), keeping the FOOTNOTE label on text runs (formulas stay FORMULA).
No abstract/footnote-specific emphasis or spacing workaround: inline-group spacing is delegated to the docling-core serializers per their contract (docling-core#693). Markdown export may show temporarily suboptimal spacing (e.g. CO 2, *in vitro* .) until docs: add integrations, revamp docs #693 merges; ground truths will be regenerated then.
Unstyled abstracts/footnotes keep the exact previous document structure (single text item).

Tests

tests/test_backend_jats.py: plain-abstract styling (italic/bold/nested/sub/sup/underline/strike), structured-abstract styling, footnote styling incl. FOOTNOTE labels, a no-styling structure-preservation test, and inline-run contract tests (no run fusion, no punctuation movement, source whitespace preserved; markdown output deliberately not asserted).
Regenerated the affected e2e groundtruth files (.md/.itxt/.json for pmc2231364, pntd.0008301, pone.0234687); the other two fixtures are untouched.
ruff check / ruff format --check clean; ty check shows only the 5 pre-existing warnings; tach check passes. (Full uv prek could not run locally due to disk constraints on this machine; the individual hooks were verified equivalently.)

Parse abstract paragraphs and footnotes into styled inline runs (bold/italic/underline/strike/sub/sup) instead of flattening them to plain text, and emit them via _emit_inline so formatting is preserved in the document. Unstyled content keeps the previous single-item structure. Extends the JATS styling tests and regenerates the affected e2e groundtruth files. Part of docling-project#3583

github-actions · 2026-09-24T03:32:39Z

✅ DCO Check Passed

Thanks @Kylinny, all your commits are properly signed off. 🎉

mergify · 2026-09-24T03:32:58Z

Merge Protections

🟢 All 2 merge protections satisfied — ready to merge.

Show 2 satisfied protections

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

🟢 Require two reviewer for test updates

When test data is updated, we require two reviewers

#approved-reviews-by >= 2

I, Junyi Yao <j.yao@wustl.edu>, hereby add my Signed-off-by to this commit: a9a0960 Signed-off-by: Junyi Yao <j.yao@wustl.edu>

The docling-core serializers join sibling inline text items with a single space, so splitting a styled run off from adjacent text that had no whitespace in the source XML (e.g. `<italic>in vitro</italic>.` or `CO2`) exported spurious spaces (`in vitro .`, `CO 2`). Strip-then-join also destroyed the information of whether a boundary originally had whitespace. Add `_coalesce_no_space_boundaries`, run between whitespace normalization and boundary stripping in `_flattened_inline_runs` (abstracts and footnotes). At each spaceless run boundary it: 1. moves trailing opening punctuation (`(`, `[`, quotes) forward onto the next run; 2. moves leading closing punctuation (`.`, `,`, `)`, ...) backward onto the previous run; 3. fuses the two runs, keeping the non-baseline formatting when exactly one run carries it. Step 3 trades precise formatting scope for text fidelity: `CO2` keeps its subscript on the fused run instead of exporting `CO 2`. Formula runs and runs with different hyperlink targets are never fused. The body-paragraph path (`_emit_inline`) is untouched. Regenerate the three affected groundtruth fixtures; their exported text (modulo styling markers) is now byte-identical to the pre-PR output. Add `test_jats_plain_abstract_styling_keeps_source_spacing` covering italic+period/comma, `CO2e`, `(ECM))-1`, parens/brackets around styled runs, and ragged source whitespace. Signed-off-by: Junyi Yao <j.yao@wustl.edu>

PeterStaar-IBM

lgtm!

codecov · 2026-09-24T04:45:43Z

Codecov Report

❌ Patch coverage is 93.33333% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
docling/backend/xml/jats_backend.py	93.33%	0 Missing and 2 partials ⚠️

📢 Thoughts on this report? Let us know!

- Sort import blocks (I001) in jats_backend.py and test_backend_jats.py. - Mark intentional curly quotes in punctuation constants (RUF001 noqa). - Condense docstrings/comments to bring jats_backend.py back under the 1500-line max-lines guardrail (1496 lines). Signed-off-by: Junyi Yao <j.yao@wustl.edu>

ceberam

Thanks @Kylinny for addressing this open topic for JATS articles.
Without entering into a detailed review, I noticed some aspects that you may want to have a look at:

The style checks still fail. Please, check the Contributor guidelines and ensure they pass locally before pushing to the remote server, using uv prek.
Reading through your inline comments I have the impression that you are implementing a workaround to an existing issue in docling-core: the space management of the inline groups in the serializers. This is an existing issue that is currently being addressed in PR docling-project/docling-core#693 and it involves the DoclingDocument, not the backend parsers. I would suggest that you stick to the following contract: respect the blank spaces in the inline groups and delegate the proper (markdown) export to docling-core serializers. The serialization will not be optimal at this point, but we will regenerate the ground-truth files once PR docling-project/docling-core#693 is finalized and merged. Please, do not re-implement the emphasis parsing just for the abstract section. Instead, reuse the current implementation for paragraphs.
Small correction to your statement:

The JATS backend flattened abstract paragraphs and footnotes to plain text via _get_text()...were silently dropped there — while body paragraphs already preserved them since #4029.

The emphasis in paragraphs is actually supported since PR #3726 . The code from this PR should be the basis of your implementation to support styling in abstracts and footnotes.

Address ceberam's review: drop the abstract/footnote-specific emphasis and spacing workaround (_normalize_segments, _coalesce_no_space_boundaries, _flattened_inline_runs) and reuse the paragraph styling implementation from docling-project#3726 instead. - Abstract elements are now walked with _walk_linear at emit time, sharing the body-paragraph code path. - Footnotes use _walk_inline_formula + _strip_segments directly. - Inline-group spacing is delegated to the docling-core serializers per their contract (docling-core#693); no fusing or punctuation movement. - Tests assert raw inline-run content/formatting instead of markdown output. - Ground truths regenerated for the new (temporarily suboptimal) spacing. Signed-off-by: Junyi Yao <j.yao@wustl.edu>

Kylinny · 2026-09-24T16:29:25Z

Thanks for the detailed review, @ceberam — all points addressed in the latest push (15fdd1e):

Dropped the workaround entirely: _normalize_segments, _coalesce_no_space_boundaries, _flattened_inline_runs, and the punctuation constants are gone.
Reuse feat(jats/inline-formula): Support emphasis elements and Inline formulas #3726's implementation: abstract  elements are now walked with _walk_linear at emit time, so abstracts share the body-paragraph code path; footnotes use _walk_inline_formula + _strip_segments directly. No run fusion, no punctuation movement.
Spacing delegated to docling-core: per the docling-core#693 contract, inline runs keep their source whitespace and serialization spacing is the serializers' job. Markdown may look suboptimal until docs: add integrations, revamp docs #693 lands (e.g. CO 2, *in vitro* .); ground truths were regenerated accordingly and will be refreshed once more after docs: add integrations, revamp docs #693.
PR description corrected: paragraph emphasis support is attributed to feat(jats/inline-formula): Support emphasis elements and Inline formulas #3726, not feat(jats): preserve external hyperlinks #4029.
Tests now assert raw inline-run content/formatting (no fused runs, style scopes faithful) rather than markdown output.
Checks: ruff check, ruff format --check, tach check clean; ty check shows only the 5 pre-existing warnings; all 62 JATS tests pass. (I verified each prek hook individually — the full uv prek run wasn't possible on this machine due to disk space, but every hook it runs was checked equivalently.)

Signed-off-by: Junyi Yao <j.yao@wustl.edu>

ceberam

Thanks @Kylinny for accepting the challenge and refactor the PR to strengthen the emphasis handling. I have some comments below, please check.

Signed-off-by: Junyi Yao <j.yao@wustl.edu>

Kylinny · 2026-10-05T22:59:35Z

Thanks @ceberam for the detailed review! All four points are addressed:

Footnotes now delegate to _emit_inline unconditionally, so single-segment footnotes skip the inline group (consistent with the abstract path) while keeping the FOOTNOTE label.
Review-context comments removed; the docling-core#693 reference is kept to a single brief mention.
Both docstrings trimmed to single-line Google style.
_normalize_xml_line_breaks is now split/join based; import re and the regex constant are gone. One note: I kept collapsing consecutive line breaks to a single space to match the old regex behavior (e.g. "a\n\n b" -> "a b"), covered by the new parametrized test.

Also added the missing footnote single-segment test you pointed out. lint-and-type is green.

ceberam

Thanks @Kylinny , all good now 🚀

Kylinny mentioned this pull request Sep 24, 2026

Improve parsing of JATS documents #3583

Closed

8 tasks done

Kylinny added 2 commits September 23, 2026 20:34

DCO Remediation Commit for Junyi Yao <j.yao@wustl.edu>

338a8fa

I, Junyi Yao <j.yao@wustl.edu>, hereby add my Signed-off-by to this commit: a9a0960 Signed-off-by: Junyi Yao <j.yao@wustl.edu>

PeterStaar-IBM requested a review from ceberam September 24, 2026 04:39

PeterStaar-IBM previously approved these changes Sep 24, 2026

View reviewed changes

Kylinny dismissed PeterStaar-IBM’s stale review via 7b99568 September 24, 2026 05:36

ceberam added the xml issue related to supported schema-specific XML formats label Sep 24, 2026

ceberam requested changes Sep 24, 2026

View reviewed changes

ceberam reviewed Sep 25, 2026

View reviewed changes

Comment thread tests/data/jats/groundtruth/pmc2231364.nxml.md Outdated

fix(jats): normalize XML line-break indentation

c50b37b

Signed-off-by: Junyi Yao <j.yao@wustl.edu>

Kylinny force-pushed the jats-abstract-footnote-styling branch 2 times, most recently from 21270ae to c50b37b Compare October 2, 2026 19:34

ceberam requested changes Oct 5, 2026

View reviewed changes

Comment thread docling/backend/xml/jats_backend.py Outdated

Comment thread docling/backend/xml/jats_backend.py Outdated

Comment thread docling/backend/xml/jats_backend.py Outdated

Comment thread docling/backend/xml/jats_backend.py Outdated

Kylinny added 2 commits October 5, 2026 13:15

fix(jats): address abstract review feedback

5ac1b45

Signed-off-by: Junyi Yao <j.yao@wustl.edu>

fix(jats): preserve footnote item labels

7c47c9b

Signed-off-by: Junyi Yao <j.yao@wustl.edu>

ceberam approved these changes Oct 6, 2026

View reviewed changes

dolfim-ibm approved these changes Oct 6, 2026

View reviewed changes

ceberam merged commit 893fdd1 into docling-project:main Oct 6, 2026
26 checks passed

matssun mentioned this pull request Oct 13, 2026

JATS: footnotes in table-wrap-foot and author-notes are dropped #4796

Open

cau-git mentioned this pull request Oct 26, 2026

feat: honor the InlineGroup whitespace contract in the backends #3891

Open

1 of 5 tasks

Conversation

Kylinny commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Changes

Tests

Uh oh!

github-actions Bot commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mergify Bot commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Protections

🟢 Enforce conventional commit

🟢 Require two reviewer for test updates

Uh oh!

PeterStaar-IBM left a comment

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ceberam left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Kylinny commented Sep 24, 2026

Uh oh!

Uh oh!

ceberam left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Kylinny commented Oct 5, 2026

Uh oh!

ceberam left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Kylinny commented Sep 24, 2026 •

edited

Loading

github-actions Bot commented Sep 24, 2026 •

edited

Loading

mergify Bot commented Sep 24, 2026 •

edited

Loading

codecov Bot commented Sep 24, 2026 •

edited

Loading

ceberam left a comment •

edited

Loading