Repository navigation
fix(html): resolve the line break sentinel outside paragraphs - #4675
Conversation
A `<br>` is replaced by a sentinel character (U+E000), which `split_by_newline()` and `get_text()` turn into a line break for paragraphs and table cells. Headings, captions, list items, `<pre>` blocks and the text of checkboxes did not resolve it, so the character was left in the text and in the exports. These items now get a line break, too. In a heading, a caption or a list item, consecutive `<br>` tags give one line break. In a `<pre>` block, each `<br>` is a newline. The text of a checkbox stays on one line. A run that only holds `<br>` tags gives no item of its own. Signed-off-by: Morten Dæhli Aslesen <morten@lagabote.no>
|
✅ DCO Check Passed Thanks @morten-lagabote, all your commits are properly signed off. 🎉 |
Merge Protections🟢 Merge protection satisfied — ready to merge. Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
github-actions
Bot
added
ai:ci-safe
AI triage (advisory)Checked commit The PR fixes the HTML backend leaving the internal U+E000 CI safety: no concern found. A maintainer must still approve CI. 2 related pull request(s) in the same code
The |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Signed-off-by: Morten Dæhli Aslesen <morten@lagabote.no>
github-actions
Bot
added
ai:ci-safe
The backend replaces each
<br>with a sentinel character (U+E000).split_by_newline()andget_text()turn it into a line break, for paragraphs and table cells. Four other places did not, so the character was left in the text and in the exports: headings and captions (to_single_text_element()), list items (_add_list_item_with_content()),<pre>blocks, and the text of checkboxes (_extract_text_excluding_tag_obj_ids()).These places now resolve the sentinel:
\n), as in a paragraph. Consecutive<br>tags give one line break, because these items hold one paragraph.<pre>block: each<br>is a newline._normalize_checkbox_text()already collapses whitespace.A run that only holds
<br>tags gives no item of its own. So in an item with formatting, a<br>between two runs gives no line break, as in a paragraph with formatting today. An<li>or a heading that only holds a<br>no longer gets the sentinel as its text.#4670 changes another part of
to_single_text_element(). The two changes merge without a conflict, and the HTML and EPUB tests pass on the combination.Issue resolved by this Pull Request:
Resolves #4671
Checklist:
Review evidence:
<h2>Line one<br>Line two</h2>gives the textLine one Line twoand the Markdown## Line one Line two.Line one\nLine two. The Markdown is## Line one Line twofor a heading and- Line one \nLine twofor a list item (fix(markdown): serialize newlines as GFM hard line breaks docling-core#721).<br>in 47 kinds of elements onmainand on this branch. Onmain, 17 of them have U+E000 in the document or in the Markdown or HTML export. On this branch, none has. Paragraphs and table cells do not change. A<br>between two formatted runs (15 shapes) leaves no empty item.test_html_line_break_outside_paragraphsfails without the fix and passes with it.HTMLDocumentBackend(HTML, MHTML, EPUB, email, Markdown, XBRL, JATS, USPTO, PPTX) pass with the existing ground truth: 324 passed, 1 skipped.