Skip to content

fix(reading-order): keep a detached hyphen when merging elements - #4180

Open
VSathveek wants to merge 1 commit into
docling-project:mainfrom
VSathveek:fix/merge-elements-detached-hyphen
Open

VSathveek wants to merge 1 commit into
docling-project:mainfrom
VSathveek:fix/merge-elements-detached-hyphen

Conversation

@VSathveek

Copy link
Copy Markdown
Contributor

What

Follow-up to #4052, which fixed the same defect in PageAssembleModel.sanitize_text.
As noted there, ReadingOrderModel._merge_elements has the parallel bug for merges
across cluster boundaries.

When two elements are merged, a trailing hyphen on the first is treated as a word
split and dropped whenever the continuation starts lowercase:

if new_item.text.endswith("\u00ad") or (
    new_item.text.endswith("-")
    and continuation_text
    and continuation_text[0].islower()
):
    new_item.text = new_item.text[:-1] + merged_elem.text

But a hyphen only splits a word when it is attached to that word ("algo-" /
"rithms"). A hyphen that follows whitespace is a literal character -- a dangling
dash, a wrapped CLI flag, a minus sign -- and dropping it silently corrupts the
text.

Fix

Require the character before the trailing hyphen to be alphanumeric before treating
it as a word split. The soft-hyphen (U+00AD) branch is unchanged -- a soft hyphen is
always a split marker. When the hyphen is detached the elements are joined with a
space, exactly as the existing else branch already does, so the character is kept.

This only narrows the existing condition, so nothing de-hyphenated today stops being
de-hyphenated.

Tests

Extended tests/test_readingorder_hyphenated_merges.py with two detached-hyphen
cases ("run -" + "prio now", "the value is -" + "five units"). Both fail on
main and pass with this change; the three existing cases pass either way.

No reference data changes: this only affects cross-element merges where a detached
hyphen meets a lowercase continuation, which none of the e2e fixtures exercise.

Signed-off-by: Varanasi Sathveek <sathveekvaranasi@gmail.com>
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

✅ DCO Check Passed

Thanks @VSathveek, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@codecov

codecov Bot commented Sep 14, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@VSathveek

Copy link
Copy Markdown
Contributor Author

Gentle ping when someone has a moment — this is the _merge_elements counterpart to #4052 (same attached-hyphen check, applied to the cross-element merge path), with two regression cases. It still merges cleanly with main and doesn't touch reference data. @ceberam, since you reviewed #4052, you may have the most context. Thanks!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants