Repository navigation
fix(md): keep backslash escapes inside the surrounding text - #4991
Krina-Paghadar wants to merge 2 commits into
Conversation
Marko splits an escaped character (snake\_case, \$5, 1\.) into its own Literal node. Each node became a separate text item of an inline group and the exporters join those items with a space, so the text came out as "snake _ case" and "1 . not a list". Merge text runs that touch each other before the items are created. Signed-off-by: Krina-Paghadar <krina007.paghadar@gmail.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
|
✅ DCO Check Passed Thanks @Krina-Paghadar, all your commits are properly signed off. 🎉 |
Merge Protections🔴 1 of 2 protections blocking · waiting on 👀 reviews
🔴 Require two reviewer for test updatesWaiting for
This rule is failing.When test data is updated, we require two reviewers
Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
AI triage (advisory)Checked commit The PR fixes backslash escapes in the Markdown backend. It adds CI safety: no concern found. A maintainer must still approve CI. 3 related pull request(s) in the same code
Reference data changes
Assessment: expected. The fix merges previously split text nodes, so existing reference outputs that recorded the wrong spaced text (e.g. The |
There was a problem hiding this comment.
AI first review (advisory): no blocker found
This review covers the full diff at 57075d90dd. It read the code and did not run it.
The PR fixes a real bug: marko splits backslash escapes into separate Literal nodes, causing exporters to add spaces around them. The merge-before-walk approach is correct. The main thing to check is the & round-trip: a Literal('&') node is stored as & in the merged RawText so that the later unescape() call at md_backend.py:884 converts it back to &; the test at line 1333 confirms this path.
A maintainer makes the merge decision. This review never approves the pull request.
|
Ready for review. The 3 failing Markdown tests also fail on main with docling-core 2.104.0 (see #4982). Happy to adjust anything. |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
@Krina-Paghadar we needto fix the regression-tests |
The groundtruth held the old output of the backend: "2 . strange" for an escaped marker, and inline groups numbered in the old order. Regenerated with docling-core 2.103.1 (the locked version) and DOCLING_GEN_TEST_DATA=1. The only content change is "2. strange" and the marker of an ordered list item that has an escaped character. The other lines are renumbered group references. Signed-off-by: Krina-Paghadar <krina007.paghadar@gmail.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Thanks @PeterStaar-IBM. The stored groundtruth still held the old buggy output (2 . strange). I regenerated it with the locked docling-core 2.103.1 (DOCLING_GEN_TEST_DATA=1) and pushed it in 48f1cf3. The content changes are 2. strange and one ordered list item that now gets the marker 1.. The rest is renumbered group references. All Markdown tests pass locally with the locked version. Could you please re-run the CI? |
Issue resolved by this Pull Request:
Resolves #4990
marko parses a backslash escape (
snake\_case,\$5,1\.) as aLiteralnode between twoRawTextnodes. The Markdown backend made one text item per node and wrapped them in an inline group, and the exporters join the items of an inline group with a space. The text came out assnake _ case.This change merges
RawText/Literalruns that touch each other into one run (_merge_adjacent_text_runs) before the items are created. An escaped&stays literal text. Inline code and emphasis stay separate items.Checklist:
Review evidence:
Before:
snake\_case hereexports assnake \_ case here;1\. not a listexports as1 . not a list. After:snake_case here,1. not a list.Checks: the new test
test_backslash_escapes_stay_in_their_text_runfails without the change and passes with it. Intests/test_backend_markdown.py, 40 pass and 3 fail, and the same 3 fail onmain(docling-core 2.104.0, see #4982).ruff checkandruff formatpass.