Repository navigation
Conversation
docling-core 2.104.0 escapes CodeItem text in the HTML serializer. Older versions let the Markdown backend's HTML round trip decode code text a second time, so `&` inside code became `&`. The Markdown groundtruth already expects the literal code text, which only 2.104.0 produces, and the lock already resolves 2.104.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
The Markdown backend puts raw HTML blocks into CodeItem text, serializes the document to HTML, and strips the <pre><code> wrapper to splice the HTML back. Since docling-core 2.104.0 escapes code text, convert() unescapes each block once. That assumes the serializer escaped it exactly once: with docling-core 2.103.1, which the floor allowed, entities in the block were decoded twice (`&` became `&`). Store the raw blocks on the backend and put only an index between the markers. convert() replaces each marked <pre><code> block with the stored HTML, so the result no longer depends on how the serializer treats code text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
|
✅ DCO Check Passed Thanks @cau-git, all your commits are properly signed off. 🎉 |
Merge Protections🟢 Merge protection satisfied — ready to merge. Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
AI triage (advisory)Checked commit This PR stores raw HTML blocks in a list and inserts numeric index placeholders into code items during Markdown conversion. It restores the stored HTML blocks by index to prevent incorrect entity decoding. Topic labels added: CI safety: needs maintainer care before you approve CI.
Possible duplicates
1 related pull request(s) in the same code
The |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Summary
The Markdown backend converts documents that contain raw HTML blocks by round-tripping through HTML. It puts each raw block into a
CodeItembetween markers, serializes the document to HTML, and swaps each marked<pre><code>block back for the raw HTML.Since docling-core 2.104.0 (docling-core#894) the HTML serializer escapes code text. #4771 adapted to this by unescaping each marked block once. That assumes the serializer escaped the block exactly once, which only holds for 2.104.0 and later:
main&becomes&Change
fix(md): the backend stores the raw blocks in a list and puts only an index between the markers.convert()replaces each marked block with the stored HTML, so the result no longer depends on how the serializer treats code text. That includes any future change to its escaping.build: raises the floor todocling-core>=2.104.0. The Markdown groundtruth (escaped_characters.md) already expects literal&inside code spans and code blocks. Only 2.104.0 produces that; older cores decode code text a second time in the round trip. The lock already resolves 2.104.0, so only the specifiers change.Testing
test_html_block_entities_are_decoded_exactly_once: the entities in a raw HTML block reach the output decoded exactly once.escaped_charactersgolden, which needs 2.104.0; that is why the floor is raised.Checklist:
🤖 Generated with Claude Code