Repository navigation
fix: preserve rich heading context in hierarchical chunks - #924
PierreMesure wants to merge 1 commit into
Conversation
Render inline heading content once when capturing heading metadata. Consume the rendered children so that they do not become body chunks. Keep plain heading text unchanged and cover empty sections and sibling headings. Signed-off-by: Pierre Mesure <pierre@mesu.re>
|
✅ DCO Check Passed Thanks @PierreMesure, all your commits are properly signed off. 🎉 |
Merge Protections🔴 1 of 2 protections blocking · waiting on 👀 reviews
🔴 Require two reviewer for test updatesWaiting for
This rule is failing.When test data is updated, we require two reviewers
Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
geoHeil
left a comment
There was a problem hiding this comment.
Reviewed e98e5bde7f14f1ad68749cdbdc783061cbcc12cc. No blocker found.
A title or section header with an empty text and one InlineGroup, FormulaItem or CodeItem child now gets its heading text from that child. The child is marked as visited, so it no longer becomes a separate body chunk. This fixes the heading metadata defect in docling-project/docling#4908.
Validation on Python 3.10:
tests/test_hierarchical_chunker.py,tests/test_hybrid_chunker.py,tests/test_serialization.py: 286 passed, 1 xfailed.- With the base chunker, 9 of the 10 new cases fail. The tests detect the defect.
- Extra cases: an inline heading
my_var *x*with the formulaa_bgivesmy_var *x* $a_b$with no escaping. A heading with non-emptytextand an inline child keeps the base behavior.
Suggestion, not blocking: an empty heading with one plain TextItem child, or with several direct children, still gives the heading "". Base has the same behavior, and the DOTS and Nemotron output uses an InlineGroup.
CI passes at this head. I did not run the full repository suite.
Titles and section headers can store their rich text in an
InlineGroupwhile their owntextfield is empty.HierarchicalChunkerreads only that empty field, so body chunks lose their heading metadata. It also emits the heading's inline children as separate body chunks.Capture serialized inline heading content when the heading text is empty. Reuse the captured text for ordinary body chunks and for empty-section heading emission. Mark the inline children as visited so they do not become body chunks. Keep the existing text path for plain headings.
Fixes #925.
This addresses the heading metadata blocker in docling-project/docling#4908. Restoring the parent's text in the adapter alone is not sufficient: the current Markdown serializer then duplicates the heading and escapes math underscores. The downstream PR still needs a core release containing this fix and a dependency update.
Validation on Python 3.12.10:
make validatepasses for the changed core files, using the downstream checkout's development environment.No reference fixtures were regenerated. The full repository suites, hybrid chunker model/tokenizer suites and model inference were not run.