Repository navigation
PDF: URL char-by-char spacing + metadata field leakage in references #3882
Description
Bug
Two related text-spacing artifacts in PDF extraction:
Artifact 1 — URL character spacing: URLs in reference sections are rendered with single-character spacing, where each character is separated by a space:
h t t p s : / / d o i . o r g / 1 0 . 1 0 1 6 / j . p e q u i n . 2 0 2 3 . 1 2 . 0 0 2
Artifact 2 — Metadata field leakage: PDF metadata field names (WGROUP, STRING, PUBLICATION, PAGE, etc.) are extracted as spaced-character text and appear in the references section:
W G R O U P : S T R I N G : P U B L I C A T I O N
P A G E : S T R I N G : A R T I C L E / C H A P T E R
Steps to reproduce
from docling.document_converter import DocumentConverter
converter = DocumentConverter()
result = converter.convert("verkerk_forensic_necropsies_2026.pdf")
md = result.document.export_to_markdown()
# Artifact 1: URLs have character spacing
assert "h t t p s" in md # Bug: URL mangled
# Artifact 2: metadata fields leaked as text
assert "W G R O U P" in md # Bug: metadata leakedExpected behavior
- URLs should be extracted as clean URLs:
https://doi.org/10.1016/... - PDF metadata fields should not appear in the text output — they are structural, not content
Source location
- URL spacing:
page_assemble_model.py:118 sanitize_text— no URL-aware run un-spacing. The PDF producer stored URLs with explicit character spacing (one glyph perTjoperator), and the extractor normalized these to word breaks. - Metadata leakage:
backend/docling_parse_backend.py— the backend pulls all text from the page without discarding/Artifactmarked-content or non-ContentsTjops.
Suggestion
- For URLs: detect runs matching URL patterns (
https?://,doi.org) and collapse inter-character spaces - For metadata: discard text runs matching known PDF metadata field patterns (single-char-spaced uppercase sequences like
Type,Subtype,Filter,WGROUP,STRING)
Docling version
2.115.0
Regression test PDF
verkerk_forensic_necropsies_2026.pdf(4.4 MB) — Springer journal article, both artifacts visible in the references section
Activity
dosubot commentedon Jul 26, 2026
Hi @bfuerea! I'm Dosu and I'm helping the IBM Docling team.
Both issues you've identified trace back to the
docling-parseC++ library, not the Python-side code. Here's what I found:Artifact 1 — URL character spacing: You're correct that
sanitize_text()has no URL-aware spacing collapse logic. It only handles dehyphenation, typographic normalization, and ligature expansion — none of which address single-character-spaced runs [1]. The text arrives already spaced fromdocling-parse'stextline_cells, so the Python layer has no opportunity to distinguish intentional word breaks from per-glyphTjoperator spacing.Artifact 2 — Metadata field leakage: The backend consumes whatever
textline_cellsdocling-parseprovides, with no filtering of/Artifactmarked-content sequences. A similar class of artifact leakage was reported in issue #861, where PDF-internal sequences like/89and/.notdefappeared in extracted text — the maintainer confirmed fixes were applied indocling-parseat that time, establishing precedent for handling this at the C++ level.Existing related work:
Your suggested fixes are reasonable, but note that a Python-side URL pattern match in
sanitize_text()would only be a partial workaround — the root cause is thatdocling-parseis emitting per-glyph text cells without collapsing them. Similarly, metadata filtering in the Python backend would work, but ideallydocling-parsewould honor/Artifactmarked-content tags natively. You may want to cross-file the artifact filtering issue ondocling-parseas well.To reply, just mention @dosu.
Share context across your team and agents. Try Dosu.
fix: fall back to real metrics when a visible glyph declares zero width
wittjeff commentedon Sep 2, 2026
Diagnosis of artifact 1 (URL character spacing), from the content stream of the article named above. It is CC BY 4.0, so I could fetch and test it.
This is not a font-metrics problem, and docling-parse#328 (the zero-
/Widthsfix behind #4018) does not touch it: the file has no zero-width fonts and that branch leaves its output unchanged.What InDesign 18.3 (Adobe PDF Library 17) actually wrote, page 23:
Between every character of the URL there is a real space glyph, tagged with
/ActualTextU+200B (ZERO WIDTH SPACE). InDesign inserts these as line-break opportunities inside long URLs; the/ActualTextdeclares that the glyph stands for a zero-width character, not a space. docling-parse honors/ActualTextsince docling-parse#300, so the cell becomes U+200B, but the cell is still kept as a separate glyph with the space's advance, so word assembly never mergeshandt, and the line comes out ash t t p s. Withapply_actual_text=Falsethe raw space glyphs remain, with the same visible result.The fix belongs in docling-parse's
apply_actual_text: a replacement string consisting only of zero-width format characters (U+200B, U+200C, U+200D, U+2060, U+FEFF) should take the same path as an empty/ActualText, which already removes the span's cells as non-textual content. With those cells gone the neighbouring glyphs are adjacent and merge intohttps://doi.org/...normally. URL-pattern post-processing in docling, as suggested above, would miss the same construct wherever InDesign applies it outside URLs.I have not traced artifact 2 (the
W G R O U P : S T R I N Gleakage) but it shows the identical spacing, so it is likely the same construct on text InDesign emitted from a hidden field. Happy to contribute the file to the docling-parse regression corpus for this case; I have offered it on docling-parse#328.