Repository navigation
DOCX: hidden text (w:vanish) is emitted as body content #4957
Description
Bug
The DOCX backend ignores the Hidden font property (w:rPr/w:vanish). Hidden runs, hidden paragraphs and hidden text in table cells are emitted on the body layer and appear in the default Markdown export, although Word does not show or print them.
This is the DOCX counterpart of #4647 (hidden PPTX slide → ContentLayer.INVISIBLE) and #4733 (hidden PPTX shape → ContentLayer.INVISIBLE). msword_backend.py has no reference to vanish on main (29caea8).
It matters for RAG pipelines: hidden text goes into chunks and embeddings as if it were visible content. Hidden text is also a common way to put instructions for an LLM into a document that a human reviewer does not see.
Steps to reproduce
"""Reproduce: the DOCX backend ignores w:vanish (hidden text) and emits it as body content."""
import sys
from importlib.metadata import version
from io import BytesIO
from docling.datamodel.base_models import InputFormat
from docling.document_converter import DocumentConverter
from docling_core.types.doc import ContentLayer, TableItem, TextItem
from docling_core.types.io import DocumentStream
from docx import Document
from docx.oxml import OxmlElement
def hide(run) -> None:
run._r.get_or_add_rPr().append(OxmlElement("w:vanish")) # Font > Hidden
def build() -> BytesIO:
doc = Document()
p = doc.add_paragraph("Shown start. ")
hide(p.add_run("Hidden run. "))
p.add_run("Shown end.")
hide(doc.add_paragraph().add_run("Hidden paragraph."))
table = doc.add_table(rows=1, cols=2)
table.cell(0, 0).text = "Shown cell"
hide(table.cell(0, 1).paragraphs[0].add_run("Hidden cell"))
buf = BytesIO()
doc.save(buf)
buf.seek(0)
return buf
result = DocumentConverter(allowed_formats=[InputFormat.DOCX]).convert(
DocumentStream(name="hidden_text.docx", stream=build())
)
print(f"docling-slim {version('docling-slim')}, docling-core {version('docling-core')}, python {sys.version.split()[0]}")
for item, _ in result.document.iterate_items(included_content_layers=set(ContentLayer)):
if isinstance(item, TextItem):
print(f"{item.text!r}: {item.content_layer.value}")
elif isinstance(item, TableItem):
print(f"table cells {[c.text for c in item.data.table_cells]}: {item.content_layer.value}")
print("default markdown:", repr(result.document.export_to_markdown()))Output:
docling-slim 2.141.0, docling-core 2.102.0, python 3.14.7
'Shown start. Hidden run. Shown end.': body
'Hidden paragraph.': body
table cells ['Shown cell', 'Hidden cell']: body
default markdown: 'Shown start. Hidden run. Shown end.\n\nHidden paragraph.\n\n| Shown cell | Hidden cell |\n|--------------|---------------|'
Expected
- A fully hidden paragraph: on
ContentLayer.INVISIBLEwith its text, the same treatment as a hidden PPTX slide (fix(pptx): move hidden slides to the invisible content layer #4647) or shape (fix(pptx): move hidden shapes to the invisible content layer #4733). - A hidden run inside a shown paragraph: not in the paragraph text (
'Shown start. Shown end.'). ADocItemhas one content layer, so the run cannot go toINVISIBLEby itself. - Hidden text in a table cell: not in the cell text, for the same reason.
w:vanish can also come from a character style or the paragraph style (styles.xml), not only from direct formatting, so the check must use the effective run property.
I have a fix with regression tests and will open a PR that references this issue.
Out of scope: w:webHidden (hidden only in Web Layout, for example TOC page numbers) and w:specVanish.
Docling version
docling-slim 2.141.0 (main at 29caea8), docling-core 2.102.0
Python version
3.14.7 (Windows 11)
Activity