Repository navigation
feat(docx): convert embedded Word, Excel and PowerPoint objects - #4716
DanielNg0729 wants to merge 10 commits into
Conversation
Word stores an embedded Office document as a package part behind an OLE object. The backend kept only the preview picture of the object, so the content of the document was lost. Add the opt-in MsWordBackendOptions.process_embedded_objects option. When it is set, the backend converts each embedded .docx, .xlsx or .pptx part with its own backend and copies the result into a group, after the preview picture of the object. - The copy remaps captions, footnotes, references, comments and rich table cells, because DoclingDocument.add_document keeps their old indexes. It also removes the page provenance, because a Word document has no pages. - The embedded document uses the default options of its backend, so the objects embedded in it are not converted. - Each part is converted once, also when several objects share it. - Linked objects, binary OLE objects and documents that fail to convert keep only their preview picture, with a warning for a failure. Refs docling-project#2051 Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
Add a fixture made in Microsoft Word. It embeds a workbook, a presentation, a Word document that embeds a workbook of its own, and a text file. The author name and local paths were removed from it. The tests check the default behaviour, the position and content of each embedded document, the single conversion level, DocTags export, rich table cell references, broken and linked objects, a missing format extra, and a part that two objects share. Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
|
✅ DCO Check Passed Thanks @DanielNg0729, all your commits are properly signed off. 🎉 |
Merge Protections🔴 1 of 2 protections blocking · waiting on 👀 reviews
🔴 Require two reviewer for test updatesWaiting for
This rule is failing.When test data is updated, we require two reviewers
Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
AI triage (advisory)Checked commit This PR adds support for converting embedded Word, Excel, and PowerPoint OLE objects in DOCX files. It introduces backend options to enable this feature and configure limits for object count, total decompressed bytes, and nesting depth. CI safety: no concern found. A maintainer must still approve CI. Reference data changes
Assessment: expected. The reference files are added for the new test document with embedded objects. The |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
Hi Dr @ceberam, would you mind taking a look at this PR when you have a moment? I’d really appreciate your feedback for this PR. Thank you very much |
…objects Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
Signed-off-by: Nguyen Hoang Duong <danielnguyenh07@gmail.com>
There was a problem hiding this comment.
Thanks @DanielNg0729 for suggesting this PR and contributing to new exciting features for Docling!
First of all, to put this PR in the context of the 200+ opened PRs, I would like to understand its motivation: is it to address a real need with the documents your work with? Or just to address the issue #2051 ? By the way, I closed it since we haven't received any response since more than 1 year ago and we understand that the user does not see a need for that feature.
Parsing embedded documents opens some challenges and also requires a broader contract on how to deal with them (FYI @dol since you also review this PR). For instance:
- Do we want to parse any inserted documents in a
.docxfile? Why only office files and not.pdffiles? - How many inserted files should we accept in a
.docxdocument and what about deep nested documents? I see that you explicitly hard-coded the depth to 1, but is this is an acceptable long-term contract? We should at least explain why we set this limitation. Also, see below my comment related to security on this topic. - Do we want to enable this feature to the Office formats only or to others that can also embed attachments?
I think these are important decisions since they can set a precedent for other backends and formats.
Having said that, here are some points that should be addressed with the current PR implementation:
Style check fails — trailing whitespace in blank lines
The style check fails and there is a conflict with msword_backend.py. Please rebase to main.
Zip-bomb risk when converting embedded documents from untrusted sources
When process_embedded_objects=True, the backend passes each embedded part's bytes as BytesIO(part.blob) to the relevant backend. None of the three paths apply size or member-count limits:
-
Embedded
.docx:load_msword_file(BytesIO(part.blob))opens a newZipFileover those bytes. Docling's zip-bomb guards (_sanitize_docx,_normalize_strict_ooxml) run only when the embedded document is Strict OOXML or contains fragment-only rels. For the common case — a plain Transitional-namespace DOCX without fragment rels —python-docxexpands the inner archive with no size limit. -
Embedded
.xlsx:openpyxl.load_workbookopens a newZipFilewith no size limit. -
Embedded
.pptx:python-pptx'sPackage.openopens a newZipFilewith no size limit.
In all three cases, a crafted embedded part whose inner members have a high compression ratio can exhaust available memory when processed. Because part.blob is already in memory as an opaque byte sequence (python-docx loads it when reading the outer DOCX), the outer-document zip-bomb checks do not see the inner archive's expansion ratio.
Fixture created by an unusual path — does it represent real Word output?
The fixture docx_embedded_objects.docx was assembled manually: it was created by COM automation (InlineShapes.AddOLEObject), exported as Flat OPC XML (Document.WordOpenXML), and then repacked into a .docx by hand. The PR description says Word could not save the file directly.
Examining the fixture reveals that the inner embedded Word document (word/embeddings/Microsoft_Word_Document.docx) carries a Flat OPC artifact: its rels files declare the xmlns:pkg="http://schemas.microsoft.com/office/2006/xmlPackage" namespace, which is never used in the document content but is a remnant of the Flat OPC export step. No other .docx in the test suite has this artifact. The outer file and the embedded .xlsx and .pptx are clean.
The artifact is harmless to parsing, and the OOXML structure of the embedded objects (the v:imagedata / o:OLEObject / package relationship pattern) matches what Word produces. However, the fixture cannot be opened and re-saved in Word to verify round-trip fidelity, and it is not clear that the inner Word document as packed would survive being opened by Word.
The concern is practical: the test exercises the code path, but it does not confirm that the feature works on files that real users create and share. It would strengthen the test base to include a fixture that was saved directly by Word (or LibreOffice, which also supports Insert→OLE Object). A file that Word can open and re-save is a much stronger confidence signal than one produced by an unusual automation path.
RichTableCell.ref remapping does not guard against a missing key
_remap_refs silently drops any reference whose target is not in copies. The corresponding line for RichTableCell:
cell.ref = copies[cell.ref.cref].get_ref()does not have this guard and raises KeyError if cell.ref.cref is absent from copies. Today this cannot happen because RichTableCell.ref always points to a group created as a direct child of the table, and the table is always a body item. But the asymmetry with _remap_refs is a latent risk if any future backend produces a RichTableCell with a ref outside the body tree.
Assertions in runtime code
First of all, there is an issue with the position of the assertion assert isinstance(backend, DeclarativeDocumentBackend). The assertion is inside the try block whose except Exception handler logs a warning and returns None. An AssertionError would be silently swallowed, hiding a programming error as a benign "cannot be converted" warning.
In any case, we should avoid using assertions in the runtime code because they are ignore in optimized python environments.
Test functions and files
I don't think we should create a new test module for this feature, as we shouldn't in general for any new feature or fix that is suggested in a PR. Otherwise the test suite will become unmanageable and inefficient (similar edge cases get dispersed, fixtures are not reused...).
Regarding the location of the test file, I don't see the issue in adding it in tests/data/docx/sources and let the end-to-end pipeline include it. Actually this would be helpful to see how the conversion is rendered in Markdown for instance.
…objects Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
The merge of main through the GitHub editor left whitespace on four blank lines, and the lint job failed. Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
A RichTableCell of an embedded document raised KeyError when its target was not in the copied tree. Now the cell becomes a plain TableCell that keeps its text, the same as _remap_refs drops a reference whose target was not copied. Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
An embedded package was opened without size limits, and the PowerPoint backend does not check archive limits. The number of embedded objects and the nesting depth were also fixed in the code. - Each embedded package must pass the archive limits of the Word file before any backend opens it. - New MsWordBackendOptions fields: max_embedded_objects (32), max_embedded_total_bytes (512 MiB) and max_embedding_depth (3). The count and the size are shared by all nesting levels. - An embedded Word document is converted with the same options, so its own objects are converted until max_embedding_depth. - The backend type is checked without an assertion, outside the try blocks, as SimplePipeline does. Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
- Replace the fixture with a file that Microsoft Word saved, and move it to tests/data/docx/sources. The generic e2e test converts it with process_embedded_objects=True and compares the reference files. - Move the tests into tests/test_backend_msword.py and remove the separate module. - Add tests for the limits over nesting levels, a zip bomb, a broken package, a linked object and a rich table cell whose target is not copied. Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
|
Thank you very much @ceberam, for the careful review and for the time you spent on it. I pushed new commits that I hope address your points, and I add my answers to your questions below. I may be wrong on the contract questions, so please feel free to change any of them. Motivation. My goal is to get the most out of a document. When a report embeds a workbook, a slide or another Word document, its content is lost today, so a RAG pipeline never sees it. I found #2051 when I looked at this gap. Thank you for closing it; I understand that its author did not reply. About your question:
The points on the implementation
Thank you again for your guidance. Please let me know if you prefer other defaults or another approach. I am happy to change it. |
Hi maintainers, this PR lets the Word backend convert the Excel workbooks, PowerPoint presentations and Word documents that are embedded in a
.docxfile as OLE objects. It is the Word part of #2051.Summary
On
main, the backend keeps only the preview picture of an embedded object (an EMF image). The table in an embedded workbook or the text on an embedded slide is lost, so a RAG pipeline never sees it.Issue resolved by this Pull Request:
Part of #2051
Usage
The feature is opt-in. The default output does not change.
Markdown export of the new test document, with the option set:
How Word stores an embedded document
rId5is a.../relationships/packagerelationship toword/embeddings/Microsoft_Excel_Worksheet.xlsx, a complete OOXML package. Legacy objects (for exampleExcel.Sheet.8, or a PDF or text file inserted asPackage) use an.../relationships/oleObjectrelationship to a binary compound file (oleObject1.bin).Other choices
False. The output for existing users and the reference data do not change. The same pattern asrender_chart_images.PictureItem, because the serializers do not export the children of a picture by default (traverse_pictures=False), so the content would not be in the Markdown. The group usesGroupLabel.SECTION, the label that the Excel backend gives to a sheet, and the nameembedded: <part file name>.process_embedded_objectsisFalsefor an embedded Word document. The work stays bounded without a depth counter.w:objectelements at one large part. The backend converts a part only one time. The other objects keep their preview.DoclingDocument.add_documentcopies the tree, but captions, footnotes, references, comments andRichTableCell.refstill point at the indexes of the embedded document. Withdocx_rich_cells.docxas the embedded document, 21 of 25 rich cell references point at wrong items afteradd_documentalone. The backend maps each source item to its copy (the two trees have the same shape) and rewrites these references. AFineRefkeeps its character range. docling-core 2.101.1 (main) has the sameadd_documentcode.export_to_doctags()raisedKeyError: 1on the copied table. The backend removes the provenance of the copies.current_part), so an object in a header uses the header relationships.Test data
tests/data/docx/embedded_objects/docx_embedded_objects.docx(63 KB) was made in Microsoft Word 365 through COM automation, withInlineShapes.AddOLEObject. It embeds an.xlsx, a.pptx, a.docxthat embeds an.xlsxof its own, and a.txtfile (a binaryPackageobject). All XML parts are Word's output. Word could not save here, so I exported the document withDocument.WordOpenXML(Flat OPC) and packed the parts into a.docx. Word opens the result and reports 4 embedded OLE objects (Excel.Sheet.12,PowerPoint.Show.12,Word.Document.12,Package).The fixture is not in
tests/data/docx/sources/, because the generic e2e test converts every file there and compares the pictures. The previews are EMF images, and Pillow on Windows, LibreOffice and a system without LibreOffice give different images. The new tests disable LibreOffice and check the document tree directly.Checklist: