Skip to content

fix(xlsx): read images placed in cells - #5015

Open
uommou wants to merge 4 commits into
docling-project:mainfrom
uommou:fix/xlsx-place-in-cell-images
Open

uommou wants to merge 4 commits into
docling-project:mainfrom
uommou:fix/xlsx-place-in-cell-images

Conversation

@uommou

@uommou uommou commented Oct 26, 2026

Copy link
Copy Markdown

Excel 365 can place an image in a cell (Insert → Pictures → Place in Cell).
Excel stores such an image as a rich value in xl/richData, not as a drawing.
The cell keeps only the error value #VALUE! as a fallback. openpyxl does not
read rich values, so the XLSX backend lost the image and wrote #VALUE! into
the table.

This PR follows the rich-value chain from the cell's vm attribute to the
image:

<c vm="1"> -> xl/metadata.xml (valueMetadata -> futureMetadata XLRICHVALUE)
           -> rdrichvalue.xml -> rdrichvaluestructure.xml "_localImage"
           -> richValueRel.xml -> image member
  • An image cell in a table becomes a RichTableCell with empty text. It refers
    to a PictureItem whose parent is the table, like images in table cells in
    the DOCX and ODF backends.
  • The alternative text that Excel stores with the image becomes the picture
    description (meta.description).
  • With treat_singleton_as_text, a lone image cell becomes a picture, not the
    text #VALUE!.
  • An error value without an image, such as #N/A, keeps its text.

Issue resolved by this Pull Request:
Resolves #5014

Changes

The PR has three commits:

  1. refactor(xlsx): add the table before its cells: the table is added first,
    then its cells one by one with add_table_cell, so that a cell can refer to
    an item under the table. A _cell_prov helper replaces three copies of the
    provenance code. The output does not change.
  2. fix(xlsx): read images placed in cells: the rich-data reader.
  • The rich-data parts are found from the workbook relationships that
    openpyxl has already parsed. A workbook without all four parts
    (sheetMetadata, rdRichValue, rdRichValueStructure, richValueRel)
    opens no extra archive member, so other workbooks do not change.
  • openpyxl does not keep the vm attribute, so the sheet XML is read again
    with iterparse. This occurs only for a sheet that has error cells in a
    workbook with rich data. Parsed cells and rows are removed during the
    parse, so memory does not grow with the sheet.
  • The image is attached to ExcelCell when the cell is read, so the table,
    the singleton and the section-label paths all see the same data. The
    #VALUE! text is removed there, in one place.
  • Each image member is decoded once, also when several cells use it. An
    EMF/WMF image falls back to the existing LibreOffice conversion.
  • All reads use the existing archive size limits and the safe XML parser
    options.
  1. test(xlsx): cover images placed in cells: fixture, groundtruth and tests.

RichTableCell.ref points to the PictureItem directly, without a group. An
Excel cell holds one image and nothing else, so a group would hold one item.
docling-core accepts this, because the picture's parent is the table.

Not in scope

  • Rich values with other structures, such as images from the IMAGE()
    function or linked data types. Their cells keep #VALUE!, as before.

Checklist:

  • Documentation has been updated, if necessary. Not necessary: the docs do
    not list what the XLSX backend extracts from a sheet, other than charts.
    
  • Examples have been added, if necessary. Not necessary.
  • Tests have been added, if necessary.

Review evidence:

Before and after, for a workbook saved by Excel 365 with two images placed in
A1:A2 (the sample file in the issue):

Before After
doc.pictures 0 2 (931×1280 and 2647×1765)
Table cells TableCell #VALUE! × 2 RichTableCell × 2, each referring to a picture under the table
Alternative text lost meta.description: "Cow in pasture", "Basketball player dribbling ball on court"
Markdown ` #VALUE!

New fixture tests/data/xlsx/sources/xlsx_10_place_in_cell.xlsx:

  • Sheet "Products": two image cells with alternative text, in a table with text
    and number cells, and an #N/A error cell without an image.
  • Sheet "Logo": one image cell without alternative text. It uses the same media
    file as a cell on the first sheet.
  • The file is built from OOXML parts with small generated images, so it holds no
    third-party content. Its structure is the same as the Excel 365 sample. Excel
    for Mac (16.110) opened and saved a copy of a file built the same way, kept
    the images in their cells, and docling reads that saved copy the same way.

Document tree of the fixture (.itxt groundtruth):

item-0 at level 0: unspecified: group _root_
  item-1 at level 1: sheet: group Products
    item-2 at level 2: table with [3x4]
      item-3 at level 3: picture
      item-4 at level 3: picture
  item-5 at level 1: sheet: group Logo
    item-6 at level 2: table with [1x1]
      item-7 at level 3: picture

Exports: the fixture was written to every CLI output format (md, json,
yaml, html, html_split_page, text, doctags, vtt, doclang, dclx,
latex, chunks) with the placeholder, embedded and referenced image
modes. This used the same DoclingDocument methods as docling convert; the
CLI itself was not run.

  • No export failed.
  • The JSON loads back with 3 pictures, and every RichTableCell.ref resolves.
  • The image is inside its cell in each format: <td><figure> in HTML,
    <fcel><picture> in DocTags, <picture> with <description> in DocLang.
  • With embedded, the image data is in the cell. With referenced, the linked
    files exist.
  • No output contains #VALUE!.

Reference data: only the new fixture's groundtruth is added. No existing
groundtruth changes.

Tests and checks:

  • uv run pytest tests/test_backend_msexcel.py: 35 passed.
  • The new tests and the new e2e groundtruth fail on main (3 failures) and pass
    with this change.
  • After the refactor commit alone, the 33 existing tests pass with unchanged
    groundtruth.
  • Pre-commit hooks over the branch (prek run --from-ref upstream/main, the
    hooks of make validate): passed (Ruff, ty, Tach, license headers, max lines).
  • ty check: the same warnings as main for the changed files, no new ones.
  • The module still imports without openpyxl, so the install hint is kept.
  • Not run: the full make test suite. Only the XLSX backend changed.

Add each table to the document first and then add its cells one by one
with `add_table_cell`. A cell can then refer to an item whose parent is
the table, as a rich table cell needs. A small `_cell_prov` helper builds
the provenance of a cell range, which was written out three times.

The output does not change.

Signed-off-by: chaiwon.hwang <chaiwon.hwang.dev@gmail.com>
Excel 365 can place an image in a cell (Insert > Pictures > Place in
Cell). Such an image is not a drawing: Excel stores it as a rich value in
`xl/richData`, and the cell keeps only the error value `#VALUE!` as a
fallback for older readers. openpyxl does not read rich values, so the
image was lost and the cell came out as the text `#VALUE!`.

The backend now follows the rich-value chain from the cell's `vm`
attribute through `xl/metadata.xml`, `rdrichvalue.xml`,
`rdrichvaluestructure.xml` and `richValueRel.xml` to the image. Only the
`_localImage` structure is read; other rich values, such as linked data
types, are left as they are.

- An image cell in a table becomes a `RichTableCell` with empty text,
  which refers to a `PictureItem` whose parent is the table.
- With `treat_singleton_as_text`, a lone image cell becomes a picture.
- The alternative text that Excel stores with the image becomes the
  picture description (`meta.description`).

The archive is read only when the workbook relationships name all the
rich-data parts, so other workbooks do not change.

Signed-off-by: chaiwon.hwang <chaiwon.hwang.dev@gmail.com>
Add `xlsx_10_place_in_cell.xlsx` with its groundtruth. The "Products"
sheet has two images placed in cells, each with alternative text, and a
plain `#N/A` error cell without an image. The "Logo" sheet has one image
cell without alternative text, which uses the same media file as one of
the cells on the first sheet.

The tests check that an image cell becomes a `RichTableCell` that refers
to a picture under the table, that the alternative text becomes the
picture description, that an ordinary error value keeps its text, and
that a lone image cell becomes a picture with `treat_singleton_as_text`.

Excel opened and saved a copy of a file built the same way and kept the
images in their cells.

Signed-off-by: chaiwon.hwang <chaiwon.hwang.dev@gmail.com>
@github-actions

Copy link
Copy Markdown
Contributor

✅ DCO Check Passed

Thanks @uommou, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🔴 1 of 2 protections blocking · waiting on 👀 reviews

Protection Waiting on
🔴 Require two reviewer for test updates 👀 reviews
🟢 Enforce conventional commit —

🔴 Require two reviewer for test updates

Waiting for

  • #approved-reviews-by >= 2
This rule is failing.

When test data is updated, we require two reviewers

  • #approved-reviews-by >= 2

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@github-actions github-actions Bot added ai:ci-safe AI triage found no CI-safety concern (advisory) bug Something isn't working docling-document xlsx issue related to xlsx backend labels Oct 26, 2026
@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

AI triage (advisory)

Checked commit 40faa71415. The analysis read the diff only. It did not run code from this pull request.

The PR adds support for images placed in Excel cells via the 'Place in Cell' feature by reading the rich-value chain (xl/richData) that openpyxl ignores. An image cell becomes a RichTableCell with a PictureItem, and a lone image cell with treat_singleton_as_text becomes a picture instead of the fallback #VALUE! text.

CI safety: no concern found. A maintainer must still approve CI.

1 related pull request(s) in the same code
PR State Signal Assessment
#5024 open changes the same code related: PR #5024 fixes empty-cell materialization in _find_data_tables by replacing iter_rows with direct sheet._cells iteration. Both PRs modify _find_data_tables and _find_table_bounds in the same file, so they conflict at those hunks, but they fix different bugs and neither subsumes the other.

Reference data changes

Change Files
added 3
  • added tests/data/xlsx/groundtruth/xlsx_10_place_in_cell.xlsx.itxt
  • added tests/data/xlsx/groundtruth/xlsx_10_place_in_cell.xlsx.json
  • added tests/data/xlsx/groundtruth/xlsx_10_place_in_cell.xlsx.md

Assessment: expected. The three added groundtruth files correspond to the new xlsx_10_place_in_cell.xlsx test fixture introduced by the PR. They capture the expected output of the new image-in-cell feature.

The ai:* labels are advisory. They do not approve CI and they do not add tests:full. A new push runs this check again.

@codecov

codecov Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.92593% with 39 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
docling/backend/msexcel_backend.py 75.92% 23 Missing and 16 partials ⚠️

📢 Thoughts on this report? Let us know!

@DanielNg0729 DanielNg0729 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @uommou, thank you for this PR. The rich-value chain is well explained, and it reads the images correctly in my tests. I found one case that loses an image, and I have one small question about the tree shape.

An image next to a section label is dropped. The PR now clears #VALUE! from image cells, so _split_leading_section_label no longer counts an image cell as content. When row 0 has a merged label and an image, the label is split off as text, and the data keeps only rows > 0. The image cell and its picture disappear:

import xlsxwriter

workbook = xlsxwriter.Workbook("section_image.xlsx")
sheet = workbook.add_worksheet()
sheet.merge_range("A1:B1", "Fruit list")
sheet.embed_image("C1", "icon.png", {"description": "Section icon"})
sheet.write_row("A2", ["name", "qty", "note"])
sheet.write_row("A3", ["apple", 1, "x"])
sheet.write_row("A4", ["pear", 2, "y"])
workbook.close()
Result
main one 4x3 table; C1 is #VALUE!
this PR text "Fruit list" and a 3x3 table; no picture
with the change below one 4x3 table; C1 is a RichTableCell with the picture

Would it make sense to count image cells as content there? This change fixed it for me, and tests/test_backend_msexcel.py still passes (33 passed, 2 skipped):

first_row_text_cells = [
    cell for cell in first_row_cells if cell.text.strip() or cell.image is not None
]

A small test for this case could help too. Thanks!

An image cell has no text, so the section-label split did not count it
as content. A merged label and an image in the first row made the label
split off, and the image cell was lost. Count image cells as content.

Signed-off-by: chaiwon.hwang <chaiwon.hwang.dev@gmail.com>
@github-actions github-actions Bot added ai:ci-safe AI triage found no CI-safety concern (advisory) and removed ai:ci-safe AI triage found no CI-safety concern (advisory) labels Oct 27, 2026
@uommou

uommou commented Oct 27, 2026

Copy link
Copy Markdown
Author

Thanks @DanielNg0729 for the review.
I applied your change, and added a test for it(test_image_cell_next_to_a_section_label). It builds your workbook with xlsxwriter and checks that C1 stays in one 4x3 table as a RichTableCell that refers to the picture. The test fails without the fix.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai:ci-safe AI triage found no CI-safety concern (advisory) bug Something isn't working docling-document xlsx issue related to xlsx backend

Projects

None yet

Development

Successfully merging this pull request may close these issues.

XLSX: images placed in cells ("Place in Cell") are lost, and the cells become #VALUE!

2 participants