Skip to content

fix(xlsx): keep currency, percent, and date number formats - #4628

Open
jipeng6036-del wants to merge 5 commits into
docling-project:mainfrom
jipeng6036-del:fix/4619-xlsx-number-formats
Open

jipeng6036-del wants to merge 5 commits into
docling-project:mainfrom
jipeng6036-del:fix/4619-xlsx-number-formats

Conversation

@jipeng6036-del

@jipeng6036-del jipeng6036-del commented Oct 7, 2026 •

Copy link
Copy Markdown

Part of #4619.

Excel table conversion discarded stored number formats after loading cached formula values. A price such as $12.50 became 12.5, and 10% became 0.1.

Apply supported currency, percent, and date formats to worksheet table cells. Preserve quoted and escaped literals, distinguish minutes from months, honor 12-hour time and the case of AM/PM, and support mmmmm month initials. Use decimal half-up rounding and the negative/zero format sections. Locale-dependent built-in short date and date-time formats use ISO values; unsupported complex formats retain str(value).

Chart data keeps the raw cached values, including categories and series names, without applying cell display formats. For example, worksheet cells show $1,234.50 and $99.13, while the reconstructed chart table retains 1234.5 and 99.125. The conversion regression checks both tables and their cell positions.

Formulas still use their cached results through data_only=True. Formula metadata requires a separate docling-core schema change and is not addressed here.

Validation for bb39a3b96ecc94deb582965759ac24cd2fb0f973 on Windows / Python 3.13.12:

  • python -m pytest tests/test_backend_msexcel.py -q: 66 passed, 2 skipped because LibreOffice is unavailable.
  • The added chart/date/time regressions reproduced 7 failures on ab177e14 before the follow-up fix.
  • The make validate changeset recipe, run as uv run --no-sync prek run --files docling/backend/msexcel_backend.py tests/test_backend_msexcel.py, passed all applicable hooks: Ruff, ty, Tach, module coverage, max lines, and file checks. A local python3.cmd shim points to the same venv for Windows; tracked check configuration is unchanged.
  • Both changed files compile with Python 3.10.20. This is a syntax check, not a Python 3.10 test-suite run.
  • git diff --check: passed.

The existing sales Markdown/JSON reference updates only change date-time text for an unpadded h token; the JSON grid copies match. They do not change ownership or order, and still require the repository's two-reviewer approval.

Hosted Run CI and Run Docs CI completed successfully on this exact commit (attempt 2). The Python 3.10 core/ML tests, lint, Tach, package checks, documentation build, DCO, and patch coverage passed. Windows/macOS and the optional install jobs were skipped. DanielNg0729 approved this commit on 2026-10-14; the test-reference changes still await the required second reviewer.

Excel cells were converted with str(value) after a data_only load, so
$12.50 became 12.5, 10% became 0.1, and Feb-25 became a timestamp.

Apply the stored number format for currency, percentages, and dates.
Custom formats still use str(value).

Fixes docling-project#4619

Signed-off-by: jipeng6036-del <jipeng6036@gmail.com>
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

✅ DCO Check Passed

Thanks @jipeng6036-del, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 All 2 merge protections satisfied — ready to merge.

Show 2 satisfied protections

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

🟢 Require two reviewer for test updates

When test data is updated, we require two reviewers

  • #approved-reviews-by >= 2

@codecov

codecov Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.53097% with 22 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
docling/backend/msexcel_backend.py 80.53% 11 Missing and 11 partials ⚠️

📢 Thoughts on this report? Let us know!

@DanielNg0729 DanielNg0729 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jipeng6036-del, thank you for this fix

I tried some built-in Excel formats and found a few cases where the date path now gives worse text than main.

Format Value main this PR Excel shows
mm-dd-yy (built-in 14) 2025-02-01 2025-02-01 00:00:00 02-01-25 2/1/2025 (en-US short date)
mm:ss (built-in 45) 00:07:45 00:07:45 01:45 07:45
h:mm AM/PM (built-in 18) 13:30 13:30:00 13:30 PM 1:30 PM
"Due "mmm d 2025-02-01 2025-02-01 00:00:00 01ue Feb 01 Due Feb 1

I left the details as inline comments. Please see below. Thanks.

Comment thread docling/backend/msexcel_backend.py Outdated
Comment thread docling/backend/msexcel_backend.py Outdated
Comment thread docling/backend/msexcel_backend.py Outdated
Comment thread docling/backend/msexcel_backend.py Outdated
Comment thread docling/backend/msexcel_backend.py Outdated
Comment thread docling/backend/msexcel_backend.py Outdated
Comment thread docling/backend/msexcel_backend.py Outdated
Comment thread docling/backend/msexcel_backend.py Outdated
Signed-off-by: jipeng6036-del <jipeng6036@gmail.com>
Signed-off-by: jipeng6036-del <jipeng6036@gmail.com>
@jipeng6036-del

Copy link
Copy Markdown
Author

Thanks @DanielNg0729 — the follow-up in 3513b5d addresses the cases you listed. Summary:

  • Built-in 14 mm-dd-yy (and 22 m/d/yy h:mm) now keep ISO 2025-02-01 / 2025-02-01 13:30:00 instead of the short ambiguous date.
  • mm:ss treats mm before ss as minutes (00:07:45 → 07:45).
  • Bracket stripping is limited to color and [$$-xxxx] currency locale markers, so [h]:mm:ss and [$$-409]#,##0.00 no longer lose later checks. Unsupported duration/locale date formats fall back to str(value).
  • Quoted literals stay out of date-token replacement ("Due "mmm d → Due Feb 1).
  • h/d/m are unpadded; AM/PM uses 12-hour (h:mm AM/PM → 1:30 PM).
  • cell.number_format is used directly, no getattr.
  • Negative/zero sections are applied (($5.00)).
  • Rounding uses Decimal ROUND_HALF_UP (0.125 with 0% → 13%).

Also merged current main so the xlsx CMYK picture path stays in this branch.

Happy to adjust further if any of these should still fall back instead of rendering.

@github-actions github-actions Bot added ai:ci-safe AI triage found no CI-safety concern (advisory) bug Something isn't working xlsx issue related to xlsx backend labels Oct 8, 2026
@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

AI triage (advisory)

Checked commit 43eca89a63. The analysis read the diff only. It did not run code from this pull request.

Adds number-format rendering for XLSX worksheet cells so that currency, percent, and date values display as Excel would show them instead of collapsing to raw Python values. Chart data keeps raw cached values without formatting.

CI safety: no concern found. A maintainer must still approve CI.

2 related pull request(s) in the same code
PR State Signal Assessment
#5015 open changes the same code related: PR #5015 also changes msexcel_backend.py and the test file, but it fixes image extraction from cells — a different feature with no overlap on the number-format logic.
#5024 open changes the same code related: PR #5024 also changes the same cell_text = line (and the scan loop) to avoid materializing empty cells, which is a different bug fix; both PRs would conflict on that one line but they solve different problems.

Reference data changes

Change Files
tables 2
  • tables tests/data/xlsx/groundtruth/xlsx_02_sample_sales_data.xlsm.json: table 0: 20 cell(s) changed
  • tables tests/data/xlsx/groundtruth/xlsx_02_sample_sales_data.xlsm.md: +20 / -20 lines; Markdown table rows changed

Assessment: expected. The PR applies number formats to worksheet cells, so existing sales data cells that held raw floats and datetimes now render as formatted strings; 20 changed cells in the JSON and 20 changed rows in the Markdown are the direct result of that fix.

The ai:* labels are advisory. They do not approve CI and they do not add tests:full. A new push runs this check again.

@dolfim-ibm

Copy link
Copy Markdown
Member

@DanielNg0729 can you please have another look here?

@DanielNg0729 DanielNg0729 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jipeng6036-del, thank you for the update,

I checked again on ab177e14. All the cases from my earlier review now match Excel. The workbook attached to #4619 gives $12.50, $270.00, 10% and Feb-25. The Excel tests pass here (59 passed, 2 skipped), also after a merge with current main.

Before I approve, I would like to ask for a few more changes:

  1. The chart data tables now also use the number format. Could you please confirm if this is intended? (inline comment on _resolve_reference)
  2. The description says Resolves #4619, so the issue closes when this PR merges. The formula part of #4619 is not in this PR. Could you change it to Part of #4619, so that the formula request stays open?
  3. Two small clean-ups: a comment that is now in the wrong place, and an unused import (inline comments).

Inline comments

docling/backend/msexcel_backend.py L120 (the # Maps an openpyxl chart object's tagname comment)

This comment is for _CHART_TAGNAME_TO_CLASSIFICATION, but the new block is now between the comment and the dict. So it reads as the comment for _GENERAL_NUMBER_FORMATS. Could the new constants and helpers go above this comment?

docling/backend/msexcel_backend.py L91

from_excel is not used. Ruff does not report it, because this repo ignores F401. Could you remove it?

docling/backend/msexcel_backend.py L2093

This also changes the chart data tables, because _chart_to_table_data reads the categories and values through _resolve_reference. For a bar chart over "$"#,##0.00 cells, the table now shows $1,234.50 and $99.13. On main it shows 1234.5 and 99.125, so the chart data loses precision. The PPTX and Word chart tables keep the raw numbers. Is this intended? If yes, could you add a test and one line in the description? If not, maybe this path could keep str(value).

docling/backend/msexcel_backend.py L125 (optional)

mmmmm (the first letter of the month) splits into mmmm + m, so it gives February2. Excel shows F, and main gives 2025-02-01 00:00:00. Maybe add mmmmm to the tokens, or fall back for it?

docling/backend/msexcel_backend.py L280 (optional)

Excel keeps the case of the marker, so h:mm am/pm shows 1:30 pm. Here it gives 1:30 PM.

tests/test_backend_msexcel.py L1084 (optional)

The names test_excel_review_number_format_regressions and test_xlsx_review_formats_survive_document_conversion (L1088) refer to this review. Names that describe the behaviour may be clearer later. Also, the workbook formulas_and_number_formats.xlsx at L1141 has no formulas.

Signed-off-by: jipeng6036-del <jipeng6036@gmail.com>
@github-actions github-actions Bot added ai:ci-safe AI triage found no CI-safety concern (advisory) and removed ai:ci-safe AI triage found no CI-safety concern (advisory) labels Oct 14, 2026
DanielNg0729
DanielNg0729 previously approved these changes Oct 14, 2026

@DanielNg0729 DanielNg0729 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@github-actions github-actions Bot removed the ai:ci-safe AI triage found no CI-safety concern (advisory) label Oct 28, 2026
@cau-git

cau-git commented Oct 28, 2026

Copy link
Copy Markdown
Member

@DanielNg0729 can you please re-check and approve again? I rebased it to current main after conflicts.

@github-actions github-actions Bot added the ai:ci-safe AI triage found no CI-safety concern (advisory) label Oct 28, 2026

@DanielNg0729 DanielNg0729 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much @cau-git, Looks good to me!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai:ci-safe AI triage found no CI-safety concern (advisory) bug Something isn't working xlsx issue related to xlsx backend

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants