Skip to content

fix(docx): preserve native Word (OMML) equation structure in text extraction - #2154

Open
meutex wants to merge 2 commits into
HKUDS:devfrom
meutex:meutex/issue-1793252521-preserve-native-word-omml-equation-struc
Open

meutex wants to merge 2 commits into
HKUDS:devfrom
meutex:meutex/issue-1793252521-preserve-native-word-omml-equation-struc

Conversation

@meutex

@meutex meutex commented Oct 29, 2026

Copy link
Copy Markdown

Description

Native Word equations (OMML) lost their structure during DOCX text extraction: 1/2 + 1/3 came out as 12+13, 2^3 as 23 and sqrt(9) as 9, because the DOCX walkers collected only the leaf text runs.

This change linearizes OMML in the shared DOCX extraction, so Immersive Reading, the reading agent and chat attachments all receive an unambiguous form:

  • fractions → a/b, superscripts → a^b, subscripts → a_b
  • square roots → sqrt(x), n-th roots → root(n, x)
  • delimiters keep their own characters and every item: [0, 1], (a,b)
  • compound operands are parenthesised once: (a+b)/(c-d), (a+b)^2
  • an inline m:oMath stays in place within its paragraph; a display m:oMathPara becomes its own line

deeptutor/utils/document_images.py adds linearize_omml and uses it in extract_docx_rich. The pure-OOXML fallback in deeptutor/utils/document_extractor.py reuses the same walker, so equations survive when python-docx is unavailable. No new dependency.

With the issue's math-formula-native-only.docx, all three paths (extract_material for Immersive Reading, extract_text_from_bytes for attachments, and the OOXML fallback) now give:

分数题:计算 1/2 + 1/3。
乘方题:计算 2^3。
根式题:计算 sqrt(9)。

Paragraph order, table text and embedded-image markers are unchanged.

Not covered: n-ary operators (∑, ∫), functions (sin x) and matrices still fall back to their leaf text, as before. Happy to add a notation for them if you have a preference.

Related Issues

Module(s) Affected

  • agents
  • api
  • config
  • core
  • knowledge
  • logging
  • services
  • tools
  • utils
  • web (Frontend)
  • docs (Documentation)
  • scripts
  • tests
  • Other: ...

Checklist

  • This PR targets dev (not main — see the base-branch selector above).
  • I have read and followed the contribution guidelines.
  • My code follows the project's coding standards.
  • I have run pre-commit run --all-files and fixed any issues. (Ran the Python hooks on the changed files instead: ruff check and ruff format --check with the pinned ruff 0.16.0, and scripts/check_repo_hygiene.py. All pass.)
  • I have added relevant tests for my changes.
  • I have updated the documentation (if necessary). (No documented behaviour changed.)
  • My changes do not introduce any new security vulnerabilities.

Additional Notes

On dev with this change, pytest tests/utils tests/reading: 620 passed, 6 skipped (605 passed without it). The one failure, tests/reading/test_extension_router.py::test_async_reading_action_can_be_retried_after_timeout[True], fails the same way on dev without this change (seen on Windows).

Square brackets came out as parentheses, and a delimiter with several
items, such as f(a, b), kept only the first. Delimiters now use their
begChr/endChr/sepChr (OMML defaults to "(", ")" and "|"), and an
already bracketed base is not wrapped again: (a+b)^2, not ((a+b))^2.

Also formats the new tests with the repository's ruff and drops an
unrelated .gitignore entry.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant