Skip to content

perf(email): do not copy .msg attachment content - #5025

Open
chdre wants to merge 1 commit into
docling-project:mainfrom
chdre:perf/email-msg-skip-attachment-payload
Open

chdre wants to merge 1 commit into
docling-project:mainfrom
chdre:perf/email-msg-skip-attachment-payload

Conversation

@chdre

@chdre chdre commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

EmailDocumentBackend._msg_to_rfc822_bytes converts an Outlook .msg into an RFC 822 message and copies every attachment's bytes into it. The backend only ever reads the filename and MIME type of attachments, so the copy is wasted. The attachment content is held in memory several times: in the .msg object, in the EmailMessage, as the serialized bytes, and as the parsed message.

This change passes an empty payload to add_attachment and keeps the filename and MIME type. Output is unchanged, list_attachments still works, and no new option is added.

Checklist

  • Documentation has been updated, if necessary. (not needed)
  • Examples have been added, if necessary. (not needed)
  • Tests have been added, if necessary.

Review evidence

Measured with a full DocumentConverter conversion of a 60 MB .msg with one large attachment, list_attachments on, median of 3, tracemalloc, Windows, Intel Core Ultra 9 285HX. The file is private and not part of the PR.

Peak memory Wall time Markdown
Before 723.1 MB 7.3 s identical (sha256 prefix)
After 185.9 MB 1.6 s identical

With list_attachments off, peak memory is also 723.1 MB before and 185.9 MB after, with identical output.

New tests in tests/test_backend_email.py:

  • test_msg_to_rfc822_keeps_attachment_metadata_without_payload: filenames and content types remain, and neither the raw payload nor its base64 form is in the output. It fails on main.
  • test_msg_attachment_listing_leaves_the_body_unchanged_end_to_end: the body text is the same with listing on and off, and the listed attachments are still correct.

Commands run:

  • uv run pytest tests/test_backend_email.py: 29 passed.
  • uv run prek run --files ...: all hooks passed.

Not run:

  • The full test suite.

Not included:

  • .msg files with embedded message attachments were not tested.

The attachment payload was copied into the intermediate RFC 822 message but only the filename and MIME type are ever read. Pass an empty payload instead. Output is unchanged.

Signed-off-by: Christer Dreierstad <christerdr@outlook.com>
@github-actions

Copy link
Copy Markdown
Contributor

✅ DCO Check Passed

Thanks @chdre, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@github-actions github-actions Bot added ai:ci-safe AI triage found no CI-safety concern (advisory) performance labels Oct 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

AI triage (advisory)

Checked commit 5d6d919152. The analysis read the diff only. It did not run code from this pull request.

This pull request avoids copying attachment file bytes when EmailDocumentBackend converts .msg files to RFC 822 format. It retains only attachment filenames and MIME types to reduce peak memory usage.

Topic labels added: performance.

CI safety: no concern found. A maintainer must still approve CI.

The ai:* labels are advisory. They do not approve CI and they do not add tests:full. A new push runs this check again.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai:ci-safe AI triage found no CI-safety concern (advisory) performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant