Repository navigation
Conversation
Table detection used sheet.iter_rows and sheet.cell, which make openpyxl create a Cell for every empty position in the data bounds. Read sheet._cells directly instead, so memory is proportional to the filled cells. Output is unchanged. Signed-off-by: Christer Dreierstad <christerdr@outlook.com>
|
✅ DCO Check Passed Thanks @chdre, all your commits are properly signed off. 🎉 |
Merge Protections🟢 Merge protection satisfied — ready to merge. Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
AI triage (advisory)Checked commit The PR changes xlsx table detection in CI safety: no concern found. A maintainer must still approve CI. 1 related pull request(s) in the same code
The |
There was a problem hiding this comment.
AI first review (advisory): no blocker found
This review covers the full diff at 7dbc2abc65. It read the code and did not run it.
I checked table detection in MsExcelDocumentBackend for sparse sheets, coordinate traversal order, and merged-cell fallback handling. The change avoids creating empty Cell objects in sheet._cells and keeps table extraction results intact.
A maintainer makes the merge decision. This review never approves the pull request.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Signed-off-by: Christer Dreierstad <christerdr@outlook.com>
There was a problem hiding this comment.
AI first review (advisory): no blocker found
This review covers the changes since 7dbc2abc65 at 9840b009e2. It read the code and did not run it.
The incremental change adds one test (test_threaded_comment_fallback_without_xml) for the threaded-comment XML-absent fallback path. The test logic is correct: cells with values are already in _cells after load, comments are injected in-memory, and _parse_threaded_comments correctly returns empty because no XML part exists in the saved file. The earlier review found no blockers; the original change has no new behavioral code in this diff. The human reviewer should confirm that Codecov's 5 uncovered lines do not include critical branches from this PR's new code paths.
A maintainer makes the merge decision. This review never approves the pull request.
Table detection in
MsExcelDocumentBackendcalledsheet.iter_rows(...)andsheet.cell(...)over the whole data bounds. In openpyxl both create aCellfor every position they visit and keep it insheet._cells. A sheet with a few values in a large rectangle therefore grew by one cell object per empty position, about 200 MB per million cells, and that memory was not released.This change reads
sheet._cellsdirectly in_find_data_tablesand_find_table_bounds:_MergedCellIndex.contains_at.The output does not change.
_cellswas already read directly in_find_true_data_boundsand_value_bounds.Checklist
Review evidence
Measured with
DocumentConverter().convert()on generated workbooks, median of 3, tracemalloc (Python allocations only), Windows, Intel Core Ultra 9 285HX.Dense sheets are unchanged, as expected.
New tests in
tests/test_backend_msexcel.py:test_find_data_tables_does_not_materialize_empty_cells: the sheet's cell count is unchanged by table detection, and peak memory stays under 50 MiB. It fails on main.test_sparse_merged_anchor_and_comment_only_cell_tables: output equality for merges and comments in a sparse sheet.test_missing_merge_anchor_inside_table_keeps_span: a merge whose anchor cell is missing keeps its span. It fails withoutcontains_at.Old and new
_find_data_tablesoutput was also compared on all workbooks intests/dataand on 300 random sheets (values, merges, comments, gap tolerance 0 to 3), with 0 differences.Commands run:
uv run pytest tests/test_backend_msexcel.py tests/test_backend_msexcel_charts.py: 43 passed, 2 skipped.uv run prek run --files ...: all hooks passed.Not run:
test_e2e_excel_conversionspasses unchanged.Not included:
ExcelCellandTableCell.sheet.cell(); it is already capped byMAX_CHART_TABLE_CELLS.