Repository navigation
fix(msexcel): read the series of every chart plot - #4812
dolfim-ibm merged 7 commits into
Conversation
openpyxl returns the first plot of a combination chart as the chart, so chart.series held only that plot's series. A column + line chart lost its line series, and it was classified by the column plot only. Read every plot in chart._charts: give one prediction per plot type, back-most first, and put the series of every plot in the data table. The LibreOffice render workbook now also copies the data of every plot. Write date cells in the chart data as ISO 8601 dates instead of "2012-01-01 00:00:00", and put the category axis title in the top-left cell of the data table. Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
The data tables of the charts in xlsx_01, xlsx_03_chartsheet and
xlsx_04_inflated now have the category axis title in the top-left
cell ("col-1" and "Year"). No other value changes.
Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
|
✅ DCO Check Passed Thanks @DanielNg0729, all your commits are properly signed off. 🎉 |
Merge Protections🟢 All 2 merge protections satisfied — ready to merge. Show 2 satisfied protections🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
🟢 Require two reviewer for test updatesWhen test data is updated, we require two reviewers
|
AI triage (advisory)Checked commit The PR makes the Excel backend read the series of every plot of a combination chart. It adds one classification prediction per plot type, ISO 8601 dates and the category axis title in the chart table. It moves the shared chart table helper to CI safety: no concern found. A maintainer must still approve CI. 3 related pull request(s) in the same code
Reference data changes
Assessment: expected. The code change puts the category axis title in the top-left cell of the chart table. It also writes dates in ISO 8601 and can add columns for other plots. These changes match the changed table rows and item attributes in the three XLSX reference files. The summary does not show the exact cell values, so this verdict comes from the kind of change. The |
There was a problem hiding this comment.
AI first review (advisory): no blocker found
This review covers the full diff at f4a7dac001. It read the code and did not run it.
I found no defect that should stop the merge. I read the new plot loop in docling/backend/msexcel_backend.py, the openpyxl reader and plot-area code, and the new tests. I also checked the render-workbook change and the axis lookup for pie, scatter and 3D plots.
A maintainer makes the merge decision. This review never approves the pull request.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
No blocker. One security point (shared with Unbounded chart ranges (shared with
The PowerPoint backend trims tables and caps them with Suggestions:
The behavior matches the PowerPoint backend: one prediction per plot type, the main label from the back-most plot, the axis title in the top-left cell, and ISO dates. Classification and the top-left cell are also correct for 3-D, radar, stock, surface, bubble and bar-of-pie charts. |
Move _build_chart_table, _last_populated_row and _MAX_CHART_TABLE_CELLS from the PowerPoint backend to docling/backend/utils/charts.py, as build_chart_table and MAX_CHART_TABLE_CELLS. Add datetime_to_iso, which _serial_to_iso now uses. The Excel backend uses the same helpers in the next commits. The PowerPoint output does not change. Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
Replace the tests of the private helpers _chart_cell_text and _build_standalone_chart_workbook with tests of the converter output. A stand-in for LibreOffice keeps the workbook that it gets, so the render test runs without LibreOffice. The date test now also covers a date with a time of day. Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
A chart range had no size limit. A 6 KB file whose ranges end at row 1,000,000 made the backend read every cell and build a table of 1,000,000 rows. A secondary plot, which is now read, doubled the cost. - A range without an end row, or with more cells than its share, stops at the last used row of its sheet. The last used row is found once per sheet. - The ranges of a chart share MAX_CHART_TABLE_CELLS equally, so a chart reads at most that many cells. A range that is cut logs a warning. - The data table is built with build_chart_table, which trims trailing empty rows and caps the table, as in the PowerPoint backend. Dates use datetime_to_iso. - A whole-column range such as $B:$B now reads the used rows. Before, it raised an error and the chart was lost. Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
A title that is linked to a cell keeps its text in the cell cache (c:strRef), and openpyxl then gives it an empty rich text. The category axis title was therefore empty. Read the cache first, as the PowerPoint backend does. The chart title uses the same code, so a linked chart title now also becomes the caption. Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
|
hi Dr @dolfim-ibm, thank you very much for the review, i have added some commit to resolve it. Thanks! |
Hi maintainers, this PR fixes Excel combination charts, which lose the series of every plot after the first one. It is the Excel follow-up to #4773 (issue #4772), which fixed the same problem for PowerPoint.
Problem
openpyxl's
read_chartreturns the first plot of a chart as the chart object (chart = plot._charts[0]).chart.seriesholds only the series of that plot. The other plots are only inchart._charts. For a "Clustered Column - Line on Secondary Axis" chart, the backend onmaintherefore:bar_chart),render_chart_images=True, copies only the data of the first plot into the workbook that LibreOffice draws. openpyxl writes every plot into that workbook, so the line plot is drawn without its data.The chart data also has two smaller problems that #4773 fixed for PowerPoint:
datetime, so date categories come out as2012-01-01 00:00:00.Changes
This PR changes the Excel backend only.
_chart_plotsreturns every plot inchart._charts, in drawing order. openpyxl always sets_charts, so a chart with one plot gives a list of one.2012-01-01, or2012-01-01T12:30:00when the cell has a time of day. The sheet tables do not change._chart_title_text(chart)becomes_title_text(title), so the chart title and the axis title use the same code. The chart captions do not change.docs/usage/advanced_options.mdnow covers Excel charts too.The value axis titles and the plot type of each series are not in the output.
TabularChartMetaFieldhas no place for them (see the question in #4772).Change in the reference data
Three fixtures have charts with a titled category axis. Their data tables now have the axis title in the top-left cell. No other value changes. The
.itxtfiles do not change. The second commit contains only this change.xlsx_01.xlsxcol-1xlsx_03_chartsheet.xlsxYearxlsx_04_inflated.xlsxcol-1No existing fixture has a combination chart or date categories, so the new tests make their own workbook with openpyxl.
Tests
The new file
tests/test_backend_msexcel_charts.pyfollowstests/test_backend_pptx_charts.py. It saves a column chart with a line plot on a secondary axis, date categories and a category axis title.test_combo_chart_lists_each_plot_type_and_keeps_all_series: the predictions arebar_chart,line_chart, and the table has all 6 series.test_date_categories_are_iso_dates: the categories are2012-01-01,2012-02-01, ...test_category_axis_title_heads_the_category_column: the top-left cell is the axis title.test_render_workbook_holds_the_data_of_every_plot: the render workbook has the line series data. This test does not need LibreOffice.test_chart_cell_text: a date keeps its time of day only when it has one.All of these tests fail on
main.test_chartsheetintests/test_backend_msexcel.pynow also checks that the top-left cell isYear.Checklist: