[cau-git]

Fixes #4928.

Cause

On scanned forms whose text layer holds only the filled-in values, tables lost their OCR text and column headers. Three problems kept OCR text out of the table tokens.

1. Full-page OCR: regression from #4061 (first released in 2.122.0). That change skips the native segmented-page decode in full-page OCR mode, and OCR fills a separate page.parsed_page. TableStructureModel still called page._backend.get_segmented_page(). That call re-ran the native decode and returned the native word cells that OCR is meant to replace.

2. Default and layout-region OCR: on every version. OCR merges its text into textline_cells only. word_cells stays native. The table model reads word_cells, so one native word in the table area (a filled-in value) kept all OCR text out of the table.

3. OCR engines with word output. An OCR engine may report word cells next to its line cells: lines through post_process_cells, and words with from_ocr=True in parsed_page.word_cells. Once the OCR lines were added to the table tokens, these OCR words were replaced by their coarser lines.

Fix

• Read page.parsed_page instead of the backend's segmented page. Both PDF backends cache the segmented page, so outside full-page OCR mode this is the same object as before. The native decode that fix: skip the native segmented-page decode in full-page OCR mode #4061 wanted to skip no longer runs again from the table stage.

• Build the table tokens from the finest cells of the text that the OCR merge kept:

  • A native word under a surviving OCR line is dropped.
  • An OCR word on a kept native word is dropped, as the merge rejected its line there. Otherwise it would double the native text.
  • An OCR line is dropped where OCR words cover it.

  Table tokens now follow the same OCR/native precedence as the page text lines. Before, a hidden text layer inside a table could win over OCR in the table but not elsewhere on the page.

• Number the tokens by position. Native word and OCR cell indices can collide, and the cell matcher keys tokens by id.

TableStructureModelV2 is not affected. It reads the cluster cells, which already include OCR.

Validation

Reporter's files, EasyOCR, table on file 1:

version	--force-ocr	default OCR
2.117.0 to 2.121.0	26×13, all column headers	7×9, no column headers
2.122.0 to 2.142.1	7×9, no column headers	7×9, no column headers
this branch	26×13, all column headers	26×13, all column headers

Files 2 and 3 give the same pattern. The native values land in the correct cells and are not duplicated. I bisected on released wheels with identical docling-parse, docling-core and docling-ibm-models versions. The regression stays when docling-ibm-models is pinned back to 3.15.0.

tests/data/pdf/sources/*.pdf with default OCR (EasyOCR), main vs this branch: 18 of 20 Markdown outputs are byte-identical. Two change, both in tables where OCR ran:

• 2203.01017v2.pdf: a table that is a picture now gets its OCR text ("Observer 2").
• table_misidentified_as_form.pdf: a scanned page with a hidden text layer. The table now uses the OCR text, as the body text on the same page already did. Per cell, the OCR quality is mixed.

The e2e ground-truth tests run with do_ocr = False, so their outputs do not change.

New tests in tests/test_table_structure_model.py:

• Token rules for OCR with line output and for OCR with word output.
• An end-to-end test with a word-level OCR engine plugged into StandardPdfPipeline. It reads the PDF text layer back as OCR lines and words, and its tables must equal the run without OCR, in full-page and in default OCR mode.

[github-actions]

✅ DCO Check Passed

Thanks @cau-git, all your commits are properly signed off. 🎉

[mergify]

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[github-actions]

AI triage (advisory)

Checked commit 679ef1039d. The analysis read the diff only. It did not run code from this pull request.

This PR keeps OCR text in table tokens across all OCR modes. It also updates TableStructureModel to read page.parsed_page instead of decoding native cells again.

CI safety: no concern found. A maintainer must still approve CI.

1 related pull request(s) in the same code

PR	State	Signal	Assessment
#4061	merged	referenced as #4061	related: PR #4061 skipped native cell extraction in full-page OCR mode. This PR updates TableStructureModel to read page.parsed_page and preserve OCR text in table tokens.

_{The ai:* labels are advisory. They do not approve CI and they do not add tests:full. A new push runs this check again.}

[github-actions]

AI first review (advisory): no blocker found

This review covers the full diff at f52b8f22f3. It read the code and did not run it.

A one-line change fixes the regression: TableStructureModel.predict_tables now reads page.parsed_page (which OCR has already updated in full-page OCR mode) instead of calling page._backend.get_segmented_page() (which re-runs the native decode and returns native words). The fix is correct, the Page contract supports it, and the OCR post-processing path writes to page.parsed_page before table structure runs. The one thing to check: there is no unit-level regression test for this specific code path in TableStructureModel, unlike the similar skip-cell-extraction fix in test_skip_cell_extraction.py.

_{A maintainer makes the merge decision. This review never approves the pull request.}

[github-actions]

suggestion: No regression test for the fixed code path

The fix is correct, but no unit test guards against re-introducing the old call. The project already has a precedent: tests/test_skip_cell_extraction.py tests the same page.parsed_page routing with lightweight mocks and no real PDF.

A focused test could construct a Page with parsed_page containing only OCR cells (set word_cells with from_ocr=True), set page._backend to a mock whose get_segmented_page returns a different set of cells, call a lightweight wrapper or inspect sp directly, and assert that sp equals page.parsed_page. This would catch any regression back to page._backend.get_segmented_page() without requiring the full TableFormer model.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[github-actions]

AI first review (advisory): no blocker found

This review covers the changes since f52b8f22f3 at 581f22e3e7. It read the code and did not run it.

Two of the three sub-fixes are correctly implemented. Fix #1 (page.parsed_page instead of page._backend.get_segmented_page()) still has no dedicated regression test; the new test only covers _words_and_ocr_cells (fix #2). The human reviewer should verify that the page.parsed_page path is exercised by at least the existing integration tests.

Earlier findings

• open (suggestion): No regression test for the fixed code path (page.parsed_page)

_{A maintainer makes the merge decision. This review never approves the pull request.}

[tozhangkai]

@cau-git I found the fix is incomplete - it works well with OCR engines with line output like easy ocr, but on OCR engines with word level output, the word cells are ignored. I have a fix that works well in my use case. Please review and let me know if you are Ok with it:

table_structure_model.new-fix.py