Repository navigation
Conversation
Contributor
|
✅ DCO Check Passed Thanks @Kshitij3h, all your commits are properly signed off. 🎉 |
Contributor
Merge Protections🟢 Merge protection satisfied — ready to merge. Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Kshitij3h
force-pushed
the
feat/font-sanity-ocr-heuristics
branch
from
October 7, 2026 13:53
1e213f3 to
8507e5e
Compare
2 of 3 tasks
github-actions
Bot
added
ai:ci-safe
Contributor
AI triage (advisory)Checked commit The PR moves text quality rating into a new shared module and adds checks for broken fonts. These checks find private-use characters, control characters and symbol noise inside words. In PDF-aware OCR mode, the PR sends clusters with corrupt text to OCR. It also lets OCR text replace corrupt PDF text when cells merge. CI safety: no concern found. A maintainer must still approve CI. 1 related pull request(s) in the same code
The |
Problem: PDFs with missing/corrupted CMap tables, unmapped Private Use Area (PUA) fonts, or legacy 8-bit font remapping (e.g., legacy Indic KrutiDev/ShreeLipi fonts) contain raw glyph indices that PDF parsers decode into unreadable mojibake, control codes, or replacement characters. In PDF-aware OCR modes, these regions are falsely marked as having valid programmatic text, causing the engine to skip OCR extraction and output garbled content. Solution: Extend PagePreprocessingModel text quality evaluation with robust font sanity heuristics and integrate with BaseOcrModel to automatically detect corrupted text and route those clusters to OCR engines: - Add Unicode Private Use Area (PUA BMP & Plane 15/16) detection - Add unprintable C0/C1 control characters and ANSI escape sequence detection - Add intra-word symbol noise heuristic to catch 8-bit font remapping mojibake without false-positive penalization of multilingual text (accents, umlauts, CJK, Devanagari) or natural punctuation/numbers - Update BaseOcrModel._find_pdf_aware_layout_ocr_rects to inspect text quality and schedule OCR for layout clusters containing corrupt text - Update BaseOcrModel._merge_ocr_and_pdf_cells under PDF_FIRST priority to prioritize OCR cells over corrupt PDF cells while preserving clean native text - Add unit tests in tests/test_text_quality_and_font_sanity.py covering valid multilingual text, corrupt patterns, cell merging priority, and layout OCR routing Signed-off-by: Kshitij3h <kshitijpandey3h@gmail.com>
… presentation bullets Signed-off-by: Kshitij3h <kshitijpandey3h@gmail.com>
Kshitij3h
force-pushed
the
feat/font-sanity-ocr-heuristics
branch
from
October 11, 2026 11:34
bf1b5da to
09677d8
Compare
github-actions
Bot
removed
the
ai:possible-duplicate
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
AI first review (advisory): blocker found
This review covers the full diff at 09677d871b. It read the code and did not run it.
The font sanity heuristics trigger false-positive corruption scores on valid document content such as single-character bullet points, math exponents, Windows paths, and code tokens. In PDF-first cell merging, these false positives cause Docling to overwrite clean native PDF text with OCR results. Review whether cell-level quality filtering can be scoped to avoid regressing valid programmatic text.
A maintainer makes the merge decision. This review never approves the pull request.
|
|
||
| # Genuinely corrupt intra-word mojibake noise (e.g. backslash, caret, tilde, pipe, backtick inside words) | ||
| # Legitimate characters like &, +, =, -, *, /, $, % are explicitly permitted | ||
| MOJIBAKE_SYMBOL_RE = re.compile( |
Contributor
There was a problem hiding this comment.
blocker: Intra-word mojibake regex classifies valid symbols and short tokens as corrupt
The regex MOJIBAKE_SYMBOL_RE matches common characters between alphanumeric characters, such as carets (x^2), backslashes (C:\dir), pipes (a|b), and hashes (item#1). When rate_text_quality() processes single-word or short text cells containing these patterns, mojibake_count / len(tokens) meets the 0.25 threshold and scores the text as 0.0. In _merge_ocr_and_pdf_cells(), valid PDF cells with these characters are treated as corrupted and replaced by OCR. In PagePreprocessingModel._parse_page_cells(), page parse confidence drops to zero. Refine the heuristic to avoid flagging math exponents, Windows paths, identifiers, and common technical punctuation.
| return True | ||
|
|
||
| # If PUA chars dominate the text block (>= 20%), it is an unmapped font dump | ||
| if len(pua_chars) / len(non_space) >= 0.20: |
Contributor
There was a problem hiding this comment.
blocker: Standalone PUA bullet point and icon cells trigger false positive corruption
PDF parsers frequently emit bullet points and symbols as isolated single-character text cells (for example \uf0b7 or \uf002). For a single-character cell, len(pua_chars) / len(non_space) is 1.0 (100%), which satisfies >= 0.20. _has_corrupt_pua() returns True and rate_text_quality() marks the cell as corrupt (score 0.0). In _merge_ocr_and_pdf_cells(), standard bullet and icon cells are treated as corrupt PDF cells and overwritten by OCR. Require a minimum character count or cluster length before applying percentage-based PUA thresholds, or exempt single-codepoint bullet characters.
| if priority == _MergeCellsPriority.PDF_FIRST: | ||
| prioritized_cells, secondary_cells = visible_cells, ocr_cells | ||
| clean_pdf_cells = [ | ||
| c for c in visible_cells if rate_text_quality(c.text) >= 0.5 |
Contributor
There was a problem hiding this comment.
blocker: Cell merging applies text quality filtering to code and formula cells without layout label awareness
While _find_pdf_aware_layout_ocr_rects() skips code and formula clusters, _merge_ocr_and_pdf_cells() filters all visible PDF cells through rate_text_quality(). Code and formula cells containing backslashes, carets, pipes, or hash symbols are marked as corrupt. When OCR cells exist for the page, OCR text overwrites these native PDF code and formula cells. Preserve programmatic text in code and formula regions or avoid cell-level quality filtering where layout structure marks the region as code or formula.
3 tasks done
BBC-Esq
added a commit
to BBC-Esq/docling
that referenced
this pull request
Oct 21, 2026
This PR makes both Tesseract engines (`tesseract` and `tesserocr`) more reliable on colour pages and on rotated text, and it makes the `tesseract` engine report its failures. It has four changes. I measured each one against `main` on public PDFs and on Docling's own test PDFs, on Windows 10 with Python 3.12, the Tesseract 5.5.0 CLI and tesserocr 2.10.0 (Tesseract 5.5.2). **Before and after.** With forced (full-page) OCR, on a page with yellow highlighting, Tesseract found only 45% of the words (33% of the highlighted ones) in the colour image, against 97% (99% of the highlighted ones) in a grayscale copy. The table shows the share of the ground-truth words that appear in Docling's Markdown export, for all words / for the words under a highlighter mark: | Sample | OCR mode | `main` | This PR | |---|---|---|---| | Newtown draft charter, page 23 as a scanned image | full page | 45.4% / 33.0% | 97.4% / 99.3% | | Newtown draft charter, all 47 pages | full page | 83.7% / 93.8% | 96.8% / 99.5% | | Newtown colour-code legend | full page | 97.3% / 0% | 100% / 100% | | Homer council slide, text on coloured boxes | full page | 94.1% | 95.3% | | Scanned court exhibit, page 3 | full page | 96.2% / 100% | 96.2% / 100% | | Newtown draft charter, page 23 as a scanned image | default | 98.4% / 99.6% | 99.2% / 100% | | Scanned court exhibit, page 3 | default | 96.2% / 100% | 96.2% / 100% | No sample got worse. Forced OCR (`OcrMode.FULL_PAGE`, `--force-ocr`) gains the most. The default mode reads smaller layout regions, and there the colour effect was small in these tests. **1. Grayscale input.** Tesseract thresholds a colour image channel by channel. A yellow highlighter mark is dark in the blue channel, so it can turn into a black block that hides its text. Both engines now convert each OCR region to grayscale before orientation detection and OCR. Direct Tesseract runs on Docling's 216-DPI renders, with the OCR command that the `tesseract` engine runs on `main` (all words / highlighted words): | Sample | Colour | Grayscale | |---|---|---| | Newtown draft charter, 36 highlighted pages (20,314 words, 5,109 highlighted) | 82.6% / 71.1% | 97.6% / 99.5% | | Same document, page 23 | 46.2% / 33.0% | 99.8% / 100% | | Newtown colour-code legend | 97.3% / 0% | 100% / 100% | | Scanned court exhibit, page 3 (19 highlighted words) | 89.9% / 57.9% | 95.9% / 100% | | Homer council slide | 100% | 100% | Grayscale was better on 33 of the 36 charter pages, equal on 2, and worse on 1 (page 46: 95.6% → 95.0%). On a synthetic page with three highlighted lines, the colour image gave 21% of the highlighted words and the grayscale image 100%, with each of five highlighter colours (three yellows, green and pink). With six highlighted lines it was 12% against 100%. A single highlighted line was read in full either way. On 75 ordinary pages from Docling's test PDFs (40,358 words, both arms with the DPI from change 2), grayscale still gained: 91.14% → 91.77%, with 17 pages better and 8 worse. Grayscale has a cost. In colour mode, Tesseract treats a pixel as text when any one channel is dark, so it reads light text on a strongly coloured box and light, saturated text on white; grayscale lowers the contrast of both. On a synthetic page with black body text, lines in orange, yellow, pink or light blue were read in full in colour but only 12–17% in grayscale, while red, blue, green, cyan and grey lines were read in full either way. On the real pages, the largest loss was the white labels on coloured boxes in Figure 4 of arXiv 2203.01017 (page 5): 31 words in direct Tesseract output. Those labels are inside a picture, so they do not reach the Markdown export on `main` or on this branch. I also tried Tesseract's Sauvola thresholding (`-c thresholding_method=2`) instead of plain grayscale, both with the DPI. It reads the orange, pink and light-blue lines (not the yellow ones), and it matched grayscale on the real pages: 97.6% on the highlighted charter pages, 91.7% against 91.8% on the 75 ordinary pages, and 99.1% against 97.7% on the slide, exhibit and legend. But it made more of the ordinary pages worse than `main` (16 against 8), so this PR keeps plain grayscale. **2. Resolution.** Docling renders OCR regions at 72 DPI × `scale` (216 DPI by default) but did not tell Tesseract, so Tesseract estimated the resolution from the text size; on 216-DPI renders it estimated, for example, 229 and 272 DPI. Both engines now pass the real value: `--dpi` for the CLI and `SetSourceResolution` for tesserocr. The effect is a small net gain, and it is mixed page by page (both arms with grayscale input): | Test set | Without the DPI | With the DPI | Better / worse | |---|---|---|---| | 202 layout regions from 16 documents, 41,590 words | 90.95% | 91.61% | 17 / 10 | | 75 full pages from 14 documents, 40,358 words | 91.62% | 91.77% | 10 / 8 | The largest gain was a table region, from 62.5% to 92.1% of its 331 words. The largest losses were 14 words in one region and 9 words on one page. On the Homer slide above, the DPI cost 4 words in Docling (97.6% with grayscale alone, 95.3% with both). I recommend it all the same: the net effect is positive in both sets, the largest single effect is a gain, and Tesseract no longer has to guess a resolution that Docling knows. **3. Orientation confidence.** Docling runs Tesseract's orientation and script detection (OSD) on every OCR region, and it rotated the region on any detection. OSD reports its orientation confidence as the margin between its best and second-best orientation, so a value near 0 is a close call. The new option `orientation_confidence_threshold` (default 1.0) on `TesseractCliOcrOptions` and `TesseractOcrOptions` ignores detections below it; `0` restores the old behaviour. I ran OSD on 277 layout regions from 16 documents and on 90 full pages from 17 documents, as rendered (all upright) and rotated by 90°, 180° and 270°: | | Layout regions | Full pages | |---|---|---| | Upright text turned the wrong way, `main` | 2 of 214 (confidence 0.06 and 1.17) | not measured | | Upright text turned the wrong way, this PR | 1 of 213 (confidence 0.0, now ignored) | 0 of 90 | | Rotated copies corrected, threshold 0 (old behaviour) | 632 of 639 | 268 of 270 | | Rotated copies corrected, threshold 1.0 (new default) | 619 of 639 | 265 of 270 | | Rotated copies corrected, threshold 14 (OCRmyPDF's page default) | 392 of 639 | 220 of 270 | The counts cover the regions and copies on which OSD returned a result. A threshold of 14 would drop 38% of the correct region rotations and 18% of the correct page rotations. With 1.0, the rotated copies that are no longer corrected hold 84 of the 114,209 words that Tesseract reads in all correctly rotated copies (0.07%), while the region that OSD turned the wrong way would have lost all 11 of its words. A region that is really sideways, the arXiv identifier in the margin of a paper (confidence 2.97), is still rotated and read in full. **4. Failures are reported.** When Tesseract failed on a region in the `tesseract` engine, Docling only logged an error, and the document still ended as `SUCCESS` without that text. Now the engine also adds an `ErrorItem` for the page (`DoclingComponentType.MODEL`, `FailureCategory.INFERENCE_FAILURE`), the same way the KServe OCR engine reports a failed region, so the document ends as `PARTIAL_SUCCESS`. The engine now keeps Tesseract's stderr, so the error carries Tesseract's own message. Tesseract exits with 0 on blank, tiny and all-black regions, so the error marks real failures only. In the `tesserocr` engine, a failure raises an exception, which the pipeline already records for the page. None of the reference conversions reported an error. An empty `lang` list still skips, without an error, a region whose script OSD cannot detect. OSD gives up on regions with too few characters, mostly pictures, and reporting those skips would turn 3 of 5 ordinary test documents (converted with `lang=[]`) into `PARTIAL_SUCCESS`. So this PR leaves that case unchanged. **Related pull requests:** - docling-project#4709 adds orientation detection for RapidOCR (`detect_orientation`, off by default). It addresses the same rotated-text problem for another engine and shares no code with this PR. Both PRs change `pipeline_options.py` and `docs/concepts/OCR.md`, but in different sections, and they merge cleanly. - docling-project#4654 and docling-project#4643 detect a broken PDF text layer and send it to OCR, docling-project#4654 per page and docling-project#4643 per region. This PR does not do that; the PRs complement each other. With a Tesseract engine, the pages and regions that they send to OCR get the changes in this PR. A page that docling-project#4654 replaces is OCR'd in full, which is where the grayscale input gains the most. **Review evidence:** - Reference data: unchanged. `tests/test_e2e_ocr_conversion.py` passes against the existing ground truth for every Tesseract configuration of both engines (default, layout-regions and full-page modes, and full-page mode with `lang=[]`), and for EasyOCR and RapidOCR. - New tests: 6 unit tests in `tests/test_tesseract_ocr_cli_model.py` and `tests/test_tesseract_ocr_model.py` (grayscale input, the DPI, the threshold and the error record) fail on `main` and pass here. `tests/test_e2e_tesseract_highlighted_text.py` OCRs a synthetic page with three highlighted lines (`tests/data/ocr/sources/highlighted_text.png`, 63 KB, regenerated by running the test module); on `main`, both engines miss 26 of its 42 highlighted words. - `tests/test_e2e_rotated_ocr.py`: same result as `main` (90° and 270° pass; 180° is still the expected failure of docling-project#3839). - One run of the tests above plus `test_tesseract_ocr_cli_lang`, `test_options`, `test_service_datamodels`, `test_ocr_options_force_full_page`, `test_ocr_engine_language_mapping`, `test_ocr_language` and `test_backend_webp`: 306 passed, 6 skipped, 2 xfailed (the 180° cases). - Checks: the prek hooks (ruff, ruff format, ty, tach), tach module coverage and max-lines pass. - Not run: the full test suite, Linux, macOS and Tesseract 4. **Samples.** The ground truth is the PDF text layer, or a manual transcription for image-only pages. The highlighted words are the words under a yellow highlighter mark. The public sources: - Newtown, CT, 2015 draft charter (47 pages): https://cmsarchive.civicplus.com/Newtown%20CT/NewtownCT_BBoard/News%20Archive/S052C770B.2/NEWTOWN%20CHARTER%20Revision%20Commission%202015%20Draft%20%20Charter%203.pdf - Newtown, CT, colour-code legend: https://cmsarchive.civicplus.com/Newtown%20CT/NewtownCT_BBoard/News%20Archive/S052C770B.4/Color%20Codes%207-10-15%205.pdf - City of Homer, AK, council supplemental packet (the slide is page 5): https://www.cityofhomer-ak.gov/sites/default/files/fileattachments/city_council/meeting/75366/supplemental_packet.pdf - "Page 23 as a scanned image" is page 23 of the draft charter, rasterized at 300 DPI into an image-only PDF. - Scanned court exhibit, U.S. District Court for the Northern District of California, No. 4:24-cv-04722, ECF No. 379-75 (the highlighted line is on page 3): https://storage.courtlistener.com/recap/gov.uscourts.cand.433688/gov.uscourts.cand.433688.379.75.pdf **Checklist:** - [x] Documentation has been updated, if necessary. - [x] Examples have been added, if necessary. - [x] Tests have been added, if necessary. Signed-off-by: BBC, Esquire <bbc@chintellalaw.com>
3 tasks done
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extends
PagePreprocessingModeltext quality evaluation with robust font sanity heuristics and integrates withBaseOcrModelto automatically detect corrupted text / broken font CMaps (e.g. legacy Indic KrutiDev/ShreeLipi fonts, unmapped Private Use Area fonts, or control character noise) and route those regions to OCR engines.Problem
PDFs with missing/corrupted CMap tables, unmapped Private Use Area (PUA) fonts, or legacy 8-bit font remapping contain raw glyph indices that PDF parsers decode into unreadable mojibake, control codes, or replacement characters. In PDF-aware OCR modes (e.g.
OcrMode.PDF_AWARE_LAYOUT_REGIONS), these regions are falsely treated as valid programmatic text, causing the engine to skip OCR extraction and output garbled content.Solution & Key Changes
PagePreprocessingModel):\uE000-\uF8FFin BMP and Plane 15/16).\x00-\x1F,\x7F-\x9F) and ANSI escape sequences.rate_text_quality_staticfor static inspection across pipeline stages.BaseOcrModel):_find_pdf_aware_layout_ocr_rects(), evaluate text quality for programmatic text clusters and automatically schedule OCR for clusters containing corrupt text._merge_ocr_and_pdf_cells()underPDF_FIRSTpriority, prioritize clean programmatic text over OCR, and OCR over corrupt programmatic text.tests/test_text_quality_and_font_sanity.pytesting multilingual validity, corrupt pattern heuristics, cell merging priority, and layout OCR routing.Test Plan
uv run pytest tests/test_text_quality_and_font_sanity.py(20 passed)uv run pytest tests/test_skip_cell_extraction.py tests/test_ocr_options_force_full_page.py(25 passed)uv run --group pr-fast-checks ruff format ...andruff check ...(Passed)uv run --group typecheck ty check ...(Passed)