Repository navigation
feat(ocr): detect page orientation for RapidOCR - #4709
gkswns0531 wants to merge 3 commits into
Conversation
|
✅ DCO Check Passed Thanks @gkswns0531, all your commits are properly signed off. 🎉 |
Merge Protections🟢 Merge protection satisfied — ready to merge. Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
github-actions
Bot
added
ai:ci-safe
AI triage (advisory)Checked commit Adds CI safety: no concern found. A maintainer must still approve CI. 4 related pull request(s) in the same code
The |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
AI first review (advisory): no blocking issue found
This review covers the full diff at ea16e95541. It read the code only. It did not run tests or code from this pull request.
The PR adds RapidOcrOptions.detect_orientation and orientation_min_confidence. When the option is on, each OCR region goes through Tesseract OSD and is turned upright before RapidOCR reads it. The boxes are then mapped back with the existing Tesseract helpers. I found no blocker or major problem. With the option off, the geometry code gives the same result as before. The main gaps are weak coverage of the box mapping, a skip guard in the new test that is too loose, and an OSD reader that is never closed. I did not run any test or command.
Findings outside the diff lines
- nit
docs/concepts/OCR.md:195: Name the exact install extra
The text says "thetesserocrextra". The extra inpyproject.tomlisfeat-ocr-tesserocr(fordocling-slim). Name it exactly, and add the system package hint forosd, so that readers can install it without a search.
A maintainer makes the merge decision. This review never approves the pull request.
| text = doc.export_to_markdown( | ||
| included_content_layers={ContentLayer.BODY, ContentLayer.FURNITURE} | ||
| ) | ||
| assert text.count(_MARKER) == 3, f"RapidOCR should read all three lines at {angle}°" |
There was a problem hiding this comment.
minor: New test does not check the box mapping
The test only counts _MARKER in the Markdown export. The riskiest new code is _line_rect, which maps boxes from the turned region back onto the page. A wrong rotation or offset can still give a correct text count. The text order and DoclingDocument item order are not checked either.
Add an assertion on the OCR cell rectangles for at least the 90° and 180° fixtures. For example, check that each cell lies inside the page. Alternatively, compare the rectangles with the ones that the Tesseract engine gives for the same fixture.
The 270° case may not prove the fix. The PR text says RapidOCR already reads 3 of 3 lines at 270° on main. The 90° and 180° cases are the real regression checks.
| and importlib.util.find_spec("rapidocr") is None, | ||
| reason="rapidocr not installed", | ||
| ) | ||
| _SKIP_NO_TESSEROCR = pytest.mark.skipif( |
There was a problem hiding this comment.
minor: Skip guard does not cover the osd traineddata
_SKIP_NO_TESSEROCR only checks that tesserocr can be imported. The model also needs the osd traineddata. _create_osd_reader raises ImportError without it. On a machine that has tesserocr but no osd data, the test fails instead of skipping.
Extend the guard to also check for osd. For example, check tesserocr.get_languages() inside a helper. The existing _SKIP_NO_TESSERACT guard has the same weak point, so you can match its style.
| ) | ||
|
|
||
| # Orientation detection must not change the upright page | ||
| configs.append((RapidOcrOptions(detect_orientation=True), False)) |
There was a problem hiding this comment.
minor: New e2e configs have no guard for tesserocr or osd
The two new RapidOcrOptions(detect_orientation=True) configs run without a guard. If tesserocr or osd is missing, RapidOcrModel.__init__ raises ImportError and the whole test_e2e_conversions fails. The Tesseract configs in the same list already need tesserocr, so CI is probably fine. The two new configs are still the only ones that fail with the RapidOCR-only setup.
The configs also use only the default backend. The existing RapidOCR configs loop over onnxruntime and torch. Decide whether this is intended.
Because supports_rotation is False, the rotated PDFs are skipped. These two configs therefore only prove that upright pages do not change. They do not prove the rotated-page mapping.
| ) | ||
|
|
||
| if self.options.detect_orientation: | ||
| self._osd_reader = _create_osd_reader() |
There was a problem hiding this comment.
minor: OSD reader is never closed
_create_osd_reader() creates a tesserocr.PyTessBaseAPI, but nothing calls End() on it. TesseractOcrModel.__del__ closes its readers (see tesseract_ocr_model.py, __del__). The native Tesseract memory of this reader stays allocated until the garbage collector runs the destructor. Add a __del__ that calls self._osd_reader.End() when the reader is not None, as the Tesseract model does.
| def _detect_orientation(self, image: Image.Image) -> int: | ||
| """The turn that makes the region upright, in the convention of the Tesseract engines. | ||
|
|
||
| 0 when `detect_orientation` is off, when OSD finds too little text, or when its |
There was a problem hiding this comment.
minor: Silent fallback when OSD returns None
When DetectOrientationScript() returns None, _detect_orientation returns 0 and logs nothing. TesseractOcrModel logs an error in this case. A user who turns the option on cannot tell why a page stays sideways. Add a _log.debug or _log.warning for the None case and for the low-confidence case. Keep it short and avoid per-line logs.
| return RapidOcrOptions | ||
|
|
||
|
|
||
| def _create_osd_reader() -> Any: |
There was a problem hiding this comment.
minor: OSD reader ignores a custom tessdata path
_create_osd_reader calls tesserocr.get_languages() and PyTessBaseAPI(lang="osd", ...) with the default data path. Only TESSDATA_PREFIX is honored. TesseractOcrOptions has a path option for this. RapidOcrOptions has no such option. Users with a non-default tessdata directory must set the environment variable. The error message already says this. Consider documenting it in docs/concepts/OCR.md, or add a path option later.
| return RapidOcrOptions | ||
|
|
||
|
|
||
| def _create_osd_reader() -> Any: |
There was a problem hiding this comment.
nit: Any hides the reader type
_osd_reader and _create_osd_reader use Any. AGENTS.md prefers explicit contracts. tesserocr is optional, so a TYPE_CHECKING import with tesserocr.PyTessBaseAPI | None is possible. tesseract_ocr_model.py already annotates tesserocr.PyTessBaseAPI. Use the same style if ty accepts it.
github-actions
Bot
added
the
ai:review-lgtm
| RapidOcrModelSizeNotSupportedError, | ||
| ) | ||
| from docling.models.base_ocr_model import BaseOcrModel | ||
| from docling.models.stages.ocr.tesseract_utils import ( |
There was a problem hiding this comment.
does this mean we have to also download the tesseract models/code to run RapidOCR?
There was a problem hiding this comment.
@PeterStaar-IBM
No. The import here is tesseract_utils, which only holds the angle and box helpers the Tesseract engines use. It is plain Python and does not load tesserocr. tesserocr and the osd traineddata are needed only when detect_orientation is turned on, and the option is off by default, so nothing changes for the default RapidOCR path.
What the option does: before RapidOCR reads a region, Tesseract's orientation detection (OSD) says which way the text faces. If it is confident, the region is turned upright, RapidOCR reads it, and the boxes are mapped back onto the page. This is the step the Tesseract engines already run on every region.
Why add it: today a sideways scan is read with Tesseract or ocrmac, but comes out garbled with RapidOCR. Docling's own tests reflect this gap: test_e2e_conversions runs the rotated sample PDFs for Tesseract, Tesseract CLI and ocrmac, but skips them for RapidOCR. With the option, RapidOCR users can get those pages read without switching engines.
If the extra dependency or anything else is a concern, please let me know. I'm happy to adjust the PR in whatever way works best for the project.
There was a problem hiding this comment.
@gkswns0531 Thanks for the info and I definitely support this feature. My only concern, and I would like that we double check this, is that if docling is installed with selected packages (say we select rapid-ocr here https://github.com/docling-project/docling/blob/main/pyproject.toml#L199) and not install the tesserocr (see here: https://github.com/docling-project/docling/blob/main/pyproject.toml#L218), it will break.
Addtionally, I would like to have no dependency of rapidocr on tesseract. So my question is: can tesseract_utlils be installed seperately of tesserocr or would it make sense to re-implement the orientation feature?
There was a problem hiding this comment.
@PeterStaar-IBM Thanks, I checked both points and updated the PR.
The new version has no Tesseract dependency: rapid_ocr_model.py imports nothing from the Tesseract modules, and the option needs no extra package or model. In a clean docling-slim[standard] install (RapidOCR only, no tesserocr), conversion works as before, and with detect_orientation=True it now reads the turned test page.
The orientation now comes from RapidOCR's own confidence scores. RapidOCR reads the region as it is. If it finds at least 3 lines and fewer than half of them score 0.8 or more, it reads the region again at the other three orientations, most likely one first. It keeps the first orientation that reads well with at least twice as much confident text. A region that reads well the first time is read only once.
On the scans I used for the Tesseract version, it brings back all 17 sideways pages (Tesseract OSD: 16), each with one extra read, and changes none of the other 180 pages. The details are in the PR description.
There was a problem hiding this comment.
Hi @PeterStaar-IBM, thank you for the earlier review. I've updated the PR so that RapidOCR no longer depends on Tesseract (details in my reply above). When you have a moment, would you mind taking another look? I'm happy to rebase onto main or adjust anything else.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
ea16e95 to
6be1b3c
Compare
gkswns0531
changed the title
|
|
||
| # detect_orientation: a line with at least this score is read with confidence, and a | ||
| # region with fewer lines is too small to judge its orientation. | ||
| _CONFIDENT_SCORE = 0.8 |
There was a problem hiding this comment.
Can you please rename _CONFIDENT_SCORE as _ORIENTATION_CONFIDENCE_SCORE.
This will help to identify the semantic of this variable.
There was a problem hiding this comment.
Thanks for the review, @nikos-livathinos. I've renamed "_CONFIDENT_SCORE" to "_ORIENTATION_CONFIDENCE_SCORE"
github-actions
Bot
added
ai:ci-safe
|
Hi @PeterStaar-IBM, a gentle follow-up on this PR. RapidOCR no longer depends on Tesseract (details in the review thread), and Nikos has approved it. Whenever you have a moment, would you mind taking another look so your earlier change request can be resolved? Thank you! |
This PR makes both Tesseract engines (`tesseract` and `tesserocr`) more reliable on colour pages and on rotated text, and it makes the `tesseract` engine report its failures. It has four changes. I measured each one against `main` on public PDFs and on Docling's own test PDFs, on Windows 10 with Python 3.12, the Tesseract 5.5.0 CLI and tesserocr 2.10.0 (Tesseract 5.5.2). **Before and after.** With forced (full-page) OCR, on a page with yellow highlighting, Tesseract found only 45% of the words (33% of the highlighted ones) in the colour image, against 97% (99% of the highlighted ones) in a grayscale copy. The table shows the share of the ground-truth words that appear in Docling's Markdown export, for all words / for the words under a highlighter mark: | Sample | OCR mode | `main` | This PR | |---|---|---|---| | Newtown draft charter, page 23 as a scanned image | full page | 45.4% / 33.0% | 97.4% / 99.3% | | Newtown draft charter, all 47 pages | full page | 83.7% / 93.8% | 96.8% / 99.5% | | Newtown colour-code legend | full page | 97.3% / 0% | 100% / 100% | | Homer council slide, text on coloured boxes | full page | 94.1% | 95.3% | | Scanned court exhibit, page 3 | full page | 96.2% / 100% | 96.2% / 100% | | Newtown draft charter, page 23 as a scanned image | default | 98.4% / 99.6% | 99.2% / 100% | | Scanned court exhibit, page 3 | default | 96.2% / 100% | 96.2% / 100% | No sample got worse. Forced OCR (`OcrMode.FULL_PAGE`, `--force-ocr`) gains the most. The default mode reads smaller layout regions, and there the colour effect was small in these tests. **1. Grayscale input.** Tesseract thresholds a colour image channel by channel. A yellow highlighter mark is dark in the blue channel, so it can turn into a black block that hides its text. Both engines now convert each OCR region to grayscale before orientation detection and OCR. Direct Tesseract runs on Docling's 216-DPI renders, with the OCR command that the `tesseract` engine runs on `main` (all words / highlighted words): | Sample | Colour | Grayscale | |---|---|---| | Newtown draft charter, 36 highlighted pages (20,314 words, 5,109 highlighted) | 82.6% / 71.1% | 97.6% / 99.5% | | Same document, page 23 | 46.2% / 33.0% | 99.8% / 100% | | Newtown colour-code legend | 97.3% / 0% | 100% / 100% | | Scanned court exhibit, page 3 (19 highlighted words) | 89.9% / 57.9% | 95.9% / 100% | | Homer council slide | 100% | 100% | Grayscale was better on 33 of the 36 charter pages, equal on 2, and worse on 1 (page 46: 95.6% → 95.0%). On a synthetic page with three highlighted lines, the colour image gave 21% of the highlighted words and the grayscale image 100%, with each of five highlighter colours (three yellows, green and pink). With six highlighted lines it was 12% against 100%. A single highlighted line was read in full either way. On 75 ordinary pages from Docling's test PDFs (40,358 words, both arms with the DPI from change 2), grayscale still gained: 91.14% → 91.77%, with 17 pages better and 8 worse. Grayscale has a cost. In colour mode, Tesseract treats a pixel as text when any one channel is dark, so it reads light text on a strongly coloured box and light, saturated text on white; grayscale lowers the contrast of both. On a synthetic page with black body text, lines in orange, yellow, pink or light blue were read in full in colour but only 12–17% in grayscale, while red, blue, green, cyan and grey lines were read in full either way. On the real pages, the largest loss was the white labels on coloured boxes in Figure 4 of arXiv 2203.01017 (page 5): 31 words in direct Tesseract output. Those labels are inside a picture, so they do not reach the Markdown export on `main` or on this branch. I also tried Tesseract's Sauvola thresholding (`-c thresholding_method=2`) instead of plain grayscale, both with the DPI. It reads the orange, pink and light-blue lines (not the yellow ones), and it matched grayscale on the real pages: 97.6% on the highlighted charter pages, 91.7% against 91.8% on the 75 ordinary pages, and 99.1% against 97.7% on the slide, exhibit and legend. But it made more of the ordinary pages worse than `main` (16 against 8), so this PR keeps plain grayscale. **2. Resolution.** Docling renders OCR regions at 72 DPI × `scale` (216 DPI by default) but did not tell Tesseract, so Tesseract estimated the resolution from the text size; on 216-DPI renders it estimated, for example, 229 and 272 DPI. Both engines now pass the real value: `--dpi` for the CLI and `SetSourceResolution` for tesserocr. The effect is a small net gain, and it is mixed page by page (both arms with grayscale input): | Test set | Without the DPI | With the DPI | Better / worse | |---|---|---|---| | 202 layout regions from 16 documents, 41,590 words | 90.95% | 91.61% | 17 / 10 | | 75 full pages from 14 documents, 40,358 words | 91.62% | 91.77% | 10 / 8 | The largest gain was a table region, from 62.5% to 92.1% of its 331 words. The largest losses were 14 words in one region and 9 words on one page. On the Homer slide above, the DPI cost 4 words in Docling (97.6% with grayscale alone, 95.3% with both). I recommend it all the same: the net effect is positive in both sets, the largest single effect is a gain, and Tesseract no longer has to guess a resolution that Docling knows. **3. Orientation confidence.** Docling runs Tesseract's orientation and script detection (OSD) on every OCR region, and it rotated the region on any detection. OSD reports its orientation confidence as the margin between its best and second-best orientation, so a value near 0 is a close call. The new option `orientation_confidence_threshold` (default 1.0) on `TesseractCliOcrOptions` and `TesseractOcrOptions` ignores detections below it; `0` restores the old behaviour. I ran OSD on 277 layout regions from 16 documents and on 90 full pages from 17 documents, as rendered (all upright) and rotated by 90°, 180° and 270°: | | Layout regions | Full pages | |---|---|---| | Upright text turned the wrong way, `main` | 2 of 214 (confidence 0.06 and 1.17) | not measured | | Upright text turned the wrong way, this PR | 1 of 213 (confidence 0.0, now ignored) | 0 of 90 | | Rotated copies corrected, threshold 0 (old behaviour) | 632 of 639 | 268 of 270 | | Rotated copies corrected, threshold 1.0 (new default) | 619 of 639 | 265 of 270 | | Rotated copies corrected, threshold 14 (OCRmyPDF's page default) | 392 of 639 | 220 of 270 | The counts cover the regions and copies on which OSD returned a result. A threshold of 14 would drop 38% of the correct region rotations and 18% of the correct page rotations. With 1.0, the rotated copies that are no longer corrected hold 84 of the 114,209 words that Tesseract reads in all correctly rotated copies (0.07%), while the region that OSD turned the wrong way would have lost all 11 of its words. A region that is really sideways, the arXiv identifier in the margin of a paper (confidence 2.97), is still rotated and read in full. **4. Failures are reported.** When Tesseract failed on a region in the `tesseract` engine, Docling only logged an error, and the document still ended as `SUCCESS` without that text. Now the engine also adds an `ErrorItem` for the page (`DoclingComponentType.MODEL`, `FailureCategory.INFERENCE_FAILURE`), the same way the KServe OCR engine reports a failed region, so the document ends as `PARTIAL_SUCCESS`. The engine now keeps Tesseract's stderr, so the error carries Tesseract's own message. Tesseract exits with 0 on blank, tiny and all-black regions, so the error marks real failures only. In the `tesserocr` engine, a failure raises an exception, which the pipeline already records for the page. None of the reference conversions reported an error. An empty `lang` list still skips, without an error, a region whose script OSD cannot detect. OSD gives up on regions with too few characters, mostly pictures, and reporting those skips would turn 3 of 5 ordinary test documents (converted with `lang=[]`) into `PARTIAL_SUCCESS`. So this PR leaves that case unchanged. **Related pull requests:** - docling-project#4709 adds orientation detection for RapidOCR (`detect_orientation`, off by default). It addresses the same rotated-text problem for another engine and shares no code with this PR. Both PRs change `pipeline_options.py` and `docs/concepts/OCR.md`, but in different sections, and they merge cleanly. - docling-project#4654 and docling-project#4643 detect a broken PDF text layer and send it to OCR, docling-project#4654 per page and docling-project#4643 per region. This PR does not do that; the PRs complement each other. With a Tesseract engine, the pages and regions that they send to OCR get the changes in this PR. A page that docling-project#4654 replaces is OCR'd in full, which is where the grayscale input gains the most. **Review evidence:** - Reference data: unchanged. `tests/test_e2e_ocr_conversion.py` passes against the existing ground truth for every Tesseract configuration of both engines (default, layout-regions and full-page modes, and full-page mode with `lang=[]`), and for EasyOCR and RapidOCR. - New tests: 6 unit tests in `tests/test_tesseract_ocr_cli_model.py` and `tests/test_tesseract_ocr_model.py` (grayscale input, the DPI, the threshold and the error record) fail on `main` and pass here. `tests/test_e2e_tesseract_highlighted_text.py` OCRs a synthetic page with three highlighted lines (`tests/data/ocr/sources/highlighted_text.png`, 63 KB, regenerated by running the test module); on `main`, both engines miss 26 of its 42 highlighted words. - `tests/test_e2e_rotated_ocr.py`: same result as `main` (90° and 270° pass; 180° is still the expected failure of docling-project#3839). - One run of the tests above plus `test_tesseract_ocr_cli_lang`, `test_options`, `test_service_datamodels`, `test_ocr_options_force_full_page`, `test_ocr_engine_language_mapping`, `test_ocr_language` and `test_backend_webp`: 306 passed, 6 skipped, 2 xfailed (the 180° cases). - Checks: the prek hooks (ruff, ruff format, ty, tach), tach module coverage and max-lines pass. - Not run: the full test suite, Linux, macOS and Tesseract 4. **Samples.** The ground truth is the PDF text layer, or a manual transcription for image-only pages. The highlighted words are the words under a yellow highlighter mark. The public sources: - Newtown, CT, 2015 draft charter (47 pages): https://cmsarchive.civicplus.com/Newtown%20CT/NewtownCT_BBoard/News%20Archive/S052C770B.2/NEWTOWN%20CHARTER%20Revision%20Commission%202015%20Draft%20%20Charter%203.pdf - Newtown, CT, colour-code legend: https://cmsarchive.civicplus.com/Newtown%20CT/NewtownCT_BBoard/News%20Archive/S052C770B.4/Color%20Codes%207-10-15%205.pdf - City of Homer, AK, council supplemental packet (the slide is page 5): https://www.cityofhomer-ak.gov/sites/default/files/fileattachments/city_council/meeting/75366/supplemental_packet.pdf - "Page 23 as a scanned image" is page 23 of the draft charter, rasterized at 300 DPI into an image-only PDF. - Scanned court exhibit, U.S. District Court for the Northern District of California, No. 4:24-cv-04722, ECF No. 379-75 (the highlighted line is on page 3): https://storage.courtlistener.com/recap/gov.uscourts.cand.433688/gov.uscourts.cand.433688.379.75.pdf **Checklist:** - [x] Documentation has been updated, if necessary. - [x] Examples have been added, if necessary. - [x] Tests have been added, if necessary. Signed-off-by: BBC, Esquire <bbc@chintellalaw.com>
RapidOCR has no page orientation step. It turns tall line crops a quarter turn counter-clockwise, and its line classifier only flips single lines by 180 degrees, so a scan turned counter-clockwise or upside down is read as garbage, or only in part. Add RapidOcrOptions.detect_orientation, off by default. RapidOCR reads each OCR region as it is. If it finds at least 3 lines and fewer than half score 0.8 or more, it reads the region again at the other three orientations and keeps the first one in which at least half of the lines score 0.8 or more and the text with such scores is at least twice as long. The boxes are mapped back onto the page with rotate_bounding_box. No extra package or model is needed. With the option on, the reader keeps every line (text_score 0.0), so that the weak lines count in the decision, and text_score is applied after it. Signed-off-by: Hanjun Cho <gkswns0531@gmail.com>
Signed-off-by: Hanjun Cho <gkswns0531@gmail.com>
66ca5e0 to
0c20575
Compare
github-actions
Bot
removed
the
ai:ci-safe
|
Rebased onto main to resolve a conflict with #4920; its |
github-actions
Bot
added
the
ai:ci-safe
terminal_width sets only Click's width. The Rich console that prints the `docling options` table reads COLUMNS and otherwise falls back to 80 columns under CliRunner, so the new `detect_orientation` key widened the Key column and truncated the Type column to "[tiny|small|mediu…". Signed-off-by: Hanjun Cho <gkswns0531@gmail.com>
github-actions
Bot
removed
the
ai:ci-safe
|
The core test failure was |
github-actions
Bot
added
the
ai:ci-safe
Adds
RapidOcrOptions.detect_orientation, off by default. Following the review, it now uses only RapidOCR's own confidence scores, so it needs no extra package or model, andrapid_ocr_model.pyimports nothing from the Tesseract modules.RapidOCR reads each OCR region as it is. If it finds at least 3 lines and fewer than half of them score 0.8 or more, it reads the region again at the other three orientations, most likely one first. It keeps the first orientation in which at least half of the lines score 0.8 or more and the text with such scores is at least twice as long. The boxes are mapped back onto the page with
rotate_bounding_box. A region that reads well is read only once.With the option on, the reader keeps every line (
text_score0.0), so that the weak lines count in the decision, andtext_scoreis applied after it. With the option off, nothing changes.Issue resolved by this Pull Request:
Resolves #4708
Checklist:
docs/concepts/OCR.md)Review evidence:
test_e2e_conversionsgets two RapidOCR configs with the option on, in the default and the full-page mode. They match the existing ground truth (checked with the onnxruntime RapidOCR configs of the test, with the option off and on). No reference data changed.ocr_test.pdf,ocr_test_rotated_{90,180,270}.pdfand the three dense fixtures, in both modes.docling-slim[standard]install (RapidOCR with the torch backend, no tesserocr, no onnxruntime) reads the 90° fixture with the option on: 0 → 3 lines.use_cls=False, full-page OCR, one page per conversion): all 17 sideways pages are read again, from 2,384 to 31,847 characters on lines scored 0.8 or higher (95–99% of the best of the four orientations). None of 119 upright pages and 61 pages with little text changes; 2 and 4 of them are read a second time. On a sideways page the option adds one read: 20 s of CPU on top of 12 s.prek runon the changed files passes.pytest tests/test_e2e_rotated_ocr.py tests/test_rapid_ocr_model.py tests/test_ocr_options_force_full_page.py: 32 passed, 3 skipped (no tesseract binary on this machine), 1 xfailed (OCR-recognized text is dropped from output for 180°-rotated images, even with force_full_page_ocr #3839). Not run here: the Tesseract, EasyOCR and torch-backend configs oftest_e2e_conversions.