Repository navigation
fix: resolve /CIDToGIDMap streams for CIDFontType2 glyph lookup - #426
Merged
Merged
Conversation
CIDFontType2 subsets that carry no cmap table address their glyphs only through a /CIDToGIDMap stream (ISO 32000-1, 9.7.4.2). Blend2D refuses such a face (no character mapping), and the FreeType fallback honoured only an /Identity map, so every glyph fell through to a system font, or to a stroked outline box where none is installed. Decode the stream (2 bytes per CID, big-endian GID; CIDs past its end map to GID 0) into the embedded font blob, and use it wherever the CID identity was used: FreeType glyph resolution and Blend2D's char-code glyph recovery. Identity and absent maps behave as before. Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
Contributor
|
✅ DCO Check Passed Thanks @cau-git, all your commits are properly signed off. 🎉 |
Contributor
Merge Protections🟢 Merge protection satisfied — ready to merge. Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
PeterStaar-IBM
previously approved these changes
Oct 27, 2026
The renderer indexed the /CIDToGIDMap stream with the content-stream character code, which is the CID only under Identity-H/V. Fonts whose /Encoding is an embedded or predefined CMap drew the wrong glyphs: "50-99" came out as "xso" and two greater-or-equal signs. Compose the map with the encoding's code -> CID when the font blob is built. Identity-H/V use the map as is; an embedded CMap reuses the code -> CID table that the codespace scan already parses; a predefined CMap takes it from its cmap resource. When no complete table is known (an embedded CMap with usecmap, or any other encoding), the stream is not applied and Unicode shaping draws the text as before. Rename the blob helpers to has_code_to_gid()/code_to_gid(), and drop the constructor default that no caller used. Correct the test's account of why Blend2D rejects these subsets: they lack a name table (BL_ERROR_FONT_MISSING_IMPORTANT_TABLE), not a character mapping. Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
The five pages whose /CIDToGIDMap stream fonts now render with their embedded glyphs get new reference images in docling-project/regression-dataset-for-docling-parse@21374211. Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
cau-git
marked this pull request as ready for review
October 27, 2026 18:38
PeterStaar-IBM
approved these changes
Oct 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
CIDFontType2 (TrueType) subsets that address their glyphs only through a
/CIDToGIDMapstream could not be drawn from the embedded program. This is valid per ISO 32000-1 §9.7.4.2 and common in PDFs from Office-like producers.face_res=65613=BL_ERROR_FONT_MISSING_IMPORTANT_TABLE. The subsets have nonametable, soNameImpl::init()fails. They also lackcmapandpost./Identitymap. Withoutcmap/post, the name, builtin-encoding and Unicode lookups all fail.Example: Amazon 2024 annual report p.36 and Q4 2024 earnings p.8. pdfium renders them correctly.
Changes
page_font.h:resolve_code_to_gid_map()decodes the/CIDToGIDMapstream (/CIDFontType2only) and composes it with the/EncodingCMap, so the renderer can map the content-stream character code straight to a glyph: GID = map[CMap(code)].init_embedded_cmap()already parses is now kept (left empty if the CMap usesusecmap)./UniJIS-UCS2-H): via the existing CMap resources (font_cid::get_code_to_cid()).embedded_font_blob.h: stores the composed map;has_code_to_gid()/code_to_gid(code).freetype_font_cache.h,blend2d_renderer.h: glyph resolution and char-code recovery use it; GID 0 from an explicit map counts as unmapped.tests/test_unit_cid_to_gid_map.py:/UniJIS-UCS2-H} × {FreeType path, Blend2D path}, with distinct code, CID and GID (code → CID → GID differs from code → GID).Identity/absent maps, CFF CIDFonts and simple fonts behave as before.
Testing
uv run pytest ./testsgives 410 passed, 2 skipped. Unit tests: 300 passed, 2 skipped.Reference images
The renderer groundtruth of the 5 improved pages is refreshed, since they now draw the embedded glyphs instead of a substitute face: regression-dataset-for-docling-parse@21374211. The commit updates
groundtruth/render/pages/{3af8995b53546b34_0010,4796728975040539044-1,94e3fe046fa21a9c_0001,9574724313327355313-1,font_03}.pdf.page_no_1.full_page.pngand nothing else.HF_DATASET_REVISIONis bumped accordingly. With the new pin, all 940 reference pages match within a mean abs error of 0.05.Known limits / follow-ups
/CIDToGIDMapcombined with a non-identity encoding still uses the code as the GID. This is pre-existing and left unchanged so other references do not move.usecmapskip the map (Unicode shaping, as before).