[ceberam]

Closes #3991
Supersedes #4454

Summary

Previously, the MarkdownDocumentBackend parsed table cells as plain text using a custom text-buffer and regex splitting approach, discarding all inline markup. A cell containing **bold** or [link](url) was stored as a TableCell with the text bold or link, with no record of the formatting or hyperlink.

This PR upgrades Markdown table parsing to be strictly AST-driven using Marko's GitHub Flavored Markdown (gfm) extension, preserving inline markup by emitting RichTableCell objects for formatted cells.

Changes

GFM Table AST & RichTableCell Support

• GFM Extension: The Markdown parser is initialized with the gfm extension (Markdown(extensions=["gfm"])), generating structured AST nodes (Table → TableRow → TableCell) with inline children per cell.
• AST-Driven Table Parsing: Replaced the legacy text-buffer table parsing subsystem (_split_table_row, _starts_pipeless_table, _close_table, etc.) with _parse_gfm_table.
• Rich vs. Plain Cell Selection:
  • Plain cells (RawText/Literal children only) continue to produce a TableCell, keeping the common case lightweight with zero overhead.
  • Rich cells produce a RichTableCell backed by an InlineGroup parented under the table.
• Inline Element Walker (_iterate_cell_inline): Traverses cell inline AST nodes and generates TextItem, CodeItem, or PictureItem children. Preserves bold, italic, strikethrough, inline code spans, hyperlinks (including GFM bare-URL autolinks), and embedded images. Only content permitted inside table cells per the GFM specification is processed.

Shared Formatting & Link Helpers

Extracted reusable static helpers shared between block-level prose and table-cell inline element walkers:

• _apply_formatting(current, *, bold, italic, strikethrough): Returns an updated Formatting instance without mutating parent references in place.
• _resolve_link_dest(dest): Parses and validates link destination strings into AnyUrl or Path objects via TypeAdapter.

Modernized Typing & Codebase Consistency

• Modernized type annotations across md_backend.py to Python 3.10+ union syntax (X | None, X | Y), eliminating Union and Optional imports.
• Updated _CreationPayload discriminated union to use _HeadingCreationPayload | _ListItemCreationPayload while preserving Annotated[..., Field(discriminator="kind")].
• Cleaned up obsolete buffer state variables (in_table, in_pipeless_table, md_table_buffer) and unneeded inline comments.

Tests

• Added 5 comprehensive integration tests in tests/test_backend_markdown.py:
  • test_rich_table_cell_bold_and_italic: Verifies bold/italic in headers and data cells, mixed formatting, and plain cell fallback.
  • test_rich_table_cell_inline_code: Verifies code spans become CodeItem instances with literal unescaped text.
  • test_rich_table_cell_hyperlink: Verifies inline links attach hyperlinks to cell items.
  • test_rich_table_cell_autolinks: Verifies angle-bracket <https://...> and GFM bare URLs.
  • test_rich_table_cell_no_stray_body_items: Verifies rich cell groups are parented under the table and do not leak into doc.body.
• Updated test expectations for backslash-escaped pipes inside code spans in test_convert_table_keeps_inline_code_spans.
• Regenerated ground truth references for inline_and_formatting.md (.yaml and .md).

Checklist:

• Documentation has been updated, if necessary.
• Examples have been added, if necessary.
• Tests have been added, if necessary.

[github-actions]

✅ DCO Check Passed

Thanks @ceberam, all your commits are properly signed off. 🎉

[mergify]

Merge Protections

🟢 All 2 merge protections satisfied — ready to merge.

Show 2 satisfied protections

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

🟢 Require two reviewer for test updates

When test data is updated, we require two reviewers

• #approved-reviews-by >= 2

[codecov]

Codecov Report

❌ Patch coverage is 81.74603% with 23 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
docling/backend/md_backend.py	81.74%	18 Missing and 5 partials ⚠️

📢 Thoughts on this report? Let us know!

[dolfim-ibm]

lgtm

[dolfim-ibm]

@ceberam could this be related to #4479 as well?

[ceberam]

@ceberam could this be related to #4479 as well?

It is indeed related to #4479 and also #4454 and other issues in the past around Markdown tables.
With this PR we are leveraging the GFM extension of the marko library, which is the library that we already used in the Markdown backend. Instead of a manual string manipulation to parse tables (and the challenges to deal with complex edge cases like this one), we can leverage the Markdown(extensions=["gfm"]):

• marko natively understands the GFM Table specification.
• The parser itself produces a clean, structured tree of table nodes.

[DanielNg0729]

hi Dr @ceberam, thank you very much for this PR. Moving the tables to marko's GFM AST is a big improvement. a <br> in a cell no longer splits the table into pieces, a table inside a list item stays in place, and links, code and images in cells are kept.

I tried the branch locally and have a few questions, please see the comments here. I may well be missing context, so please feel free to disregard any of them.

Thank you very much again!

[ceberam]

Thanks @DanielNg0729 for you detailed review and good suggestions. I have added some commits to address them. Please, feel free to have a second look.

[DanielNg0729]

Thank you Dr @ceberam, looks good to me now!