Skip to content

fix(markdown): keep GFM task-list checkboxes and paragraph [x] markers - #4935

Open
Siddharth-DWT wants to merge 2 commits into
docling-project:mainfrom
Siddharth-DWT:fix/md-task-list-checkboxes
Open

Siddharth-DWT wants to merge 2 commits into
docling-project:mainfrom
Siddharth-DWT:fix/md-task-list-checkboxes

Conversation

@Siddharth-DWT

@Siddharth-DWT Siddharth-DWT commented Oct 21, 2026 •

Copy link
Copy Markdown

marko's GFM Paragraph strips a leading [x] / [ ] from every paragraph and keeps it only in checked, which the Markdown backend never read. Task-list items lost their marker, and any paragraph starting with one lost that text.

_GfmParagraph is now based on marko's plain Paragraph (with override = True, and the existing table-aware break_paragraph), so the marker is never stripped. Task items stay list_items with the marker in their text.

Following review, this replaces the first version, which turned task items into checkbox_* text items: that dropped ordered-list numbers on export, broke the HTML list structure, still lost the marker when an item did not start with a paragraph, and relied on attribute probing. Keeping the marker as text has none of those problems. Representing task items as structured checkboxes inside a list would need list-item checkbox support in docling-core's serializers, so it is better as a separate change.

Issue resolved by this Pull Request:
Resolves #4930

Checklist:

  • Documentation has been updated, if necessary. (not needed)
  • Examples have been added, if necessary. (not needed)
  • Tests have been added, if necessary.

Review evidence:

Markdown Before After
- [x] done item / - [ ] open item list items "done item" / "open item" list items "[x] done item" / "[ ] open item"; Markdown - [x] done item, HTML <li>[x] done item</li>
7. [x] seven / 8. [ ] eight / 9. nine 7. seven / 8. eight / 9. nine 7. [x] seven / 8. [ ] eight / 9. nine
- [x] parent with nested - [ ] child markers lost - [x] parent / - [ ] child
[x] I accept the terms, > [x] quoted, a second paragraph [x] ... in an item, - # Title followed by [x] ... marker lost marker kept

Tests (tests/test_backend_markdown.py): test_task_list_items_keep_their_markers (labels, Markdown and HTML export), test_ordered_task_list_items_keep_their_numbers, test_nested_task_list_items_keep_their_markers, and test_task_markers_outside_list_items_are_kept (5 cases, including the heading-first item). All 8 fail on main and pass with the change.

Checks run:

  • pytest tests/test_backend_markdown.py tests/test_backend_markdown_images.py: 75 passed, including test_convert_valid against the reference data. No reference data changes: none of tests/data/md/sources contains task markers.
  • ruff check / ruff format --check (0.15.12): clean.
  • ty check (0.0.33) on md_backend.py: same 4 diagnostics as on main.
  • Not run: the full test suite and make validate.

Written with AI assistance; I reviewed the change and ran the checks myself.

marko's GFM paragraph strips a leading "[x] " / "[ ] " from every paragraph
and records it in `checked`, which the Markdown backend never read. Task-list
items lost their state, and any paragraph starting with the marker lost that
text.

Only a list item's first paragraph now treats the marker as a task marker; the
backend turns those items into checkbox_selected / checkbox_unselected items in
the list, as the HTML backend does. Elsewhere the marker stays text.

Fixes docling-project#4930

Signed-off-by: Siddharth-DWT <mernhash@gmail.com>
@github-actions

Copy link
Copy Markdown
Contributor

✅ DCO Check Passed

Thanks @Siddharth-DWT, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@github-actions github-actions Bot added ai:ci-safe AI triage found no CI-safety concern (advisory) bug Something isn't working markdown issue related to markdown backend labels Oct 21, 2026
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

AI triage (advisory)

Checked commit 36e2adb928. The analysis read the diff only. It did not run code from this pull request.

The PR changes _GfmParagraph to inherit from marko.block.Paragraph instead of the GFM paragraph so that task-list markers ([x]/[ ]) are no longer stripped from list items or standalone paragraphs. Four new tests in test_backend_markdown.py verify the markers are preserved.

CI safety: no concern found. A maintainer must still approve CI.

3 related pull request(s) in the same code
PR State Signal Assessment
#4991 open changes the same code related: PR #4991 also edits md_backend.py and appends tests at the same location, but it fixes backslash-escape splitting into separate items — a different bug with different code changes.
#4848 open changes the same code related: PR #4848 also edits md_backend.py and appends tests at the same location, but it refactors HTML-block handling — a different feature area with no overlap in logic.
#4401 merged referenced as #4401 related: PR #4401 (merged) fixed GFM task-list rendering in html_backend.py; this PR fixes the same semantic issue in md_backend.py and references #4401 as the target structure, but the two backends are independent.

The ai:* labels are advisory. They do not approve CI and they do not add tests:full. A new push runs this check again.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI first review (advisory): no blocker found

This review covers the full diff at 9134a0886e. It read the code and did not run it.

The fix is correct for the main cases: bullet task lists, nested task lists, and [x] paragraphs outside list items. The main risk is ordered task lists. They now export with the wrong marker, and the PR does not test them. Check that the author accepts the change for ordered lists.

A maintainer makes the merge decision. This review never approves the pull request.

Comment thread docling/backend/md_backend.py Outdated
start = list_start_by_ref.get(parent_ref, 1)
count = list_item_counter_by_ref.get(parent_ref, 0)
marker = f"{start + count}."
checked: bool | None = getattr(child, "checked", None)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Ordered task-list items lose their number on export

Trigger: 1. [x] a\n2. [ ] b. The code makes every checked item a checkbox_* text item through doc.add_text, and it drops enumerated and marker. The docling_core text serializer always writes - [x] or - [ ] for these labels (markdown.py:345-348). Before the PR, the Markdown export kept 1. a and 2. b. After the PR, ordered task items export as bullets. In a mixed list, a plain item can still be numbered by its position in the list. The PR table and the tests cover bullet lists only. Add an ordered-list case to the tests. Either accept the new export and say so in the PR, or keep list_item for enumerated lists.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 36e2adb: task items are no longer converted to checkbox items, so ordered lists keep their numbers (test_ordered_task_list_items_keep_their_numbers).

Comment thread docling/backend/md_backend.py Outdated
@classmethod
def parse(cls, source):
state = source.state
opens_list_item = isinstance(state, marko.block.ListItem) and not getattr(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: "First paragraph" check is wrong when a non-paragraph block comes first, and it probes attributes

The flag _docling_has_paragraph is set only when a paragraph is parsed. In - # Title\n\n [x] text, the heading comes first, so the paragraph counts as the "first" one. checked is set and the marker is stripped. The list-item branch in _iterate_elements needs children[0] to be a Paragraph, so it is skipped. The [x] text is then lost, which is the defect this PR fixes. GFM allows the marker only when the paragraph is the first block of the item. AGENTS.md also asks to avoid getattr, setattr, and hasattr probing. The code uses them on a marko ListItem and on _ParsedLines. Prefer a typed approach, for example a field on a small _GfmParagraph subclass, or a check that the item has no earlier block.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 36e2adb: the marker is never stripped, so the heading-first case keeps it (covered in test_task_markers_outside_list_items_are_kept), and the attribute probing is gone.

@github-actions github-actions Bot added the ai:review-lgtm AI first review found no blocker or major issue (advisory) label Oct 21, 2026
@codecov

codecov Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@geoHeil geoHeil left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 9134a0886efd6abd33c7123e0e4e2614c3fa3a0e. Changes are still needed despite green CI and full patch coverage.

The doc.add_text() checkbox path discards ordered-list markers. 7. [x] seven\n8. [ ] eight\n9. nine exports as - [x] seven\n- [ ] eight\n9. nine. It also changes - [x] done\n- plain into HTML with <ul><p>done</p><li>plain</li></ul>. Preserve the list-item contract in both exports.

The earlier first-block finding also remains open: - # header\n\n [x] ordinary second block still loses [x]. That loss exists on base, so it is an incomplete fix rather than a new regression. See the open ordered-list and first-block discussions: #4935 (comment) and #4935 (comment).

All 73 Markdown and image tests pass locally. The current tests do not detect these cases.

The new export defect is in the checkbox representation.

Local make check stops on file-length violations that already exist on base. The other read-only checks pass. I made no source or reference-data changes.

Review feedback: turning task items into checkbox text items dropped ordered
list numbers on export, broke the HTML list structure, still lost the marker
when the item did not start with a paragraph, and probed attributes.

Base _GfmParagraph on marko's plain Paragraph instead of GFM's, so the
"[x] " / "[ ] " marker is never stripped. Task items stay list items with
the marker in their text, which keeps numbering and both exports intact,
and no paragraph loses its leading marker.

Signed-off-by: Siddharth-DWT <mernhash@gmail.com>
@github-actions github-actions Bot removed ai:ci-safe AI triage found no CI-safety concern (advisory) ai:review-lgtm AI first review found no blocker or major issue (advisory) labels Oct 24, 2026
@Siddharth-DWT

Copy link
Copy Markdown
Author

Thanks for the careful review @geoHeil. You're right on all points, and the checkbox-item approach was the root of them, so I've replaced it in 36e2adb.

_GfmParagraph is now based on marko's plain Paragraph, so the marker is never stripped and task items stay ordinary list items with [x] / [ ] in their text:

  • `7. [x] seven
  1. eight
  2. nine` exports unchanged, numbers included.
  • `- [x] done

  • plainexports to HTML as

    • done
    • plain
    `.

  • `- # header

    [x] ordinary second blockkeeps its[x]`.

  • No getattr / setattr / hasattr probing remains; the backend change is 7 lines.

Tests cover all of these (8 cases; they fail on main). The trade-off is that the checked state is text, not a structured checkbox. Doing that properly inside lists needs serializer support in docling-core, so I'd keep it for a follow-up if you want it.

@github-actions github-actions Bot added ai:ci-safe AI triage found no CI-safety concern (advisory) ai:review-lgtm AI first review found no blocker or major issue (advisory) labels Oct 24, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI first review (advisory): no blocker found

This review covers the changes since 9134a0886e at 36e2adb928. It read the code and did not run it.

The updated changes fix the earlier findings by subclassing marko.block.Paragraph in docling/backend/md_backend.py:66. This subclass keeps task-list checkbox markers and ordered-list numbers inside the element text. I checked the subclassing of marko.block.Paragraph and the added tests for nested and ordered lists in tests/test_backend_markdown.py:1323. The code behaves correctly.

Earlier findings

  • fixed (suggestion): Ordered task-list items lose their number on export
  • fixed (suggestion): "First paragraph" check is wrong when a non-paragraph block comes first, and it probes attributes

A maintainer makes the merge decision. This review never approves the pull request.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai:ci-safe AI triage found no CI-safety concern (advisory) ai:review-lgtm AI first review found no blocker or major issue (advisory) bug Something isn't working markdown issue related to markdown backend

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Markdown backend drops GFM task-list checkboxes and deletes a leading [x] from paragraphs

3 participants