Skip to content

feat: add opt-in sparse table record serializer - #2

Draft
sks wants to merge 2 commits into
mainfrom
feat/record-table-serializer
Draft

sks wants to merge 2 commits into
mainfrom
feat/record-table-serializer

Conversation

@sks

@sks sks commented Oct 12, 2026

Copy link
Copy Markdown

Summary

  • Add RecordTableSerializer as an explicit choice for MarkdownDocSerializer. It reads source cells without constructing a padded table grid. The default Markdown export and structured document do not change.
  • Remove the fork-specific serving package, image, publication workflow, and version pins from this branch. Do not add a replacement server integration.
  • Document the record format and its limits. Add unit tests for sparse cells, explicit empty fields, spans, rich and nested cells, captions, and unchanged default exports. Add a CSV conversion test.

Review evidence

Before: the fork replaced the Markdown serializer at process startup and shipped a branded Serve image. After: an SDK caller opts in for one export; no process-wide replacement or branded files remain. Record text is not a Markdown pipe table. Conversion and final output still use memory.

  • uv run --no-sync pytest -q tests/test_table_records.py tests/test_backend_csv.py: 59 passed.
  • make validate: passed.
  • make check: not clean. It stops at check_max_lines.py on 13 files that already exceed the 1,000-line limit on upstream main. No changed file exceeds that limit. Ruff, ty (warnings), and Tach completed before that stop.
  • Image/Serve tests: not run. This PR removes that integration rather than replacing its deployment behavior.

Upstream ownership

The general Markdown serializer lives in docling-core. This fork PR provides a neutral opt-in SDK implementation. A later contribution to docling-project/docling-core should settle its long-term home and public API. It does not enable this format in Docling Serve.

  • Documentation updated
  • Tests added
  • Upstream Core design reviewed

@sks
sks force-pushed the feat/record-table-serializer branch from 794b8d8 to 09c4a1e Compare October 12, 2026 03:42
@sks
sks force-pushed the feat/record-table-serializer branch from 09c4a1e to 9138eed Compare October 12, 2026 05:58
@sks sks mentioned this pull request Oct 12, 2026
2 tasks done
## Summary

- Add opt-in record-oriented table output for Markdown exports. The
default Markdown table format does not change.
- Support `--markdown-table-format records` in `convert` and
`convert-remote`. Remote conversion still applies the option only when
writing Markdown locally.
- Provide `export_to_record_markdown()` and `save_as_record_markdown()`
for packaged SDK users. Keep image modes and other output formats
unchanged.
- Remove the fork-specific Serve integration from PR #1. The serializer
does not replace a process-wide Core serializer.

This branch is based on the current `stackgenhq/main`. [Draft PR
#2](#2) contains the initial
serializer. The [branch
comparison](feat/record-table-serializer...feat/record-table-export-dmain)
now shows only the 11-file opt-in addition, without unrelated upstream
commits.

## Checks

- Selected tests: 164 passed, 2 deselected (`test_cli_convert`), 91
warnings.
- `make validate` passed on the 11-file opt-in changeset.
- `git diff --check` passed.
- Full repository test suite and `make check` were not run for this PR.

This export option does not address conversion memory use or add a
Docling Serve-wide option. A general Markdown serializer API remains a
`docling-core` design decision.

- [x] Documentation updated
- [x] Tests added

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant