[chaliy]

What changed

PDF extraction now reconstructs reading order, columns, headings/lists and ruled or borderless tables, including merged and wrapped cells. Optional versioned layout JSON exposes source words and semantic blocks/lines/cells linked to IR provenance. Tagged headings and ActualText are honored; repeated margin text can be suppressed.

Textless pages support opt-in local OCR using Rust Hayro rendering. Tesseract is the default; pdf-ocr-ocrs adds Rust ocrs/RTen inference with explicit local ONNX paths. Rust, CLI, Node and Python expose the options; WASM gains text-layer PDF extraction, layout and tables. Python also gains structured document extraction.

Why

PDFs need useful structured Markdown and source geometry across columns, tables and scanned pages. OCR uses local assets and works without Poppler; model downloads happen only through the separate pinned setup script.

Before / After

Evidence	Before	After
Public PDF document score	58.36%	65.53%
Source layout	unavailable	version-1 positioned words, reading-order blocks and table cells
Scanned PDFs without text layers	empty output	opt-in OCR; 56.0% mean document score on the existing three scans

The optional ocrs backend scores 54.9% on the same scans, compared with Tesseract's 56.0%. Three fresh measured runs after warm-up, one inference thread: median document time 12.78s versus 4.28s; median sampled peak process-tree RSS 227.6 versus 36.2 MiB. Tesseract remains the default. Both return valid geometry; a generated fixture verifies known word positions. Public scan box comparisons measure agreement, not ground-truth localization accuracy.

Validation:

• just pre-pr: formatting, Clippy, 825 default workspace tests, knowledge checks (including pinned okf-lint 0.1.1), script tests, examples and cargo-deny passed.
• All features: 827 tests passed; two external-asset/helper fixtures are ignored by default and were run explicitly.
• Eight explicit OCR tests, Node/Python/WASM checks, 178 public default cases with no regressions, and 40 authored PDF eval cases passed. Saved eval reports retain corpus/model/binary provenance and per-document regressions. A final run after stream-guard hardening matched all six saved structured OCR outputs exactly.
• cargo audit, documentation build and the core package publication dry-run passed. Ordinary benchmark results are saved alongside the OCR comparison.
• Security regressions cover decode/analysis/storage/grid limits, tagged content, geometry, helper cleanup, model files, escaped dictionary keys/names, and compressed prefixes before image codecs.

Usage: --json --layout for geometry; --ocr for Tesseract; build with --features pdf-ocr-ocrs and use --ocr --ocr-backend ocrs with both local model paths for Rust inference.

Risk

High: PDF interpretation, reading-order heuristics and native image decoding are expanded. Table inference remains conservative. Rendering and RTen deadlines are cooperative; hostile documents still require host CPU/memory isolation. Inline, JBIG2, CCITT and DecodeParms-based OCR images fail explicitly pending bounded decoder support.

The optional default ocrs model supports Latin/English recognition. Its approximately 12 MB weights are separate CC-BY-SA 4.0 deployment assets with size/SHA256 pins and attribution; they are not embedded in packages. Extraction never downloads models or silently changes engines.

Breaking: exhaustive Rust MarkkitBuilder literals need the new PDF options. Builder methods/default update syntax remain valid. The IR source link is additive; the committed schema and all binding documentation are updated.

Checklist

• Tests added or updated
• Backward compatibility considered