[chaliy]

Requested by Михайло · project thread

What changed

• extract_document on HTML now fills Extraction::title (was always None for HTML) and a new Extraction::metadata: description, author, published date, language, canonical URL, site name, lead image.
• Sources: <meta> (name/property/itemprop), Open Graph, Twitter cards, Dublin Core, <link rel=canonical>, <html lang>, JSON-LD (incl. @graph, article types preferred). Fixed precedence table in knowledge/formats/html-extraction.md.
• markkit extract --json includes format, title, metadata for --file and stdin. (URL path stays Markdown-only; it gets the same in the follow-up URL-resolution PR.)
• Metadata exported from the crate root; Extraction is #[non_exhaustive], so additive.

First of three PRs for the roadmap's "Readability and metadata" item; next are relative URL resolution and main-content detection.

Why

Web pages carry title/author/date/canonical in head tags that the Markdown drops. LLM/RAG callers need them as structured fields.

Before / After

$ printf '<html lang=en><head><title>Hi</title><meta name=description content="D"></head><body><p>x</p></body></html>' | markkit extract --stdin --json

Before: {"markdown": "x\n", "source": "<stdin>"}
After:

{"format": "html", "markdown": "x\n", "source": "<stdin>", "title": "Hi",
 "metadata": {"description": "D", "language": "en", "author": null, "canonical_url": null, "image": null, "published": null, "site_name": null}}

Markdown output unchanged.

Risk

• Low. Metadata is read from the already-parsed DOM; Markdown rendering untouched.
• Untrusted input: values whitespace-collapsed, control chars dropped, capped at 2048 chars; URL fields drop non-http(s) schemes (javascript:, data:); JSON-LD bounded (16 scripts, 1 MiB each, 256 nodes, serde_json depth 128), invalid JSON ignored. Threat model T1 + new T2e.

Checklist

• Tests added or updated (tests/html_metadata.rs incl. security cases, proptest metadata_values_single_line, unit tests)
• Backward compatibility considered (additive field; stdin now decodes lossily instead of erroring on invalid UTF-8)

---

Generated by Claude Code