Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Fix root cause. Unsure: read more code; if stuck, ask w/ short options. Unrecogn
| formats/format-support | Formats, bytes-in API, detection order, DOCX + spreadsheet mapping |
| integrations/tool-contract | `MarkkitTool` LLM tool (`tool` feature), Everruns toolkit contract |
| security/security-model | One-page threat/invariant summary |
| security/threat-model | Threats T1-T6, mitigations, test map, resource limits, invariants S1-S12 |
| security/threat-model | Threats T1-T6, mitigations, test map, resource limits, invariants S1-S17 |
| runtimes/python-package | PyO3 `markkit` module, maturin, PyPI |
| runtimes/wasm-package | wasm-bindgen `@everruns/markkit-wasm`, browser/edge, build + tests |
| runtimes/node-package | NAPI-RS `@everruns/markkit`, npm, binding parity table |
Expand Down
13 changes: 12 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
- Relative URL resolution: links, images and metadata URLs resolve against the new
`MarkkitBuilder::base_url` (CLI `--base-url`) and the page's `<base href>`; `extract_url` uses
the final URL after redirects. Absolute and fragment-only URLs are left unchanged.
- Math: `$tex$` and `$$` blocks from MathML (`application/x-tex` annotation), KaTeX and MathJax output.
- Math: `` $`tex`$ `` and ` ```math ` blocks from MathML (`application/x-tex` annotation), KaTeX and MathJax output.
- IR transforms: `MarkkitBuilder::transform(..)` runs `Transform` passes after parsing and before
rendering, for every format. Built-ins `transform::NormalizeHeadings` and `transform::DropImages`;
`transform::from_fn` for closures; `Document::visit` / `visit_mut` to walk the tree.
Expand Down Expand Up @@ -64,6 +64,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
(`max_redirects`, default 5), the body is bounded by `max_input_size` while streaming, and the whole
fetch by `max_processing_secs`. New builder options `allow_private_networks(true)` and `allow_host(..)`;
refusals return the new `Error::UrlBlocked`.
- Markdown output is now escaped: text from documents can no longer emit raw HTML (`<img onerror>`),
links, emphasis, headings or list markers; code blocks use a fence longer than any backtick run
inside. Links with schemes other than `http`, `https`, `mailto`, `tel`, `ftp` or relative URLs
(e.g. `javascript:`) and images other than `http`/`https`/relative/raster `data:image/*` are
dropped, keeping their text. URLs percent-encode `( ) < > \` and spaces.
- Markdown structure from documents can no longer break out of its container: code blocks in table
cells and headings are flattened to one line, list item bodies are indented by the marker width
(nested content under `1.` items now nests correctly), and nested links render as text.
- The JSON IR of DOCX and spreadsheet tables now holds raw text (pipes were pre-escaped before).
- Math TeX (taken from page markup) renders as code, GitHub's `` $`tex`$ `` and ` ```math `
forms, instead of `$tex$` / `$$`, so it can no longer emit raw HTML or links in plain GFM.

### Added

Expand Down
18 changes: 18 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ napi-derive = "3"
tempfile = "3"
criterion = { version = "0.8", features = ["html_reports"] }
proptest = "1"
pulldown-cmark = { version = "0.13", default-features = false }

[profile.release]
lto = "thin"
Expand Down
1 change: 1 addition & 0 deletions crates/markkit/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ anyhow.workspace = true
tokio = { workspace = true, features = ["macros", "rt-multi-thread", "net", "io-util", "time"] }
tempfile.workspace = true
proptest.workspace = true
pulldown-cmark.workspace = true
criterion.workspace = true
tower = { workspace = true }
zip = { workspace = true }
Expand Down
6 changes: 2 additions & 4 deletions crates/markkit/src/docx.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ use crate::config::MarkkitBuilder;
use crate::error::Error;
use crate::ir::{Document, ListItem, Node, Source, TableCell};
use crate::limits::Budget;
use crate::render::escape_table_cell;
use crate::xml::{self, Element, XmlError};

/// Result of parsing a DOCX package.
Expand Down Expand Up @@ -883,11 +882,10 @@ impl Ctx<'_> {
}
}

/// Make cell content table-safe: no line breaks, pipes escaped.
/// Make cell content single-line: hard breaks become spaces (the renderer escapes pipes).
fn table_cell(nodes: Vec<Node>) -> Vec<Node> {
fn fix(n: Node) -> Node {
match n {
Node::Text(t) => Node::Text(escape_table_cell(&t)),
Node::LineBreak => Node::Text(" ".into()),
Node::Bold { children } => Node::Bold {
children: children.into_iter().map(fix).collect(),
Expand All @@ -903,7 +901,7 @@ fn table_cell(nodes: Vec<Node>) -> Vec<Node> {
title,
children,
} => Node::Link {
url: url.replace('|', "%7C"),
url,
title,
children: children.into_iter().map(fix).collect(),
},
Expand Down
Loading
Loading