Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
- XLSX packages missing `_rels/.rels` no longer fail inside calamine.
- Release builds use `panic = "unwind"`, so a parser panic becomes an error instead of aborting the
CLI or the host Python/Node process.
- Text that only looks like a block marker is no longer escaped: `4.6 out of 5` stays `4.6`, not
`4\.6`; same for `#tag` and `+1`. Markers followed by whitespace (`1. `, `# `, `+ `) are still
escaped.
- Exclude selectors now apply to `<li>`, `<tr>`, `<th>` and `<td>`; they were ignored there.
- Indented HTML (most real web pages) now keeps blank lines between blocks; previously
headings, paragraphs, lists and tables after the first ran together with a leading space
Expand Down
22 changes: 19 additions & 3 deletions crates/markkit/src/render/escape.rs
Original file line number Diff line number Diff line change
Expand Up @@ -42,17 +42,33 @@ pub(crate) fn push_text(buf: &mut String, s: &str, pos: Position) -> bool {
continue;
}
line_start = false;
// A marker only opens a block when followed by whitespace. At the end of
// the run the next node may supply it, so escape there too.
let opens = |at: usize| chars.get(at).is_none_or(|n| *n == ' ' || *n == '\t');
match c {
'#' | '>' | '-' | '+' | '=' => {
// `-` (thematic break, setext), `=` (setext), `>` (quote): always.
'>' | '-' | '=' => {
buf.push('\\');
buf.push(c);
i += 1;
continue;
}
'#' | '+' => {
let run = chars[i..].iter().take_while(|d| **d == c).count();
if c == '#' && opens(i + run) || c == '+' && opens(i + 1) {
buf.push('\\');
buf.push(c);
i += 1;
continue;
}
}
'0'..='9' => {
// Ordered list marker: up to 9 digits then `.` or `)`.
// Ordered list marker: up to 9 digits, `.` or `)`, then whitespace.
let digits = chars[i..].iter().take_while(|d| d.is_ascii_digit()).count();
if digits <= 9 && matches!(chars.get(i + digits), Some('.') | Some(')')) {
if digits <= 9
&& matches!(chars.get(i + digits), Some('.') | Some(')'))
&& opens(i + digits + 1)
{
buf.extend(&chars[i..i + digits]);
buf.push('\\');
buf.push(chars[i + digits]);
Expand Down
36 changes: 36 additions & 0 deletions crates/markkit/tests/output_safety.rs
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,42 @@ fn line_start_block_markers_are_escaped() {
}
}

#[test]
fn line_start_marker_lookalikes_render_verbatim() {
// Not block markers: no whitespace after `#`, `+`, or the ordered `.`/`)`.
for s in [
"4.6 out of 5 stars",
"3.14 is pi",
"1)x",
"#hashtag",
"##tag",
"+1 for this",
"2024.10.27 release",
] {
let md = render(vec![para(vec![text(s)])]);
assert_eq!(md, format!("{s}\n"), "{s:?} over-escaped");
assert_eq!(block_tags(&md), vec!["Paragraph"], "{s:?} -> {md:?}");
}
}

#[test]
fn line_start_marker_at_run_end_stays_escaped() {
// The next inline node may start with whitespace and complete the marker.
for (a, b) in [
("1.", " x"),
("#", " h"),
("##", " h"),
("+", " li"),
("7)", "\tx"),
] {
let md = render(vec![para(vec![text(a), text(b)])]);
assert_eq!(block_tags(&md), vec!["Paragraph"], "{a:?}+{b:?} -> {md:?}");
let md = render(vec![para(vec![text(a)])]);
assert_eq!(block_tags(&md), vec!["Paragraph"], "{a:?} -> {md:?}");
assert_eq!(visible_text(&md), a, "{a:?} -> {md:?}");
}
}

#[test]
fn line_start_after_hard_break_is_escaped() {
let md = render(vec![para(vec![
Expand Down
1 change: 1 addition & 0 deletions knowledge/log.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
## 2026-10-27

* **XLSB support (#11)**: Excel binary workbooks are read by a new bounds-checked BIFF12 reader (`spreadsheet/xlsb.rs`) instead of being rejected; calamine's XLSB reader was not used because it panics on malformed records. Same caps, hidden-sheet and macro rules as XLSX. See [Format Support](formats/format-support.md) and [Threat Model](security/threat-model.md) T6c.
* **Over-escaping fix**: line-start `#`, `+` and ordered markers are escaped only when followed by whitespace or the end of the text run, so `4.6 out of 5` no longer renders `4\.6` (eval gap). `> - =` stay always escaped. See [Threat Model](security/threat-model.md) T1, [Evaluation Harness](operations/eval.md).
* **MCP server**: `markkit mcp` serves `MarkkitTool` over MCP stdio (initialize, ping, tools/list, tools/call); line size capped from `--max-size`, tool failures as `isError` results. New [MCP Server](integrations/mcp-server.md) concept; MCP messages row in the [Threat Model](security/threat-model.md).
* **Spreadsheet follow-ups (#11)**: single-row sheets render as data under column letters; XLSX/ODS/XLS titles come from document properties; XLSB content type no longer maps to XLSX; XLSX without `_rels/.rels` parses; trailing-newline output documented as intended; release profile switched to `panic = "unwind"` so `catch_unwind` is effective in markkit's own binaries. See [Format Support](formats/format-support.md) and [Threat Model](security/threat-model.md) T6c.
* **Plain text and Markdown input**: text is always on and goes through the IR; Markdown (`markdown` feature) is re-parsed with pulldown-cmark rather than passed through, so raw HTML never reaches output. Neither is sniffed from bytes. Shares the call's `Limits` budget (timeout, nodes, nesting depth). New threat T7 and invariant S18. See [Format Support](formats/format-support.md) and [Threat Model](security/threat-model.md).
Expand Down
2 changes: 1 addition & 1 deletion knowledge/operations/eval.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ collapsed to one column and dropped every `<td>`, fixed by IR v2 row-header
question title and body are dropped (forum 81% default vs 62%).
- **`main_content` on the wiki page** keeps the TOC, `[edit]` links and
categories.
- **Over-escaping** (since #23): a paragraph starting `4.6 out of 5` renders
- **Over-escaping** (since #23, fixed 2026-10-27): a paragraph starting `4.6 out of 5` renders
`4\.6 ...`; `4.6` is not an ordered-list marker (needs `4.` + space), so the
backslash is noise for text consumers (commerce, both modes; before #23
main_content passed the page).
Expand Down
3 changes: 2 additions & 1 deletion knowledge/security/threat-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,8 @@ that renders as headings, links, code fences or tables).
- Output is markdown text with no HTML pass-through, enforced at render time by
`render/escape.rs` for every format (the IR keeps raw text):
- Text escapes `` ` * [ ] < ~ | ``, `_` at word boundaries, entity-like `&`, a
trailing `\` or `!`, and block markers (`# > - + =`, `1.`/`1)`) at line start.
trailing `\` or `!`, and block markers at line start: `> - =` always, `#` runs,
`+` and `1.`/`1)` only before whitespace or the end of the text run.
Newlines inside text become spaces.
- Link schemes allowlisted (`http`, `https`, `mailto`, `tel`, `ftp`, relative);
image schemes `http`, `https`, relative and raster `data:image/*`. Others are
Expand Down
Loading