Repository navigation
feat(ir): math nodes and IR JSON Schema - #34
Merged
Merged
Conversation
chaliy
force-pushed
the
claude/project-thread-2rc0mr
branch
2 times, most recently
from
October 27, 2026 00:20
322a5dd to
487e394
Compare
- Node::Math / MathBlock (TeX), rendered as $tex$ / $$ blocks with dollar escaping - HTML: MathML x-tex annotation, KaTeX and MathJax wrappers - schema/ir.schema.json generated via schemars (json-schema feature), drift-checked in tests; just schema regenerates
chaliy
force-pushed
the
claude/project-thread-2rc0mr
branch
from
October 27, 2026 00:29
487e394 to
f38631a
Compare
2 tasks done
chaliy
added a commit
that referenced
this pull request
Oct 27, 2026
<!-- claude-projects-attribution: {"github_login":"chaliy"} -->
_Requested by **Михайло** · [project
thread](https://claude.ai/code/project/chan_01MUtksmx61ypcr6wBFEWsbM?thread=cmsg_01MUtksmx61ypcr6wBFEWsbMDLeuybnBNqgVL9B9kMmARU)_
## What changed
Markdown output is now safe to render, for every input format. The PR is
rebased onto current `main`, so it covers IR v2 (footnotes, definition
lists, figures, task items, table spans), math (#34), transforms (#36),
relative URL resolution (#31) and the limits from #27. #27's output and
deadline checks are kept in the reworked renderer.
- **Text renders verbatim.** These characters are escaped:
- backtick, `*`, `_` (at word boundaries), `[`, `]`, the less-than sign,
`~`, `|` and entity-like `&`;
- line-start block markers: `#`, `>`, `-`, `+`, `=`, `1.`.
Document text can no longer produce raw HTML, links, emphasis, headings,
lists or tables.
- **URL scheme allowlist.** Links keep `http`, `https`, `mailto`, `tel`,
`ftp` and relative URLs. Images keep `http`, `https`, relative URLs and
raster `data:image/*`. Any other scheme is dropped and its link text or
alt text is kept: `javascript:`, `vbscript:`, `data:text/html`, `file:`.
Scheme detection ignores whitespace and control characters and reads the
whole scheme.
- **Destinations and titles.** Parentheses, angle brackets, backslash
and spaces are percent-encoded. Entity-like `&` is escaped, so
`javascript:` cannot decode into a scheme. Empty destinations are
written `<>`. Titles escape quotes, backslash and less-than.
- **Code.** Fences and spans are longer than any backtick run inside
them, and adjacent spans cannot merge their fences. The code-block
language keeps only its first word.
- **Math renders as code.** TeX comes from page markup, and plain GFM
has no math syntax, so `$tex$` let an annotation containing an img tag
or a link through. Math now uses GitHub's forms: `` $`tex`$ `` inline
and a ```` ```math ```` fence for display math. GitHub renders both as
math. Every other renderer shows them as inert code.
- **Structure stays in its container.**
- Block nodes inside inline content are flattened to one line. Examples:
a code block or display math in a table cell, a paragraph in a heading.
- List item bodies are indented by the marker width, so content nested
under `1.` items now nests correctly.
- Every block starts on its own line, and hard breaks never produce a
blank line.
- Nested links render as text.
- **Raw text in the IR.** The DOCX and spreadsheet parsers stop
pre-escaping table pipes, so the JSON IR holds raw text. The renderer
escapes pipes in table context.
All escaping lives in the new `render/escape.rs`, with the design
decisions in its header. The new invariant is S17, since #27 took S14 to
S16.
## Why
Threat model T1 promised "no HTML pass-through", but the renderer
escaped nothing. On `main`, a probe page showed four problems:
- it produced a raw img tag with an onerror handler;
- it kept `javascript:` links;
- a triple-backtick line inside a `pre` block closed the fence;
- links broke on `)`.
A fuzzing pass against the new oracle then found eight more ways content
could escape its container. The cargo-fuzz harness comes in a follow-up
PR. Each finding is a regression test here.
## Before / After
Input (HTML) is a paragraph containing `# not heading *x*`, followed by:
- an escaped img tag with an onerror handler,
- a link to `javascript:alert(1)`,
- a link to `http://x/a)b`,
- a `pre/code` block holding a triple-backtick line, then `evil`, then
another triple-backtick line.
Before:
- the paragraph rendered as a heading with emphasis and a live img tag;
- the script link was kept;
- the second URL was split at `)`;
- the inner backticks closed the fence, so `evil` ended up outside the
code block.
After:
~~~text
\# not heading \*x\* \[img tag with backslash-escaped less-than, shown
as text]
click[p](http://x/a%29b)
````
```
evil
```
````
~~~
The new `tests/output_safety.rs` has 39 tests, 4 of them for math. 22 of
them failed on the old `main`. A pulldown-cmark (GFM) oracle checks four
things:
- no raw HTML,
- no unsafe destination,
- the expected block structure,
- verbatim text.
Proptests cover arbitrary text, URLs and code. #27's `limits` tests pass
in debug and release, including the deep-nesting tests under a small
stack.
`just pre-pr` passes locally except `cargo deny`, which is not installed
here; the CI Audit job runs it. New dev-dependency: `pulldown-cmark`
(MIT), used in tests only.
## Risk
- Medium.
- Output changes:
- Text with Markdown-significant characters is now backslash-escaped.
- Links and images with schemes outside the allowlist are dropped.
- Empty link targets are now written `<>`.
- Content nested under ordered list items is now indented 3 or more
spaces.
- Math now renders as `` $`tex`$ `` and ```` ```math ````.
- Tests updated to match:
- one golden fixture;
- the four math assertions from #34;
- #31's `javascript:` pass-through test, now
`absolute_dangerous_schemes_are_not_resolved`. Base resolution still
never rewrites the URL, and the allowlist then drops it.
## Checklist
- [x] Tests added or updated
- [x] Backward compatibility considered (CHANGELOG Unreleased entry)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Михайло · project thread
What changed
Final IR v2 pieces from the roadmap: math and a published JSON Schema.
Node::Math { tex }(inline) andNode::MathBlock { tex }(display). HTML maps MathML via itsapplication/x-texannotation, plus KaTeX (.katex/.katex-display) and MathJax (<mjx-container>) wrappers. Their duplicated visual HTML is dropped. MathML without TeX keeps its text.$tex$(one line, trimmed, bare$escaped) and$$fenced blocks ($$inside is split, so it can't close the fence).schema/ir.schema.json(draft 2020-12) is generated from the Rust types withschemarsbehind a newjson-schemafeature (markkit::ir::json_schema()).ir::tests::committed_schema_is_currentfails CI when the file drifts;just schemaregenerates it.Why
Roadmap "Richer IR": math was the remaining construct, and a schema lets Python, Node and other consumers rely on the IR JSON rather than only the Markdown.
Before / After
KaTeX page (
<span class="katex">…<annotation encoding="application/x-tex">x^2</annotation>…<span class="katex-html">x2</span></span>):Before:
Inline x2x^2x2.(MathML text + annotation + visual HTML all leak through)After:
Risk
Nodeis#[non_exhaustive], so this isn't breaking).schemarsis optional and off by default.$…$or$$(html_integration::math_dollars_cannot_break_out, renderer unit tests).Checklist
foundations/ir.mdincl. JSON Schema section,formats/html-extraction.md, log), CHANGELOGGenerated by Claude Code