Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
32f53d3
fix(spreadsheet): single-row sheets, workbook titles, XLSB detection
chaliy Oct 26, 2026
c913ef0
fix(spreadsheet): single-row sheets, workbook titles, XLSB detection
chaliy Oct 26, 2026
b7fad42
chore: merge stale branch head
chaliy Oct 26, 2026
7322499
fix(spreadsheet): satisfy clippy chunks_exact_to_as_chunks
chaliy Oct 27, 2026
119c757
chore: merge origin/main
chaliy Oct 27, 2026
465cb08
Merge remote-tracking branch 'origin/main' into claude/project-thread…
chaliy Oct 27, 2026
f4b06d9
Merge remote-tracking branch 'origin/main' into claude/project-thread…
chaliy Oct 27, 2026
86b7104
chore(knowledge): tidy log after merge
chaliy Oct 27, 2026
28f4d17
Merge remote-tracking branch 'origin/main' into claude/project-thread…
chaliy Oct 27, 2026
3a62597
Merge remote-tracking branch 'origin/main' into claude/project-thread…
chaliy Oct 27, 2026
e0365e9
Merge remote-tracking branch 'origin/main' into claude/project-thread…
chaliy Oct 27, 2026
a6a7d7d
feat(spreadsheet): read XLSB workbooks with a bounds-checked BIFF12 r…
chaliy Oct 27, 2026
f36f38e
chore: supersede merged PR history
chaliy Oct 27, 2026
7130864
chore: merge origin/main; XLSB honours the parse time budget
chaliy Oct 27, 2026
50b5aea
chore: merge origin/main
chaliy Oct 27, 2026
7b9703c
feat(pptx): convert PowerPoint presentations slide by slide
chaliy Oct 27, 2026
c422071
chore: supersede merged PR history
chaliy Oct 27, 2026
5e2469b
test(js): use legacy .ppt as the unsupported-format example
chaliy Oct 27, 2026
c058ec4
chore: merge origin/main
chaliy Oct 27, 2026
41e90ee
chore(knowledge): drop trailing whitespace
chaliy Oct 27, 2026
f4daf3b
feat(api): structured warnings; Markkit is Clone + Debug
chaliy Oct 27, 2026
6415396
feat(limits): structured ResourceLimit errors with LimitKind
chaliy Oct 27, 2026
e67a42f
chore: supersede merged PR history
chaliy Oct 27, 2026
0a2bb5a
Merge remote-tracking branch 'origin/main' into claude/project-thread…
chaliy Oct 27, 2026
c2a3a86
Merge remote-tracking branch 'origin/main' into claude/project-thread…
chaliy Oct 27, 2026
94a0e8f
chore(deps): make tokio optional
chaliy Oct 27, 2026
22b4c89
chore: supersede merged PR history
chaliy Oct 27, 2026
226b552
Merge remote-tracking branch 'origin/main' into claude/project-thread…
chaliy Oct 27, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,14 @@ jobs:
echo "HTTP client leaked into the docx,spreadsheet,markdown,email feature set"
exit 1
fi
if cargo tree -p markkit --no-default-features --features docx,spreadsheet -e normal | grep -E 'tokio'; then
echo "tokio leaked into the docx,spreadsheet feature set"
exit 1
fi
if cargo tree -p markkit -e normal | grep -E 'tokio'; then
echo "tokio leaked into the default feature set"
exit 1
fi
- name: Run examples
run: |
for ex in basic custom_selectors configuration ir_access transforms; do
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ just pre-pr # Pre-PR checks
- Stable Rust, pinned in `rust-toolchain.toml`
- `cargo fmt`, `cargo clippy --workspace --all-targets -- -D warnings`
- Licenses/advisories: `cargo deny check` (`deny.toml`), `cargo audit`
- Format deps stay feature-gated; `http_client` opt-in (never default); default and `docx,spreadsheet` builds must not link reqwest/hyper (CI checks)
- Format deps stay feature-gated; `http_client` opt-in (never default); default and `docx,spreadsheet` builds must not link reqwest/hyper or tokio (CI checks)

### Pre-PR Checklist

Expand Down
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
collapsing).
- Spreadsheets make a third fewer allocations per cell (XLSX values are formatted without an
intermediate copy and trimmed in place).
- Dependencies: `tokio` is optional, pulled in only by `http_client` and `tool`, so default and
`docx,spreadsheet` builds no longer link it. CI checks both trees. `serde_json` stays required
(always-on `.ipynb` input).
- **Breaking**: limit fields moved from `MarkkitBuilder` to `MarkkitBuilder::limits`
(`config().max_input_size` → `config().limits.max_input_size`); `max_processing_secs` field is
now `limits.timeout: Duration`. Builder setters are unchanged, plus new `timeout`,
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,8 @@ let markkit = Markkit::builder()
| `tool` | No | `markkit::tool::MarkkitTool` LLM agent tool (Everruns toolkit contract) |
| `logging` | No | Reserved: pulls in `tracing`, emits no events yet |

`http_client` is opt-in: the default build has no HTTP/TLS stack in the dependency tree.
`http_client` is opt-in: the default build has no HTTP/TLS stack and no tokio in the dependency
tree.
Enable it to get `extract_url` (the CLI and the Python/Node bindings ship with it on):

```toml
Expand Down
8 changes: 4 additions & 4 deletions crates/markkit/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,17 @@ spreadsheet = ["dep:calamine", "dep:zip", "dep:quick-xml", "dep:cfb"]
markdown = ["dep:pulldown-cmark"]
email = ["dep:mail-parser"]
# Opt-in: the default build has no network stack (no reqwest/hyper/TLS).
http_client = ["dep:reqwest", "tokio/net"]
http_client = ["dep:reqwest", "dep:tokio", "tokio/net"]
logging = ["dep:tracing"]
# LLM tool contract (`markkit::tool`): Everruns toolkit-library shape.
tool = ["dep:tower", "dep:futures-core", "dep:base64"]
tool = ["dep:tower", "dep:futures-core", "dep:base64", "dep:tokio", "tokio/sync"]
# JSON Schema for the IR (`markkit::ir::json_schema()`)
json-schema = ["dep:schemars"]
all_formats = ["html", "pdf", "docx", "pptx", "spreadsheet", "markdown", "email"]

[dependencies]
# Async
tokio = { workspace = true, features = ["sync", "io-util"] }
# Async (optional): only URL fetching and the tool's output channel need tokio.
tokio = { workspace = true, optional = true }

# Error handling
thiserror.workspace = true
Expand Down
2 changes: 1 addition & 1 deletion knowledge/foundations/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ It follows a pipeline architecture: **detect → parse → normalize → convert
2. **Streaming where possible**: Process large documents without loading entirely into memory
3. **Configurable output**: Users control what gets included/excluded
4. **Safe by default**: All input is treated as untrusted; no code execution
5. **Feature-gated formats**: Only compile what you need
5. **Feature-gated formats**: Only compile what you need. `tokio` is optional (`http_client`, `tool`). `serde_json` stays required: always-on Jupyter notebook input parses JSON

## Pipeline

Expand Down
1 change: 1 addition & 0 deletions knowledge/log.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
* **Over-escaping fix**: line-start `#`, `+` and ordered markers are escaped only when followed by whitespace or the end of the text run, so `4.6 out of 5` no longer renders `4\.6` (eval gap). `> - =` stay always escaped. See [Threat Model](security/threat-model.md) T1, [Evaluation Harness](operations/eval.md).
* **MCP server**: `markkit mcp` serves `MarkkitTool` over MCP stdio (initialize, ping, tools/list, tools/call); line size capped from `--max-size`, tool failures as `isError` results. New [MCP Server](integrations/mcp-server.md) concept; MCP messages row in the [Threat Model](security/threat-model.md).
* **Nesting cost**: HTML parsing was quadratic in nesting depth (html5ever) and exponential in nested tables (descendant row selection duplicated inner rows); text flattened past `max_nesting_depth` leaked `<script>`/`<style>` bodies. Fixed with a pre-scan nesting cap, direct-structure table rows and a non-content-aware flattener; new invariant S21. Found by `markkit-bench`. See [HTML Extraction](formats/html-extraction.md) and [Threat Model](security/threat-model.md) T2c.
* **Optional tokio (#11)**: `tokio` only via `http_client` / `tool`; CI asserts it is not in the default or `docx,spreadsheet` tree. `serde_json` stays required because notebook input is always on. See [Markkit Architecture](foundations/architecture.md).
* **Spreadsheet follow-ups (#11)**: single-row sheets render as data under column letters; XLSX/ODS/XLS titles come from document properties; XLSB content type no longer maps to XLSX; XLSX without `_rels/.rels` parses; trailing-newline output documented as intended; release profile switched to `panic = "unwind"` so `catch_unwind` is effective in markkit's own binaries. See [Format Support](formats/format-support.md) and [Threat Model](security/threat-model.md) T6c.
* **Plain text and Markdown input**: text is always on and goes through the IR; Markdown (`markdown` feature) is re-parsed with pulldown-cmark rather than passed through, so raw HTML never reaches output. Neither is sniffed from bytes. Shares the call's `Limits` budget (timeout, nodes, nesting depth). New threat T7 and invariant S18. See [Format Support](formats/format-support.md) and [Threat Model](security/threat-model.md).
* **Eval harness on mira**: `markkit-eval` is now a mira study (modeled on bashkit-eval). `markkit_extract` scores markkit's HTML output on ten realistic pages with typed deterministic checks and one scorer per metric; a `mode` axis compares defaults with `main_content` (73.0% vs 86.8% mean, noise 34% vs 82%); dated results committed in `crates/markkit-eval/results/`; gaps recorded (`main_content` drops the forum question, `4.6` over-escaped as `4\.6`, `tfoot`/`caption` dropped, edit links in headings). See [Evaluation Harness](operations/eval.md).
Expand Down
Loading