diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index c8b31c6..546d882 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -89,6 +89,14 @@ jobs: echo "HTTP client leaked into the docx,spreadsheet,markdown,email feature set" exit 1 fi + if cargo tree -p markkit --no-default-features --features docx,spreadsheet -e normal | grep -E 'tokio'; then + echo "tokio leaked into the docx,spreadsheet feature set" + exit 1 + fi + if cargo tree -p markkit -e normal | grep -E 'tokio'; then + echo "tokio leaked into the default feature set" + exit 1 + fi - name: Run examples run: | for ex in basic custom_selectors configuration ir_access transforms; do diff --git a/AGENTS.md b/AGENTS.md index 3b96a7b..760fc07 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -98,7 +98,7 @@ just pre-pr # Pre-PR checks - Stable Rust, pinned in `rust-toolchain.toml` - `cargo fmt`, `cargo clippy --workspace --all-targets -- -D warnings` - Licenses/advisories: `cargo deny check` (`deny.toml`), `cargo audit` -- Format deps stay feature-gated; `http_client` opt-in (never default); default and `docx,spreadsheet` builds must not link reqwest/hyper (CI checks) +- Format deps stay feature-gated; `http_client` opt-in (never default); default and `docx,spreadsheet` builds must not link reqwest/hyper or tokio (CI checks) ### Pre-PR Checklist diff --git a/CHANGELOG.md b/CHANGELOG.md index ed6ea4d..4da6eef 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -157,6 +157,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/). collapsing). - Spreadsheets make a third fewer allocations per cell (XLSX values are formatted without an intermediate copy and trimmed in place). +- Dependencies: `tokio` is optional, pulled in only by `http_client` and `tool`, so default and + `docx,spreadsheet` builds no longer link it. CI checks both trees. `serde_json` stays required + (always-on `.ipynb` input). - **Breaking**: limit fields moved from `MarkkitBuilder` to `MarkkitBuilder::limits` (`config().max_input_size` → `config().limits.max_input_size`); `max_processing_secs` field is now `limits.timeout: Duration`. Builder setters are unchanged, plus new `timeout`, diff --git a/README.md b/README.md index 5d8a009..bec1676 100644 --- a/README.md +++ b/README.md @@ -209,7 +209,8 @@ let markkit = Markkit::builder() | `tool` | No | `markkit::tool::MarkkitTool` LLM agent tool (Everruns toolkit contract) | | `logging` | No | Reserved: pulls in `tracing`, emits no events yet | -`http_client` is opt-in: the default build has no HTTP/TLS stack in the dependency tree. +`http_client` is opt-in: the default build has no HTTP/TLS stack and no tokio in the dependency +tree. Enable it to get `extract_url` (the CLI and the Python/Node bindings ship with it on): ```toml diff --git a/crates/markkit/Cargo.toml b/crates/markkit/Cargo.toml index be276f4..9922b0a 100644 --- a/crates/markkit/Cargo.toml +++ b/crates/markkit/Cargo.toml @@ -23,17 +23,17 @@ spreadsheet = ["dep:calamine", "dep:zip", "dep:quick-xml", "dep:cfb"] markdown = ["dep:pulldown-cmark"] email = ["dep:mail-parser"] # Opt-in: the default build has no network stack (no reqwest/hyper/TLS). -http_client = ["dep:reqwest", "tokio/net"] +http_client = ["dep:reqwest", "dep:tokio", "tokio/net"] logging = ["dep:tracing"] # LLM tool contract (`markkit::tool`): Everruns toolkit-library shape. -tool = ["dep:tower", "dep:futures-core", "dep:base64"] +tool = ["dep:tower", "dep:futures-core", "dep:base64", "dep:tokio", "tokio/sync"] # JSON Schema for the IR (`markkit::ir::json_schema()`) json-schema = ["dep:schemars"] all_formats = ["html", "pdf", "docx", "pptx", "spreadsheet", "markdown", "email"] [dependencies] -# Async -tokio = { workspace = true, features = ["sync", "io-util"] } +# Async (optional): only URL fetching and the tool's output channel need tokio. +tokio = { workspace = true, optional = true } # Error handling thiserror.workspace = true diff --git a/knowledge/foundations/architecture.md b/knowledge/foundations/architecture.md index 3e1f993..cdaa4c4 100644 --- a/knowledge/foundations/architecture.md +++ b/knowledge/foundations/architecture.md @@ -21,7 +21,7 @@ It follows a pipeline architecture: **detect → parse → normalize → convert 2. **Streaming where possible**: Process large documents without loading entirely into memory 3. **Configurable output**: Users control what gets included/excluded 4. **Safe by default**: All input is treated as untrusted; no code execution -5. **Feature-gated formats**: Only compile what you need +5. **Feature-gated formats**: Only compile what you need. `tokio` is optional (`http_client`, `tool`). `serde_json` stays required: always-on Jupyter notebook input parses JSON ## Pipeline diff --git a/knowledge/log.md b/knowledge/log.md index af7b3d2..2be446c 100644 --- a/knowledge/log.md +++ b/knowledge/log.md @@ -10,6 +10,7 @@ * **Over-escaping fix**: line-start `#`, `+` and ordered markers are escaped only when followed by whitespace or the end of the text run, so `4.6 out of 5` no longer renders `4\.6` (eval gap). `> - =` stay always escaped. See [Threat Model](security/threat-model.md) T1, [Evaluation Harness](operations/eval.md). * **MCP server**: `markkit mcp` serves `MarkkitTool` over MCP stdio (initialize, ping, tools/list, tools/call); line size capped from `--max-size`, tool failures as `isError` results. New [MCP Server](integrations/mcp-server.md) concept; MCP messages row in the [Threat Model](security/threat-model.md). * **Nesting cost**: HTML parsing was quadratic in nesting depth (html5ever) and exponential in nested tables (descendant row selection duplicated inner rows); text flattened past `max_nesting_depth` leaked `