fix(security): block SSRF and bound extract_url fetches

.github/workflows/coverage.yml

@@ -22,7 +22,9 @@ jobs:
       - name: Install cargo-tarpaulin
         run: cargo install cargo-tarpaulin
       - name: Generate coverage
-        run: cargo tarpaulin --workspace --out xml
+        # llvm engine: the default ptrace engine segfaults on multi-threaded
+        # tokio socket tests (tests/url_security.rs)
+        run: cargo tarpaulin --workspace --engine llvm --out xml
       - name: Upload coverage
         uses: codecov/codecov-action@v4
         with:

AGENTS.md

@@ -29,7 +29,7 @@ Fix root cause. Unsure: read more code; if stuck, ask w/ short options. Unrecogn
 | formats/html-extraction | HTML element mapping, stripping, whitespace, depth limits |
 | formats/format-support | Formats, bytes-in API, detection order, DOCX + spreadsheet mapping |
 | security/security-model | One-page threat/invariant summary |
-| security/threat-model | Threats T1-T6, mitigations, test map, resource limits, invariants S1-S11 |
+| security/threat-model | Threats T1-T6, mitigations, test map, resource limits, invariants S1-S12 |
 | runtimes/python-package | PyO3 `markkit` module, maturin, PyPI |
 | runtimes/node-package | NAPI-RS `@everruns/markkit`, npm, binding parity table |
 | operations/testing | Test layout, categories, CI jobs |

CHANGELOG.md

@@ -7,12 +7,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
 
 ## [Unreleased]
 
+### Security
+
+- `extract_url` (CLI, Python and Node `extract_url` too) now refuses non-public destinations by default:
+  loopback, private, link-local / cloud metadata and other reserved addresses, checked on the URL,
+  after DNS resolution and on every redirect. Only `http`/`https` are fetched. Redirects are capped
+  (`max_redirects`, default 5), the body is bounded by `max_input_size` while streaming, and the whole
+  fetch by `max_processing_secs`. New builder options `allow_private_networks(true)` and `allow_host(..)`;
+  refusals return the new `Error::UrlBlocked`.
+
 ### Changed
 
 - Design docs moved from `specs/` to an OKF v0.2 knowledge bundle in `knowledge/`, validated in CI.
 - CI workflows declare read-only default permissions and do not persist checkout credentials.
 - CI runs every example; `just bench` now runs the criterion benchmarks.
-- README, SECURITY and binding docs corrected: Python module is `markkit`, URL fetching has no private-network blocking yet.
+- README, SECURITY and binding docs corrected: Python module is `markkit`.
 - Dependencies: drop unused `async-trait`, `futures`, `regex`, `unicode-segmentation`, `encoding_rs`,
   `base64`, `mime`, `lol_html` (core) and `tracing-subscriber`, plus unused deps of the CLI, bindings
   and stub crates. Upgrade scraper 0.29, html5ever 0.40, quick-xml 0.42 (now a single version shared

README.md

@@ -168,17 +168,20 @@ markkit = { version = "0.1", default-features = false, features = ["docx", "spre
 
 Office-document limits are configurable on the builder: `max_archive_entries`,
 `max_archive_entry_size`, `max_archive_total_size`, `max_spreadsheet_cells`,
-`max_sheet_rows`, `max_sheet_cols`, `include_hidden_sheets`.
+`max_sheet_rows`, `max_sheet_cols`, `include_hidden_sheets`. URL fetching:
+`allow_private_networks`, `allow_host`, `max_redirects`.
 
 ## Security
 
 Markkit treats every input as untrusted. HTML scripts, styles, forms, iframes and
 SVG are stripped; office containers are decompressed under measured byte limits;
 spreadsheet grids are pre-scanned so sparse cells cannot force huge allocations;
 XML entities are never expanded; macros and formulas are never executed; hidden
-sheets are skipped by default. URL fetching does not yet block private networks,
-so embedders fetching attacker-chosen URLs should fetch themselves and call
-`extract_document` (build with `default-features = false`).
+sheets are skipped by default. `extract_url` accepts only http/https, refuses
+loopback, private, link-local and cloud-metadata addresses (also after DNS
+resolution and on every redirect), caps redirects, and bounds the body by
+`max_input_size` and the fetch by `max_processing_secs`. Opt in to internal
+hosts with `allow_private_networks(true)`; restrict hosts with `allow_host(..)`.
 
 See the [threat model](knowledge/security/threat-model.md) and the
 [security policy](SECURITY.md) for reporting vulnerabilities.

SECURITY.md

@@ -32,7 +32,7 @@ Markkit processes untrusted input (HTML, DOCX, spreadsheets) and converts it to
 - **No code execution**: scripts, styles, forms, iframes and SVG are stripped; macros and formulas are never executed; XML entities are never expanded
 - **Resource limits**: input size, HTML/XML nesting depth, archive entry count and decompressed size, spreadsheet grid cells, rows and columns
 - **Explicit failure**: malformed input returns errors, never panics; unsupported formats fail explicitly
-- **URL fetching**: enabled by the default `http_client` feature; private-network blocking and allowlists are not implemented yet, so fetch attacker-chosen URLs yourself and build with `default-features = false`
+- **URL fetching**: enabled by the default `http_client` feature; http/https only, non-public addresses refused by default (after DNS and on every redirect), optional host allowlist, bounded redirects, body size and fetch time. Behind an environment HTTP proxy, the proxy resolves names, so restrict egress there too
 
 Full analysis: [threat model](knowledge/security/threat-model.md).
 

crates/markkit/Cargo.toml

@@ -19,7 +19,7 @@ html = ["dep:scraper", "dep:html5ever"]
 pdf = []
 docx = ["dep:zip", "dep:quick-xml"]
 spreadsheet = ["dep:calamine", "dep:zip", "dep:quick-xml", "dep:cfb"]
-http_client = ["dep:reqwest"]
+http_client = ["dep:reqwest", "tokio/net"]
 logging = ["dep:tracing"]
 all_formats = ["html", "pdf", "docx", "spreadsheet"]
 
@@ -54,7 +54,7 @@ tracing = { workspace = true, optional = true }
 
 [dev-dependencies]
 anyhow.workspace = true
-tokio = { workspace = true, features = ["macros", "rt-multi-thread"] }
+tokio = { workspace = true, features = ["macros", "rt-multi-thread", "net", "io-util", "time"] }
 tempfile.workspace = true
 proptest.workspace = true
 criterion.workspace = true

crates/markkit/src/config.rs

@@ -32,6 +32,14 @@ pub struct MarkkitBuilder {
     pub max_spreadsheet_cells: u64,
     /// Render hidden / very hidden sheets. Default: false (skipped with a warning).
     pub include_hidden_sheets: bool,
+    /// Let `extract_url` reach loopback, private, link-local and other
+    /// non-public addresses. Default: false (SSRF protection).
+    pub allow_private_networks: bool,
+    /// Hosts `extract_url` may contact, matched exactly and case-insensitively
+    /// on every hop (empty = any public host).
+    pub allowed_hosts: Vec<String>,
+    /// Maximum redirects `extract_url` follows. Default: 5.
+    pub max_redirects: usize,
 }
 
 impl Default for MarkkitBuilder {
@@ -58,6 +66,9 @@ impl Default for MarkkitBuilder {
             max_sheet_cols: 256,
             max_spreadsheet_cells: 4_000_000,
             include_hidden_sheets: false,
+            allow_private_networks: false,
+            allowed_hosts: Vec::new(),
+            max_redirects: 5,
         }
     }
 }
@@ -128,6 +139,24 @@ impl MarkkitBuilder {
         self
     }
 
+    /// Allow `extract_url` to reach non-public addresses (loopback, RFC 1918,
+    /// link-local, cloud metadata, ...). Only for trusted URLs.
+    pub fn allow_private_networks(mut self, yes: bool) -> Self {
+        self.allow_private_networks = yes;
+        self
+    }
+
+    /// Restrict `extract_url` to this host (repeatable). Checked on every redirect hop.
+    pub fn allow_host(mut self, host: impl Into<String>) -> Self {
+        self.allowed_hosts.push(host.into());
+        self
+    }
+
+    pub fn max_redirects(mut self, n: usize) -> Self {
+        self.max_redirects = n;
+        self
+    }
+
     pub fn build(self) -> Markkit {
         Markkit::new(self)
     }

crates/markkit/src/error.rs

@@ -16,6 +16,11 @@ pub enum Error {
     #[error("HTTP error: {0}")]
     Http(String),
 
+    /// `extract_url` refused a URL: disallowed scheme, host outside the
+    /// allowlist, or an address on a non-public network.
+    #[error("URL blocked: {0}")]
+    UrlBlocked(String),
+
     #[error("Encoding error: {0}")]
     Encoding(String),
 

crates/markkit/src/extract.rs

@@ -235,25 +235,20 @@ impl Markkit {
         ))
     }
 
-    /// Extract markdown from a URL by fetching and converting.
+    /// Fetch a URL and extract markdown from the response.
+    ///
+    /// Treats the URL as untrusted (threat model T3): only `http`/`https`;
+    /// loopback, private, link-local and other non-public addresses are
+    /// refused with [`Error::UrlBlocked`] on every hop (including after DNS
+    /// resolution) unless [`MarkkitBuilder::allow_private_networks`] is set;
+    /// [`MarkkitBuilder::allow_host`] restricts hosts; redirects capped by
+    /// `max_redirects`; body capped by `max_input_size`; whole fetch capped
+    /// by `max_processing_secs` ([`Error::ResourceLimit`]).
     #[cfg(feature = "http_client")]
     pub async fn extract_url(&self, url: &str) -> crate::Result<String> {
-        let response = reqwest::get(url)
+        let fetched = crate::fetch::fetch(url, &self.config).await?;
+        self.extract_bytes(&fetched.bytes, fetched.content_type.as_deref())
             .await
-            .map_err(|e| Error::Http(e.to_string()))?;
-
-        let content_type = response
-            .headers()
-            .get("content-type")
-            .and_then(|v| v.to_str().ok())
-            .map(|s| s.to_string());
-
-        let bytes = response
-            .bytes()
-            .await
-            .map_err(|e| Error::Http(e.to_string()))?;
-
-        self.extract_bytes(&bytes, content_type.as_deref()).await
     }
 
     /// PDF extraction is not implemented yet; report it explicitly rather than