[chaliy]

What changed

extract_url (and so the CLI, Python extract_url and Node extractUrl) now treats the URL as untrusted:

• Non-public destinations refused by default with the new Error::UrlBlocked. That covers loopback, RFC 1918, link-local / cloud metadata (169.254.169.254), CGNAT, reserved and multicast ranges, the IPv6 equivalents, and IPv4 embedded in IPv6 (mapped, NAT64, 6to4). The check runs at three layers:
  • on the URL itself, before any I/O;
  • on every redirect hop;
  • in a DNS resolver that drops non-public addresses, so DNS rebinding can't reach them.
• Only http/https are fetched.
• Redirects are capped by max_redirects (default 5).
• The body is bounded by max_input_size. A declared Content-Length over the limit is rejected up front; a body without one is stopped while streaming. Before this change the whole body was buffered first.
• The whole fetch is bounded by max_processing_secs. Before this change there was no timeout.
• New builder options: allow_private_networks(true) for trusted internal URLs, and allow_host(..) for an exact, case-insensitive host allowlist that is checked on every hop.

Why

Threat model T3 (SSRF) was open. With the default http_client feature, anyone passing an attacker-chosen URL could reach internal services or cloud metadata, or exhaust memory and time.

Before / After

Before: markkit extract http://169.254.169.254/latest/meta-data/ sent the request, and no limits applied to the body or the time taken.

After:

$ markkit extract http://169.254.169.254/latest/meta-data/
Error: URL blocked: address 169.254.169.254 is not public
$ markkit extract http://localhost:8080/
Error: URL blocked: host 'localhost' is a loopback name
$ markkit extract file:///etc/passwd
Error: URL blocked: scheme 'file' not allowed (http/https only)

The new tests/url_security.rs has 14 tests, run against a local raw-TCP server. Before the fix, 9 of them failed. Now all 14 pass, as do 5 new unit tests in fetch::tests. just pre-pr is green.

Risk

• Medium
• This changes behavior for callers that fetch internal hosts on purpose. They now need allow_private_networks(true). The CLI and the bindings have no opt-in flag yet.
• Residual risk: when an HTTP(S)_PROXY is set in the environment, the proxy resolves names itself, so the resolver filter can't apply. The URL, allowlist and redirect checks still do. This is documented in T3 and SECURITY.md.

Checklist

• Tests added or updated
• Backward compatibility considered (CHANGELOG Security entry, new Error variant)

---

Generated by Claude Code