Repository navigation
feat(bindings): expose every resource limit in CLI, Python, Node and WASM - #40
Merged
Merged
Conversation
Move every resource limit into one markkit::Limits struct (bashkit ExecutionLimits style) on MarkkitBuilder. New limits, generous defaults: - timeout (60s) now bounds parsing and rendering, not only URL fetch - max_output_size (128 MiB), checked per block/row; stops table amplification - max_nodes (5M) for HTML DOM and DOCX XML nodes - max_nesting_depth (256) configurable, hard ceiling 1024 for stack safety Breaking: limit fields moved to builder.limits; max_processing_secs field is now limits.timeout (setter kept).
…-mx19ge # Conflicts: # CHANGELOG.md # crates/markkit/src/extract.rs
…-mx19ge # Conflicts: # crates/markkit/src/config.rs # crates/markkit/src/extract.rs # crates/markkit/src/html/parser.rs
…-mx19ge # Conflicts: # CHANGELOG.md # crates/markkit/src/extract.rs # crates/markkit/src/spreadsheet.rs
…-mx19ge # Conflicts: # knowledge/security/threat-model.md
…WASM CLI: --max-output-size, --timeout, --max-nesting-depth, --max-nodes plus archive, sheet and redirect flags under a Limits help heading. Python: markkit.Limits passed as limits= to every function. Node: MarkkitOptions gains timeout (seconds), maxOutputSize, maxNodes, maxNestingDepth. WASM Options gains maxOutputSize, maxNodes, maxNestingDepth (no timeout: no clock on wasm32).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Михайло · project thread
What changed
This is the follow-up to #27. Every
markkit::Limitsfield can now be set outside Rust:markkit extractgets a "Limits" help section with one flag per limit:--max-input-size(--max-sizestill works as an alias),--max-output-size,--timeout(seconds, fractions allowed),--max-nesting-depth,--max-nodes, the archive and sheet flags, and--max-redirects.markkit.Limits(...)class, a frozen keyword-only constructor with readable attributes. Bad values raiseValueError/OverflowError. Every function acceptslimits=.MarkkitOptionsgets flat keystimeout(seconds, float, and it wins overmaxProcessingSecs),maxOutputSize,maxNodesandmaxNestingDepth. A negative or NaNtimeoutis rejected.OptionsgetsmaxOutputSize,maxNodesandmaxNestingDepth. There is notimeout, because wasm32 has no clock. Unknown keys throw, so passingtimeoutfails loudly instead of being ignored.baseUrlandmainContent(feat(html): resolve relative URLs against a base URL #31, feat(html): Readability-style main-content detection #38), and Pythonextract_html_sync_with_optionsgetsbase_url=andmain_content=, so the bindings match the core builder.The binding READMEs, the knowledge runtime docs and parity table, CHANGELOG and log are updated.
Why
The user asked for limits on every aspect. #27 added them to the core, and this PR makes them reachable from every runtime.
Before / After
Before: the CLI had only
--max-size. Python had no limit arguments, and Node and WASM had no output, node or depth caps.After (CLI, table-amplification input that renders 12 MB of Markdown):
Python smoke run:
Test results: Node
options.test.cjspasses 9 of 9, WASM tests pass 19 of 19, and CLI unit and integration tests pass.Risk
--max-sizeis kept.Checklist
🤖 Generated with Claude Code