Make default HTML conversion allocation-light

[dereuromark]

Outcome

Make ordinary Djot-to-HTML conversion substantially faster without weakening the public AST, extension behavior, or the authoritative parser.

The default facade now attempts a conservative borrowed-source layout for common ASCII documents up to 64 KiB. If the complete document is accepted, it publishes byte-identical HTML without constructing public AST nodes. Any ambiguous or unsupported boundary rejects the entire attempt and re-runs through the existing parser and renderer.

Why this is so much faster

The previous convert() route always paid for the complete owned pipeline:

1. split and classify blocks;
2. allocate a Document plus block and inline node objects;
3. attach attributes and source-derived state;
4. recursively dispatch every node through HtmlRenderer;
5. concatenate the rendered fragments.

That architecture remains the source of truth, but most application traffic does not need an inspectable AST. The new route removes the dominant costs for the common facade call:

• source text is borrowed as string slices instead of copied into nodes;
• block layout is recognized in one conservative pass;
• common non-nested inline markup is tokenized by PCRE in native code instead of walking every byte in PHP;
• heading IDs are derived through the shared tracker, preserving deduplication;
• no output is returned until the whole document has been accepted;
• rejection is fail-closed, so syntax uncertainty affects speed only, never semantics.

The 64 KiB and ASCII limits are deliberate parity boundaries, not parser limits. Rich inline syntax, lazy or ambiguous headings, lists, tables, quotes, Unicode, attributes, profiles, safety modes, warnings, source lines, extensions, listeners, output transformers, custom parser/renderer access, alternate renderers, and explicit parse() calls all retain the owned AST pipeline.

convertFile() now uses the same facade route.

Performance

PHP 8.5.9 NTS, CLI OPcache enabled, Node.js 22.22.2, Rust 1.97.1, Go 1.22.2. Warmed in-process trials; 20 measured iterations and 5 warmups. The cross-language runners now generate the exact same paragraph-safe 51,179-byte Djot fixture in every runtime.

Djot implementation	median	throughput	vs djot-php
Rust jotdown 0.7	1.20 ms	40.5 MB/s	6.22x faster
PHP djot-php	7.46 ms	6.5 MB/s	baseline
JavaScript @djot/djot 0.3	10.99 ms	4.4 MB/s	1.47x slower
Go godjot 1.0.6	17.47 ms	2.9 MB/s	2.34x slower

So this makes djot-php faster than the reference JavaScript implementation and godjot on the shared common-document workload. Native Rust remains faster.

Against djot-php itself, a separate 57,410-byte representative document measured:

route	median
owned AST via an explicitly configured converter	49.5 ms
borrowed default facade	6.5 ms
improvement	7.7x

The HTML was byte-identical.

Reproduce the cross-language comparison with:

node tests/benchmark/compare-languages.mjs --djot-only --iterations=20 --warmup=5

The benchmark cleanup in this PR also:

• makes PHP, JavaScript, Rust, and Go generate the same size-bounded fixture without truncating through markup;
• enables CLI OPcache for the PHP comparison;
• uses cargo run so custom Cargo target directories work;
• uses Go module mode and reports the pinned godjot 1.0.6 version;
• excludes unrelated Python Markdown libraries under --djot-only.

Correctness

• 2,815 tests pass with 18,458 assertions.
• 13 focused borrowed-layout tests pass with 30 assertions.
• Every official corpus document admitted by the facade is byte-identical to the authoritative converter.
• A deterministic 2,000-document mixed-inline probe admitted 568 conservative documents with zero mismatches.
• Nested emphasis, empty list markers, definition-title ambiguity, custom renderers, unsafe URLs, Unicode, headings with lazy continuation, lists, tables, quotes, and attributes are explicitly covered as parity or fallback boundaries.
• PHPStan and PHPCS pass for all changed PHP production and test files.

Validation

vendor/bin/phpunit
vendor/bin/phpstan analyse src/DjotConverter.php src/Performance/BorrowedHtmlLayout.php tests/BorrowedHtmlLayoutTest.php --no-progress
vendor/bin/phpcs src/DjotConverter.php src/Performance/BorrowedHtmlLayout.php tests/BorrowedHtmlLayoutTest.php tests/benchmark/benchmark.php
node --check tests/benchmark/benchmark-js.mjs
node --check tests/benchmark/compare-languages.mjs

[codecov]

Codecov Report

❌ Patch coverage is 86.55660% with 57 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.26%. Comparing base (1d5e7ce) to head (cd96f16).

Files with missing lines	Patch %	Lines
src/Performance/BorrowedHtmlLayout.php	85.45%	57 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master     #276      +/-   ##
============================================
- Coverage     92.49%   92.26%   -0.24%     
- Complexity     3708     3896     +188     
============================================
  Files           110      111       +1     
  Lines         10501    10923     +422     
============================================
+ Hits           9713    10078     +365     
- Misses          788      845      +57     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.