Benchmark coverage: missing constructions, wellformed levels, eight entry points

[mathieu17g]

Closes #138.

The tables of PERFORMANCE-v0.4.md and the README's access-pattern table rest on one generated XMark-style document, and it carries none of these:

• a reference, named or numeric, in text or in an attribute value;
• an attribute value that needs §3.3.3 white-space normalization;
• a comment, a CDATA section or a processing instruction;
• a DOCTYPE.

XMarkGenerator.jl now has an opt-in feature for each. Placement is deterministic, every k-th word, item or person, and draws nothing from the random stream, so a generated twin keeps the XMark-style document's elements and attributes in the same order: a difference between the two belongs to the construction alone. The default document is byte-identical to the one behind the published tables. One correction to the issue's diagnosis: the generator's escaping path had no caller, so nothing would have been escaped had the vocabulary needed it; the special words now reach the document through that path.

What the scripts gain, after the entity twin of #130 in section (6) of profile.jl:

• profile.jl, section (7): every reader over the XMark-style document and two twins. The escaped twin replaces one drawn word in ten by one carrying a predefined entity or a character reference, and gives every item and person a note attribute to decode or to normalize. The markup twin has a CDATA section, a comment and a processing instruction per item and person, under a DOCTYPE holding the schema's 74 element and 14 attribute-list declarations and no entity, which parse_dtd reads.
• profile.jl, section (8): the three wellformed levels on the XMark-style document, on the escaped twin, and on the XMark-style document's character data alone.
• flatnode_bench.jl: the other seven entry points that had no cell, on the XMark-style document's 1000th item addressed in every reader.
• benchmarks.jl: nothing measured changes, but every cell that builds a C tree now frees it per sample, outside the timing (see below), and benchmarks_results.md is regenerated.
• PERFORMANCE-v0.4.md: Tables 6 to 8, replacing the :strict note that no script reproduced; the four parts its tables fall into, as section headings; the XMark paper and the XML Benchmarks survey cited at the first mention, and the paper's §4.4 where the twins are introduced; and "XMark-style document" for what the text called a corpus, a word the XMark literature does not use for a single document. Every figure of the document and of the README is republished from one campaign, with the C trees freed.

What the twins show

	document	escaped twin	markup twin
Cursor stream	24.8 ms · 1	38.2 ms · 269,015	26.1 ms · 5
parse → Node	49.8 ms · 2,526,927	76.5 ms (GC 11.8) · 2,904,925	50.1 ms · 2,566,935
parse → Node{SubString}	44.1 ms · 2,416,965	46.3 ms · 2,429,766	46.3 ms · 2,456,973
LazyNode walk	136 ms (GC 0.5) · 272,762	155 ms (GC 1.0) · 541,776	139 ms (GC 0.5) · 272,762
LazyNode attribute sweep	142 ms (GC 0.5) · 272,762	148 ms (GC 0.5) · 314,362	145 ms (GC 0.4) · 272,762
FlatNode walk	3.82 ms · 0	14.1 ms · 269,014	3.89 ms · 0
EzXML StreamReader (libxml2)	67.3 ms (GC 1.0) · 1,172,072	76.3 ms (GC 0.9) · 1,172,697	79.0 ms (GC 0.9) · 1,188,708
EzXML parsexml (libxml2)	38.3 ms · —	47.6 ms · —	45.0 ms · —

• The escaped column is the decode path: a decoded value allocates about six times, the same +269,014 on Cursor, LazyNode and FlatNode for the 46,583 text tokens carrying a reference. unescape copies its argument to a String, runs a regular-expression replace and copies the result.
• Node{SubString} does not decode; its +12,801 are the §3.3.3 normalization of the 3,200 attribute values carrying a literal tab or newline, four allocations each.
• The markup column costs its extra nodes and nothing more, plus the prolog probe of a DOCTYPE that declares no entity: four allocations and 6 µs at every entry, which Include internal entity replacement text per XML 1.0 §4.4: &e; should resolve in content and attribute values #130 measured on a document with no DOCTYPE at all.
• Parsing that DOCTYPE's 88 declarations takes 8.1 µs.
• libxml2, on the same three documents: its reader touches node names only, so its escaped column is the lexing of a reference, not its decoding; its DOM build decodes, for 9 ms over the plain build against 27 ms for parse → Node, and takes the markup twin's comments, processing instructions, CDATA sections and DOCTYPE for 7 ms over it, where parse → Node shows no difference beyond noise.

What the levels show

parse(…, Node; wellformed = …)	:structural	:strict	ratio
the XMark-style document, text share 57 %	48.5 ms	59.6 ms	1.2×
its escaped twin, 81,799 references	67.5 ms (GC 5.1)	103 ms (GC 19.3)	1.3×
its character data alone, 8.1 MB	0.49 ms	7.0 ms	14×

• The note said ~1.1× on the XMark-style document and up to ~20× on a pure-text document; measured, 1.2× and 14×, the ratios taken on the time net of the GC share.
• It described one check where :strict runs two: the character-range scan, in proportion to the text share, and the reference check, only on tokens that carry a &, which the XMark-style document never had.
• The reference check is a regular-expression match per reference: 583,076 allocations on the escaped twin, seven per reference. The character-range scan allocates nothing.
• libxml2 has no levels and always enforces well-formedness in full: 37 ms on the plain document, 45 ms on the escaped twin, 4.0 ms on its character data alone, where the C library parses, checks and builds in a little over half the time of the :strict character-range scan by itself.

The entry points

The issue's eight entry points had no cell, so nothing said what any of them costs, or which reader pays for what. They are now measured on one item element, the 1000th of the XMark-style document, addressed in every reader. The allocation counts are the absolutes: most of these answer in nanoseconds and allocate nothing, so any later change that makes one of them allocate will show.

entry point	reader	time	allocations
sourcespan	FlatNode	3.5 ns	0
sourcespan	LazyNode	1.26 µs	1
splicetext	FlatNode	281 µs	3, the 13.5 MiB result
splicetext	LazyNode	311 µs	4, the 13.5 MiB result
issamenode	FlatNode	2.2 ns	0
issamenode	LazyNode	1.4 ns	0
depth	FlatNode	1.8 ns	0
depth(child, root)	Node	4.32 ms	2
siblings(child, root)	Node	5.26 ms	15
foreach_attr	LazyNode	36.0 ns	0
eachattribute	LazyNode	69.7 ns	0
xpath, /site/regions/asia/item[500]	Node	2.5 µs	26
xpath, //item[@featured='yes']	Node	8.58 ms	60, 17.4 MiB
parse_dtd, the schema's 88 declarations	—	8.1 µs	372

What the numbers say:

• The same question costs nanoseconds, microseconds or milliseconds depending on what the reader keeps. FlatNode stores each node's span and parent, so sourcespan and depth are one lookup. LazyNode keeps only the source, so sourcespan has to scan to the element's end. Node keeps no parent link, so depth and siblings search the whole tree down from the root: 4.32 ms where FlatNode answers in 1.8 ns.
• splicetext returns the document with one node replaced, so it costs a copy of the document whichever reader asks: the 13.5 MiB in the allocation column is the result itself.
• The two attribute loops of LazyNode are allocation-free: foreach_attr yields raw tokens, eachattribute decoded pairs, and on attributes that need no decoding the pair costs about twice the tokens, 70 ns against 36.
• xpath over the descendant axis allocates 17 MiB for a query that returns 303 elements; the positional path, which returns one, allocates 26 times.

A measurement defect found on the way

The libxml2 rows first added to Table 6 did not reproduce: the same DOM build read 55 ms in one section and 94 ms in the next. The cause is not libxml2, which is stable through a bare xmlReadMemory/xmlFreeDoc loop, nor the Julia collector, which gcsample = true rules out. EzXML attaches its finalizer to the document's node, not to the document, so the finalize(doc) the benchmark cells relied on frees nothing:

12 parses of the escaped twin	resident set size, the memory the process holds in RAM
xmlReadMemory / xmlFreeDoc by ccall	flat at 425 MiB
EzXML.parsexml + finalize(doc)	+144 MiB per parse, 2 GiB at the end
EzXML.parsexml + finalize(doc.node)	flat at 425 MiB

A benchmark loop allocates two Julia objects per sample, so the collector never runs inside a cell and the orphaned trees pile up: about 8 GiB per 5 s cell, tens of gigabytes over a run, until macOS compresses and swaps. The cells that allocate most, the escaped and markup twins, were the first to suffer; every libxml2 cell was pessimistic. Every C-tree cell of the three scripts now frees its tree per sample, outside the timing, through the document's node (benchmarks.jl's comment described exactly this risk and believed finalize(doc) covered it), and LightXML's extraction cell moves its free out of the timed region for the same reason. Every figure of README.md and PERFORMANCE-v0.4.md is republished from one campaign with the fix in place:

	published before	freed per sample
libxml2 DOM build, Table 5 and the README	47.6 ms	37.3 ms
libxml2 full extraction, Table 2	65 ms	58 ms
LightXML extraction, Table 2, free out of the timing	66 ms	47 ms
libxml2 DOM build, escaped twin, Table 6	55 to 94 ms	47.6 ms
README cross-library, Parse, medium, EzXML	46.7 ms	39.7 ms

The XML.jl figures move within noise.

Left out, and why

The real-world documents the issue lists as candidates are not added:

• Treebank, for the text-share extreme the :strict claim needed: the XMark-style document's character data alone gives it;
• OpenStreetMap, for §3.3.3: the note attributes give the normalization its values;
• DBLP, entity-dense: its entities are declared in an external subset, which is §4.4 inclusion is internal-subset only, and the dense real-world case declares externally #137.

Three of the findings above are candidates for issues of their own rather than fixes here: the decode path's six allocations per value, the reference check's seven per reference, and a character-range scan that alone takes longer than libxml2's full parse of the same text.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.47%. Comparing base (730be7f) to head (3302462).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #140   +/-   ##
=======================================
  Coverage   96.47%   96.47%           
=======================================
  Files          15       15           
  Lines        2494     2494           
=======================================
  Hits         2406     2406           
  Misses         88       88           

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.