From 28f550939a8eb254aef6988c643c274f43fbb166 Mon Sep 17 00:00:00 2001 From: Diogo Sousa Date: Wed, 28 Oct 2026 14:06:18 +0000 Subject: [PATCH] Restore the tree construction tests used by the feedback tests. The html5lib-tests submodule update in f3093c9 moved to a revision without the `tree-construction` directory, since those tests are now maintained in web-platform-tests (WPT). Because `for_each_test_file()` accepted an empty glob, the feedback tests silently went from 1,809 inputs to the 31 in `tests/data/regression/`, and CI stayed green. We now keep a copy of the tests in `tests/data/tree-construction/`, taken from html5lib-tests 9329e64 (the last revision before they were removed), and `for_each_test_file()` fails when nothing matches. The current WPT version of these tests has five inputs with CDATA sections inside SVG/MathML integration points (e.g. `x`) that we tokenize as bogus comments, so switching to it needs a fix first. --- .gitattributes | 3 + tests/data/tree-construction/README.md | 25 + tests/data/tree-construction/adoption01.dat | 354 +++ tests/data/tree-construction/adoption02.dat | 56 + tests/data/tree-construction/blocks.dat | 695 +++++ tests/data/tree-construction/comments01.dat | 217 ++ tests/data/tree-construction/doctype01.dat | 474 +++ tests/data/tree-construction/domjs-unsafe.dat | Bin 0 -> 10356 bytes tests/data/tree-construction/entities01.dat | 943 ++++++ tests/data/tree-construction/entities02.dat | 309 ++ .../tree-construction/foreign-fragment.dat | 645 ++++ .../data/tree-construction/html5test-com.dat | 301 ++ tests/data/tree-construction/inbody01.dat | 54 + tests/data/tree-construction/isindex.dat | 49 + tests/data/tree-construction/main-element.dat | 46 + tests/data/tree-construction/math.dat | 104 + .../tree-construction/menuitem-element.dat | 241 ++ .../namespace-sensitivity.dat | 22 + tests/data/tree-construction/noscript01.dat | 237 ++ ...pending-spec-changes-plain-text-unsafe.dat | Bin 0 -> 927 bytes .../pending-spec-changes.dat | 46 + .../tree-construction/plain-text-unsafe.dat | Bin 0 -> 9486 bytes tests/data/tree-construction/quirks01.dat | 53 + tests/data/tree-construction/ruby.dat | 302 ++ tests/data/tree-construction/scriptdata01.dat | 372 +++ .../data/tree-construction/search-element.dat | 46 + tests/data/tree-construction/svg.dat | 104 + tests/data/tree-construction/tables01.dat | 323 ++ tests/data/tree-construction/template.dat | 1673 +++++++++++ tests/data/tree-construction/tests1.dat | 1958 +++++++++++++ tests/data/tree-construction/tests10.dat | 853 ++++++ tests/data/tree-construction/tests11.dat | 523 ++++ tests/data/tree-construction/tests12.dat | 62 + tests/data/tree-construction/tests14.dat | 75 + tests/data/tree-construction/tests15.dat | 216 ++ tests/data/tree-construction/tests16.dat | 2602 +++++++++++++++++ tests/data/tree-construction/tests17.dat | 177 ++ tests/data/tree-construction/tests18.dat | 572 ++++ tests/data/tree-construction/tests19.dat | 1398 +++++++++ tests/data/tree-construction/tests2.dat | 831 ++++++ tests/data/tree-construction/tests20.dat | 842 ++++++ tests/data/tree-construction/tests21.dat | 306 ++ tests/data/tree-construction/tests22.dat | 190 ++ tests/data/tree-construction/tests23.dat | 168 ++ tests/data/tree-construction/tests24.dat | 79 + tests/data/tree-construction/tests25.dat | 288 ++ tests/data/tree-construction/tests26.dat | 453 +++ tests/data/tree-construction/tests3.dat | 305 ++ tests/data/tree-construction/tests4.dat | 74 + tests/data/tree-construction/tests5.dat | 210 ++ tests/data/tree-construction/tests6.dat | 663 +++++ tests/data/tree-construction/tests7.dat | 453 +++ tests/data/tree-construction/tests8.dat | 165 ++ tests/data/tree-construction/tests9.dat | 482 +++ .../tree-construction/tests_innerHTML_1.dat | 844 ++++++ tests/data/tree-construction/tricky01.dat | 336 +++ .../tree-construction/void-in-phrasing.dat | 151 + tests/data/tree-construction/webkit01.dat | 783 +++++ tests/data/tree-construction/webkit02.dat | 775 +++++ .../html5lib_tests/feedback_tests/mod.rs | 2 +- tests/harness/suites/mod.rs | 9 +- 61 files changed, 24537 insertions(+), 2 deletions(-) create mode 100644 tests/data/tree-construction/README.md create mode 100644 tests/data/tree-construction/adoption01.dat create mode 100644 tests/data/tree-construction/adoption02.dat create mode 100644 tests/data/tree-construction/blocks.dat create mode 100644 tests/data/tree-construction/comments01.dat create mode 100644 tests/data/tree-construction/doctype01.dat create mode 100644 tests/data/tree-construction/domjs-unsafe.dat create mode 100644 tests/data/tree-construction/entities01.dat create mode 100644 tests/data/tree-construction/entities02.dat create mode 100644 tests/data/tree-construction/foreign-fragment.dat create mode 100644 tests/data/tree-construction/html5test-com.dat create mode 100644 tests/data/tree-construction/inbody01.dat create mode 100644 tests/data/tree-construction/isindex.dat create mode 100644 tests/data/tree-construction/main-element.dat create mode 100644 tests/data/tree-construction/math.dat create mode 100644 tests/data/tree-construction/menuitem-element.dat create mode 100644 tests/data/tree-construction/namespace-sensitivity.dat create mode 100644 tests/data/tree-construction/noscript01.dat create mode 100644 tests/data/tree-construction/pending-spec-changes-plain-text-unsafe.dat create mode 100644 tests/data/tree-construction/pending-spec-changes.dat create mode 100644 tests/data/tree-construction/plain-text-unsafe.dat create mode 100644 tests/data/tree-construction/quirks01.dat create mode 100644 tests/data/tree-construction/ruby.dat create mode 100644 tests/data/tree-construction/scriptdata01.dat create mode 100644 tests/data/tree-construction/search-element.dat create mode 100644 tests/data/tree-construction/svg.dat create mode 100644 tests/data/tree-construction/tables01.dat create mode 100644 tests/data/tree-construction/template.dat create mode 100644 tests/data/tree-construction/tests1.dat create mode 100644 tests/data/tree-construction/tests10.dat create mode 100644 tests/data/tree-construction/tests11.dat create mode 100644 tests/data/tree-construction/tests12.dat create mode 100644 tests/data/tree-construction/tests14.dat create mode 100644 tests/data/tree-construction/tests15.dat create mode 100644 tests/data/tree-construction/tests16.dat create mode 100644 tests/data/tree-construction/tests17.dat create mode 100644 tests/data/tree-construction/tests18.dat create mode 100644 tests/data/tree-construction/tests19.dat create mode 100644 tests/data/tree-construction/tests2.dat create mode 100644 tests/data/tree-construction/tests20.dat create mode 100644 tests/data/tree-construction/tests21.dat create mode 100644 tests/data/tree-construction/tests22.dat create mode 100644 tests/data/tree-construction/tests23.dat create mode 100644 tests/data/tree-construction/tests24.dat create mode 100644 tests/data/tree-construction/tests25.dat create mode 100644 tests/data/tree-construction/tests26.dat create mode 100644 tests/data/tree-construction/tests3.dat create mode 100644 tests/data/tree-construction/tests4.dat create mode 100644 tests/data/tree-construction/tests5.dat create mode 100644 tests/data/tree-construction/tests6.dat create mode 100644 tests/data/tree-construction/tests7.dat create mode 100644 tests/data/tree-construction/tests8.dat create mode 100644 tests/data/tree-construction/tests9.dat create mode 100644 tests/data/tree-construction/tests_innerHTML_1.dat create mode 100644 tests/data/tree-construction/tricky01.dat create mode 100644 tests/data/tree-construction/void-in-phrasing.dat create mode 100644 tests/data/tree-construction/webkit01.dat create mode 100644 tests/data/tree-construction/webkit02.dat diff --git a/.gitattributes b/.gitattributes index 6f479225..24a0444d 100644 --- a/.gitattributes +++ b/.gitattributes @@ -3,3 +3,6 @@ tests/data/** linguist-vendored benches/data/** linguist-vendored fuzz/corpus/** linguist-vendored + +# Keep the tree construction tests byte-for-byte: some of them contain CR characters. +tests/data/tree-construction/*.dat -text diff diff --git a/tests/data/tree-construction/README.md b/tests/data/tree-construction/README.md new file mode 100644 index 00000000..7dbe7864 --- /dev/null +++ b/tests/data/tree-construction/README.md @@ -0,0 +1,25 @@ +# Tree construction tests + +These are the html5lib tree construction tests, copied from [html5lib-tests][html5lib-tests] at +revision `9329e64694e7835d0dcff9811e22856ef6ad16f9`, the last one before they were removed from +that repository. They are now maintained in [web-platform-tests][wpt] (WPT). + +We only use the inputs (the `#data` sections): the feedback tests compare our tokens against the +ones html5ever produces for the same input. The scripted tests are not included (`scripted/` in +html5lib-tests, `scripted_*.dat` in WPT), since they require script execution. + +To replace the files with the ones from WPT revision `$rev`, run this from the root of the +repository (and update the source revision above): + +```sh +url=https://raw.githubusercontent.com/web-platform-tests/wpt/$rev/html/syntax/parsing/resources +rm tests/data/tree-construction/*.dat +gh api "repos/web-platform-tests/wpt/contents/html/syntax/parsing/resources?ref=$rev" \ + --jq '.[].name | select(endswith(".dat") and (startswith("scripted_") | not))' | + while read -r name; do + curl -sSfLo "tests/data/tree-construction/$name" "$url/$name" + done +``` + +[html5lib-tests]: https://github.com/html5lib/html5lib-tests +[wpt]: https://github.com/web-platform-tests/wpt/tree/master/html/syntax/parsing/resources diff --git a/tests/data/tree-construction/adoption01.dat b/tests/data/tree-construction/adoption01.dat new file mode 100644 index 00000000..38f98efd --- /dev/null +++ b/tests/data/tree-construction/adoption01.dat @@ -0,0 +1,354 @@ +#data +

+#errors +(1,3): expected-doctype-but-got-start-tag +(1,10): adoption-agency-1.3 +#document +| +| +| +| +|

+| + +#data +1

23

+#errors +(1,3): expected-doctype-but-got-start-tag +(1,12): adoption-agency-1.3 +#document +| +| +| +| +| "1" +|

+| +| "2" +| "3" + +#data +1 +#errors +(1,3): expected-doctype-but-got-start-tag +(1,17): adoption-agency-1.3 +#document +| +| +| +| +| "1" +|