From 82621e65f34f233826711f2d6130b10ec6dc2ce2 Mon Sep 17 00:00:00 2001 From: DonislawDev Date: Wed, 14 Oct 2026 16:23:50 +0200 Subject: [PATCH 1/3] html: a page on one line, and styles in a sheet or in attributes formatting=minified puts the whole page on one line with no newline at the end, and it shows the same as the page in lines. The catalogue of every element keeps a space where a line ended, and its pre keeps its line break as a character reference. The manifest says line_ending none. styles=embedded puts one list of CSS rules in a style element, in the head of a page and at the very start of a fragment. styles=inline puts the same rules in a style attribute on every element they are for, outside the catalogue. No rule names anything to fetch. Every piece of markup a page writes is now one entry of an array, and the other spellings are made from the plain one by two rules, so the default bytes are unchanged. The structural check is told both settings and asserts them, four new guards, three golden hashes. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 19 +- README.md | 2 +- internal/format/htmlfile/blocks.go | 43 +-- internal/format/htmlfile/catalogue.go | 12 +- internal/format/htmlfile/characters.go | 5 +- internal/format/htmlfile/html.go | 25 +- internal/format/htmlfile/markup.go | 265 ++++++++++++++++++ internal/format/htmlfile/page.go | 65 ++--- internal/format/htmlfile/pictures.go | 30 +- internal/format/htmlfile/settings.go | 65 ++++- internal/guard/generatorbytes_test.go | 11 + internal/guard/htmlsettings_test.go | 49 +++- internal/guard/htmlstyles_test.go | 224 +++++++++++++++ internal/guard/parity_test.go | 2 + internal/guard/testdata/generator-golden.json | 12 + internal/gui/text/locale/registry/en.json | 39 ++- internal/gui/text/locale/registry/pl.json | 9 +- internal/gui/text/locale/said/en.json | 16 +- internal/gui/text/locale/said/pl.json | 4 +- internal/oracle/strict.py | 119 ++++++-- 20 files changed, 890 insertions(+), 126 deletions(-) create mode 100644 internal/format/htmlfile/markup.go create mode 100644 internal/guard/htmlstyles_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 6f814876..d5364767 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -160,7 +160,7 @@ because it turns other people's test suites red. - **HTML: pictures inside the page.** `inline`, `inline_format` and `inline_size` - the same settings, with the same meaning, as the pictures of an `eml` message - write real pictures of any picture format into the - page as `data:` URLs, each on a line of its own above the blocks, so + page as `data:` URLs, each in a paragraph of its own above the blocks, so opening the page fetches nothing. Each picture is the very file its format writes on its own. The Nu HTML Checker finds nothing wrong with a page of any of the nine formats, Chromium shows every format but `tiff`, and @@ -168,6 +168,23 @@ because it turns other people's test suites red. a picture past about 7.1 MB unless told to read huge trees - on Linux lxml took over 20 seconds over such a page, and with a 15 MB picture it lost the rest of the page as well. Files made without pictures keep their bytes. +- **HTML: a page on one line, and styles.** `formatting=minified` puts the + whole page on one line with no newline at the end, the way a bundler + writes it, and the page shows the same as one written in lines. A program + reading line by line gets the whole page as one line - Go's + `bufio.Scanner` refuses a line past 64 KB unless given a larger buffer - + and text read by joining what the elements hold runs the last word of one + block into the first word of the next. The manifest says `line_ending: + none` for such a page. `styles=embedded` puts CSS rules in a style element, + in the head of a page and at the very start of a fragment, and + `styles=inline` puts the same rules in a style attribute on every heading, + paragraph, list, table and quote the page is filled with. The Nu HTML + Checker finds nothing wrong with either, and Chromium shows both the same. + DOMPurify with its defaults drops the style element without listing it as + removed and keeps the attributes, and lxml's `text_content` and + BeautifulSoup on html5lib count the rules as text of the page. No rule + names anything to fetch. Files made with the earlier settings keep their + bytes. - **A length of time as a setting.** `duration`, `change_interval` and `keyframe_interval` take `10s`, `1m30s`, `1h`, `500ms` or `59.9s`, to the millisecond. A bare number is refused rather than read as seconds, and so is a length that does not diff --git a/README.md b/README.md index 2acef44b..f3c68a64 100644 --- a/README.md +++ b/README.md @@ -621,7 +621,7 @@ recipe. `tfg formats ` prints the allowed range or list for each: | `geojson` | `geometry`, `formatting`, `precision`, `altitude`, `vertices`, `holes`, `winding`, `antimeridian`, `unlocated`, `ids`, `bbox` | | `eml` | `body`, `text_encoding`, `charset`, `headers`, `recipients`, `line_endings`, `attachments`, `attachment_format`, `attachment_size`, `filename_style`, `message_depth`, `inline`, `inline_format`, `inline_size` | | `svg` | `width`, `height` | -| `html` | `structure`, `nesting`, `characters`, `elements`, `bom`, `inline`, `inline_format`, `inline_size` | +| `html` | `structure`, `nesting`, `characters`, `elements`, `bom`, `formatting`, `styles`, `inline`, `inline_format`, `inline_size` | | `yaml`, `toml` | none in this build - the document is a fixed shape, and a size is the only thing to ask for | ``` diff --git a/internal/format/htmlfile/blocks.go b/internal/format/htmlfile/blocks.go index 3b797a7f..799e3cde 100644 --- a/internal/format/htmlfile/blocks.go +++ b/internal/format/htmlfile/blocks.go @@ -25,51 +25,56 @@ type blocks struct { // tail is what the closing record ends with, which is the only thing the // shape of the file changes down here. tail string + // m is how the blocks are spelled. The words drawn are the same in every + // spelling, so one seed fills a page of one line and a page of many with + // the same blocks. + m *markup } // Shortest is the smallest closing record: an empty paragraph plus whatever // this shape closes after it. -func (b blocks) Shortest() int64 { return int64(len(paraOpen) + len(b.tail)) } +func (b blocks) Shortest() int64 { return int64(len(b.m[paraOpen]) + len(b.tail)) } -func (blocks) Append(dst []byte, rng *rand.Rand) []byte { +func (b blocks) Append(dst []byte, rng *rand.Rand) []byte { + m := b.m switch rng.IntN(5) { case 0: - dst = append(dst, "

"...) + dst = append(dst, m[h2Open]...) dst = appendPhrase(dst, rng, 2+rng.IntN(3)) - return append(dst, "

\n"...) + return append(dst, m[h2Close]...) case 1: - dst = append(dst, "
    \n"...) + dst = append(dst, m[ulOpen]...) for i, n := 0, 2+rng.IntN(3); i < n; i++ { - dst = append(dst, "
  • "...) + dst = append(dst, m[liOpen]...) dst = appendPhrase(dst, rng, 3+rng.IntN(4)) - dst = append(dst, "
    • \n"...) + dst = append(dst, m[liClose]...) } - return append(dst, "
\n"...) + return append(dst, m[ulClose]...) case 2: - dst = append(dst, "\n\n"...) + dst = append(dst, m[tableOpen]...) for i, n := 0, 2+rng.IntN(3); i < n; i++ { - dst = append(dst, "\n"...) + dst = append(dst, m[rowClose]...) } - return append(dst, "
idfieldvalue
"...) + dst = append(dst, m[rowOpen]...) dst = strconv.AppendInt(dst, int64(i+1), 10) - dst = append(dst, ""...) + dst = append(dst, m[cellSep]...) dst = append(dst, words[rng.IntN(len(words))]...) - dst = append(dst, ""...) + dst = append(dst, m[cellSep]...) dst = append(dst, words[rng.IntN(len(words))]...) - dst = append(dst, "
\n"...) + return append(dst, m[tableClose]...) case 3: // A real page carries entities, and an ampersand left raw is the // classic way a document stops being well formed. The fixture carries // one on purpose. - dst = append(dst, "

"...) + dst = append(dst, m[quoteOpen]...) dst = appendPhrase(dst, rng, 4+rng.IntN(4)) dst = append(dst, " & "...) dst = appendPhrase(dst, rng, 2) - return append(dst, "

\n"...) + return append(dst, m[quoteClose]...) default: - dst = append(dst, paraOpen...) + dst = append(dst, m[paraOpen]...) dst = appendPhrase(dst, rng, 12+rng.IntN(12)) - return append(dst, paraClose...) + return append(dst, m[paraClose]...) } } @@ -79,7 +84,7 @@ func (blocks) Discard() {} func (b blocks) AppendExact(dst []byte, rng *rand.Rand, n int64) []byte { start := len(dst) - dst = append(dst, paraOpen...) + dst = append(dst, b.m[paraOpen]...) used := int64(len(dst)-start) + int64(len(b.tail)) dst = appendFiller(dst, n-used) return append(dst, b.tail...) diff --git a/internal/format/htmlfile/catalogue.go b/internal/format/htmlfile/catalogue.go index 67950605..b8c24f7e 100644 --- a/internal/format/htmlfile/catalogue.go +++ b/internal/format/htmlfile/catalogue.go @@ -29,6 +29,11 @@ const ( smallPicture = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAICAIAAABLbSncAAAAEUlEQVR42mNwaDiAFTEMLQkAYvNgAQlPC90AAAAASUVORK5CYII=" sound = "data:audio/wav;base64,UklGRjQAAABXQVZFZm10IBAAAAABAAEAQB8AAEAfAAABAAgAZGF0YRAAAACAgICAgICAgICAgICAgICA" + // preformatted is the text of the pre element, the one place in the + // catalogue where a line break is text rather than layout. A page on one + // line has to keep it, so it is named here and found by name. + preformatted = "preformatted text\n keeps its spaces" + // catalogueHead is what the catalogue puts in the head of a page. base // names no address, so it changes nothing a link resolves to, and the icon // is empty data so a browser asks nobody for one. @@ -52,8 +57,7 @@ const (
Fifth level
Sixth level

A quotation in a block.

-
preformatted   text
-  keeps   its spaces
+
` + preformatted + `
Term
Its description.
  1. three
  2. two
@@ -143,7 +147,7 @@ func (s settings) catalogueHeadFor() string { if !s.every || s.shape != structureDocument { return "" } - return catalogueHead + return s.spelling()[headCatalogue] } // catalogueBlocks is what the catalogue adds to the body. @@ -151,5 +155,5 @@ func (s settings) catalogueBlocks() string { if !s.every { return "" } - return catalogueBody + return s.spelling()[bodyCatalogue] } diff --git a/internal/format/htmlfile/characters.go b/internal/format/htmlfile/characters.go index 5520a8a4..67ce624d 100644 --- a/internal/format/htmlfile/characters.go +++ b/internal/format/htmlfile/characters.go @@ -29,10 +29,11 @@ const ( `and entities & < > "   © € € 😀.

` + "\n" ) -// internationalBlocks is what the setting adds, nothing for ascii. +// internationalBlocks is what the setting adds, nothing for ascii, in the +// spelling of the page. func (s settings) internationalBlocks() string { if !s.international { return "" } - return international + return s.spelling()[scripts] } diff --git a/internal/format/htmlfile/html.go b/internal/format/htmlfile/html.go index c6efe840..b3a0d4cb 100644 --- a/internal/format/htmlfile/html.go +++ b/internal/format/htmlfile/html.go @@ -56,10 +56,9 @@ func init() { // than a comment. Label: format.LabelVisible, Oracle: "python-html", - // Style sheets and a page on one line come later - // (docs/HTML-2-2026-10-14.md section 8). Declaring only what is here - // is what makes a recipe asking for them fail loudly instead of - // quietly producing something else. + // Links come later, if at all (docs/HTML-2-2026-10-14.md section 3, + // B4). Declaring only what is here is what makes a recipe asking for + // them fail loudly instead of quietly producing something else. Properties: properties(), GeneratorVersion: generatorVersion, Generator: generator{}, @@ -89,7 +88,7 @@ func (generator) PlanContext(ctx context.Context, r format.Request) (format.Plan return format.Plan{}, err } - pictures, pictureBytes, err := planPictures(ctx, r, s.pictures) + pictures, pictureBytes, err := planPictures(ctx, r, s.pictures, s.spelling()) if err != nil { return format.Plan{}, err } @@ -110,8 +109,10 @@ func (generator) PlanContext(ctx context.Context, r format.Request) (format.Plan Exact: true, Determinism: format.DeterminismByte, Properties: map[string]any{ - "encoding": "utf-8", - "line_ending": "lf", + "encoding": "utf-8", + // none for a page on one line: it has no line to end, and lf + // would describe a file that is not there. + "line_ending": s.lineEnding(), "doctype": doctype, "language": "en", }, @@ -154,9 +155,10 @@ func belowMinimum(s settings, requested, min int64) error { // the reason names, whatever else was asked. reason = core.Says("html.MinimumWithPictures", "the file holds whole blocks and every picture in base64, which is about a third larger than the picture itself, and together they need that much") case s.adds(): - reason = core.Says("html.MinimumWithSettings", "the file holds whole blocks and what %s, %s, %s and %s add to them, and together they need that much", + reason = core.Says("html.MinimumWithSettings", "the file holds whole blocks and what %s, %s, %s, %s and %s add to them, and together they need that much", core.A("Nesting", core.LabelTerm(settingNesting)), core.A("Characters", core.LabelTerm(settingCharacters)), - core.A("Elements", core.LabelTerm(settingElements)), core.A("BOM", core.LabelTerm(textenc.SettingBOM))) + core.A("Elements", core.LabelTerm(settingElements)), core.A("BOM", core.LabelTerm(textenc.SettingBOM)), + core.A("Styles", core.LabelTerm(settingStyles))) case s.shape == structureFragment: reason = core.Says("html.MinimumFragment", "a fragment holds whole blocks, and one of them needs that much") } @@ -178,7 +180,8 @@ func belowMinimum(s settings, requested, min int64) error { // fit and what does it cost", and Plan answers "is this request askable at all". func labelledHead(s settings, r format.Request, min int64) (head, heading string, note *format.Note) { label := core.Label("html", r.Bytes, r.Seed) - heading = "

" + label + "

\n" + m := s.spelling() + heading = m[labelOpen] + label + m[labelClose] plain := s.prologue() head, extra := plain, int64(0) @@ -229,5 +232,5 @@ func minimumBytes() int64 { return minimumFor(settings{shape: structureDocument} // whole block, computed from the pieces Write uses rather than written down so // it cannot drift away from the template. func minimumFor(s settings) int64 { - return int64(len(s.prologue())) + s.openingBytes() + int64(len(paraOpen)) + s.tailBytes() + return int64(len(s.prologue())) + s.openingBytes() + int64(len(s.spelling()[paraOpen])) + s.tailBytes() } diff --git a/internal/format/htmlfile/markup.go b/internal/format/htmlfile/markup.go new file mode 100644 index 00000000..d3baf7bc --- /dev/null +++ b/internal/format/htmlfile/markup.go @@ -0,0 +1,265 @@ +package htmlfile + +import ( + "regexp" + "strings" +) + +// How a page is spelled: every piece of markup it is written with, for one +// layout and one way of carrying its styles (docs/HTML-4-2026-10-14.md). +// +// Held as finished strings, the way the JSON layouts hold theirs, because the +// arithmetic that lands a page on its byte count has to measure what it writes +// rather than predict it. The plain spelling is the bytes every page has been +// written with, and every other spelling is made from it by two rules - the +// rules of the sheet put on each element, and the line breaks taken out. The +// pieces sit in one array, so a rule reaches every piece there is, including +// one added later, and there is no list of them to forget one on. + +// markup is one spelling of a page, a piece to each part. +type markup [parts]string + +// part names one piece of a page's markup. +type part int + +// The pieces a page writes itself, then those it does not. Everything before +// ownParts is spelled by the rules below, and the catalogue of every element, +// written by hand once, keeps its look and has a rule of its own. +const ( + // The skeleton in two halves, the style sheet between them or in front of + // a fragment, and what closes the body and the document. + headOpen part = iota + headClose + sheet + bodyClose + // The label, one level of nesting, and a paragraph. + labelOpen + labelClose + divOpen + divClose + paraOpen + paraClose + // The blocks that fill a page. + h2Open + h2Close + ulOpen + liOpen + liClose + ulClose + tableOpen + rowOpen + cellSep + rowClose + tableClose + quoteOpen + quoteClose + // The paragraphs in other scripts, and a picture's line around its name + // and its base64. + scripts + pictureLead + pictureTail + + ownParts +) + +// The catalogue's two pieces, after the page's own. +const ( + headCatalogue = ownParts + iota + bodyCatalogue + parts +) + +// plain is how a page has been spelled since 2026-08-01, the golden hashes +// holding it there. The skeleton is in two halves so the catalogue and the +// style sheet can go into the head between them, and the title is empty until a +// label fills it - the empty pair is what the smallest document carries. +var plain = markup{ + headOpen: `` + "\n" + + `` + "\n" + + `` + "\n" + + `` + "\n" + + emptyTitle + "\n", + headClose: `` + "\n" + + `` + "\n", + bodyClose: "\n\n", + labelOpen: "

", + labelClose: "

\n", + // One level of nesting, each on a line of its own. + divOpen: "
\n", + divClose: "
\n", + paraOpen: "

", + paraClose: "

\n", + h2Open: "

", + h2Close: "

\n", + ulOpen: "
    \n", + liOpen: "
  • ", + liClose: "
    • \n", + ulClose: "
\n", + tableOpen: "\n\n", + rowOpen: "\n", + // The table closes and the quote holds a paragraph. + tableClose: "
idfieldvalue
", + cellSep: "", + rowClose: "
\n", + quoteOpen: "

", + quoteClose: "

\n", + scripts: international, + pictureLead: `

`, + pictureTail: `

` + "\n", + headCatalogue: catalogueHead, + bodyCatalogue: catalogueBody, +} + +// rule is the look of one element: its declarations, property and value. +type rule struct { + element string + declarations [][2]string +} + +// rules are the styles a page carries, one list for both places they go. A +// font every system has, colours and spacing, and nothing that names an +// address: a page that fetches a font or a picture for its style reaches a +// network when it is opened (D16). No rule for div either, or the levels of +// nesting would each push the content further in. +var rules = []rule{ + {"h1", [][2]string{{"font-family", "sans-serif"}, {"font-size", "28px"}, {"color", "#0b3d91"}, {"border-bottom", "2px solid #0b3d91"}, {"margin", "0 0 16px"}}}, + {"h2", [][2]string{{"font-family", "sans-serif"}, {"font-size", "20px"}, {"color", "#0b3d91"}, {"margin", "16px 0 8px"}}}, + {"p", [][2]string{{"font-family", "sans-serif"}, {"color", "#222222"}, {"line-height", "1.5"}, {"margin", "0 0 12px"}}}, + {"ul", [][2]string{{"margin", "0 0 12px"}, {"padding-left", "24px"}}}, + {"li", [][2]string{{"font-family", "sans-serif"}, {"color", "#222222"}, {"margin", "4px 0"}}}, + {"table", [][2]string{{"border-collapse", "collapse"}, {"margin", "0 0 12px"}}}, + {"th", [][2]string{{"font-family", "sans-serif"}, {"background", "#0b3d91"}, {"color", "#ffffff"}, {"text-align", "left"}, {"padding", "4px 8px"}}}, + {"td", [][2]string{{"font-family", "sans-serif"}, {"border", "1px solid #cccccc"}, {"padding", "4px 8px"}}}, + {"blockquote", [][2]string{{"border-left", "4px solid #cccccc"}, {"margin", "12px 0"}, {"padding-left", "12px"}, {"color", "#555555"}}}, +} + +// css is a rule's declarations, readable on a page of lines and as short as +// they go on a page of one. +func (r rule) css(minified bool) string { + colon, sep := ": ", "; " + if minified { + colon, sep = ":", ";" + } + parts := make([]string, len(r.declarations)) + for i, d := range r.declarations { + parts[i] = d[0] + colon + d[1] + } + return strings.Join(parts, sep) +} + +// sheetOf is the rules as a style element: one rule to a line, or all of them +// on one. +func sheetOf(minified bool) string { + var b strings.Builder + if minified { + b.WriteString("") + return b.String() + } + b.WriteString("\n") + return b.String() +} + +// styledTag finds the start tag of every element a rule is for, in one pass, so +// a tag given an attribute is never found again. The space or the bracket after +// the name keeps p from matching pre and th from matching thead. +var styledTag = func() *regexp.Regexp { + names := make([]string, len(rules)) + for i, r := range rules { + names[i] = r.element + } + return regexp.MustCompile(`<(` + strings.Join(names, "|") + `)([ >])`) +}() + +// withStyles puts each element's rule on it as a style attribute, first among +// its attributes. +func withStyles(s string, minified bool) string { + css := make(map[string]string, len(rules)) + for _, r := range rules { + css[r.element] = r.css(minified) + } + return styledTag.ReplaceAllStringFunc(s, func(tag string) string { + m := styledTag.FindStringSubmatch(tag) + return "<" + m[1] + ` style="` + css[m[1]] + `"` + m[2] + }) +} + +// oneLineCatalogue is the catalogue on one line. Where a line ended it keeps a +// space, because between two inline elements a line break shows as one - taken +// out, 52 of its 575 elements move on the screen (Chromium 152, 2026-10-14). +// The one break that is text, in the pre element, becomes a reference to the +// character, which is what keeps its text the same. +func oneLineCatalogue(body string) string { + body = strings.Replace(body, preformatted, strings.ReplaceAll(preformatted, "\n", " "), 1) + return strings.ReplaceAll(body, "\n", " ") +} + +// spell makes the spelling of one layout and one way of carrying styles out of +// the plain one. +func spell(minified bool, styles string) *markup { + m := plain + if styles == stylesInline { + for p := range ownParts { + m[p] = withStyles(m[p], minified) + } + } + if styles == stylesEmbedded { + m[sheet] = sheetOf(minified) + } + if minified { + for p := range ownParts { + m[p] = strings.ReplaceAll(m[p], "\n", "") + } + m[headCatalogue] = strings.ReplaceAll(m[headCatalogue], "\n", "") + m[bodyCatalogue] = oneLineCatalogue(m[bodyCatalogue]) + } + return &m +} + +// spellings are made once, when the program starts: the engine holds the plan +// of every file at once, and a plan holds a pointer to one of these rather +// than a copy of the attributes on every element. +var spellings = func() map[spellingKey]*markup { + out := map[spellingKey]*markup{} + for _, minified := range []bool{false, true} { + for _, styles := range []string{stylesNone, stylesEmbedded, stylesInline} { + out[spellingKey{minified, styles}] = spell(minified, styles) + } + } + return out +}() + +type spellingKey struct { + minified bool + styles string +} + +// styleName is the value of styles, none for settings made without one. +func (s settings) styleName() string { + if s.styles == "" { + return stylesNone + } + return s.styles +} + +// spelling is how these settings spell a page. +func (s settings) spelling() *markup { + return spellings[spellingKey{s.minified, s.styleName()}] +} + +// lineEnding is what the manifest says the lines of the page end with: none for +// a page on one line, which has no line to end, where lf would describe a file +// that is not there. +func (s settings) lineEnding() string { + if s.minified { + return "none" + } + return "lf" +} diff --git a/internal/format/htmlfile/page.go b/internal/format/htmlfile/page.go index 409623aa..b0ebf906 100644 --- a/internal/format/htmlfile/page.go +++ b/internal/format/htmlfile/page.go @@ -8,32 +8,11 @@ import "strings" // nesting, the paragraphs in other scripts, the catalogue, the blocks, and the // closing paragraph with everything it closes after it. -const ( - // The skeleton of a page, in two halves so the catalogue can put what it - // needs into the head between them. Joined they are the bytes every page - // opened with before there were settings. - headOpen = `` + "\n" + - `` + "\n" + - `` + "\n" + - `` + "\n" + - `` + "\n" - headClose = `` + "\n" + - `` + "\n" +// The markup itself - the skeleton, the label, the levels and the blocks - is +// spelled in markup.go, in every layout and every way of carrying styles. - // The title is filled in per file when a label is wanted, so the empty pair - // above is what the smallest document carries. - emptyTitle = `` - - paraOpen = "

" - paraClose = "

\n" - bodyClose = "\n\n" - - // One level of nesting, each on a line of its own. A page whose levels - // shared a line would be a page of one very long line, which is a different - // question with a setting of its own to come. - divOpen = "
\n" - divClose = "
\n" -) +// emptyTitle is the title a page carries until a label fills it in. +const emptyTitle = `` // byteOrderMark is the UTF-8 mark, the only one a page may carry: the standard // requires UTF-8 (docs/HTML-STRUCTURE-2026-09-08.md section 1). Written as its @@ -41,12 +20,17 @@ const ( var byteOrderMark = string([]byte{0xEF, 0xBB, 0xBF}) // prologue is everything in front of the label: the mark, and the skeleton down -// to the body with whatever the catalogue puts in the head - or for a fragment -// nothing but the mark. +// to the body with whatever the catalogue and the style sheet put in the head - +// or for a fragment the mark and the sheet. A fragment has no head, and a sheet +// at its very start is one a parser puts into the head it makes up, where the +// Nu HTML Checker finds nothing wrong with it. Further down it would be a +// sheet in the body, which the checker calls an error rule by rule +// (docs/HTML-4-2026-10-14.md section 1). func (s settings) prologue() string { - p := "" + m := s.spelling() + p := m[sheet] if s.shape == structureDocument { - p = headOpen + s.catalogueHeadFor() + headClose + p = m[headOpen] + s.catalogueHeadFor() + m[sheet] + m[headClose] } if s.bom { p = byteOrderMark + p @@ -59,12 +43,12 @@ func (s settings) prologue() string { // kept in the plan - a hundred thousand levels are 600 KB, and the engine holds // the plan of every file at once. func (s settings) opening() string { - return strings.Repeat(divOpen, s.nesting) + s.internationalBlocks() + s.catalogueBlocks() + return strings.Repeat(s.spelling()[divOpen], s.nesting) + s.internationalBlocks() + s.catalogueBlocks() } // openingBytes is the length of opening, worked out without building it. func (s settings) openingBytes() int64 { - return int64(s.nesting*len(divOpen) + len(s.internationalBlocks()) + len(s.catalogueBlocks())) + return int64(s.nesting*len(s.spelling()[divOpen]) + len(s.internationalBlocks()) + len(s.catalogueBlocks())) } // blocks is the body builder for these settings. @@ -76,23 +60,28 @@ func (s settings) openingBytes() int64 { // nested a thousand deep with not one - the right size, deterministic, // and nonsense. func (s settings) blocks() blocks { - tail := paraClose + strings.Repeat(divClose, s.nesting) + m := s.spelling() + tail := m[paraClose] + strings.Repeat(m[divClose], s.nesting) if s.shape == structureDocument { - tail += bodyClose + tail += m[bodyClose] } - return blocks{tail: tail} + return blocks{tail: tail, m: m} } // tailBytes is the length of what the last record ends with, worked out // without building it. func (s settings) tailBytes() int64 { - n := len(paraClose) + s.nesting*len(divClose) + m := s.spelling() + n := len(m[paraClose]) + s.nesting*len(m[divClose]) if s.shape == structureDocument { - n += len(bodyClose) + n += len(m[bodyClose]) } return int64(n) } // adds says the settings put something into the page beyond the skeleton and -// the blocks, which is what the refusal of a size too small has to name. -func (s settings) adds() bool { return s.nesting > 0 || s.international || s.every || s.bom } +// the blocks, which is what the refusal of a size too small has to name. A page +// on one line is smaller than the same page on many, so formatting adds nothing. +func (s settings) adds() bool { + return s.nesting > 0 || s.international || s.every || s.bom || s.styleName() != stylesNone +} diff --git a/internal/format/htmlfile/pictures.go b/internal/format/htmlfile/pictures.go index 80d67482..5e23ebc6 100644 --- a/internal/format/htmlfile/pictures.go +++ b/internal/format/htmlfile/pictures.go @@ -13,8 +13,9 @@ import ( ) // Pictures inside the page: real files of a picture format, each in a data: -// URL (RFC 2397) on a line of its own, the way a page saved as one file -// carries them. The settings are the ones a mail message has, with the same +// URL (RFC 2397) in a paragraph of its own - on a line of its own too, unless +// the whole page is one line - the way a page saved as one file carries them. +// The settings are the ones a mail message has, with the same // names and the same meaning (package picture). // // Measured on 2026-10-14 on pages from this program (docs/HTML-3-2026-10-14.md @@ -32,14 +33,11 @@ const picturesGroup = "Pictures in the page" // reason. const maxPage = int64(1) << 60 -// pictureClose ends the line a picture sits on. -const pictureClose = `">

` + "\n" - // pictureProperties are the settings of the pictures the page shows. func pictureProperties() []format.Property { return picture.Properties(picturesGroup, picture.Details{ Count: "How many pictures the page shows, each one a real file of its format written into the page as a data: URL, so opening the page fetches nothing. " + - "They sit above the blocks, one to a line, and each says which it is in its alt text.", + "They sit above the blocks, each in a paragraph of its own, and each says which it is in its alt text.", Format: "The format of the pictures. Any format whose files are pictures, such as png, jpg or gif. Chromium shows every one of them but tiff.", Size: "How big each picture is. Inside the page it is base64 on one line, about a third larger. " + "lxml, and BeautifulSoup on top of it, leave out a picture past about 7.1 MB unless told to read huge trees. " + @@ -48,23 +46,23 @@ func pictureProperties() []format.Property { } // pictureLine is one picture, planned: the file, and the line it sits on up -// to where the base64 starts. +// to where the base64 starts and after it ends. type pictureLine struct { - desc format.Descriptor - plan format.Plan - open string + desc format.Descriptor + plan format.Plan + open, close string } // bytes is what the line takes in the page. func (p pictureLine) bytes() int64 { - return int64(len(p.open)) + base64Length(p.plan.Bytes) + int64(len(pictureClose)) + return int64(len(p.open)) + base64Length(p.plan.Bytes) + int64(len(p.close)) } // planPictures plans the pictures, each with the seed of its place (RC4), so // asking for one more changes none of the others. The format is asked whether // it is a picture even when none are asked for: a value the setting cannot // take is a mistake either way, and eml refuses it the same way. -func planPictures(ctx context.Context, r format.Request, s picture.Settings) ([]pictureLine, int64, error) { +func planPictures(ctx context.Context, r format.Request, s picture.Settings, m *markup) ([]pictureLine, int64, error) { desc, ok, err := picture.Of(s.Format) if err != nil { return nil, 0, err @@ -87,7 +85,7 @@ func planPictures(ctx context.Context, r format.Request, s picture.Settings) ([] if cp.Bytes > maxPage { return nil, 0, pageTooLarge(cp.Bytes) } - line := pictureLine{desc: desc, plan: cp, open: pictureOpen(desc, i+1)} + line := pictureLine{desc: desc, plan: cp, open: pictureOpen(m, desc, i+1), close: m[pictureTail]} if sum += line.bytes(); sum > maxPage { return nil, 0, pageTooLarge(sum) } @@ -99,9 +97,9 @@ func planPictures(ctx context.Context, r format.Request, s picture.Settings) ([] // pictureOpen is a picture's line up to its base64. The alt text is the name // the picture has in a mail message, inline_0001.png, so a tester reading // either finds the same picture by the same name. -func pictureOpen(desc format.Descriptor, n int) string { +func pictureOpen(m *markup, desc format.Descriptor, n int) string { name := fmt.Sprintf("inline_%04d%s", n, desc.Extension) - return `

` + name + `\n", n) + end := strings.Repeat(""+nl, n) if props["structure"] != "fragment" { - end += "\n\n" + end += "" + nl + "" + nl } return end } +// htmlEmptyEnd matches the end of the smallest file of these settings: an +// empty closing paragraph, with the style it carries when styles are inline, +// and what closes after it. +func htmlEmptyEnd(props map[string]string) *regexp.Regexp { + open := `

` + if props["styles"] == "inline" { + open = `

` + } + return regexp.MustCompile(open + `

` + regexp.QuoteMeta(htmlLineEnd(props)+htmlClosing(props)) + `\z`) +} + // TestEveryHTMLSettingAnswersForItsOwnMinimum holds the arithmetic of every // path. // @@ -67,8 +95,8 @@ func TestEveryHTMLSettingAnswersForItsOwnMinimum(t *testing.T) { for _, props := range htmlCombos { floor := d.SmallestAccepted(format.Request{Seed: 7741, Label: true, Properties: props}) body := writeEncoded(t, "html", floor, props) - if want := "

\n" + htmlClosing(props); !bytes.HasSuffix(body, []byte(want)) { - t.Errorf("%v: the smallest file, %d B, does not end in an empty closing paragraph and %q, so its floor is not its own", + if want := htmlEmptyEnd(props); !want.Match(body) { + t.Errorf("%v: the smallest file, %d B, does not end in an empty closing paragraph and what closes after it (%s), so its floor is not its own", props, floor, want) } _, err := d.Generator.Plan(format.Request{Bytes: floor - 1, Seed: 7741, Label: true, Properties: props}) @@ -141,11 +169,14 @@ func TestEveryHTMLSettingIsRecordedInTheManifest(t *testing.T) { want map[string]any }{ {nil, map[string]any{"structure": "document", "nesting": 0, "characters": "ascii", "elements": "basic", "bom": false, + "formatting": "lines", "styles": "none", "line_ending": "lf", "inline": 0, "inline_format": "png", "inline_size": int64(8192), "inline_bytes": int64(0), "inline_encoded_bytes": int64(0)}}, {map[string]string{"structure": "fragment", "nesting": "7", "characters": "international", "elements": "every", "bom": "true", - "inline": "2", "inline_format": "gif", "inline_size": "1kb"}, + "formatting": "minified", "styles": "inline", "inline": "2", "inline_format": "gif", "inline_size": "1kb"}, map[string]any{"structure": "fragment", "nesting": 7, "characters": "international", "elements": "every", "bom": true, + "formatting": "minified", "styles": "inline", "line_ending": "none", "inline": 2, "inline_format": "gif", "inline_size": int64(1024), "inline_bytes": int64(2048), "inline_encoded_bytes": int64(2736)}}, + {map[string]string{"styles": "embedded"}, map[string]any{"formatting": "lines", "styles": "embedded", "line_ending": "lf"}}, } { p, err := d.Generator.Plan(format.Request{Bytes: 65536, Seed: 7741, Label: true, Properties: c.props}) if err != nil { @@ -164,7 +195,7 @@ func TestEveryHTMLSettingIsRecordedInTheManifest(t *testing.T) { // routes through the parser and have to meet. func TestTheDefaultHTMLSettingsAreTheBytesItAlwaysWrote(t *testing.T) { spoken := map[string]string{"nesting": "0", "characters": "ascii", "elements": "basic", "bom": "false", - "inline": "0", "inline_format": "png", "inline_size": "8kb"} + "formatting": "lines", "styles": "none", "inline": "0", "inline_format": "png", "inline_size": "8kb"} for _, size := range []int64{118, 119, 4096} { if !bytes.Equal(writeEncoded(t, "html", size, nil), writeEncoded(t, "html", size, spoken)) { t.Errorf("at %d B: saying nothing and saying every default produce different bytes", size) @@ -192,6 +223,12 @@ func TestTheStructuralCheckIsToldEveryHTMLSetting(t *testing.T) { {map[string]string{}, map[string]string{"inline": "1"}}, {map[string]string{"inline": "1", "inline_format": "gif", "inline_size": "1kb"}, map[string]string{"inline": "1", "inline_size": "1kb"}}, {map[string]string{"inline": "1", "inline_size": "1kb"}, map[string]string{"inline": "1", "inline_size": "2kb"}}, + {map[string]string{"formatting": "minified"}, map[string]string{}}, + {map[string]string{}, map[string]string{"formatting": "minified"}}, + {map[string]string{"styles": "embedded"}, map[string]string{}}, + {map[string]string{"styles": "inline"}, map[string]string{"styles": "embedded"}}, + {map[string]string{}, map[string]string{"styles": "inline"}}, + {map[string]string{"styles": "embedded", "structure": "fragment"}, map[string]string{"styles": "inline", "structure": "fragment"}}, } { path := filepath.Join(dir, strconv.Itoa(i)+".html") if err := os.WriteFile(path, writeEncoded(t, "html", 32768, c.made), 0o600); err != nil { diff --git a/internal/guard/htmlstyles_test.go b/internal/guard/htmlstyles_test.go new file mode 100644 index 00000000..f5a2ed6d --- /dev/null +++ b/internal/guard/htmlstyles_test.go @@ -0,0 +1,224 @@ +package guard + +import ( + "bytes" + "maps" + "regexp" + "slices" + "strings" + "testing" +) + +// A page on one line and a page with styles, both 2026-10-14 +// (docs/HTML-4-2026-10-14.md). Every guard here reads the BYTES, for the reason +// htmlfragment_test.go gives, and the names of the elements the rules are for +// are written here from the setting's meaning rather than taken from the +// generator. + +// htmlStyledElements are the elements the rules of a page are for. +var htmlStyledElements = []string{"h1", "h2", "p", "ul", "li", "table", "th", "td", "blockquote"} + +var ( + // htmlAnyTag is one tag of any kind, for taking every tag out of a text. + htmlAnyTag = regexp.MustCompile(`<[^>]*>`) + // htmlPreText is the text of a pre element as the catalogue writes it. + htmlPreText = regexp.MustCompile(`(?s)
(.*?)
`) + // htmlSheetRule is one rule of a sheet, readable or short, and + // htmlStyledStart one start tag of an element a rule is for, its + // attributes optional so a bare one is found as bare. + htmlSheetRule = regexp.MustCompile(`([a-z0-9]+) ?\{ ?([^}]*?) ?\}`) + htmlStyledStart = regexp.MustCompile(`<(` + strings.Join(htmlStyledElements, "|") + `)(\s[^>]*)?>`) +) + +// htmlTagNames is the tags of a page in order, a closing one with its slash, +// and none of their attributes. +func htmlTagNames(body []byte) []string { + var out []string + for _, m := range htmlNamedTag.FindAllSubmatch(body, -1) { + out = append(out, string(m[1])+strings.ToLower(string(m[2]))) + } + return out +} + +// htmlCatalogueOf is where the catalogue of every element sits in a page, from +// its header to the end of its footer. +func htmlCatalogueOf(t *testing.T, body []byte) (start, end int) { + t.Helper() + start, end = bytes.Index(body, []byte("
")), bytes.LastIndex(body, []byte("")) + if start < 0 || end < start { + t.Fatal("the page carries no catalogue to read") + } + return start, end + len("") +} + +// TestAMinifiedPageIsTheSamePageOnOneLine is the claim of formatting=minified. +// +// The page on one line has no line break anywhere, and it is the same page: +// the same tags in the same order as the page of lines with the same seed, as +// far as that one goes before its closing paragraph. A block on one line takes +// fewer bytes, so at the same size it holds the same blocks and more of them - +// core.FillRecords takes a block whenever it fits, and a shorter one fits +// wherever the longer one did. +func TestAMinifiedPageIsTheSamePageOnOneLine(t *testing.T) { + walked := 0 + for _, props := range htmlCombos { + if props["formatting"] != "" { + continue + } + walked++ + one := maps.Clone(props) + one["formatting"] = "minified" + lines, page := writeEncoded(t, "html", 65536, props), writeEncoded(t, "html", 65536, one) + if at := bytes.IndexAny(page, "\r\n"); at >= 0 { + t.Errorf("%v: the page of one line has a line break at byte %d", props, at) + } + if bytes.Count(lines, []byte("\n")) < 2 { + t.Errorf("%v: the page of lines is not lines, so nothing was taken out of it", props) + } + want, got := htmlTagNames(lines), htmlTagNames(page) + cut := lastPara(want) + if cut == 0 { + t.Fatalf("%v: the page of lines has no closing paragraph to compare up to", props) + } + want = want[:cut] + if len(got) < len(want) || !slices.Equal(got[:len(want)], want) { + t.Errorf("%v: the page of one line is not the page of lines - %d tags against %d, and they part at tag %d", + props, len(got), len(want), partAt(got, want)) + } + } + if walked < 8 { + t.Fatalf("only %d of the settings walked are pages of lines, so this compared almost nothing", walked) + } +} + +// lastPara is where the last paragraph opens, the closing one. +func lastPara(tags []string) int { + at := 0 + for i, tag := range tags { + if tag == "p" { + at = i + } + } + return at +} + +// partAt is the first place two lists of tags differ. +func partAt(a, b []string) int { + i := 0 + for i < len(a) && i < len(b) && a[i] == b[i] { + i++ + } + return i +} + +// TestTheCatalogueOnOneLineReadsAsItDoesOnMany holds the one place a line +// break is not just layout. +// +// Between two elements a line break shows as a space, and taken out it runs +// the text on each side together - 52 of the catalogue's 575 elements moved +// in Chromium when it was. So every tag is taken out with nothing in its place +// and the words have to be the same, and the line break inside pre, which is +// text, has to come back from its character reference. +func TestTheCatalogueOnOneLineReadsAsItDoesOnMany(t *testing.T) { + for _, shape := range []string{"document", "fragment"} { + props := map[string]string{"elements": "every", "structure": shape} + one := maps.Clone(props) + one["formatting"] = "minified" + lines, page := writeEncoded(t, "html", 32768, props), writeEncoded(t, "html", 32768, one) + a, b := htmlCatalogueOf(t, lines) + lines = lines[a:b] + a, b = htmlCatalogueOf(t, page) + page = bytes.ReplaceAll(page[a:b], []byte(" "), []byte("\n")) + + want := strings.Fields(string(htmlAnyTag.ReplaceAll(lines, nil))) + got := strings.Fields(string(htmlAnyTag.ReplaceAll(page, nil))) + if len(want) < 100 { + t.Fatalf("%s: the catalogue holds %d words, so there was hardly anything to compare", shape, len(want)) + } + if !slices.Equal(got, want) { + t.Errorf("%s: the catalogue on one line reads differently, from word %d on", shape, partAt(got, want)) + } + pre, onePre := htmlPreText.FindSubmatch(lines), htmlPreText.FindSubmatch(page) + if pre == nil || !bytes.Contains(pre[1], []byte("\n")) { + t.Fatalf("%s: the catalogue has no pre with a line break in its text, so nothing here asks about one", shape) + } + if onePre == nil || !bytes.Equal(onePre[1], pre[1]) { + t.Errorf("%s: the text of pre on one line is not the text it has on many", shape) + } + } +} + +// TestInlineStylesAreTheRulesOfTheSheet is the claim that inline and embedded +// are one look in two places. +// +// The rules are read out of the embedded sheet, and on the inline page every +// element they are for - the label, the paragraphs in other scripts, the +// pictures' paragraphs and the blocks, everything but the catalogue - carries +// exactly its rule. Each element has to be met at least once, so the guard +// cannot pass by finding none. +func TestInlineStylesAreTheRulesOfTheSheet(t *testing.T) { + for _, layout := range []string{"lines", "minified"} { + props := map[string]string{"formatting": layout, "characters": "international", "elements": "every", + "inline": "1", "inline_size": "1kb", "styles": "embedded"} + withSheet := writeEncoded(t, "html", 65536, props) + // The page's own sheet is the last one in the head - the catalogue's + // comes first. + at := bytes.LastIndex(withSheet, []byte(""))], -1) { + sheet[string(m[1])] = string(m[2]) + } + if len(sheet) != len(htmlStyledElements) { + t.Fatalf("%s: the sheet holds %d rules, and there are %d elements they are for", layout, len(sheet), len(htmlStyledElements)) + } + + props["styles"] = "inline" + page := writeEncoded(t, "html", 65536, props) + a, b := htmlCatalogueOf(t, page) + met := map[string]int{} + for _, m := range htmlStyledStart.FindAllSubmatch(append(slices.Clip(page[:a]), page[b:]...), -1) { + name, attrs := string(m[1]), string(m[2]) + if css, ok := strings.CutPrefix(attrs, ` style="`); !ok || !strings.HasPrefix(css, sheet[name]+`"`) { + t.Errorf("%s: a %s carries %q and its rule is %q", layout, name, attrs, sheet[name]) + } + met[name]++ + } + for _, name := range htmlStyledElements { + if met[name] == 0 { + t.Errorf("%s: no %s outside the catalogue, so its rule went unchecked", layout, name) + } + } + } +} + +// TestAStyleSheetSitsWhereAParserPutsItInTheHead is where embedded puts the +// rules, and that no style names anything to fetch. +// +// In the head of a page, and at the very start of a fragment - behind the mark +// when there is one - where a parser puts it into the head it makes up. Anywhere +// further down is a sheet in the body, which the Nu HTML Checker calls an error +// rule by rule (docs/HTML-4-2026-10-14.md section 1). +func TestAStyleSheetSitsWhereAParserPutsItInTheHead(t *testing.T) { + page := writeEncoded(t, "html", 16384, map[string]string{"styles": "embedded"}) + if s, h := bytes.Index(page, []byte("")), bytes.Index(page, []byte("")); s < 0 || h < s { + t.Errorf("the sheet of a page is not in its head") + } + for _, bom := range []string{"false", "true"} { + fragment := writeEncoded(t, "html", 16384, map[string]string{"styles": "embedded", "structure": "fragment", "bom": bom}) + if !bytes.HasPrefix(bytes.TrimPrefix(fragment, []byte{0xEF, 0xBB, 0xBF}), []byte("" + nl) + if end < 0: + fail("styles=embedded was ordered and the fragment has no style sheet to read past") + body = body[end + len("" + nl):] + if re.match(r"]", body): # The label sits outside the levels, so a reader that drops the deep # ones still shows which file it read. - body = body[body.index("\n") + len("\n"):] + body = body[body.index("" + nl) + len("" + nl):] + div_open, div_close = "
" + nl, "
" + nl opened = 0 - while body.startswith("
\n", opened * 6): + while body.startswith(div_open, opened * len(div_open)): opened += 1 closed, end = 0, len(body) - while end >= 7 and body[end - 7:end] == "
\n": - closed, end = closed + 1, end - 7 + while end >= len(div_close) and body[end - len(div_close):end] == div_close: + closed, end = closed + 1, end - len(div_close) if opened != want or closed != want: fail(f"nesting={want} was ordered and the content opens {opened} levels and closes {closed}") + # The paragraphs and the pictures are looked for as written, so the style + # attributes come off first. That they are there at all is html_styles' + # question, asked before this one. + bare = re.sub(r' style="[^"]*"', "", text) if settings.get("styles") == "inline" else text chars = settings.get("characters", "ascii") if chars == "international": for piece in HTML_INTERNATIONAL: - if piece not in text: + if piece not in bare: fail(f"characters=international was ordered and the page does not carry {ascii(piece)}") elif chars == "ascii": stray = sorted({c for c in text if c != "\n" and not (" " <= c <= "~")}) @@ -755,7 +767,7 @@ def html_settings(text, shape, settings): fail(f"the html check was told elements={elements!r}, which is not one this tool writes") if "script" in names: fail("the page carries a script, which no setting of this format writes") - html_pictures(text, settings) + html_pictures(bare, settings, nl) # The picture formats a page may carry and what each is called inside it - its @@ -769,8 +781,9 @@ def html_settings(text, shape, settings): } # One picture's line, as the generator writes it: the base64 on one line, # because the Nu HTML Checker calls a line break inside src an error -# (docs/HTML-3-2026-10-14.md section 1). -HTML_PICTURE = re.compile(r'

inline_([0-9]{4,5})(\.[a-z]+)

\n') +# (docs/HTML-3-2026-10-14.md section 1). What ends the line is added where it +# is known - a line break, or nothing on a page of one line. +HTML_PICTURE = r'

inline_([0-9]{4,5})(\.[a-z]+)

' def html_size(word): @@ -781,7 +794,7 @@ def html_size(word): return int(match.group(1)) * {None: 1, "b": 1, "kb": 1024, "mb": 1024 ** 2, "gb": 1024 ** 3}[match.group(2)] -def html_pictures(text, settings): +def html_pictures(text, settings, nl): """The pictures inline ordered: as many as were asked, numbered in order, each of the format and size asked and each a file its own check accepts. @@ -797,11 +810,11 @@ def html_pictures(text, settings): fail(f"the html check was told inline_format={kind!r}, which is not a picture format") size = html_size(settings.get("inline_size", "8kb")) ext, media = HTML_PICTURE_TYPES[kind] - found = HTML_PICTURE.findall(text) + found = re.findall(HTML_PICTURE + re.escape(nl), text) if len(found) != int(count): fail(f"inline={count} was ordered and the page carries {len(found)} pictures") if "base64," in text and text.count('inline_') != len(found): - fail((.*?)", text, re.DOTALL) + re.findall(r' style="([^"]*)"', text): + if HTML_FETCHING_CSS.search(css): + fail(f"the styles of the page name something to fetch: {css[:80]!r}") + + if styles == "none": + if own_sheets or attributes: + fail(f"styles=none was ordered and the page carries {own_sheets} style sheets and {attributes} style attributes") + return + if styles == "embedded": + if own_sheets != 1 or attributes: + fail(f"styles=embedded was ordered and the page carries {own_sheets} style sheets of its own and {attributes} style attributes") + at = text.rfind("", at)] + missing = [e for e in HTML_STYLED if not re.search(r"(^|[}\s>])" + e + r"\s*\{", sheet)] + if missing: + fail(f"styles=embedded was ordered and the sheet has no rule for {missing}") + if shape == "fragment" and at != 0: + fail("styles=embedded was ordered and the sheet is not where a fragment opens") + if shape == "document" and not at < text.find(""): + fail("styles=embedded was ordered and the sheet is not in the head of the page") + return + if own_sheets: + fail(f"styles=inline was ordered and the page carries {own_sheets} style sheets of its own") + outside = text + if every: + outside = text[:text.find("
")] + text[text.rfind(""):] + # The attributes are optional in the pattern, so a bare

is found and + # counted as bare - and the name has to end where they start, so p does + # not find pre nor th find thead. + tags = re.findall(r"<(" + "|".join(HTML_STYLED) + r")(\s[^>]*)?>", outside) + bare = [name for name, rest in tags if not rest.startswith(' style="')] + if not tags or bare: + fail(f"styles=inline was ordered and {len(bare)} of {len(tags)} elements it is for carry no style attribute: {bare[:6]}") + + def check_html(data, settings): """Balanced, the shape that was ordered, and with real blocks in it. @@ -883,7 +968,9 @@ def check_html(data, settings): shape = settings.get("structure", "document") if shape not in ("document", "fragment"): fail(f"the html check was told structure={shape!r}, which is not one this tool writes") - html_settings(text, shape, settings) + nl = html_layout(text, settings) + html_styles(text, shape, settings) + html_settings(text, shape, settings, nl) opens = text.lower().startswith("") closes = text.rstrip().endswith("") From 501e6585e3c5181c0608c837b6da815340bc2e96 Mon Sep 17 00:00:00 2001 From: DonislawDev Date: Wed, 14 Oct 2026 16:43:22 +0200 Subject: [PATCH 2/3] html check: refuse styles that declare nothing and a stray carriage return The structural check took "p {}" and style="" for styles in both layouts, and a guard compared an empty rule with an empty attribute and called them one look. A rule and an attribute now have to hold one declaration, a property and a value. The rules of a sheet are read with a lookbehind, because on one line the brace of the last rule is all that stands before the next. A carriage return was refused only on a page of ASCII, so one in a page of other scripts went through, and a fragment of CRLF lines ended in a traceback. A page of lines now refuses any carriage return first. A guard hands the check pages from the program broken each of those ways and asks for a verdict, and a canary gives it an honest page on one line with a sheet. Co-Authored-By: Claude Opus 5.5 --- internal/guard/htmlsettings_test.go | 4 ++ internal/guard/htmlstyles_test.go | 70 +++++++++++++++++++++++++++++ internal/oracle/strict.py | 24 ++++++++-- 3 files changed, 94 insertions(+), 4 deletions(-) diff --git a/internal/guard/htmlsettings_test.go b/internal/guard/htmlsettings_test.go index 58e1c5d1..0abc3dc4 100644 --- a/internal/guard/htmlsettings_test.go +++ b/internal/guard/htmlsettings_test.go @@ -229,6 +229,10 @@ func TestTheStructuralCheckIsToldEveryHTMLSetting(t *testing.T) { {map[string]string{"styles": "inline"}, map[string]string{"styles": "embedded"}}, {map[string]string{}, map[string]string{"styles": "inline"}}, {map[string]string{"styles": "embedded", "structure": "fragment"}, map[string]string{"styles": "inline", "structure": "fragment"}}, + // On one line a rule follows the brace of the last one with nothing + // between them - the honest half of this row is what keeps the checker + // reading every rule there (review of #179). + {map[string]string{"styles": "embedded", "formatting": "minified"}, map[string]string{"styles": "inline", "formatting": "minified"}}, } { path := filepath.Join(dir, strconv.Itoa(i)+".html") if err := os.WriteFile(path, writeEncoded(t, "html", 32768, c.made), 0o600); err != nil { diff --git a/internal/guard/htmlstyles_test.go b/internal/guard/htmlstyles_test.go index f5a2ed6d..2958a183 100644 --- a/internal/guard/htmlstyles_test.go +++ b/internal/guard/htmlstyles_test.go @@ -3,10 +3,15 @@ package guard import ( "bytes" "maps" + "os" + "path/filepath" "regexp" "slices" + "strconv" "strings" "testing" + + "github.com/donislawdev/TestingFilesGenerator/internal/oracle" ) // A page on one line and a page with styles, both 2026-10-14 @@ -28,6 +33,9 @@ var ( // attributes optional so a bare one is found as bare. htmlSheetRule = regexp.MustCompile(`([a-z0-9]+) ?\{ ?([^}]*?) ?\}`) htmlStyledStart = regexp.MustCompile(`<(` + strings.Join(htmlStyledElements, "|") + `)(\s[^>]*)?>`) + // htmlDeclares is one declaration, a property and a value - what a rule + // has to hold to style anything at all. + htmlDeclares = regexp.MustCompile(`[a-z-]+\s*:\s*[^;\s]`) ) // htmlTagNames is the tags of a page in order, a closing one with its slash, @@ -171,6 +179,14 @@ func TestInlineStylesAreTheRulesOfTheSheet(t *testing.T) { if len(sheet) != len(htmlStyledElements) { t.Fatalf("%s: the sheet holds %d rules, and there are %d elements they are for", layout, len(sheet), len(htmlStyledElements)) } + // An empty rule matches an empty attribute, so without this a sheet and + // attributes that both say nothing would pass as one look (review of + // #179). + for name, css := range sheet { + if !htmlDeclares.MatchString(css) { + t.Errorf("%s: the rule for %s declares nothing: %q", layout, name, css) + } + } props["styles"] = "inline" page := writeEncoded(t, "html", 65536, props) @@ -222,3 +238,57 @@ func TestAStyleSheetSitsWhereAParserPutsItInTheHead(t *testing.T) { } } } + +// TestTheStructuralCheckRefusesStylesThatSayNothingAndStrayCarriageReturns +// holds the two holes the review of #179 found in the checker, each with a +// verdict and not a traceback. +// +// A carriage return was refused only on a page of ASCII, so one in a page of +// other scripts went through, and a fragment of CRLF lines ended in a +// traceback. A sheet was asked for the names of its rules and an element for +// the start of its attribute, so "p {}" and style="" passed - in both layouts. +// Every page here is one the program wrote, broken one way. +func TestTheStructuralCheckRefusesStylesThatSayNothingAndStrayCarriageReturns(t *testing.T) { + type broken struct { + props map[string]string + spoil func([]byte) []byte + verdict string + } + emptyRules := regexp.MustCompile(`\{[^}]*\}`) + emptyAttrs := regexp.MustCompile(` style="[^"]*"`) + oneCR := func(b []byte) []byte { + at := bytes.Index(b, []byte("

")) + return append(append(append([]byte{}, b[:at]...), '\r'), b[at:]...) + } + cases := []broken{ + {map[string]string{"characters": "international"}, oneCR, "formatting=lines was ordered and the page has 1 carriage returns"}, + {map[string]string{"characters": "international", "structure": "fragment"}, + func(b []byte) []byte { return bytes.ReplaceAll(b, []byte("\n"), []byte("\r\n")) }, "formatting=lines was ordered and the page has"}, + } + for _, layout := range []string{"lines", "minified"} { + cases = append(cases, + broken{map[string]string{"formatting": layout, "styles": "embedded"}, + func(b []byte) []byte { return emptyRules.ReplaceAll(b, []byte("{}")) }, "styles=embedded was ordered and the sheet has no rule that declares anything"}, + broken{map[string]string{"formatting": layout, "styles": "inline"}, + func(b []byte) []byte { return emptyAttrs.ReplaceAll(b, []byte(` style=""`)) }, "styles=inline was ordered and"}) + } + dir := t.TempDir() + for i, c := range cases { + page := writeEncoded(t, "html", 16384, c.props) + bad := c.spoil(page) + if bytes.Equal(bad, page) { + t.Fatalf("%v: breaking the page changed nothing, so there is nothing to refuse", c.props) + } + path := filepath.Join(dir, strconv.Itoa(i)+".html") + if err := os.WriteFile(path, bad, 0o600); err != nil { + t.Fatal(err) + } + res := oracle.Strict("html", path, htmlSettingWords(c.props)...) + if !res.Available { + t.Skip("the structural check needs python") + } + if res.Err == nil || !strings.HasPrefix(res.Err.Error(), c.verdict) { + t.Errorf("%v: a page broken on purpose was answered with %v, and %q was the verdict owed", c.props, res.Err, c.verdict) + } + } +} diff --git a/internal/oracle/strict.py b/internal/oracle/strict.py index 0f455323..1e689107 100644 --- a/internal/oracle/strict.py +++ b/internal/oracle/strict.py @@ -866,6 +866,11 @@ def html_layout(text, settings): return "" if layout != "lines": fail(f"the html check was told formatting={layout!r}, which is not one this tool writes") + # Asked here and not left to the ASCII check, which sees a carriage return + # only on a page of ASCII - one in a page of other scripts went through, and + # a fragment of CRLF lines ended in a traceback further down (review of #179). + if "\r" in text: + fail(f"formatting=lines was ordered and the page has {text.count(chr(13))} carriage returns, where its lines end in LF") if not text.endswith("\n") or text.count("\n") < 2: fail("formatting=lines was ordered and the page is not lines ending in a line break") return "\n" @@ -877,6 +882,10 @@ def html_layout(text, settings): # when it is opened (decision D16). HTML_STYLED = ["h1", "h2", "p", "ul", "li", "table", "th", "td", "blockquote"] HTML_FETCHING_CSS = re.compile(r"url\(|@import|@font-face", re.IGNORECASE) +# One declaration, a property and a value. A rule or an attribute without one +# styles nothing - "p {}", style="" and style=";;" all name an element and say +# nothing about it (review of #179). +HTML_DECLARATION = re.compile(r"[a-z-]+\s*:\s*[^;\s]") def html_styles(text, shape, settings): @@ -904,9 +913,15 @@ def html_styles(text, shape, settings): fail(f"styles=embedded was ordered and the page carries {own_sheets} style sheets of its own and {attributes} style attributes") at = text.rfind("", at)] - missing = [e for e in HTML_STYLED if not re.search(r"(^|[}\s>])" + e + r"\s*\{", sheet)] + declared = {} + # A lookbehind and not a group: on one line the brace that ends a rule + # is all that stands before the next, and a group would eat it and + # skip every other rule. + for name, body in re.findall(r"(?"): @@ -921,9 +936,10 @@ def html_styles(text, shape, settings): # counted as bare - and the name has to end where they start, so p does # not find pre nor th find thead. tags = re.findall(r"<(" + "|".join(HTML_STYLED) + r")(\s[^>]*)?>", outside) - bare = [name for name, rest in tags if not rest.startswith(' style="')] + bare = [name for name, rest in tags + if not (rest.startswith(' style="') and HTML_DECLARATION.search(rest[len(' style="'):rest.find('"', 8)]))] if not tags or bare: - fail(f"styles=inline was ordered and {len(bare)} of {len(tags)} elements it is for carry no style attribute: {bare[:6]}") + fail(f"styles=inline was ordered and {len(bare)} of {len(tags)} elements it is for carry no style that declares anything: {bare[:6]}") def check_html(data, settings): From 24afa87346db2285e4238b45af623e02117158ed Mon Sep 17 00:00:00 2001 From: DonislawDev Date: Wed, 14 Oct 2026 16:45:23 +0200 Subject: [PATCH 3/3] site: the html settings formatting and styles in every language The formats page lists every setting of every format, and the guard that holds the site to what the program renders went red on all three systems once html had two settings more. Co-Authored-By: Claude Opus 5.5 --- web/public/ar/formats/index.html | 10 ++++++++++ web/public/cs/formaty/index.html | 10 ++++++++++ web/public/de/formate/index.html | 10 ++++++++++ web/public/es/formatos/index.html | 10 ++++++++++ web/public/formats/index.html | 10 ++++++++++ web/public/fr/formats/index.html | 10 ++++++++++ web/public/hi/formats/index.html | 10 ++++++++++ web/public/id/format/index.html | 10 ++++++++++ web/public/it/formati/index.html | 10 ++++++++++ web/public/ja/formats/index.html | 10 ++++++++++ web/public/ko/formats/index.html | 10 ++++++++++ web/public/nl/formaten/index.html | 10 ++++++++++ web/public/pl/formaty/index.html | 10 ++++++++++ web/public/pt-br/formatos/index.html | 10 ++++++++++ web/public/ro/formate/index.html | 10 ++++++++++ web/public/ru/formats/index.html | 10 ++++++++++ web/public/th/formats/index.html | 10 ++++++++++ web/public/tr/bicimler/index.html | 10 ++++++++++ web/public/uk/formats/index.html | 10 ++++++++++ web/public/vi/dinh-dang/index.html | 10 ++++++++++ web/public/zh-hans/formats/index.html | 10 ++++++++++ web/public/zh-hant/formats/index.html | 10 ++++++++++ 22 files changed, 220 insertions(+) diff --git a/web/public/ar/formats/index.html b/web/public/ar/formats/index.html index 5cf07dc9..91ddff25 100644 --- a/web/public/ar/formats/index.html +++ b/web/public/ar/formats/index.html @@ -676,6 +676,16 @@

الإعدادات التي تقبلها كل صيغة

bom صحيح أو خاطئ + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/cs/formaty/index.html b/web/public/cs/formaty/index.html index deee58d5..651c7ce9 100644 --- a/web/public/cs/formaty/index.html +++ b/web/public/cs/formaty/index.html @@ -677,6 +677,16 @@

Nastavení, která každý formát přijímá

bom pravda nebo nepravda + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/de/formate/index.html b/web/public/de/formate/index.html index f8164358..5825ad81 100644 --- a/web/public/de/formate/index.html +++ b/web/public/de/formate/index.html @@ -679,6 +679,16 @@

Einstellungen, die jedes Format kennt

bom wahr oder falsch + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/es/formatos/index.html b/web/public/es/formatos/index.html index 7f2b432a..91faaad5 100644 --- a/web/public/es/formatos/index.html +++ b/web/public/es/formatos/index.html @@ -680,6 +680,16 @@

Ajustes que admite cada formato

bom verdadero o falso + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/formats/index.html b/web/public/formats/index.html index f110ce72..1db9b6ab 100644 --- a/web/public/formats/index.html +++ b/web/public/formats/index.html @@ -678,6 +678,16 @@

Settings each format accepts

bom true or false + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/fr/formats/index.html b/web/public/fr/formats/index.html index 4aa62344..ffa77ce1 100644 --- a/web/public/fr/formats/index.html +++ b/web/public/fr/formats/index.html @@ -680,6 +680,16 @@

Réglages que chaque format accepte

bom vrai ou faux + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/hi/formats/index.html b/web/public/hi/formats/index.html index 243bde49..a77113b0 100644 --- a/web/public/hi/formats/index.html +++ b/web/public/hi/formats/index.html @@ -678,6 +678,16 @@

हर फ़ॉर्मैट की स्वीकार की जा bom सही या गलत + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/id/format/index.html b/web/public/id/format/index.html index d0b7ac63..20d21738 100644 --- a/web/public/id/format/index.html +++ b/web/public/id/format/index.html @@ -678,6 +678,16 @@

Pengaturan yang diterima setiap format

bom benar atau salah + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/it/formati/index.html b/web/public/it/formati/index.html index 74c69ad8..55573f73 100644 --- a/web/public/it/formati/index.html +++ b/web/public/it/formati/index.html @@ -680,6 +680,16 @@

Impostazioni che ogni formato accetta

bom vero o falso + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/ja/formats/index.html b/web/public/ja/formats/index.html index 540680fd..ec0c2bd7 100644 --- a/web/public/ja/formats/index.html +++ b/web/public/ja/formats/index.html @@ -670,6 +670,16 @@

形式ごとに受け付ける設定

bom 真または偽 + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/ko/formats/index.html b/web/public/ko/formats/index.html index bd578940..ba69d828 100644 --- a/web/public/ko/formats/index.html +++ b/web/public/ko/formats/index.html @@ -671,6 +671,16 @@

각 형식이 받는 설정

bom 참 또는 거짓 + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/nl/formaten/index.html b/web/public/nl/formaten/index.html index 5e0076f1..862ac708 100644 --- a/web/public/nl/formaten/index.html +++ b/web/public/nl/formaten/index.html @@ -680,6 +680,16 @@

Instellingen die elk formaat accepteert

bom waar of onwaar + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/pl/formaty/index.html b/web/public/pl/formaty/index.html index a2615714..f5d08d56 100644 --- a/web/public/pl/formaty/index.html +++ b/web/public/pl/formaty/index.html @@ -678,6 +678,16 @@

Ustawienia, które przyjmuje każdy format

bom prawda albo fałsz + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/pt-br/formatos/index.html b/web/public/pt-br/formatos/index.html index 0af97539..6fad45d4 100644 --- a/web/public/pt-br/formatos/index.html +++ b/web/public/pt-br/formatos/index.html @@ -679,6 +679,16 @@

Configurações que cada formato aceita

bom verdadeiro ou falso + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/ro/formate/index.html b/web/public/ro/formate/index.html index dd595dbe..dfd44c71 100644 --- a/web/public/ro/formate/index.html +++ b/web/public/ro/formate/index.html @@ -679,6 +679,16 @@

Setările pe care le acceptă fiecare format

bom adevărat sau fals + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/ru/formats/index.html b/web/public/ru/formats/index.html index 77c31f3c..581a1b2e 100644 --- a/web/public/ru/formats/index.html +++ b/web/public/ru/formats/index.html @@ -678,6 +678,16 @@

Настройки, которые принимает каждый форм bom истина или ложь + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/th/formats/index.html b/web/public/th/formats/index.html index abf61438..70a91baf 100644 --- a/web/public/th/formats/index.html +++ b/web/public/th/formats/index.html @@ -676,6 +676,16 @@

การตั้งค่าที่แต่ละรูปแบบร bom จริงหรือเท็จ + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/tr/bicimler/index.html b/web/public/tr/bicimler/index.html index 861d3ca2..65c3609a 100644 --- a/web/public/tr/bicimler/index.html +++ b/web/public/tr/bicimler/index.html @@ -678,6 +678,16 @@

Her biçimin kabul ettiği ayarlar

bom doğru veya yanlış + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/uk/formats/index.html b/web/public/uk/formats/index.html index 352fe8b9..b355a8aa 100644 --- a/web/public/uk/formats/index.html +++ b/web/public/uk/formats/index.html @@ -678,6 +678,16 @@

Налаштування, які приймає кожен формат

bom істина або хибність + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/vi/dinh-dang/index.html b/web/public/vi/dinh-dang/index.html index ee8e5b30..2879ac8b 100644 --- a/web/public/vi/dinh-dang/index.html +++ b/web/public/vi/dinh-dang/index.html @@ -676,6 +676,16 @@

Các thiết lập mỗi định dạng nhận

bom đúng hoặc sai + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/zh-hans/formats/index.html b/web/public/zh-hans/formats/index.html index 34752965..952a0956 100644 --- a/web/public/zh-hans/formats/index.html +++ b/web/public/zh-hans/formats/index.html @@ -670,6 +670,16 @@

每种格式接受的设置

bom 真或假 + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline diff --git a/web/public/zh-hant/formats/index.html b/web/public/zh-hant/formats/index.html index 89629c61..f39e4907 100644 --- a/web/public/zh-hant/formats/index.html +++ b/web/public/zh-hant/formats/index.html @@ -670,6 +670,16 @@

每種格式接受的設定

bom 真或假 + + + formatting + lines, minified + + + + styles + embedded, inline, none + inline