Repository navigation
fix(document): parse text responses with consistent decoding - #761
Merged
Merged
Conversation
lanyue-llk
force-pushed
the
codex/pr640-reopen-engineering
branch
from
September 22, 2026 12:44
a4b6756 to
cec1eda
Compare
lanyue-llk
force-pushed
the
codex/pr640-reopen-text
branch
from
September 22, 2026 12:44
449e21c to
15df7ed
Compare
This was referenced Sep 22, 2026
Sequential Navigation Soak A/B✅ HEAD completed the 200-navigation resilience and memory observation. One browser process, one target, and one CDP session navigate CSDN → SegmentFault → Huaban → example.com repeatedly. This run issued 200 Common ancestor Session resilience
A public-page failure is reported separately from an unrecoverable session. The soak requires all navigation attempts, zero failed recovery, zero lifecycle/network ordering violations, and complete resource evidence. Process-tree memory
HEAD memory by 50-navigation quarter
Raw reports and per-navigation boundary samples: workflow run and full Public-site timing and memory are observational. Use the A/B deltas and trend shape as evidence, not as a deterministic performance threshold. |
Spider Bench A/B✅ All benchmark browser service runs completed; results are informational. Public HEAD: 189 / 240 rows; 39 / 48 sites produced rows; 9 unexpected empty sites. ✅ Deterministic fixture contract is clean. Common ancestor Public 48-site run · informational
Public-site content, timing, and memory are noisy. This single A/B run reports evidence only; site outcome counts explain missing rows without treating them as a deterministic regression. CPU and memory timelinesThese are bounded 40-point views of the same complete process-tree samples used by the HTML report. CPU uses 100% per occupied logical core; memory lines are RSS first and PSS second when complete PSS samples are available. Basexychart-beta
title "Base CPU"
x-axis "Elapsed seconds" [1.543, 2.544, 3.045, 4.548, 5.55, 7.052, 8.554, 10.055, 11.558, 13.06, 14.061, 15.563, 17.068, 18.571, 20.075, 21.575, 22.576, 24.079, 25.582, 27.085, 28.086, 28.587, 30.089, 31.091, 32.593, 34.095, 35.597, 37.098, 38.601, 39.602, 41.104, 42.606, 44.109, 44.611, 45.614, 47.116, 48.117, 49.619]
y-axis "CPU percent" 0 --> 200
line [131.88, 97.87, 123.84, 101.81, 169.63, 151.59, 87.86, 167.8, 79.94, 91.73, 183.84, 39.98, 27.94, 21.97, 43.78, 4, 39.94, 101.87, 107.74, 131.71, 105.88, 105.87, 127.8, 151.72, 91.95, 123.85, 127.89, 35.97, 101.93, 7.98, 149.58, 73.91, 15.99, 0, 85.75, 137.61, 21.99, 105.88]
xychart-beta
title "Base memory: RSS then PSS"
x-axis "Elapsed seconds" [0.042, 1.543, 2.544, 3.045, 4.548, 5.55, 7.052, 8.554, 10.055, 11.558, 13.06, 14.061, 15.563, 17.068, 18.571, 20.075, 21.575, 22.576, 24.079, 25.582, 27.085, 28.086, 28.587, 30.089, 31.091, 32.593, 34.095, 35.597, 37.098, 38.601, 39.602, 41.104, 42.606, 44.109, 44.611, 45.614, 47.116, 48.117, 49.619, 50.91]
y-axis "Memory MiB" 0 --> 500
line [11.89, 106.79, 115.53, 121.28, 139.5, 173.64, 186.18, 161.77, 168.95, 175.14, 163.53, 154.34, 156.86, 139.03, 148.06, 139.49, 145.36, 144.71, 140.39, 157.83, 170.48, 276.29, 207.42, 205.07, 172.16, 164.73, 164, 142.74, 143.55, 145.76, 141.27, 157.59, 152.77, 137.57, 137.57, 151.85, 151.37, 146.79, 166.69, 0]
line [8.21, 102.75, 111.01, 117.25, 135.86, 170.03, 181.24, 157.62, 164.83, 171.01, 159.43, 150.2, 152.76, 134.9, 143.99, 135.38, 141.21, 140.55, 136.31, 153.71, 166.54, 272.23, 202.81, 201.01, 168.65, 159.8, 159.93, 138.63, 139.46, 141.7, 137.2, 153.13, 148.7, 133.51, 133.52, 147.8, 148.04, 142.83, 162.78, 0]
HEADxychart-beta
title "HEAD CPU"
x-axis "Elapsed seconds" [1.536, 3.039, 4.541, 5.042, 6.043, 6.543, 8.045, 9.548, 11.052, 13.054, 14.556, 16.059, 17.559, 19.06, 21.064, 22.566, 24.066, 25.568, 27.069, 29.072, 30.574, 32.075, 33.577, 35.078, 36.581, 38.584, 40.085, 41.588, 43.089, 44.59, 45.592, 46.593, 48.095, 49.597, 51.099, 52.602, 54.605, 56.106]
y-axis "CPU percent" 0 --> 500
line [123.9, 117.81, 153.67, 211.6, 147.98, 181.88, 91.83, 49.9, 159.68, 163.71, 83.86, 117.75, 24, 51.89, 59.91, 117.88, 137.94, 105.87, 151.73, 137.96, 125.67, 93.94, 93.76, 111.93, 21.97, 123.71, 47.9, 51.9, 21.98, 27.94, 0, 2, 21.95, 69.88, 13.98, 127.76, 73.79, 127.78]
xychart-beta
title "HEAD memory: RSS then PSS"
x-axis "Elapsed seconds" [0.035, 1.536, 3.039, 4.541, 5.042, 6.043, 6.543, 8.045, 9.548, 11.052, 13.054, 14.556, 16.059, 17.559, 19.06, 21.064, 22.566, 24.066, 25.568, 27.069, 29.072, 30.574, 32.075, 33.577, 35.078, 36.581, 38.584, 40.085, 41.588, 43.089, 44.59, 45.592, 46.593, 48.095, 49.597, 51.099, 52.602, 54.605, 56.106, 57.478]
y-axis "Memory MiB" 0 --> 500
line [12.18, 107.98, 124.44, 146.7, 150.03, 185.79, 162.72, 173.78, 157.16, 167.68, 169.1, 167.21, 146.12, 156, 144.35, 135, 172.03, 158.88, 267.69, 205.01, 159.61, 167.77, 191.41, 145.59, 152.04, 151.82, 247.5, 236.48, 241.56, 241.11, 245.77, 242.41, 241.45, 241.4, 186.65, 173.25, 156.77, 147.23, 167.82, 0]
line [8.87, 104.39, 121.21, 143.06, 146.48, 182.18, 159.12, 170.69, 153.73, 164.28, 165.25, 163.08, 142.57, 152.55, 140.9, 131.54, 168.59, 155.71, 264.38, 201.58, 156.55, 164.35, 188, 142.29, 148.64, 148.39, 244.04, 233.05, 238, 237.7, 242.35, 238.99, 238.02, 237.98, 183.23, 169.83, 154.17, 143.81, 164.42, 0]
Deterministic fixture · required contractThe fixture exercises 15 routes: 8 are expected to emit rows and 7 intentionally exercise empty, loading, or timeout behavior.
Full HTML, JSON, CSV, logs, and page snapshots: workflow run and full The public-web A/B is informational. The exact HEAD fixture contract runs as a separate required CI check; timing and public-site content deltas remain non-blocking. |
CI Regression ReportSource CI run · source state at render:
Release regression — ✅ HEAD/base failures 0/0; raw binary −0.014607%Package and image size
Startup latency and PSS
Concurrency matrix
Frontend differential — ✅ 1,020/1,020 cases matched; 0 issues
Agent episodes · Moli vs Chromium — ✅ 8/8 Moli episodes passed; 0 failures
Runtime and CDP session contracts — ✅ 26 contract cases; 0 failures
CDP smoke — ✅ 50/50 groups passed; 574 scenariosWorkers: 4 · cumulative group time: 132.51 s · failed groups: 0
WebMainBench · 545 pages — ✅ 545/545 completed; 0 unexpected failures
Frozen HTML · Linux · Diagnostics: All artifact fields are parsed by the trusted default-branch renderer; missing or invalid inputs remain visible as unavailable. |
lanyue-llk
force-pushed
the
codex/pr640-reopen-engineering
branch
from
September 24, 2026 04:21
19f651f to
7649f13
Compare
lanyue-llk
force-pushed
the
codex/pr640-reopen-text
branch
from
September 24, 2026 04:23
15df7ed to
f4fcb91
Compare
lanyue-llk
force-pushed
the
codex/pr640-reopen-engineering
branch
from
September 24, 2026 05:10
7649f13 to
6185ba5
Compare
lanyue-llk
force-pushed
the
codex/pr640-reopen-text
branch
from
September 24, 2026 05:10
f4fcb91 to
877778e
Compare
ldm0
force-pushed
the
codex/pr640-reopen-engineering
branch
from
September 28, 2026 20:51
008b31d to
2009cd7
Compare
ldm0
force-pushed
the
codex/pr640-reopen-text
branch
from
September 29, 2026 00:13
877778e to
116d497
Compare
Keep valid inherited text encodings ahead of heuristic detection while preserving BOM, transport charset, and JSON default precedence. Share the response decoder between streaming main documents, asynchronous child loads, data URLs, and blocking child snapshots. Classify navigation XML MIME types independently of DOMParser's allowlist, including +xml subtypes in parser routing and text exclusions. Add regressions for inherited UTF-8, literal encoding declarations, XML namespaces, and DOMParser rejection of navigation-only XML types. Validation: - cargo fmt --all - cargo clippy --workspace --all-targets --all-features -- -D warnings - NEXTEST_TEST_THREADS=8 cargo nextest run --no-fail-fast 18,952 passed, 16 skipped. Two existing timing tests failed under the default concurrency, then passed individually and in the full run above.
Keep network XML response bytes intact through the existing raw-body renderer path. Decode both network and external XML bodies with the shared response decoder, then install the selected encoding before document bootstrap. Remove the obsolete ready-body capture variant used by the old XML path. Cover matching main-document and iframe text, namespaces, and characterSet for Atom, RSS, and text/*+xml responses with Windows-1252 headers, XML declarations, and UTF-16LE/BE BOMs. Exercise the external input as single-byte chunks and verify that BOMs override a conflicting transport charset. Validation: - Before the fix, the network regression returned caf� / UTF-8 instead of café / windows-1252. - cargo fmt --all - cargo clippy --workspace --all-targets --all-features -- -D warnings - Six focused XML tests passed. - NEXTEST_TEST_THREADS=8 cargo nextest run --no-fail-fast: 18,954 passed, 16 skipped.
Give document decoding explicit HTML, XML, and text declaration policies. XML ignores HTML meta elements, retains BOM and transport charset precedence, and selects an initial XML declaration or UTF-16 signature before its UTF-8 default. Keep partial declarations and signatures buffered across chunks. Cover conflicting meta declarations with and without an XML declaration, including main documents, iframes, and external raw input. Adapt seven Chromium XML/XHTML navigation scenarios and three decoder boundary tests from Chromium a03603fe9af6230a12f1b2fb2c18a7d003a0d937; source paths are recorded next to the tests. These include KOI8-R script inheritance, CP1251 XML, BOM-less UTF-16, supplementary Unicode characters, and BOM boundaries. Validation: - cargo fmt --all - cargo clippy --workspace --all-targets --all-features -- -D warnings - XML and Chromium regression tests passed. - NEXTEST_TEST_THREADS=8 cargo nextest run --no-fail-fast: 18,980 passed, 16 skipped.
ldm0
force-pushed
the
codex/pr640-reopen-text
branch
from
September 29, 2026 08:12
c5a0275 to
5039308
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
JSON 等文本响应被当作 HTML 解析，生成错误的节点结构，并可能执行脚本或触发资源预加载。编码、流式分块及主/子文档的不同处理路径又可能改变最终文本。本 PR 在文档构造边界统一 MIME 分类、解码和文本解析语义。
一图看懂
图示说明问题、修复机制与行为边界；点击可查看原图。验收归属不等于当前提交通过结果，版本和测试证据见下文。
行为与机制
声明 MIME 优先于 URL 后缀；文本内容按字面文本进入文档，不先转义包装再解析。decoder/parser/预加载策略使用一致模式，分块边界不改变结果。文本文档不生成 doctype，显式保持标准模式；保留 main 已有的纯文本子文档语义。
代码阅读路线
moli-web-mimeclassification/sniffing 和moli-encoding/src/document.rs：响应类型及编码/BOM。moli-parser/src/html.rs、session/stream 与 renderer 的document_response_decoder.rs、live_document_parser：流式文本、换行与 DOM 构造。moli-protocol/src/conn/runtime_load.rs：主/子文档统一入口与原始字节保留。Case 与回归契约
负责原第 11 类
784。测试应覆盖 JSON/text MIME、BOM 与编码、跨块换行、NUL、实体、字面的关闭 pre 内容、MIME/URL 后缀冲突、主文档/iframe 一致性。伪 HTML 不能执行脚本或请求图片；正常 HTML 必须有会执行脚本的正对照，防止通过禁用全部执行来通过文本测试。历史受测
ca2a8d39572644de1211e3aba30d239abf0ffb17的 784：10 个冻结动作全部完成、1/1、零重试。逐题审计。独立 HTTP + CDP 主/子文档 12 场景通过，浏览器场景结果、新增解析场景红绿证据。这些结果对应所列版本，不改标为最终 head 的复跑。已合入；最后 head
50393080f266afb383fab210ebeb6a9a5ed2b863，base main，最终提交门禁见 Checks。来源 #740。旧报告中 #755 的分支依赖是历史状态。Reviewer 重点
MIME 决策、decoder、parser 和预加载模式是否一致？分块、iframe 和字面伪 HTML 是否与主文档共享语义？正常 HTML/XML 等其他类型是否保持自身解析行为？
整组验收与证据入口
这组修复来自 #640，对应 1.1.9 的 34 个失败 case，加上 1.1.8 已失败的 386，共 35 个唯一 case。按最终职责分配为：#756 布局 27、#757 输入 4、#758 网络 3、#761 文本 1；#755、#759、#760 以各自独立功能契约验收，不重复认领这 35 个 case。