Skip to content

fix(html2md): retain complex tables and referenced fragments - #1336

Open
lanyue-llk wants to merge 3 commits into
codex/content-math-formattingfrom
codex/content-tables-fragments
Open

lanyue-llk wants to merge 3 commits into
codex/content-math-formattingfrom
codex/content-tables-fragments

Conversation

@lanyue-llk

@lanyue-llk lanyue-llk commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

问题与方案

Markdown 管道表格无法表达跨行、跨列和嵌套表格。原转换把这些结构展开成文字,丢失单元格之间的归属关系。文内链接也可能保留 #fragment 却丢掉目标;代码中的链接和特殊标题会出现类似损失。

简单表格继续使用 Markdown;复杂表格保留经过筛选的内容 HTML,保留行列、标题、合并单元格和嵌套关系。HTML 路径复用媒体、表单与公式转换,文字保持转义。仅保留实际被引用的片段目标,并遵循 ID 优先和精确匹配后再解码的规则。代码内的链接及被引用目标使用 HTML 保留语义。

根因、方案与验收

问题 根因 方案依据 泛化验收
合并和嵌套单元格失去归属 管道表格无法表达跨度和嵌套,展平又抹掉结构 可完整表示的简单表格用 Markdown,其余保留内容 HTML rowspan/colspan、日历空单元格、分组关系、嵌套表格、表头及展示性布局表
HTML 后备路径与普通转换漂移 后备转换再次独立解释媒体、表单和公式 复用 #1334、#1335 的共享规则,仅在标记和转义阶段分流 同一内容放入普通、合并、嵌套表格;资源安全、当前 textarea 值、checkbox 与 MathML
片段链接保留了地址,却丢失目标 转换删除目标身份;过度解码或把普通 name 当锚点 保留实际引用的 ID 和传统 <a name>,精确匹配优先于解码,ID 优先 编码片段、重复身份、传统命名锚点、普通元素 name 的反向控制、代码与 MathML 内的目标
代码链接或特殊标题丢失 一律围栏或只选取一个后代元素 逐节点保留语义;含结构的代码和标题采用内容 HTML 可导航的 pre、代码内术语链接、嵌套及空标题、块内容;深树的小栈及复制量边界

修复机制图

复杂表格与文内链接:问题、修复路径与验证场景

查看原图

flowchart LR
  A[表格结构] --> B{管道表格能完整表示?}
  B -->|能| C[Markdown 表格]
  B -->|不能| D[内容 HTML 表格]
  E[媒体、表单与公式规则] --> D
  F[文内链接] --> G[匹配实际目标并保留锚点]
  G --> D
  G --> C
Loading

范围与依赖

这是 #919 的表格与片段链接部分,直接基于 #1335;#1335 已包含 #1334 的媒体与表单规则。复杂表格复用两组转换,本 PR 的 diff 只包含表格与片段增量。DOM 接口增加默认的布局边界查询,#1337 的运行时适配提供真实 CSS 状态。

验证

  • cargo fmt --all
  • cargo test -p moli-html2md --locked:全部通过。
  • cargo clippy -p moli-html2md --all-targets --locked -- -D warnings:通过。
  • 红绿验证:合并单元格的日历和分组归属在旧转换中丢失,本修改保留关系。
  • 原生 DOM 导出:22 个场景通过。嵌套布局表格检查正文、元数据顺序及导出前后 DOM 不变;该验收随表格转换一起提交,不依赖后续 fix(renderer): export live rendered content through shared markdown conversion #1337。
  • CLI 导出:4 个场景通过,涵盖布局表格、数据表格和无表头表格,以及开启和关闭布局的调用。CLI 验收与本组表格转换一起提交。

测试覆盖合并及嵌套表格、媒体与控件在两种表格路径中的一致性、片段编码及目标优先级、代码内链接、嵌套标题、深层结构的小线程栈与复制量边界。已有测试保留原排列,减少无关 diff。完整仓库测试交由 CI。

与 #1305 的关系

#1305 直接基于 main,负责执行调用者明确指定的脚本,等待 Promise 和导航后走标准导出路径。这四组修复负责导出内容的完整性与语义。双方没有代码前置依赖;组合后,交互后的 DOM 会使用这些共享转换规则。

默认只读采集和明确交互采集是不同的测量协议。报告中的历史成绩仍绑定原调用协议;不能给单一方法补交互后替换同一行成绩。

报告

完整 WebMainBench 报告 保留 #919 已发布的运行。其源码为 721bd2e329f96c06573db6e6b257a149cf17a6fe,该测量不对应本拆分分支。

PR stack

GitHub 原生依赖链:#1334 → #1335 → #1336 → #1337 → #1380 → #1428 → #1431 → #1432。

每个 PR 的 GitHub base 都是左侧紧邻分支,因此 Files changed 只显示本层增量。按从左到右的顺序合入。#919 是保留的历史完整组合,不是这条链的依赖;#1429 独立面向 main,只提供通用 CI 评测入口。

@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Spider Bench A/B

✅ All benchmark browser service runs completed; results are informational.

Public HEAD: 189 / 240 rows; 39 / 48 sites produced rows; 9 unexpected empty sites.

✅ Deterministic fixture contract is clean.

Common ancestor cd0193d92881 → HEAD 59c3c4d39cd2; benchmark order: base-first; workflow: success.

Public 48-site run · informational

Metric Base HEAD Δ
Run status success success —
Extracted rows / contract 184 / 240 189 / 240 +5 (+2.72%)
Contract fill 76.67% 78.75% +2.08 pp (+2.71%)
Sites with rows 38 / 48 39 / 48 +1 (+2.63%)
Sites meeting row contract 36 / 48 37 / 48 +1 (+2.78%)
Unexpected empty sites 10 9 -1 (-10.00%)
Partial-row sites 2 2 0 (0.00%)
Site outcomes extracted: 36; HTTP error + rows: 2; empty: 7; HTTP error + empty: 1; snapshot error + empty: 2 extracted: 38; HTTP error + rows: 1; empty: 6; HTTP error + empty: 1; snapshot error + empty: 2 —
Observed duration 45.93 s 48.35 s +2.42 s (+5.27%)
Estimated CPU 47.24 s 49.67 s +2.43 s (+5.15%)
Peak PSS 302.46 MiB 272.43 MiB -30.03 MiB (-9.93%)
Peak RSS 305.21 MiB 276.30 MiB -28.91 MiB (-9.47%)
Peak threads 26 27 +1 (+3.85%)
Mismatched sites 0 0 0
Timeout with items 0 0 0
Category rows Base HEAD Δ
news 35 / 40 35 / 40 0
stocks 20 / 40 20 / 40 0
tech 35 / 40 35 / 40 0
sports 27 / 40 32 / 40 +5
games 37 / 40 37 / 40 0
life 30 / 40 30 / 40 0

Public-site content, timing, and memory are noisy. This single A/B run reports evidence only; site outcome counts explain missing rows without treating them as a deterministic regression.

CPU and memory timelines

These are bounded 40-point views of the same complete process-tree samples used by the HTML report. CPU uses 100% per occupied logical core; memory lines are RSS first and PSS second when complete PSS samples are available.

Base

xychart-beta
    title "Base CPU"
    x-axis "Elapsed seconds" [1.037, 1.537, 2.539, 3.54, 4.041, 4.542, 5.042, 6.545, 7.546, 9.049, 10.05, 11.552, 13.052, 14.054, 15.555, 16.556, 18.058, 19.059, 20.562, 21.563, 23.065, 24.568, 25.569, 27.071, 28.073, 29.574, 30.575, 32.077, 33.078, 34.58, 36.083, 37.083, 38.585, 39.587, 41.089, 42.09, 43.594, 44.595]
    y-axis "CPU percent" 0 --> 500
    line [0, 41.98, 141.77, 163.84, 23.95, 75.81, 129.86, 195.86, 153.78, 133.62, 125.82, 141.79, 85.96, 123.88, 49.94, 87.9, 69.94, 49.96, 113.71, 107.87, 141.87, 111.86, 107.77, 145.84, 117.82, 129.84, 181.78, 67.91, 97.84, 37.96, 139.81, 80.01, 7.99, 15.98, 97.86, 109.83, 53.9, 137.78]
Loading
xychart-beta
    title "Base memory: RSS then PSS"
    x-axis "Elapsed seconds" [0.036, 1.037, 1.537, 2.539, 3.54, 4.041, 4.542, 5.042, 6.545, 7.546, 9.049, 10.05, 11.552, 13.052, 14.054, 15.555, 16.556, 18.058, 19.059, 20.562, 21.563, 23.065, 24.568, 25.569, 27.071, 28.073, 29.574, 30.575, 32.077, 33.078, 34.58, 36.083, 37.083, 38.585, 39.587, 41.089, 42.09, 43.594, 44.595, 45.928]
    y-axis "Memory MiB" 0 --> 500
    line [12.32, 54.82, 91.12, 139.43, 131.66, 131.79, 117.38, 134.92, 149.55, 181.32, 166.42, 186.55, 151.64, 182.89, 187.97, 170.78, 162.13, 162.2, 145.5, 184.18, 196.32, 164.78, 305.21, 203.31, 147.51, 138.91, 202.23, 168.68, 154.14, 160.18, 148.68, 201.75, 178.75, 148.69, 147.8, 164.77, 174.95, 150.77, 172.9, 0]
    line [9.16, 51.21, 87.55, 135.84, 128.37, 128.22, 113.85, 131.35, 146.14, 177.64, 162.43, 183.26, 148.16, 178.87, 184.37, 167.28, 158.63, 158.61, 141.78, 180.79, 192.96, 161.08, 302.46, 199.68, 143.98, 135.29, 198.85, 165.05, 150.34, 156.41, 145.04, 197.98, 174.92, 144.99, 144.1, 160.87, 171.25, 147.31, 169.32, 0]
Loading

HEAD

xychart-beta
    title "HEAD CPU"
    x-axis "Elapsed seconds" [1.037, 1.538, 2.539, 4.04, 5.543, 6.544, 8.046, 9.047, 9.548, 11.049, 12.051, 13.553, 15.056, 16.058, 17.561, 19.063, 20.064, 21.565, 23.067, 24.569, 25.57, 27.071, 27.572, 28.573, 29.575, 31.076, 32.578, 33.578, 35.08, 36.582, 37.583, 39.086, 40.586, 42.086, 43.086, 44.588, 46.091, 47.093]
    y-axis "CPU percent" 0 --> 500
    line [6, 115.74, 118, 110.01, 147.76, 161.91, 69.89, 193.72, 153.82, 141.89, 177.65, 141.64, 25.96, 0, 5.98, 105.92, 131.83, 55.93, 129.85, 135.77, 109.85, 99.86, 115.86, 99.95, 111.89, 113.87, 171.93, 147.97, 105.86, 105.85, 17.98, 117.69, 84.07, 8, 68, 117.95, 127.77, 119.76]
Loading
xychart-beta
    title "HEAD memory: RSS then PSS"
    x-axis "Elapsed seconds" [0.037, 1.037, 1.538, 2.539, 4.04, 5.543, 6.544, 8.046, 9.047, 9.548, 11.049, 12.051, 13.553, 15.056, 16.058, 17.561, 19.063, 20.064, 21.565, 23.067, 24.569, 25.57, 27.071, 27.572, 28.573, 29.575, 31.076, 32.578, 33.578, 35.08, 36.582, 37.583, 39.086, 40.586, 42.086, 43.086, 44.588, 46.091, 47.093, 48.349]
    y-axis "Memory MiB" 0 --> 500
    line [12.32, 63.98, 110.93, 127.37, 140.24, 166.98, 175.62, 190.64, 185.97, 173.47, 185.2, 185.14, 161.71, 149.64, 146.86, 147.68, 165.72, 172.92, 149.62, 157.93, 196.68, 176.81, 270.14, 276.3, 211.57, 233.25, 155.63, 202.76, 170.07, 157.17, 153.41, 155.16, 232.31, 271.44, 169.23, 181.21, 195.05, 162.99, 178.66, 0]
    line [9.24, 60.54, 107.5, 124.17, 136.84, 163.63, 172.25, 187.23, 183.09, 170.06, 181.76, 181.72, 158.72, 146.35, 143.43, 144.25, 162.4, 169.49, 145.4, 154.57, 193.31, 173.12, 266.77, 272.43, 208.11, 229.68, 152.25, 199.32, 166.55, 153.64, 149.88, 151.65, 228.86, 267.94, 165.71, 177.7, 191.52, 159.07, 175.22, 0]
Loading

Deterministic fixture · required contract

The fixture exercises 15 routes: 8 are expected to emit rows and 7 intentionally exercise empty, loading, or timeout behavior.

Metric Base HEAD Δ
Run status success success —
Extracted rows / contract 40 / 40 40 / 40 0 (0.00%)
Contract fill 100.00% 100.00% 0.00 pp (0.00%)
Sites with rows 8 / 15 8 / 15 0 (0.00%)
Sites meeting row contract 15 / 15 15 / 15 0 (0.00%)
Unexpected empty sites 0 0 0
Partial-row sites 0 0 0
Site outcomes extracted: 7; HTTP error + rows: 1; empty: 3; timeout + empty: 4 extracted: 7; HTTP error + rows: 1; empty: 3; timeout + empty: 4 —
Observed duration 19.87 s 19.86 s -0.01 s (-0.06%)
Estimated CPU 0.44 s 0.44 s 0.00 s (0.00%)
Peak PSS 65.75 MiB 66.19 MiB +0.44 MiB (+0.67%)
Peak RSS 69.48 MiB 69.75 MiB +0.28 MiB (+0.40%)
Peak threads 18 18 0 (0.00%)
Mismatched sites 0 0 0
Timeout with items 0 0 0

Full HTML, JSON, CSV, logs, and page snapshots: workflow run and full spider-bench-results artifact.

The public-web A/B is informational. The exact HEAD fixture contract runs as a separate required CI check; timing and public-site content deltas remain non-blocking.

@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

CI Regression Report

Source CI run · source state at render: in_progress · artifacts: 6/6

Check Status Signal
Release regression ✅ HEAD/base failures 0/0; raw binary +0.019646%
Frontend differential ✅ 1,020/1,020 cases matched; 0 issues
Agent episodes ✅ 8/8 Moli episodes passed; 0 failures
Runtime/CDP contracts ✅ 26 contract cases; 0 failures
CDP smoke ✅ 52/52 groups passed; 621 scenarios
WebMainBench ✅ 545/545 completed; 0 unexpected failures
Release regression — ✅ HEAD/base failures 0/0; raw binary +0.019646%

Package and image size

Metric Base HEAD Delta Delta %
Raw binary 148.85 MiB 148.88 MiB +29.95 KiB +0.019646%
Stripped binary 102.50 MiB 102.53 MiB +28.00 KiB +0.026677%
gzip binary 51.07 MiB 51.08 MiB +10.02 KiB +0.019167%
Rootfs 155.75 MiB 155.78 MiB +30.00 KiB +0.018810%
gzip rootfs 59.96 MiB 59.97 MiB +18.43 KiB +0.030025%

Startup latency and PSS

Case Base p50 HEAD p50 Delta Base PSS HEAD PSS Delta
Serve ready 27.22 ms 27.94 ms +0.72 ms 4.68 MiB 4.32 MiB −370.00 KiB
CDP first page 50.75 ms 52.14 ms +1.40 ms 4.55 MiB 4.24 MiB −310.50 KiB
CDP warm page 12.01 ms 12.81 ms +0.81 ms 49.46 MiB 49.39 MiB −67.00 KiB
CLI about:blank 14.28 ms 16.12 ms +1.84 ms 4.05 MiB 4.08 MiB +30.50 KiB
CLI local JS 18.33 ms 19.30 ms +0.98 ms 4.12 MiB 4.12 MiB −2.50 KiB

Concurrency matrix

Concurrency Base p50 HEAD p50 Delta Base PSS HEAD PSS Delta HEAD failures Unstable B/H
1 18.85 ms 19.79 ms +0.94 ms 805.00 KiB 794.00 KiB −11.00 KiB 0 0/0
5 23.09 ms 22.54 ms −0.54 ms 22.88 MiB 27.78 MiB +4.91 MiB 0 0/0
10 22.88 ms 22.39 ms −0.49 ms 27.26 MiB 13.33 MiB −13.93 MiB 0 0/0
25 24.16 ms 23.33 ms −0.84 ms 38.29 MiB 17.50 MiB −20.79 MiB 0 0/0
100 24.35 ms 21.78 ms −2.57 ms 18.09 MiB 23.16 MiB +5.08 MiB 0 0/0
Frontend differential — ✅ 1,020/1,020 cases matched; 0 issues
Match DOM mismatch Diagnostic mismatch Moli error Reference error Infrastructure error Mismatched frames Duration
1,020 0 0 0 0 0 0 6m 49.47s
Agent episodes · Moli vs Chromium — ✅ 8/8 Moli episodes passed; 0 failures
Fast contract Episodes Passed Assertions Ready p50 Episode p50
Moli 8 8 47/47 4.26 ms 24.96 ms
Chromium 8 8 47/47 308.31 ms 96.12 ms
Operation p50 Chromium Moli Delta Delta %
navigate 46.09 ms 5.73 ms −40.36 ms −87.57%
observe 0.85 ms 0.91 ms +0.06 ms +6.64%
fill 4.45 ms 3.52 ms −0.93 ms −20.97%
click 5.16 ms 4.17 ms −0.99 ms −19.21%
Idle-resource episode Peak RSS Peak PSS Average CPU Peak CPU Peak processes Sampler
Moli 65.61 MiB 63.80 MiB 0.79% 10.00% 1 ✅
Chromium 1308.32 MiB 429.66 MiB 2.67% 80.58% 13 ⚠️ PSS partial
Runtime and CDP session contracts — ✅ 26 contract cases; 0 failures
Suite Cases Failures Median case p50 Median case p95 Median peak PSS p50
Synthetic fetch 13 0 18.46 ms 19.32 ms 614.00 KiB
Long-lived CDP session 13 0 17.91 ms 18.95 ms —
CDP smoke — ✅ 52/52 groups passed; 621 scenarios

Workers: 4 · cumulative group time: 162.66 s · failed groups: 0

Slowest group Scenarios Duration Status
scrollbar-visibility 22 21.45 s ✅
target-lifecycle 24 12.79 s ✅
chromium-cdp 56 11.05 s ✅
dom-input 27 10.66 s ✅
multi-page 45 8.86 s ✅
WebMainBench · 545 pages — ✅ 545/545 completed; 0 unexpected failures
Completed Successful Expected DNS failure Unexpected failures Panics Crashes Timeouts Empty output
545/545 544 1 0 0 0 0 0

Frozen HTML · Linux · --wait done · 45 seconds per page · no retries. Content quality is evaluated separately.

Diagnostics: webmainbench-results — every page's Markdown, stderr, and exit status. Source run and artifacts

All artifact fields are parsed by the trusted default-branch renderer; missing or invalid inputs remain visible as unavailable.

@lanyue-llk
lanyue-llk force-pushed the codex/content-tables-fragments branch from fd2db36 to 6ed184d Compare October 9, 2026 11:44
@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Sequential Navigation Soak A/B

✅ HEAD completed the 200-navigation resilience and memory observation.

One browser process, one target, and one CDP session navigate CSDN → SegmentFault → Huaban → example.com repeatedly. This run issued 200 Page.navigate commands per binary.

Common ancestor cd0193d928 → HEAD 59c3c4d39c; benchmark order: base-first; workflow: success.

Session resilience

Metric Base HEAD
Attempted / planned 159 / 200 200 / 200
Direct observable pass 157 / 159 200 / 200
Failures / recovered 2 / 1 0 / 0
Recovery failures 1 0
Lifecycle order violations 0 0
Network order violations 0 0
Wall time 557.9 s 530.6 s

A public-page failure is reported separately from an unrecoverable session. The soak requires all navigation attempts, zero failed recovery, zero lifecycle/network ordering violations, and complete resource evidence.

Process-tree memory

Metric Base HEAD Δ
Boundary samples 159 200 41
Boundary RSS samples 159 200 41
Boundary PSS samples 159 200 41
Periodic peak RSS 259.4 MiB 260.7 MiB +1.3 MiB
Periodic peak PSS 255.4 MiB 258.0 MiB +2.6 MiB
RSS · first 10 avg 178.2 MiB 178.0 MiB -0.2 MiB
RSS · last 10 avg 146.5 MiB 161.7 MiB +15.2 MiB
RSS · last − first window -31.7 MiB -16.3 MiB +15.4 MiB
RSS · warm slope / 100 nav -20.6 MiB -17.1 MiB +3.4 MiB
PSS · first 10 avg 175.8 MiB 174.2 MiB -1.6 MiB
PSS · last 10 avg 143.2 MiB 157.9 MiB +14.6 MiB
PSS · last − first window -32.5 MiB -16.3 MiB +16.2 MiB
PSS · warm slope / 100 nav -19.8 MiB -17.9 MiB +1.9 MiB
Peak file descriptors 58 59 1
Peak threads 31 31 0

HEAD memory by 50-navigation quarter

HEAD range Avg RSS Final RSS Peak RSS Avg PSS Final PSS Peak FD
1–50 178.6 MiB 154.2 MiB 233.6 MiB 175.3 MiB 152.1 MiB 54
51–100 175.2 MiB 107.0 MiB 234.9 MiB 172.3 MiB 103.1 MiB 53
101–150 156.2 MiB 136.3 MiB 209.4 MiB 152.6 MiB 132.4 MiB 54
151–200 159.1 MiB 148.3 MiB 211.7 MiB 155.2 MiB 144.4 MiB 53

Raw reports and per-navigation boundary samples: workflow run and full sequential-navigation-soak-results artifact.

Public-site timing and memory are observational. Use the A/B deltas and trend shape as evidence, not as a deterministic performance threshold.

@lanyue-llk
lanyue-llk changed the base branch from codex/content-math-formatting to codex/content-conversion-prerequisites October 9, 2026 12:19
@lanyue-llk
lanyue-llk force-pushed the codex/content-tables-fragments branch from 6ed184d to 22760a4 Compare October 9, 2026 12:19
@lanyue-llk
lanyue-llk force-pushed the codex/content-tables-fragments branch from 13a8b62 to da9b680 Compare October 9, 2026 13:27
@lanyue-llk
lanyue-llk added this pull request to stack #1381 October 10, 2026 08:55
@lanyue-llk
lanyue-llk removed this pull request from stack #1381 October 10, 2026 09:01
@lanyue-llk
lanyue-llk force-pushed the codex/content-tables-fragments branch from ba33592 to a863725 Compare October 10, 2026 09:04
@lanyue-llk
lanyue-llk changed the base branch from codex/content-conversion-prerequisites to codex/content-math-formatting October 10, 2026 09:04
@lanyue-llk
lanyue-llk added this pull request to stack #1382 October 10, 2026 09:05
@lanyue-llk
lanyue-llk force-pushed the codex/content-tables-fragments branch from a863725 to eb8ccd8 Compare October 12, 2026 03:01
@lanyue-llk

Copy link
Copy Markdown
Contributor Author

Closing after the consolidation audit. This belongs to an earlier experimental Markdown stack and was not part of the measured fixed build. Current failure evidence does not justify carrying its broad table and fragment expansion into #1501, so it is intentionally not merged.

@lanyue-llk

Copy link
Copy Markdown
Contributor Author

Reopened after correcting the consolidation scope: this is an independent pre-existing stack and should not have been closed as part of #1501 cleanup. Its original scope and base relationship are preserved; this does not add it to #1501.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant