Skip to content

fix(reliability): harden diagnostics and test infrastructure - #755

Merged
ldm0 merged 5 commits into
mainfrom
codex/pr640-reopen-engineering
Sep 28, 2026
Merged

ldm0 merged 5 commits into
mainfrom
codex/pr640-reopen-engineering

Conversation

@lanyue-llk

@lanyue-llk lanyue-llk commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

启动器关闭 stderr 后,记录页面脚本错误可能触发日志组件对损坏管道的二次写入,导致浏览器中止。本 PR 将诊断输出失败限制在日志边界,同时修正 CI 发布归档处理及跨平台测试夹具;正常错误日志与原有功能断言仍须保留。

一图看懂

PR #755 · 日志故障隔离与可靠验证

图示说明问题、修复机制与行为边界;点击可查看原图。验收归属不等于当前提交通过结果,版本和测试证据见下文。

修复机制

使用 tracing 标准配置处理诊断写入失败,不吞掉页面脚本错误。字体相关布局夹具采用确定输入,SQLite 路径夹具分别验证平台可表达的真实路径语义;网络夹具等待真实 socket 接入,避免调度竞态。CI 解包前检查下载和归档有效性,不能把下载错误页当作发布包。

代码阅读路线

  • moli/src/telemetry.rs 与 moli/tests/fetch_cli.rs:日志配置和真实子进程,关闭 stderr 的失败路径与正常日志正对照。
  • .github/scripts/unpack-ci-release.sh / .test.cjs、相关 action:下载失败、无效归档及成功解包的行为。
  • moli-fetch/src/tests/support.rs、layout phase4/phase5 测试、renderer 的 network/file_and_xhr.rs:固定环境不等于删除语义断言;重点审查 socket 时序、字体几何及路径处理。
  • .github/workflows/cancel-pr-ci.yml:CI 任务取消边界,避免影响应继续工作的运行。

验收

本 PR 不认领历史 35 个功能 case。验收使用真实进程与独立夹具契约:关闭 stderr 后继续完成页面,正常 stderr 仍输出脚本错误;归档脚本需有坏输入与正常输入;路径和网络用例仍验证真实数据。

历史 22e161c8(base de1093d9c)的 Rust 门禁:18,416 通过、0 失败、13 既有跳过;CI 脚本 13/13。真实 release 对照为旧版关闭 stderr 后 SIGABRT,修复版 exit 0 且内容完整。stderr 红绿证据、Rust 门禁、脚本用例。这些数字对应所列历史版本。

PR 已合入;最后 head 7ab3bb8ba3345b75936c8d82138bbff75ea57a22,base main,最终门禁见 Checks。原修复来源 #734。该工程前提已进入 main,剩余 PR 不再以 #755 分支作为直接 base。

Reviewer 重点

损坏 stderr 是否仅影响诊断输出?正常脚本错误是否仍可观察?夹具是否仍验证相同功能,CI 是否只取消正确范围内的过期运行?

整组验收与证据入口

这组修复来自 #640,对应 1.1.9 的 34 个失败 case,加上 1.1.8 已失败的 386,共 35 个唯一 case。按最终职责分配为:#756 布局 27、#757 输入 4、#758 网络 3、#761 文本 1;#755、#759、#760 以各自独立功能契约验收,不重复认领这 35 个 case。

@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

CI Regression Report

Source CI run · source state at render: in_progress · artifacts: 6/6

Check Status Signal
Release regression ✅ HEAD/base failures 0/0; raw binary 0.00%
Frontend differential ✅ 1,020/1,020 cases matched; 0 issues
Agent episodes ✅ 8/8 Moli episodes passed; 0 failures
Runtime/CDP contracts ✅ 26 contract cases; 0 failures
CDP smoke ✅ 50/50 groups passed; 574 scenarios
WebMainBench ✅ 545/545 completed; 0 unexpected failures
Release regression — ✅ HEAD/base failures 0/0; raw binary 0.00%

Package and image size

Metric Base HEAD Delta Delta %
Raw binary 146.59 MiB 146.59 MiB 0 B 0.00%
Stripped binary 100.57 MiB 100.57 MiB 0 B 0.00%
gzip binary 50.17 MiB 50.17 MiB −105 B −0.000200%
Rootfs 153.49 MiB 153.49 MiB 0 B 0.00%
gzip rootfs 58.95 MiB 58.95 MiB +151 B +0.000244%

Startup latency and PSS

Case Base p50 HEAD p50 Delta Base PSS HEAD PSS Delta
Serve ready 28.73 ms 4.89 ms −23.84 ms 3.82 MiB 8.37 MiB +4.55 MiB
CDP first page 58.26 ms 61.05 ms +2.79 ms 3.93 MiB 6.84 MiB +2.91 MiB
CDP warm page 17.57 ms 19.20 ms +1.63 ms 48.93 MiB 48.60 MiB −339.50 KiB
CLI about:blank 18.40 ms 21.39 ms +2.99 ms 5.08 MiB 3.73 MiB −1.35 MiB
CLI local JS 25.79 ms 20.10 ms −5.70 ms 3.20 MiB 2.92 MiB −286.50 KiB

Concurrency matrix

Concurrency Base p50 HEAD p50 Delta Base PSS HEAD PSS Delta HEAD failures Unstable B/H
1 17.62 ms 20.53 ms +2.91 ms 467.00 KiB 471.00 KiB +4.00 KiB 0 0/0
5 19.51 ms 24.74 ms +5.22 ms 27.01 MiB 13.35 MiB −13.66 MiB 0 0/0
10 20.54 ms 34.41 ms +13.88 ms 19.95 MiB 11.14 MiB −8.81 MiB 0 0/0
25 20.30 ms 31.63 ms +11.33 ms 25.39 MiB 8.66 MiB −16.73 MiB 0 0/0
100 23.58 ms 31.61 ms +8.03 ms 11.74 MiB 10.04 MiB −1.71 MiB 0 0/0
Frontend differential — ✅ 1,020/1,020 cases matched; 0 issues
Match DOM mismatch Diagnostic mismatch Moli error Reference error Infrastructure error Mismatched frames Duration
1,020 0 0 0 0 0 0 7m 36.51s
Agent episodes · Moli vs Chromium — ✅ 8/8 Moli episodes passed; 0 failures
Fast contract Episodes Passed Assertions Ready p50 Episode p50
Moli 8 8 47/47 29.82 ms 33.62 ms
Chromium 8 8 47/47 348.25 ms 119.86 ms
Operation p50 Chromium Moli Delta Delta %
navigate 70.72 ms 7.85 ms −62.87 ms −88.90%
observe 2.00 ms 1.98 ms −0.02 ms −0.83%
fill 6.05 ms 4.16 ms −1.89 ms −31.20%
click 7.58 ms 6.89 ms −0.69 ms −9.14%
Idle-resource episode Peak RSS Peak PSS Average CPU Peak CPU Peak processes Sampler
Moli 64.14 MiB 61.49 MiB 0.70% 9.99% 1 ✅
Chromium 1283.10 MiB 331.48 MiB 1.96% 92.04% 13 ✅
Runtime and CDP session contracts — ✅ 26 contract cases; 0 failures
Suite Cases Failures Median case p50 Median case p95 Median peak PSS p50
Synthetic fetch 13 0 24.85 ms 29.19 ms 536.00 KiB
Long-lived CDP session 13 0 21.57 ms 23.58 ms —
CDP smoke — ✅ 50/50 groups passed; 574 scenarios

Workers: 4 · cumulative group time: 168.79 s · failed groups: 0

Slowest group Scenarios Duration Status
target-lifecycle 24 18.74 s ✅
dom-input 27 14.26 s ✅
multi-page 45 13.00 s ✅
chromium-cdp 56 11.15 s ✅
multi-context 17 8.32 s ✅
WebMainBench · 545 pages — ✅ 545/545 completed; 0 unexpected failures
Completed Successful Expected DNS failure Unexpected failures Panics Crashes Timeouts Empty output
545/545 544 1 0 0 0 0 0

Frozen HTML · Linux · --wait done · 45 seconds per page · no retries. Content quality is evaluated separately.

Diagnostics: webmainbench-results — every page's Markdown, stderr, and exit status. Source run and artifacts

All artifact fields are parsed by the trusted default-branch renderer; missing or invalid inputs remain visible as unavailable.

Comment thread .github/actions/use-ci-release/action.yml
@lanyue-llk lanyue-llk closed this Sep 23, 2026
@lanyue-llk lanyue-llk reopened this Sep 23, 2026
@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Spider Bench A/B

✅ All benchmark browser service runs completed; results are informational.

Public HEAD: 184 / 240 rows; 38 / 48 sites produced rows; 10 unexpected empty sites.

✅ Deterministic fixture contract is clean.

Common ancestor 82e4533ae1a8 → HEAD 7ab3bb8ba334; benchmark order: head-first; workflow: success.

Public 48-site run · informational

Metric Base HEAD Δ
Run status success success —
Extracted rows / contract 184 / 240 184 / 240 0 (0.00%)
Contract fill 76.67% 76.67% 0.00 pp (0.00%)
Sites with rows 38 / 48 38 / 48 0 (0.00%)
Sites meeting row contract 36 / 48 36 / 48 0 (0.00%)
Unexpected empty sites 10 10 0 (0.00%)
Partial-row sites 2 2 0 (0.00%)
Site outcomes extracted: 37; HTTP error + rows: 1; empty: 7; HTTP error + empty: 1; snapshot error + empty: 2 extracted: 36; HTTP error + rows: 2; empty: 7; HTTP error + empty: 1; snapshot error + empty: 2 —
Observed duration 42.28 s 50.03 s +7.75 s (+18.32%)
Estimated CPU 43.85 s 45.98 s +2.13 s (+4.85%)
Peak PSS 270.96 MiB 269.20 MiB -1.76 MiB (-0.65%)
Peak RSS 275.05 MiB 272.68 MiB -2.37 MiB (-0.86%)
Peak threads 26 27 +1 (+3.85%)
Mismatched sites 0 0 0
Timeout with items 0 0 0
Category rows Base HEAD Δ
news 35 / 40 35 / 40 0
stocks 20 / 40 20 / 40 0
tech 35 / 40 35 / 40 0
sports 27 / 40 27 / 40 0
games 37 / 40 37 / 40 0
life 30 / 40 30 / 40 0

Public-site content, timing, and memory are noisy. This single A/B run reports evidence only; site outcome counts explain missing rows without treating them as a deterministic regression.

CPU and memory timelines

These are bounded 40-point views of the same complete process-tree samples used by the HTML report. CPU uses 100% per occupied logical core; memory lines are RSS first and PSS second when complete PSS samples are available.

Base

xychart-beta
    title "Base CPU"
    x-axis "Elapsed seconds" [1.035, 2.036, 2.536, 3.537, 4.538, 5.54, 6.04, 6.54, 7.04, 8.54, 9.542, 10.544, 12.044, 13.044, 14.045, 15.548, 16.549, 17.551, 19.054, 20.056, 21.559, 22.56, 23.561, 25.064, 26.065, 27.067, 28.569, 29.569, 30.572, 32.073, 33.074, 34.076, 35.577, 36.579, 38.081, 39.081, 40.082, 41.584]
    y-axis "CPU percent" 0 --> 200
    line [2, 132, 95.92, 101.91, 149.9, 129.83, 188, 128.01, 141.97, 179.93, 111.8, 153.9, 108.01, 129.92, 97.94, 117.79, 59.96, 35.93, 121.75, 41.91, 141.7, 105.97, 109.76, 143.64, 107.9, 143.86, 129.83, 97.97, 77.81, 7.98, 93.93, 115.78, 17.98, 1.99, 123.81, 135.77, 103.93, 127.85]
Loading
xychart-beta
    title "Base memory: RSS then PSS"
    x-axis "Elapsed seconds" [0.035, 1.035, 2.036, 2.536, 3.537, 4.538, 5.54, 6.04, 6.54, 7.04, 8.54, 9.542, 10.544, 12.044, 13.044, 14.045, 15.548, 16.549, 17.551, 19.054, 20.056, 21.559, 22.56, 23.561, 25.064, 26.065, 27.067, 28.569, 29.569, 30.572, 32.073, 33.074, 34.076, 35.577, 36.579, 38.081, 39.081, 40.082, 41.584, 42.281]
    y-axis "Memory MiB" 0 --> 500
    line [12.07, 60.8, 94.81, 107.5, 126.46, 142.8, 158.82, 143.22, 158.89, 176.88, 157.5, 186.79, 171.37, 172.46, 169.16, 174.69, 157.6, 151.98, 153.58, 157.21, 155.26, 163.84, 275.05, 196.03, 163.44, 162.48, 175, 187.61, 141.59, 143.32, 143.67, 170.04, 167.06, 158.2, 151.23, 196.93, 153.93, 157.95, 188.95, 0]
    line [8.73, 57.12, 91.12, 103.81, 122.81, 139.07, 155.09, 139.52, 155.18, 173.23, 153.91, 183.19, 167.98, 169.16, 165.87, 171.04, 153.97, 148.33, 149.93, 153.56, 151.62, 162.31, 270.96, 192.44, 159.84, 158.84, 171.48, 183.97, 138.01, 139.77, 139.86, 166.4, 163.31, 154.48, 147.52, 193.28, 150.28, 154.38, 185.41, 0]
Loading

HEAD

xychart-beta
    title "HEAD CPU"
    x-axis "Elapsed seconds" [1.535, 2.537, 3.037, 4.039, 5.04, 5.541, 7.042, 8.544, 9.546, 11.049, 12.549, 14.05, 15.553, 16.554, 18.057, 19.559, 21.066, 22.067, 23.569, 25.07, 26.574, 28.078, 29.081, 30.584, 32.087, 33.589, 34.591, 36.169, 37.717, 39.221, 40.723, 41.725, 42.725, 43.226, 44.727, 46.229, 47.229, 48.729]
    y-axis "CPU percent" 0 --> 500
    line [115.98, 97.74, 135.86, 127.91, 171.8, 169.66, 172.03, 85.86, 195.57, 123.78, 61.99, 133.76, 2, 2, 117.9, 5.99, 3.99, 27.96, 67.88, 144.07, 111.13, 103.8, 108.05, 93.77, 97.93, 127.51, 97.8, 17.97, 15.97, 139.67, 79.91, 2, 0, 3.99, 114, 121.72, 30, 127.93]
Loading
xychart-beta
    title "HEAD memory: RSS then PSS"
    x-axis "Elapsed seconds" [0.035, 1.535, 2.537, 3.037, 4.039, 5.04, 5.541, 7.042, 8.544, 9.546, 11.049, 12.549, 14.05, 15.553, 16.554, 18.057, 19.559, 21.066, 22.067, 23.569, 25.07, 26.574, 28.078, 29.081, 30.584, 32.087, 33.589, 34.591, 36.169, 37.717, 39.221, 40.723, 41.725, 42.725, 43.226, 44.727, 46.229, 47.229, 48.729, 50.027]
    y-axis "Memory MiB" 0 --> 500
    line [12.07, 92.41, 114.57, 121.37, 132.53, 164.53, 154.56, 158.7, 168.32, 191.53, 163.52, 164.27, 168.71, 137.18, 142.11, 154.84, 148.11, 140.85, 138.74, 149.63, 160.08, 272.68, 198.11, 156.23, 157.38, 187.65, 144.68, 148.15, 147.25, 148.54, 189.65, 164.85, 147.31, 140.31, 138.8, 152.97, 160.59, 154.76, 183.23, 0]
    line [8.79, 88.77, 110.93, 117.75, 128.89, 161.09, 150.91, 155.06, 164.79, 188.01, 159.88, 160.62, 165.33, 133.5, 138.43, 151.16, 144.42, 137.17, 135.04, 145.86, 156.39, 269.2, 194.37, 152.41, 153.26, 183.68, 140.4, 144.33, 143.55, 144.82, 185.35, 161.22, 143.63, 136.65, 135.08, 149.25, 156.87, 151.04, 179.51, 0]
Loading

Deterministic fixture · required contract

The fixture exercises 15 routes: 8 are expected to emit rows and 7 intentionally exercise empty, loading, or timeout behavior.

Metric Base HEAD Δ
Run status success success —
Extracted rows / contract 40 / 40 40 / 40 0 (0.00%)
Contract fill 100.00% 100.00% 0.00 pp (0.00%)
Sites with rows 8 / 15 8 / 15 0 (0.00%)
Sites meeting row contract 15 / 15 15 / 15 0 (0.00%)
Unexpected empty sites 0 0 0
Partial-row sites 0 0 0
Site outcomes extracted: 7; HTTP error + rows: 1; empty: 3; timeout + empty: 4 extracted: 7; HTTP error + rows: 1; empty: 3; timeout + empty: 4 —
Observed duration 19.88 s 19.90 s +0.02 s (+0.10%)
Estimated CPU 0.42 s 0.48 s +0.06 s (+14.25%)
Peak PSS 57.95 MiB 57.98 MiB +0.03 MiB (+0.06%)
Peak RSS 61.41 MiB 61.40 MiB -0.01 MiB (-0.02%)
Peak threads 18 19 +1 (+5.56%)
Mismatched sites 0 0 0
Timeout with items 0 0 0

Full HTML, JSON, CSV, logs, and page snapshots: workflow run and full spider-bench-results artifact.

The public-web A/B is informational. The exact HEAD fixture contract runs as a separate required CI check; timing and public-site content deltas remain non-blocking.

@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Sequential Navigation Soak A/B

✅ HEAD completed the 200-navigation resilience and memory observation.

One browser process, one target, and one CDP session navigate CSDN → SegmentFault → Huaban → example.com repeatedly. This run issued 200 Page.navigate commands per binary.

Common ancestor 82e4533ae1 → HEAD 7ab3bb8ba3; benchmark order: head-first; workflow: success.

Session resilience

Metric Base HEAD
Attempted / planned 200 / 200 200 / 200
Direct observable pass 199 / 200 199 / 200
Failures / recovered 1 / 1 1 / 1
Recovery failures 0 0
Lifecycle order violations 0 0
Network order violations 0 0
Wall time 321.3 s 299.5 s

A public-page failure is reported separately from an unrecoverable session. The soak requires all navigation attempts, zero failed recovery, zero lifecycle/network ordering violations, and complete resource evidence.

Process-tree memory

Metric Base HEAD Δ
Boundary samples 200 200 0
Boundary RSS samples 200 200 0
Boundary PSS samples 200 200 0
Periodic peak RSS 219.1 MiB 199.4 MiB -19.7 MiB
Periodic peak PSS 216.2 MiB 196.3 MiB -20.0 MiB
RSS · first 10 avg 166.4 MiB 145.3 MiB -21.1 MiB
RSS · last 10 avg 130.6 MiB 142.0 MiB +11.4 MiB
RSS · last − first window -35.8 MiB -3.4 MiB +32.4 MiB
RSS · warm slope / 100 nav -13.5 MiB +1.8 MiB +15.3 MiB
PSS · first 10 avg 163.7 MiB 142.2 MiB -21.5 MiB
PSS · last 10 avg 128.2 MiB 139.7 MiB +11.5 MiB
PSS · last − first window -35.6 MiB -2.5 MiB +33.1 MiB
PSS · warm slope / 100 nav -13.4 MiB +2.1 MiB +15.5 MiB
Peak file descriptors 63 62 -1
Peak threads 31 31 0

HEAD memory by 50-navigation quarter

HEAD range Avg RSS Final RSS Peak RSS Avg PSS Final PSS Peak FD
1–50 141.7 MiB 129.3 MiB 199.5 MiB 139.2 MiB 127.1 MiB 52
51–100 139.9 MiB 116.5 MiB 182.3 MiB 136.1 MiB 112.6 MiB 54
101–150 142.5 MiB 133.9 MiB 165.4 MiB 139.3 MiB 131.1 MiB 52
151–200 142.7 MiB 128.2 MiB 176.0 MiB 140.1 MiB 126.0 MiB 52

Raw reports and per-navigation boundary samples: workflow run and full sequential-navigation-soak-results artifact.

Public-site timing and memory are observational. Use the A/B deltas and trend shape as evidence, not as a deterministic performance threshold.

@lanyue-llk
lanyue-llk force-pushed the codex/pr640-reopen-engineering branch 2 times, most recently from 7649f13 to 6185ba5 Compare September 24, 2026 05:10
@ldm0
ldm0 force-pushed the codex/pr640-reopen-engineering branch from 008b31d to 2009cd7 Compare September 28, 2026 20:51
@ldm0
ldm0 merged commit 3e2b9d0 into main Sep 28, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants