Repository navigation
docs: lead README benchmarks with LexBench WebFetch - #1609
lanyue-llk wants to merge 1 commit into
Conversation
286d531 to
8ca9e66
Compare
Spider Bench A/B✅ All benchmark browser service runs completed; results are informational. Public HEAD: 184 / 240 rows; 38 / 48 sites produced rows; 10 unexpected empty sites. ✅ Deterministic fixture contract is clean. Common ancestor Public 48-site run · informational
Public-site content, timing, and memory are noisy. This single A/B run reports evidence only; site outcome counts explain missing rows without treating them as a deterministic regression. CPU and memory timelinesThese are bounded 40-point views of the same complete process-tree samples used by the HTML report. CPU uses 100% per occupied logical core; memory lines are RSS first and PSS second when complete PSS samples are available. Basexychart-beta
title "Base CPU"
x-axis "Elapsed seconds" [1.537, 3.539, 5.041, 5.541, 6.542, 8.045, 10.048, 11.55, 13.052, 15.054, 16.055, 16.556, 18.058, 19.56, 21.563, 23.065, 24.567, 25.569, 26.57, 28.073, 29.576, 31.078, 33.081, 34.583, 36.083, 38.085, 39.587, 41.089, 42.589, 44.593, 46.093, 47.593, 49.593, 51.096, 52.596, 54.099, 56.102, 57.103]
y-axis "CPU percent" 0 --> 500
line [113.71, 135.94, 159.91, 199.98, 151.75, 115.81, 135.83, 141.79, 149.83, 2, 0, 123.9, 153.83, 2, 133.96, 115.85, 139.87, 107.75, 141.96, 107.75, 145.81, 129.81, 139.8, 109.89, 28.01, 3.99, 121.86, 115.87, 8, 5.99, 0, 6, 7.99, 0, 126.02, 167.73, 109.74, 147.95]
xychart-beta
title "Base memory: RSS"
x-axis "Elapsed seconds" [0.036, 1.537, 3.539, 5.041, 5.541, 6.542, 8.045, 10.048, 11.55, 13.052, 15.054, 16.055, 16.556, 18.058, 19.56, 21.563, 23.065, 24.567, 25.569, 26.57, 28.073, 29.576, 31.078, 33.081, 34.583, 36.083, 38.085, 39.587, 41.089, 42.589, 44.593, 46.093, 47.593, 49.593, 51.096, 52.596, 54.099, 56.102, 57.103, 57.115]
y-axis "Memory MiB" 0 --> 500
line [12.66, 103.22, 126.41, 158.79, 174.37, 190.03, 198.51, 190, 176.86, 177.07, 155.6, 155.35, 162.09, 177.79, 148.23, 183.14, 198.02, 165.76, 290.05, 204.61, 226.2, 159.3, 185.58, 172.97, 166.62, 152.78, 154.7, 242.25, 226.24, 181.96, 179.95, 179.96, 179.95, 179.95, 179.95, 170.43, 195.16, 191.57, 0, 0]
HEADxychart-beta
title "HEAD CPU"
x-axis "Elapsed seconds" [1.535, 2.538, 3.039, 4.04, 5.543, 7.044, 8.547, 10.048, 11.049, 12.55, 14.053, 15.556, 17.057, 17.558, 18.058, 19.56, 21.063, 22.564, 24.067, 25.568, 26.57, 28.071, 29.573, 31.075, 32.578, 33.579, 35.08, 36.584, 38.084, 39.587, 40.587, 42.089, 43.592, 45.093, 46.596, 47.097, 47.598, 49.099]
y-axis "CPU percent" 0 --> 500
line [113.96, 111.7, 163.73, 113.89, 183.57, 153.94, 121.99, 173.8, 139.76, 109.85, 103.8, 71.87, 5.99, 0, 4, 101.89, 17.96, 2, 5.99, 61.89, 69.92, 115.88, 103.81, 99.97, 97.97, 129.81, 71.85, 79.85, 75.93, 7.99, 144.03, 91.85, 7.99, 79.89, 111.75, 209.58, 27.95, 111.96]
xychart-beta
title "HEAD memory: RSS then PSS"
x-axis "Elapsed seconds" [0.035, 1.535, 2.538, 3.039, 4.04, 5.543, 7.044, 8.547, 10.048, 11.049, 12.55, 14.053, 15.556, 17.057, 17.558, 18.058, 19.56, 21.063, 22.564, 24.067, 25.568, 26.57, 28.071, 29.573, 31.075, 32.578, 33.579, 35.08, 36.584, 38.084, 39.587, 40.587, 42.089, 43.592, 45.093, 46.596, 47.097, 47.598, 49.099, 50.236]
y-axis "Memory MiB" 0 --> 500
line [12.65, 114.92, 123.57, 137.32, 142.02, 152.49, 190.02, 159.36, 187.13, 170.34, 200.52, 173.02, 166.87, 153.19, 153.47, 152.91, 166.61, 150.9, 150.64, 149.59, 165.59, 174.74, 174.11, 287.3, 211.3, 159.82, 163.07, 193.56, 154.12, 168.17, 151.09, 195.28, 173.58, 151.01, 165.36, 184.88, 168.92, 165.88, 167.46, 0]
line [9.24, 111.21, 119.87, 133.44, 138.34, 149.13, 186.47, 155.79, 183.58, 166.77, 196.95, 169.46, 163.26, 149.58, 149.87, 149.3, 163.14, 147.3, 147.04, 145.99, 162.22, 171.16, 170.35, 283.83, 207.7, 156.28, 159.36, 189.89, 150.69, 164.68, 147.5, 186.54, 170, 147.42, 161.74, 181.34, 165.82, 162.21, 163.78, 0]
Deterministic fixture · required contractThe fixture exercises 15 routes: 8 are expected to emit rows and 7 intentionally exercise empty, loading, or timeout behavior.
Full HTML, JSON, CSV, logs, and page snapshots: workflow run and full The public-web A/B is informational. The exact HEAD fixture contract runs as a separate required CI check; timing and public-site content deltas remain non-blocking. |
CI Regression ReportSource CI run · source state at render:
Release regression — ✅ HEAD/base failures 0/0; raw binary 0.00%Package and image size
Startup latency and PSS
Concurrency matrix
Frontend differential — ✅ 1,020/1,020 cases matched; 0 issues
Agent episodes · Moli vs Chromium — ✅ 8/8 Moli episodes passed; 0 failures
Runtime and CDP session contracts — ✅ 26 contract cases; 0 failures
CDP smoke — ✅ 52/52 groups passed; 621 scenariosWorkers: 4 · cumulative group time: 196.63 s · failed groups: 0
WebMainBench · 545 pages — ✅ 545/545 completed; 0 unexpected failures
Frozen HTML · Linux · Diagnostics: All artifact fields are parsed by the trusted default-branch renderer; missing or invalid inputs remain visible as unavailable. |
Sequential Navigation Soak A/B✅ HEAD completed the 200-navigation resilience and memory observation. One browser process, one target, and one CDP session navigate CSDN → SegmentFault → Huaban → example.com repeatedly. This run issued 200 Common ancestor Session resilience
A public-page failure is reported separately from an unrecoverable session. The soak requires all navigation attempts, zero failed recovery, zero lifecycle/network ordering violations, and complete resource evidence. Process-tree memory
HEAD memory by 50-navigation quarter
Raw reports and per-navigation boundary samples: workflow run and full Public-site timing and memory are observational. Use the A/B deltas and trend shape as evidence, not as a deterministic performance threshold. |
Problem
The README leads with the historical 192-URL crawl sample. The reviewed LexBench WebFetch comparison is missing from the project overview.
Change
Replace that sample with the public five-browser, 430-task table. Show one Moli result, task counts, pass rates and the full report link. The native Markdown disclosure opens by default. English, Chinese, German, Spanish, French, Japanese and Korean use the same measurements.
This PR targets
maindirectly and changes only seven README files. It does not include #1380's WebMainBench table, benchmark reorganization or runtime changes. Other existing benchmark sections remain unchanged.Reproduce and validate
Open this branch's README at the benchmark section. The LexBench WebFetch disclosure should already be open, with five rows and exactly one Moli row: 385/430 (89.53%). The report link opens the complete task-level results. Click the disclosure or focus its summary and press Enter to close or reopen it.
All seven editions pass GitHub Markdown rendering checks for the same values, five rows, one Moli result and the default open state. No WebMainBench table or internal build history remains in this change. The result counts are copied from the published report; no evaluation calls or measured evidence were changed.
git diff --checkpasses.Report: LexBench WebFetch.