Complete WebArena-Verified Chromium and Moli trajectory replay

.agents/skills/benchmark-editorial/SKILL.md

@@ -7,7 +7,7 @@ description: Write or revise standalone articles interpreting this project's ben
 
 Produce a reader-facing article whose conclusions, examples and background remain understandable without opening the source report. The benchmark supplies evidence; the article explains what the results mean and why they matter.
 
-Keep this skill and its dependencies in the repository. Read [editorial-standard.md](references/editorial-standard.md) before writing or revising, and [html-delivery.md](references/html-delivery.md) for HTML. Use the complete [Humanizer Chinese](../humanizer-chinese/SKILL.md) for Chinese prose and [Humanizer](../humanizer/SKILL.md) for English. Apply them in embedded/file mode: only finished prose enters the article.
+Keep this skill and its dependencies in the repository. Read [editorial-standard.md](references/editorial-standard.md) before writing or revising, and [html-delivery.md](references/html-delivery.md) for HTML. Run the complete [Humanizer](../humanizer/SKILL.md) and then [Humanizer Chinese](../humanizer-chinese/SKILL.md) over Chinese prose; run Humanizer over English prose. Apply them in embedded/file mode: only finished prose enters the article.
 
 ## Choose the work being requested
 
@@ -27,9 +27,11 @@ Choose an argument that the evidence supports, including exceptions that change
 
 Develop the reader's understanding through concrete results, necessary background and inspectable examples. Explain unfamiliar concepts through their purpose and operation; a sentence excluding them from the benchmark is not an explanation. Use charts and specimens to carry data so prose can explain the result instead of repeating it. There is no fixed chapter count, case count, headline formula or required hero image.
 
+For report interpretation and summary pages, use a presentation-ready reading sequence: one finding per section, a data-bearing visual or evidence specimen as the main element, and short prose that explains it. Remove images that merely fill space. Keep full inventories and raw evidence on their functional report pages. When several methods fail for materially different reasons, give each method its own chapter with the complete reviewed distribution, mechanism, representative evidence, and cross-method distinction.
+
 When revising, check the whole article for the same underlying problem, then make the smallest coherent repair. Review any removed material for lost explanations, evidence, limitations or transitions. Broader restructuring is justified when the argument or organization is defective, not simply because a paragraph received criticism. If a needed fact is absent, investigate it or narrow the claim; do not replace it with confidence or invented precision.
 
-Run the language-appropriate Humanizer pass on headings, prose and captions. This is a prose edit, not a substitute for evidence or editorial judgment. Preserve technical meaning, natural language conventions and original measured text. Do not manufacture casual language or personal experience.
+Run both Humanizer and Humanizer Chinese on Chinese headings, prose and captions; run Humanizer on English. This is a prose edit, not a substitute for evidence or editorial judgment. Preserve technical meaning, natural language conventions and original measured text. Do not manufacture casual language or personal experience.
 
 ## Review and delivery
 

.agents/skills/benchmark-editorial/references/html-delivery.md

@@ -2,6 +2,8 @@
 
 The dashboard reading page and standalone HTML are two presentations of the same authored article. Use the current report renderer and preserve its integrity, safety and localization contracts.
 
+Treat report home and interpretation pages as reusable presentation material. Each section should communicate one conclusion through a chart, meaningful diagram, image, or inspectable evidence specimen, with concise supporting text and exact values available to copy. Do not use decorative background art or repeat the same dense table across summary, interpretation, and analysis. Result analysis owns filterable inventories and detailed distributions; result pages own logs and complete evidence.
+
 Generate Chinese by default. English prose, illustrations, rendering and language-specific checks run only on an explicit manual request. An absent English edition gets a not-generated notice and a return link, not stale translated pages. Preserve translation sources without distributing unused English assets. Apply cross-language parity checks only when both editions are generated; always test the notice in a Chinese-only release.
 
 ## Reading layout

.agents/skills/benchmark-report/SKILL.md

@@ -27,6 +27,7 @@ Repository reports default to Chinese-only output and checks. English generation
 12. Present one authoritative logical result per method and task. The frozen protocol chooses the score-bearing execution; a renderer must never silently replace that rule. Keep every physical call in the evidence chain retain accumulated execution cost separately from the declared primary latency metric. State whether primary latency measures the final call or the complete user-visible operation; retries never increase the logical-result denominator. Do not make execution ordinal labels part of the public comparison unless retry behavior is the declared subject.
 13. Apply the track-independent presentation contract in [report and release](references/report-and-release.md) to every report, including a new track. Wire semantic checks into its release command; prose instructions alone are not a release gate.
 14. In this repository, publish a finished report to the Artifact Site host and site bound by that track’s current profile, following that host’s current `for-agents.md`. Never embed a default deployment hostname or borrow a target from another report. Build and validate before the external write, bind the returned slug, share URL, and version in the current profile, use optimistic locking for later updates, and verify the unauthenticated share page. Use repository publish targets when available; do not reconstruct a target from prose or an earlier receipt. Export the whole deployed tree only when the specific report publisher requires it or when diagnosing a release failure.
+15. Make summary and interpretation routes presentation-ready. Give each section one finding and one primary visual such as a chart, mechanism diagram, evidence specimen, or meaningful image, with short supporting prose and exact values in accessible text. Do not add decorative hero art. Keep sortable inventories, logs, and full evidence on their functional pages. For Chinese report-authored prose, run both the repository Humanizer and Humanizer Chinese after the evidence pass; preserve measured language and quoted evidence.
 
 ## Stop conditions
 

.agents/skills/benchmark-report/references/report-and-release.md

@@ -8,6 +8,12 @@ For the current trajectory report, `make trajectory-replay-report EVIDENCE=/abso
 
 ## Reader-facing structure
 
+Make the home and interpretation routes presentation-ready. Each section should make one finding legible through a data-bearing chart, mechanism diagram, meaningful image, or inspectable specimen, followed by short explanatory prose. Keep exact values in accessible text. Decoration is not a visual explanation: remove images that carry no data, mechanism, topology, or evidence. Dense sortable inventories and raw logs remain on the analysis and evidence routes that own them.
+
+When multiple methods have material failures, give each one a separate interpretation chapter containing its full reviewed subtype distribution, a plain-language explanation of the dominant mechanisms, representative result evidence, and the difference from the other methods. The home page should show only a compact leading-error signature; result analysis owns complete stage and case distributions.
+
+After the factual pass, run both repository Humanizer and Humanizer Chinese over Chinese report-authored headings, prose, and captions. Run Humanizer over English. These passes may improve language only; they must preserve numbers, scope, technical meaning, quotations, and uncertainty.
+
 Organize around reader questions, not storage tables. A useful report usually makes these answers easy to find:
 
 - What was compared, on which tasks and versions?

.env.example

@@ -19,7 +19,10 @@ OBSCURA_BINARY=obscura
 # Pin the exact executable used by the cohort; do not leave this blank or rely
 # on Playwright's mutable bundled-browser fallback.
 CHROMIUM_BINARY=/Applications/Google Chrome.app/Contents/MacOS/Google Chrome
-KITESURF_CDP_ENDPOINT=wss://kitesurf.cloudflare.app/devtools/browser
+# Cloudflare Browser Run constructs the account-scoped Kitesurf CDP endpoint
+# from these values. Do not persist an Authorization header or token in evidence.
+CLOUDFLARE_ACCOUNT_ID=
+CLOUDFLARE_API_TOKEN=
 
 # Browser Trajectory Replay diagnostics and autonomous Browser Use evaluation
 OPENAI_API_KEY=
@@ -54,8 +57,6 @@ TAVILY_API_KEY=
 EXA_API_KEY=
 BROWSERBASE_API_KEY=
 BROWSERBASE_PROJECT_ID=
-CLOUDFLARE_API_TOKEN=
-CLOUDFLARE_ACCOUNT_ID=
 
 # Query-to-search-results and model-bound retrieval
 BOCHA_API_KEY=

AGENTS.md

@@ -1,5 +1,15 @@
 # Report maintenance
 
+## Change lifecycle and repository footprint
+
+- Extend the current owner before adding a parallel implementation. A second browser, cohort, dataset size, provider, language or release must select behavior through data, a registry or parameters in the existing pipeline. It must not create a browser-specific runner, copied renderer, numbered implementation or compatibility alias.
+- Keep command-line files as thin entry points. Parsing, validation, execution, aggregation and publication contracts belong in the track's shared `harness/` module so every entry point uses the same rules.
+- Git owns reusable code, compact dataset definitions, acceptance contracts, provenance annotations and path/hash publication bindings. Measured attempts, generated reports, report prose and illustrations, review payloads, diagnostics, temporary queues and run-local manifests live under the configured data home or the pinned Artifact Site release.
+- Keep one supported current implementation of each stage. Remove superseded generators, repair scripts, migrations and their exclusive tests in the same change once no supported entry point calls them. Retain a historical revision in Git only when a current versioned authority names it and reproduction requires its bytes.
+- Treat a large diff as a design signal. Before adding files, identify the existing owner, the smallest reusable extension and the data that can stay outside Git. File or line counts alone are not release gates; unexplained duplicate stages, private evidence and unreferenced historical versions are.
+- Finish structural changes with a repository inventory: search for references to removed entry points, list the supported entry points, verify that current profiles bind every external artifact, and run the behavior and release gates that exercise those public boundaries.
+- Do not preserve a changed command under its old name when its inputs or meaning change. Restore the old contract or introduce the new behavior under the existing shared abstraction and migrate every caller atomically.
+
 - Keep report-specific prose, illustrations and review payloads outside Git, alongside frozen evidence. Commit compact path/hash bindings instead. The replay build reads `research/trajectory-replay-content.json` from the `--content` directory (default: the data home's `report-assets/trajectory-replay/`); restore missing content from the pinned Artifact Site version, not an unversioned latest report. Do not silently fall back to a repository copy. Compact task definitions, acceptance contracts and provenance annotations remain versioned inputs.
 - Remove retired report-specific generators and their exclusive tests when no supported entry point calls them. Preserve published reports and historical evidence. Keep behavioral tests for supported build, package and publication boundaries; do not replace them with assertions about private helper structure.
 - WCXB builds read profile-bound files from `<evidence>/report-assets/` and preserve the same relative paths in the public package; restore missing files from the profile's pinned Artifact Site version, never from an unversioned latest report. Do not silently fall back to a repository copy.
@@ -9,6 +19,9 @@
 These requirements apply to **all** report tracks; browser-specific rules below only add evidence types.
 
 - Follow the shared presentation contract in `.agents/skills/benchmark-report/references/report-and-release.md`. Metric labels, units, populations, unique directional highlights, one-summary result details, evidence dialogs and repairable error drilldown are common requirements, not Browser Use features.
+- Make report home and interpretation pages presentation-ready: each section states one finding and lets a chart, diagram, image, or inspectable specimen carry the main information; prose explains the evidence briefly. Keep exact values available as text. Remove decorative imagery that does not encode data, mechanism, topology, or evidence. Preserve dense tables and logs on the result-analysis and evidence pages that own them instead of turning audit material into decoration.
+- When several compared browsers or methods have material failures, give each one a dedicated interpretation chapter with its complete reviewed failure distribution, plain-language mechanism, representative evidence, and the difference from the other methods. Put only a compact leading-error summary on the home page; keep full stage and case inventories in result analysis.
+- Run both repository Humanizer and Humanizer Chinese over finished Chinese report prose, headings, and captions after the factual pass. Run Humanizer over English prose. Treat these as language edits: they must preserve measurements, technical meaning, quoted evidence, and uncertainty.
 - Highlight the unique column maximum/minimum from raw values; tied or unavailable values stay neutral. A scoped numerical extreme is not a full-benchmark ranking. Do not remove highlights merely because successful-page populations differ; show the counts and scope.
 - Mean/P50/P95 share one explicitly stated population and parallel labels. Retain retry execution cost separately when the current contract selects final-call latency; one logical result remains one sample.
 - For a before/after build comparison, runs on different dates cannot supply the authoritative speed difference. Predeclare the complete task scope, alternate call order within each task, run serially on a quiet host, bind both binary hashes and all physical calls, and select a round before seeing results. If an earlier timing is corrected, retain it privately, supersede every affected logical timing, and make the report table and detail agree. Release checks must reject an absent or partial timing receipt.

Makefile

@@ -3,7 +3,7 @@ PY := $(UV) run --frozen python
 TEST_PY := $(UV) run --frozen --all-extras python
 WEBMAINBENCH_SAMPLE ?= evaluation/public-webmainbench-200.json
 
-.PHONY: setup import-env runtime-check naming-check test test-python test-ui fixture-smoke ci report report-check popular-sites-report popular-sites-report-check wcxb-release-check resource-image public-benchmark-report public-benchmark-sample webmainbench-run webmainbench-score webmainbench-review webmainbench-assemble browser-use-review browser-use-report browser-use-report-check browser-use-visual-check browser-use-release-check trajectory-replay-report trajectory-replay-report-check trajectory-replay-visual-check trajectory-replay-release-check trajectory-replay-package trajectory-replay-publish public-webarena-report public-webarena-report-audit public-webarena-report-release
+.PHONY: setup import-env runtime-check naming-check test test-python test-ui fixture-smoke ci report report-check popular-sites-report popular-sites-report-check wcxb-release-check resource-image public-benchmark-report public-benchmark-sample webmainbench-run webmainbench-score webmainbench-review webmainbench-assemble browser-use-review browser-use-report browser-use-report-check browser-use-visual-check browser-use-release-check trajectory-replay-report trajectory-replay-report-check trajectory-replay-visual-check trajectory-replay-release-check trajectory-replay-package trajectory-replay-publish webarena-catalog-report webarena-catalog-release-check public-webarena-report public-webarena-report-audit public-webarena-report-release
 
 setup:
 	$(UV) sync --frozen --all-extras
@@ -100,6 +100,13 @@ trajectory-replay-package: runtime-check
 trajectory-replay-publish: runtime-check
 	$(PY) scripts/publish_webarena_replay_artifact_site.py
 
+webarena-catalog-report: runtime-check
+	@test -n "$(EVIDENCE_ROOT)" -a -n "$(LEDGER)" -a -n "$(SCOPE)" -a -n "$(CATALOG)" -a -n "$(SUITE)" -a -n "$(SITE_MAP)" -a -n "$(OUTPUT)" -a -n "$(ARCHIVE)" || (echo 'EVIDENCE_ROOT, LEDGER, SCOPE, CATALOG, SUITE, SITE_MAP, OUTPUT and ARCHIVE are required' >&2; exit 2)
+	$(PY) scripts/build_webarena_catalog_report.py --evidence-root "$(EVIDENCE_ROOT)" --ledger "$(LEDGER)" --scope "$(SCOPE)" --catalog "$(CATALOG)" --portable-suite "$(SUITE)" --site-map "$(SITE_MAP)" --output "$(OUTPUT)" --archive "$(ARCHIVE)"
+
+webarena-catalog-release-check: runtime-check
+	$(PY) scripts/check_webarena_catalog_release.py $(if $(DATA_HOME),--data-home "$(DATA_HOME)",)
+
 public-webarena-report: runtime-check
 	$(PY) scripts/release_webarena_trajectory_report.py $(if $(FORMAL_ROOT),--formal-root "$(FORMAL_ROOT)",)
 

deploy/webarena/README.md

@@ -59,7 +59,8 @@ Nominatim 的 `flatnode.file` 导入／增量更新缓存。评测使用固定
 镜像自带的 Rails OSM 对象库没有节点、道路或关系记录；搜索结果来自已恢复的
 Nominatim 库，直接点击原本会得到 404。对象详情入口从同一份冻结数据按 OSM
 类型和 ID 查询，展示名称、地址、坐标及可用的附加标签，保留原来的 `/node/`、
-`/way/`、`/relation/` 地址。它只展示 Nominatim 收录的对象；缺失对象仍返回 404，
+`/way/`、`/relation/` 地址。Nominatim 不索引的评测所需路线 relation 只接受绑定到
+固定 Wikidata revision、OSM ID 和来源哈希的精确记录；其它缺失对象仍返回 404，
 不会合成不存在的内容或声称提供完整 OSM 版本历史和成员关系。
 
 三个原始归档约 174.5 GiB。初始化逐包下载、校验 SHA-256、稀疏解包并删除归档，