Add autonomous WebArena-Verified browser evaluation

.agents/skills/benchmark-report/SKILL.md

@@ -22,6 +22,8 @@ Read the repository's `AGENTS.md` first. Project rules override this general ski
 9. Rerun only calls whose inputs, provider route/version, runtime contract, or source validity changed. Recompute downstream scores and reports without provider calls when only scoring, annotations, or presentation changed. Never retry merely to obtain a better result. A reference or verdict correction must identify the shared root cause and add executable positive/negative controls through the production scorer; bind those regression IDs in the revision receipt instead of patching stored verdicts.
 10. Build the public report from the frozen evidence. Every page should answer a reader question. Show task and run identifiers, the exact expected-versus-actual evidence behind a verdict, denominators and unavailable metrics, method differences, limits, concise reproduction instructions, and a compact review history.
 11. Release only after code tests, evidence integrity, report links, responsive/visual checks, secret scans, target-site checks, and deployed-output verification pass. Preserve earlier sites unless replacement was explicitly requested.
+12. Present one authoritative logical result per method and task. The frozen protocol chooses the score-bearing execution; a renderer must never silently replace that rule. Keep every physical call in the evidence chain and aggregate all latency and cost. Do not make execution ordinal labels part of the public comparison unless retry behavior is the declared subject.
+13. In this repository, a finished report release defaults to Artifact Site at `https://artifact-hub.app.lexmount.com`, following its current [`for-agents.md`](https://artifact-hub.app.lexmount.com/for-agents.md). Build and validate before the external write, bind the returned slug, share URL, and version in the current profile, use optimistic locking for later updates, and verify the unauthenticated share page. Use repository publish targets when available; do not reconstruct a target from prose or an earlier receipt. Export the whole deployed tree only when the specific report publisher requires it or when diagnosing a release failure.
 
 ## Stop conditions
 

.agents/skills/benchmark-report/references/report-and-release.md

@@ -18,14 +18,32 @@ Keep environment and reproduction together and deduplicated. Show repository/ver
 
 For a multilingual report, generate every language from one semantic report model. Keep route inventory, section order, tables, values, filters and evidence links aligned one-to-one. Localize navigation and explanatory prose for the target language instead of translating mechanically; preserve task names, IDs, URLs, references, provider outputs and other measured evidence unless the benchmark explicitly evaluates translated content. A summary-only secondary language must be labeled as a summary, not presented as the full report.
 
-Use a compact review page resembling release history: current reviewed coverage and unresolved count, then one card or row per review round with scope, findings, corrections, and remaining work. Do not publish agent work logs, giant execution manifests, or downloads merely because the evidence exists.
+Use a compact review page resembling release history: one card or row per review round with scope, findings, corrections, status, and remaining work. Keep per-result review evidence on result routes or in the evidence store; do not turn the review history into another task-result inventory.
+
+Expose one logical result per method and task. A declared additional execution is an orchestration detail: preserve its immutable evidence, follow the score authority frozen before the run, and add every execution's latency and cost. Do not split the public result table by execution ordinal unless that distinction is the preregistered research question.
+
+Keep dataset inventory separate from measured results. Dataset pages explain the task, selection, reference and acceptance contract; result pages own method verdicts, timing, trajectories and run-derived solvability observations. Define participants as reproducible method stacks and render comparisons from a registry. When the participant set can grow beyond two, group each task's pass/fail participants and provide one baseline selector plus multi-select comparison participants and explicit outcome relations; optional pairwise aggregates may supplement this view. Do not hard-code a two-column comparison or keep adding one wide column per participant.
+
+Name report tracks by the procedure a reader must distinguish, such as autonomous use versus deterministic trajectory replay, rather than by an internal module layer. Once a repository selects canonical track names, migrate code, commands, report routes and checks together; do not keep compatibility aliases unless external callers are explicitly in scope.
+
+On result-detail pages, use the opening summary for the verdict, a concise failure locator, and short scalar facts. Put long evaluator messages, expected/actual payloads, traces, and logs in their dedicated evidence sections; never force diagnostic prose or raw JSON into a narrow KPI card.
+
+For self-hosted web environments, include a deployment and network topology, image identities, included/excluded sites and compatibility layers. Distinguish configured preparation from independently bound preparation receipts. Split environment preparation, browser-session setup, agent run, tool-call wall time, evidence finalization and evaluator time at collection; label historical mixed timers by their actual boundaries and never call wrapper/CDP wall time pure browser-core speed.
+
+Never publish user prompts, assistant reasoning, plans, internal notes, conversational back-and-forth, or tool-work narration. A formal report explains the benchmark and evidence; it is not the authoring notebook.
+
+Write every title and paragraph for the reader, not as a reply to author feedback. Convert corrections into neutral artifact facts; never mention who requested a change, quote review conversation, or use a rebuttal-style heading when a descriptive heading works. Review history may record an objective finding and resolution, while release history records only reader-visible effects.
 
 ## Claims and rankings
 
 Pair every ranking with its task family, denominator, acceptance meaning, a typical input-to-output route, and limits. Keep task acceptance separate from continuous quality, latency, cost, and title/body diagnostics. Do not call conditional averages global quality or explain a correlation as a verified mechanism.
 
 Prefer concise tables for exact comparisons and prose for interpretation. Remove repeated caveats after placing each limitation where it changes interpretation. Run a factual pass before a prose/humanization pass; never let editing change numbers or scope.
 
+For a standard numeric comparison, show mean, P50, and P95 when all three are useful. P50 is the median, so never present “median” and P50 as separate statistics. Declare whether higher or lower is better and visually mark only a unique, directionally valid winner; leave ties, identifiers, call counts, and metrics without a universal direction neutral. Do not rank raw totals when unequal workloads make them non-comparable without also exposing the workload or normalized value.
+
+Do not add a second static inventory when existing filters expose the same result slice. Keep one searchable result table and use focused summaries only when they add a distinct analytical relationship.
+
 ## Release gates
 
 Before publication:
@@ -39,6 +57,10 @@ Before publication:
 7. fetch the unauthenticated share URL and compare the deployed file tree or archive with the submitted package;
 8. record a concise release entry with score/denominator effects and remaining limits.
 
+The responsive/visual gate must cover every core route class, both languages, desktop and narrow widths, and light and dark themes. Check that short headers do not wrap one character per line, long cells receive deliberate width or scrolling, filters remain operable, code and evidence blocks have readable contrast, and text-heavy diagrams use scalable/vector rendering or retain a readable minimum width. A screenshot of only the home page is insufficient.
+
+Render release history like a report, not a heading plus a raw table: show the current version summary first, keep evidence/handling/impact details expandable, retain older versions in a compact history, and provide the structured record download. Each entry states what changed, why, the formal handling, and the score, denominator, latency, evidence, and limitation effects.
+
 Credentials belong in the approved secret store, never HTML or Git. If a service requires an account, list the account and official acquisition location in the report. Do not include live keys.
 
 Publication authorization does not imply permission to overwrite another report. Bind the release target in the current profile and reject receipts for a different service or site. Return the public share link, stable site ID, PR/change link, verification result, and unresolved count.

.env.example

@@ -19,12 +19,13 @@ OBSCURA_BINARY=obscura
 CHROMIUM_BINARY=/Applications/Google Chrome.app/Contents/MacOS/Google Chrome
 KITESURF_CDP_ENDPOINT=wss://kitesurf.cloudflare.app/devtools/browser
 
-# Browser Use optional model-based diagnostics only (not Web Fetch scoring)
+# Browser Trajectory Replay diagnostics and autonomous Browser Use evaluation
 OPENAI_API_KEY=
-OPENAI_BASE_URL=
+OPENAI_BASE_URL=https://api.openai.com/v1
 
-# Optional publishing override; default: ~/.config/artifact-hub/token
-ARTIFACT_HUB_TOKEN=
+# Optional Artifact Site publishing override; default:
+# ~/.config/artifact-site/tokens/<target-host>
+ARTIFACT_SITE_TOKEN=
 # Set explicitly in the environment for both report build and packaging.
 WEBFETCH_REPORT_PUBLIC_URL=
 

.gitignore

@@ -29,6 +29,7 @@ run.log
 *.egg-info/
 .pytest_cache/
 .ruff_cache/
-.browser-use-eval-venv/
+artifacts/trajectory_replay/
 artifacts/browser_use/
-/docs/reports/browser-use-evaluation-*
+/docs/reports/trajectory-replay-evaluation-*
+.ai/topics/