Repository navigation
Extract Web Fetch benchmark code into pinned LexBench package - #54
Conversation
|
本轮独立审查完成,固定 head 为 发现 1 个本 PR 的 P2 回退:历史输入路径别名绕过绑定和禁止修改检查。已用临时独立 checkout 实际写入并读出未绑定的本地数据;相同路径在拆分前会被拒绝修改。另有独立包 CLI 的输入缺口,已在 lexbench-webfetch#1 单独定位。 现场验证范围:
测试结论不覆盖上述跳过用例,也不包含真实供应商调用或完整历史采集材料的重新认证。发现的问题通过额外场景复现,现有测试通过不代表这两个输入边界已经正确。除已定位问题外,本轮没有发现其他需要修改的实质问题。没有提交、推送或合并代码;本轮不跟进后续新 head。 |
|
第二轮独立复审结论:上轮历史输入问题部分修复,P2 仍未关闭。 已从 GitHub 核对并固定审查 head 但目录链接与其后面的 本轮现场测试:
除这个尚未解决的路径问题,本轮没有发现其他实质问题。未重复第一轮已通过且未受本次修改影响的完整慢检查,未进行真实供应商评测。只使用本会话独立 checkout,没有修改、提交、推送或合并代码。 |
|
第三轮独立复审完成:上轮遗留的 P2 已修复,本轮没有发现新的实质问题。 已从 GitHub 核对并固定审查 head 修复覆盖了共同原因:先解析实际父目录,正确处理目录链接后面的 本轮独立验证:
关联 HTTP 测试首次被沙箱禁止监听本机地址;在允许本机测试服务的环境重跑同一组场景后,全部通过。没有修改产品代码,也没有因环境错误跳过这些场景。 LexBench 仍为 |
There was a problem hiding this comment.
Reviewed head 7ea42b812438a6e38894070b126b8177cace21ff against base 637bd5171b296af6431b7a570f3d8a48ec5c671f. One P2 regression remains, detailed inline: relocating the manifests disables the existing checkout-input CI guard for the historical suites. Fix this before merging.
The ownership split is appropriate: LexBench owns task definitions and deterministic acceptance; Browser Eval retains execution, evidence and reporting. Direct imports and one pinned implementation provide a sufficient shared boundary. The guard needs a small extension to its existing discovery logic, with no additional abstraction or compatibility layer.
Independent validation: 22 relocated definitions are byte-identical to base; all 894 active scoring controls preserve their complete metrics and verdicts except the implementation fingerprint. The previous path fix passes 132 definition aliases, 30 dataset aliases and 60 file-link scenarios. Four isolated wheel installation variants, 25 actual-Chrome reference-capture tests, static checks, report UI, native adapters and the fixture CLI pass. Across the full test sweep and two macOS alias cases rerun under their required system temp root, 5,406 tests and 2,322 subtests passed; 82 environment/historical-input cases skipped. No product code was changed, pushed or merged.
|
本轮独立复审通过,审查 head 为 根因确实是发现范围缩小:本地 manifest 移走后,原来的扫描无法再找到历史冻结目录。当前修改从现有 compact binding 补充五个历史 manifest 的逻辑路径,继续通过安装包校验绑定哈希,并检查原 checkout 目录;本地候选 manifest 的扫描也保留。修改集中在原检查器,没有复制输入规则或另建一套验收流程。检查器实现 · 对应回归场景 现场验证结果:
本轮只在独立 checkout 和临时源码快照中复现,没有访问原开发工作区,没有修改产品代码、提交、推送或合并。未重复先前已通过且未受本次修改影响的完整慢检查,也未进行真实供应商评测。 |
Self-authored Web Fetch tasks and scoring were mixed with Browser Eval's execution code. Move their ownership to LexBench Web Fetch, and make Browser Eval import its pinned package directly. LexBench can validate inputs and score saved responses or delivered files without Browser Eval.
Remove the local task model, scorer, scenario compiler, task-authoring rules and frozen definitions. Move their scenario tests with them. Browser Eval retains adapters, execution policy, reference/evidence lifecycle, experiment profiles and reporting. Frozen inputs, certificates and measured results retain their original bytes and bindings.
Pin the dependency, lockfile and compact input binding to LexBench main commit
6d8d72637bb96f4938b9a2f7c46d0c7dffd74945, merged through lexbench-webfetch#1.uv sync --frozenfetches the Git source, builds a wheel and installs the Python package. A sibling checkout is not required.Historical input paths resolve through the installed owner, including equivalent paths and directory links. Keep the existing checkout-input CI guard effective after the manifests move: discover historical manifests from the compact binding, validate their installed hashes, and check the original checkout directories alongside local candidate manifests. Accidental dataset, source and reference copies remain forbidden.
Refresh the matching English/Chinese READMEs, repository About text, architecture and contribution guidance. LexBench includes licensing, citation/provenance, security/contribution guidance, CI and source/wheel packaging; third-party captures and measured evidence remain external.
Validation
make cigate, all four wheel installation variants, real Chromium reference capture and Python 3.12–3.14 base compatibility.5dd76e4e. It independently repeats the public CI scenarios and verifies the installed main commit, with 116 related tests and 23 subtests passing. The corresponding review discussion is resolved.