You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Task 566: the repository check expects path do-it-myself, the README check expects do-it-myself-with #55
Task 566's creation check expects project[path] "do-it-myself" (GitLab's path for the name "Do it myself"), but its README check expects the commit at __GITLAB__//api/v4/projects/byteblaze%2Fdo-it-myself-with/repository/commits. A correct run creates one repository and commits to it, so one of the two checks always fails. The original WebArena task read the README from byteblaze/Do-it-myself.
My run created do-it-myself (creation check passed) and committed the README to byteblaze%2Fdo-it-myself (201). README check: "No matching network event found that satisfies all expected criteria."
Reproduce in the folder of task-566-paths.zip (the README is the expected content, so only the path differs): uvx --from webarena-verified==1.2.3 webarena-verified eval-tasks --task-ids 566 --output-dir out-do-it-myself --config config.json. With out-do-it-myself-with, the creation check fails instead.
Fix: __GITLAB__/api/v4/projects/byteblaze%2Fdo-it-myself/repository/commits. The README content check has a separate fault, fixed in #52.
Activity