Repository navigation
test(gui-user): match four scenario specs to the shipped Customize, memory and shortcuts UI - #22893
Conversation
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
…emory and shortcuts UI (#22733)
25539f1 to
359f7cd
Compare
|
self-added: yes
🤖 AI-assisted comment 🤖 |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the change only rewords four scenario specs to match the dashboard as it ships, and it adds tests that check the quoted labels against the en catalog. Notes
[DESIGN-REVIEWED] 359f7cd Description read[DESCRIPTION-READ] 3e1cbd341dba1c7c8faa4d376059670b18afadc1e02d069857df1332460bc8fc That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
|
Review-ready at 359f7cd: PR Readiness passed (104 checks, 0 failing, label 🤖 AI-assisted comment 🤖 |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: test (5 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: four GUI-user scenario files updated to the labels the dashboard ships (Customize, memory browser, shortcuts), plus two tests that read those labels from the en locale catalog; test files only.
Problem / Motivation
Goal: Four nightly GUI test scenarios describe the dashboard as it ships today, so they pass when the product works.
In the nightly run 39577973904 (#22733), the tester could not find screens these four scenarios name:
capabilities-agents-list-and-open-editorexpects an editor with a grouped rail, an Overview diagram and a "Chat with this crewmate" button. The shipped editor has a flat rail that opens on General.customize-tabsexpects an "Add crewmate" button, one crewmate card, and "Customize" as the large page title. The Custom agents pane is also supposed to start with its filter row.memory-open-browser-from-overviewexpects a "Consolidation idle (hours)" field on the first card.settings-shortcutsjudges the "Enable shortcuts" switch on the last screenshot. The scenario's own scroll step moves that switch off screen.Why it matters
These four scenarios fail every night, whatever the product does, and each failure costs two paid attempts. They also hide real regressions on the same surfaces.
Not a goal
What changed (motivation → approach → change)
Each scenario now names what the dashboard draws. The text came from the shipped en catalog and the nightly screenshots, and each stale line traces to the commit that changed that screen:
capabilities-agents-list-and-open-editordefault+nova-sky; "New crewmate"customize-tabsdefault+nova-sky; "New crewmate"; the "New sessions use" picker sits above the Custom agents filter rowmemory-open-browser-from-overviewsettings-shortcutsTwo unit tests pin the quotes to the en catalog and refuse the retired readings.
Backwards compatibility
Compatible: only scenario text and tests change. No loader, harness or product code changes.
Removes nothing: no scenario, key or check is removed; the four scenarios keep their names, tiers and limits.
Tests
In
test/gui_user/test_scenarios_and_report.py:test_customize_and_memory_scenarios_quote_the_shipped_labelsreads the button, filter, editor rail, "Edit avatar", "Customize", and memory card and field labels fromen.json/en.manual.json. It checks that the scenarios quote them, and that "Add crewmate", "Chat with this crewmate", "Who it is", "Overview" and "Consolidation idle" are gone.test_shortcuts_scenario_judges_everything_on_the_frame_it_lands_onchecks that the shortcuts scenario does not scroll and that every expectation names what the first frame shows (the switch and "Chat Navigation"), since the harness judges expectations on the last screenshot.Both fail on
origin/mainand pass here. They passed 10 repeats in a row;test/gui_user/: 314 passed.Manual verification
N/A: the paid GUI suite is not rerun for this PR. The new wording was checked against the 10-21 nightly screenshots (Customize Crewmates list, Custom agents pane, memory browser, Shortcuts pane) and the shipped en catalog. The next nightly run is the end-to-end check.
Related Issues
Part of #22733
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)