|
| 1 | +--- |
| 2 | +name: uitest |
| 3 | +description: Write, update, run, or debug vscode-java-dependency (Project Manager for Java) UI/E2E tests using AutoTest YAML plans. Use when the user asks for a UI test, E2E test, VS Code UI validation, Java Projects tree/view test, referenced-library test, or autotest plan. |
| 4 | +--- |
| 5 | + |
| 6 | +# UI/E2E tests with AutoTest |
| 7 | + |
| 8 | +Use this skill to add or update UI/E2E coverage for `vscode-java-dependency` (Project Manager for Java). |
| 9 | + |
| 10 | +The repository uses `@vscjava/vscode-autotest`: YAML plans in `test/e2e-plans/*.yaml` launch VS Code, install the Extension Pack for Java (`vscjava.vscode-java-pack`) plus a local VSIX of this extension, execute user-facing actions against the Java Projects view, capture screenshots, and write `test-results/<plan>/results.json`. |
| 11 | + |
| 12 | +## Prerequisites (local) |
| 13 | + |
| 14 | +- Node.js >= 18 and JDK 21+ installed and on `PATH` (JDK 21 is required to build the `jdtls.ext` OSGi bundle). |
| 15 | +- Close any running VS Code instance before running a plan locally; a running instance can block AutoTest from launching its own VS Code. |
| 16 | +- Workspace fixtures are in-repo — no external clones are needed. Plans reference `../maven` (`test/maven`, a `maven-archetype-quickstart` project) or `../invisible` (`test/invisible`, an unmanaged-folder project). |
| 17 | + |
| 18 | +## Workflow |
| 19 | + |
| 20 | +1. Identify the scenario and search `test/e2e-plans/*.yaml` for an existing plan that already covers the area (project explorer, view modes, classpath, export jar, new types, file operations, delete, copy paths, refresh, build lifecycle, autorefresh). |
| 21 | +2. Update the existing plan when possible. Create a new `test/e2e-plans/java-dep-<scenario>.yaml` only when no existing plan fits. |
| 22 | +3. Use stable AutoTest actions and deterministic verifiers. Do not add raw Playwright tests or screenshot-only checks. |
| 23 | +4. Validate the plan: |
| 24 | + |
| 25 | +```powershell |
| 26 | +npx -y @vscjava/vscode-autotest validate test\e2e-plans\<name>.yaml |
| 27 | +``` |
| 28 | + |
| 29 | +5. If validating the current branch, build the OSGi bundle and package the extension: |
| 30 | + |
| 31 | +```powershell |
| 32 | +npm install # first time only; on later iterations run just the two commands below |
| 33 | +npm run build-server |
| 34 | +npx @vscode/vsce package -o vscode-java-dependency.vsix |
| 35 | +``` |
| 36 | + |
| 37 | +6. Run the plan against the packaged VSIX: |
| 38 | + |
| 39 | +```powershell |
| 40 | +npx -y @vscjava/vscode-autotest run test\e2e-plans\<name>.yaml --vsix vscode-java-dependency.vsix --output test-results\<name> |
| 41 | +``` |
| 42 | + |
| 43 | + Add `--no-llm` to skip natural-language `verify:` fields and rely only on deterministic verifiers for a fast local loop. Run the whole suite with `npm run test-e2e` (`autotest run-all test/e2e-plans --no-llm`). |
| 44 | + |
| 45 | +7. Inspect `test-results/<name>/results.json` and `test-results/<name>/screenshots/`. |
| 46 | +8. Iterate based on the failure cause: |
| 47 | + - **Incorrect plan**: fix the YAML and rerun step 6. No rebuild is needed. |
| 48 | + - **Product code fix**: after editing extension source (`src/**`) or the OSGi bundle (`jdtls.ext/**`), re-run step 5 (rebuild + repackage the VSIX) before rerunning step 6. Never rerun against a stale VSIX. |
| 49 | + - **Product bug (report only)**: report the observed behavior and cite the failing step, screenshot, and result reason. |
| 50 | + |
| 51 | +## Authoring rules |
| 52 | + |
| 53 | +- For most plans, use: |
| 54 | + |
| 55 | +```yaml |
| 56 | +setup: |
| 57 | + extension: "vscjava.vscode-java-pack" |
| 58 | + vscodeVersion: "stable" |
| 59 | + workspace: "../maven" |
| 60 | + settings: |
| 61 | + java.configuration.checkProjectSettingsExclusions: false |
| 62 | + workbench.startupEditor: "none" |
| 63 | +``` |
| 64 | +
|
| 65 | +- Use `--vsix vscode-java-dependency.vsix` to test current-branch changes; do not rely on a marketplace copy of `vscjava.vscode-java-dependency`. |
| 66 | +- Use `../invisible` (not `../maven`) for referenced-library / classpath commands, which only apply to unmanaged-folder projects. |
| 67 | +- Prefer `executeVSCodeCommand <commandId>` for command-driven UI (e.g. `javaProjectExplorer.focus`, `java.view.package.revealInProjectExplorer`, `workbench.actions.treeView.javaProjectExplorer.collapseAll`). |
| 68 | +- Drive the tree with `expandTreeItem <name>` and title-bar buttons with `clickViewTitleAction "Java Projects" "<action>"`. |
| 69 | +- Prefer `verifyTreeItem` for tree state, `verifyFile` for generated/modified/deleted files, `verifyEditorTab` for opened tabs, and `verifyClipboard` for copy-path commands. |
| 70 | +- Use `waitForLanguageServer` before tree interactions, and `insertLineInFile` for Java source edits that JDT LS must observe. |
| 71 | +- Free sidebar space (`closeAuxiliaryBar`, `collapseSidebarSection OUTLINE`/`TIMELINE`, `collapseWorkspaceRoot`) before asserting tree rows. |
| 72 | +- Keep step IDs unique, descriptive, and kebab-case. Omit `verify:` on steps whose only assertion is a deterministic verifier. |
| 73 | +- Avoid hard-coded coordinates and brittle DOM structure assumptions. |
| 74 | + |
| 75 | +## CI |
| 76 | + |
| 77 | +The repository workflow `.github/workflows/e2eUI.yml` runs on push and pull requests to `main`. It lints, discovers `test/e2e-plans/*.yaml` into a matrix, builds a branch VSIX per OS, runs every plan on Windows and Linux as independent matrix cells, and uploads `test-results/` artifacts plus an aggregate summary. |
| 78 | + |
| 79 | +Each plan surfaces as its own PR check, so a new `test/e2e-plans/*.yaml` is picked up automatically without editing the workflow. |
0 commit comments