Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# AGENTS

## Entrypoint
- CLI entrypoint is `bin/fire` (executes `bin/fire.php`); it requires Composer autoload to exist, so run `composer install` before local runs.

## Config + discovery
- `fire.yml` and optional `fire.local.yml` are read from the project root (four directories above `vendor/fourkitchens/fire`); if missing, only `InitCommand.php` is registered and the CLI prompts to run `fire init`.
- Local env auto-detection sets `local_environment` based on `.lando.yml` or `.ddev/config.yaml` in the project root.

## Command sources
- Core commands live in `src/Robo/Plugin/Commands/*Command.php` and are discovered via Robo `CommandFileDiscovery`.
- Project-level custom commands are loaded from `fire/src/Commands` in the project root (path is computed by stripping `vendor/fourkitchens/` from the package path).

## Config template
- `fire init` uses the template at `assets/templates/fire.yml`; update it if you change required config fields.
79 changes: 78 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,10 @@ Example:

Alias: `vlec`

- `vrt:playwright:init`: Configure Playwright VRT scaffolding using Automated Testing Kit.

Alias: `vpinit`

- `vrt:reference`: Takes new reference screeshots from the reference URL.

Alias: `vref`
Expand All @@ -210,11 +214,84 @@ Example:
**2. Full:** It replaces all existing code and allows you to write the command from scratch.

You can also create a new command, just choice the "Custom" option at the prompt when it ask you for for the command you want to overwrite, then respond to the questions, now a new command should have been created in the custom path, by default only a task is added to cleans the Drupal cache, but from this file, you can add your custom tasks.

- `platform:uli`: This command allows you to generate a one-time login URL for any environment hosted on Pantheon, Acquia, or Platform.sh.

Alias: `puli`

## Playwright VRT

FIRE supports Playwright-based visual regression testing as an alternative to the deprecated Backstop workflow.

### Initialize Playwright VRT

Set `ATK_HOME` to the Playwright test root before running the init command:

```
export ATK_HOME=./tests/playwright
fire vrt:playwright:init
```

The init command installs the required Drupal packages, runs the Automated Testing Kit Playwright scaffold, copies FIRE's Playwright templates, installs Node dependencies, installs `@fkbender/playwright-vrt-scripts`, installs Playwright browsers, and adds these npm scripts to `tests/playwright/package.json`:

```
"scripts": {
"vrt": "playwright-vrt",
"vrt:local": "playwright-vrt-local",
"vrt:ci": "playwright-vrt-ci"
}
```

If `NVM_DIR` is set and `tests/playwright/.nvmrc` exists, FIRE sources NVM and runs the npm commands using the Node version defined in `.nvmrc`.

### Environment Settings

During initialization, FIRE asks for:

- Baseline URL
- Baseline Terminus env
- Baseline Terminus site
- Candidate URL, usually the local URL

Those values are written to `tests/playwright/.env`:

```
BASELINE_URL="https://www.aft.org"
CANDIDATE_URL="http://aft-main.lndo.site"
BASELINE_TERMINUS_ENV=live
BASELINE_TERMINUS_SITE=aft-main
```

The `.env` file is added to `tests/playwright/.gitignore` by the init command.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to add a part here where we describe how to edit the common pages.

### Run Playwright VRT

After initialization, run Playwright VRT with:

```
fire vrt:run --tool=playwright
```

or from the Playwright test root:

```
cd tests/playwright
npm run vrt
```

`fire vrt:reference --tool=playwright` also runs the Playwright VRT npm script from `tests/playwright`.

### Automatic Tool Detection

`vrt:run` and `vrt:reference` default to `--tool=auto`.

FIRE detects configured tools using these files:

- Backstop: `tests/backstop/backstop.json` or `tests/backstop/backstop-local.json`
- Playwright: `tests/playwright/package.json` or `tests/playwright/playwright.config.js`

If only one tool is configured, FIRE uses it automatically. If both Backstop and Playwright are configured in an interactive shell, FIRE asks which one to run. In non-interactive runs with both configured, FIRE defaults to Playwright.

## Configuration
Into your project root create a file called: `fire.yml` and iside of it speficify your global project settings.

Expand Down
3 changes: 3 additions & 0 deletions assets/templates/fire.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,6 @@ remote_canonical_env: live

# (Optional setting), the system will automatically detected your local env, currently available: ddev, lando.
#local_environment : lando

# (Optional setting), default VRT tool to use. Options: backstop, playwright.
#vrt_tool: backstop
1 change: 1 addition & 0 deletions assets/templates/playwright/.nvmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
24.15.0
155 changes: 155 additions & 0 deletions assets/templates/playwright/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
# Playwright VRT

Visual Regression Testing for this project, set up with [Fire](https://github.com/fourkitchens/fire) and powered by [`@fkbender/playwright-vrt-scripts`](https://github.com/fourkitchens/playwright-vrt-scripts).

VRT works by capturing screenshots of your site from two environments — a **baseline** (typically the live Pantheon environment) and a **candidate** (your local environment) — and comparing them pixel by pixel. Failures mean something visually changed.

---

## Drupal modules

`drupal/automated_testing_kit` and `drupal/qa_accounts` are required in your `composer.json` but **not enabled**. Enable them only after a config split is in place to ensure they cannot reach production:

```bash
ddev drush en automated_testing_kit qa_accounts -y
```

These modules are not required for VRT. They are needed for ATK-based E2E tests (Phase 2).

---

## Prerequisites

- Node (version pinned in `.nvmrc`)
- Playwright browsers installed (`npx playwright install --with-deps` — done automatically by `fire vrt:playwright:init`)
- A populated `.env` file (see below)

### `.env`

This file is gitignored and must exist locally. It is generated automatically by `fire vrt:playwright:init`, but can be created manually if needed:

```
BASELINE_URL="https://live-yoursite.pantheonsite.io"
CANDIDATE_URL="https://yoursite.ddev.site"
BASELINE_TERMINUS_ENV=live
BASELINE_TERMINUS_SITE=yoursite
```

---

## Running VRT Locally

```bash
# Full two-pass comparison: captures baseline from BASELINE_URL, candidate from CANDIDATE_URL
npm run vrt:local

# CI-style run (used in automated pipelines)
npm run vrt:ci
```

Results open automatically in your browser as an HTML report. Failing tests show a diff of baseline vs. candidate.

---

## Adding Pages to Test

Open `tests/data/vrtCommonPages.json` and add an entry:

```json
[
{
"name": "Home",
"path": "/",
"screenshotName": "home-page"
},
{
"name": "Login",
"path": "/user/login",
"screenshotName": "user-login"
},
{
"name": "News listing",
"path": "/news",
"screenshotName": "news-listing"
}
]
```

Each entry requires three fields:

| Field | Purpose |
|---|---|
| `name` | Human-readable label shown in the test report |
| `path` | Site-relative URL to visit (must start with `/`) |
| `screenshotName` | Base filename for saved screenshots (no extension, no spaces) |

The spec automatically generates tests for every page × every device profile (Desktop, iPad, iPhone 12), so adding one entry produces three new tests.

---

## Device Profiles

Tests run against three device profiles, all using Chromium:

| Profile | Viewport |
|---|---|
| Desktop Chrome | 1280 × 720 |
| iPad (gen 7) | 810 × 1080 |
| iPhone 12 | 390 × 844 |

Device profiles are defined in `tests/support/4k_utilities.js`. Firefox and WebKit are available but commented out in `playwright.config.js` — remove the comments to enable them.

---

## Running a Single Test or Page

```bash
# Run only tests matching a page name
npx playwright test --grep "Home"

# Run only desktop tests
npx playwright test --grep "desktop"

# Run all VRT tests (tagged @vrt)
npx playwright test --grep "@vrt"
```

---

## Viewing the Report

After a run, open the HTML report from inside `tests/playwright/`:

```bash
npm run report
```

Or from the project root:

```bash
npx playwright show-report tests/playwright/playwright-report
```

On CI, reports are uploaded as artifacts. Each failing test shows the baseline screenshot, the candidate screenshot, and a pixel-diff image.

---

## Troubleshooting

**Tests fail with image differences I didn't expect**

- Check that `stage_file_proxy` is enabled so images load correctly in your local environment.
- Make sure your local database is synced from a recent pull of the live environment.
- Confirm there are no pending CSS or JS builds that differ from what's on live.

**`networkidle` timeout errors**

The test waits for the page to become network-idle before screenshotting. If a page has long-polling or persistent background requests, this will time out. Add a custom wait or increase the timeout for that specific test.

**Snapshots are stale after a design change**

Once a visual change is intentional and approved, update the baseline by re-running the VRT capture against the new live environment.

**`.env` file not found**

Run `fire vrt:playwright:init` to regenerate it, or copy the values from a teammate or the project's password manager entry.
7 changes: 7 additions & 0 deletions assets/templates/playwright/css/screenshotGlobalStyle.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#toolbar-administration {
visibility: hidden;
}

.gin-secondary-toolbar__layout-container {
visibility: hidden;
}
12 changes: 12 additions & 0 deletions assets/templates/playwright/data/vrtCommonPages.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[
{
"name": "Home",
"path": "/",
"screenshotName": "home-page"
},
{
"name": "Login",
"path": "/user/login",
"screenshotName": "user-login"
}
]
37 changes: 37 additions & 0 deletions assets/templates/playwright/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
{
"name": "playwright-vrt",
"version": "1.0.0",
"description": "Playwright VRT and E2E testing",
"main": "playwright.atk.config.js",
"scripts": {
"vrt": "playwright-vrt",
"vrt:local": "playwright-vrt-local",
"vrt:ci": "playwright-vrt-ci"
},
"keywords": [],
"dependencies": {
"@eslint/eslintrc": "3.3.5",
"@eslint/js": "8.57.1",
"@fkbender/playwright-vrt-scripts": "1.0.0-alpha.2",
"@playwright/test": "1.59.1",
"axios": "1.16.0",
"cspell": "8.19.4",
"dotenv": "16.6.1",
"fast-xml-parser": "4.5.6",
"junit-report-merger": "9.0.3",
"winston": "3.19.0",
"yaml": "2.9.0"
},
"devDependencies": {
"@eslint/eslintrc": "3.3.5",
"@eslint/js": "8.57.1",
"eslint": "8.57.1",
"eslint-config-airbnb-base": "15.0.0",
"eslint-config-standard-with-typescript": "43.0.1",
"eslint-plugin-chai-friendly": "1.2.0",
"eslint-plugin-cypress": "3.6.0",
"eslint-plugin-import": "2.32.0",
"globals": "15.15.0",
"prettier": "3.8.3"
}
}
45 changes: 45 additions & 0 deletions assets/templates/playwright/playwright.atk.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
/*
* Automated Testing Kit configuration.
*/
var currentEnv = process.env.TERMINUS_ENV || 'local';
var runOnPantheon = false;
if (currentEnv != 'local') {
runOnPantheon = true;
}

module.exports = {
operatingMode: "native",
drushCmd: "__DRUSH_CMD__",
articleAddUrl: 'node/add/article',
contactUsUrl: "form/contact",
logInUrl: "user/login",
logOutUrl: "user/logout",
imageAddUrl: 'media/add/image',
mediaDeleteUrl: 'media/{mid}/delete',
mediaEditUrl: 'media/{mid}/edit',
mediaList: 'admin/content/media',
menuAddUrl: 'admin/structure/menu/manage/main/add',
menuDeleteUrl: 'admin/structure/menu/item/{mid}/delete',
menuEditUrl: 'admin/structure/menu/item/{mid}/edit',
menuListUrl: 'admin/structure/menu/manage/main',
nodeDeleteUrl: 'node/{nid}/delete',
nodeEditUrl: 'node/{nid}/edit',
pageAddUrl: 'node/add/page',
registerUrl: "user/register",
resetPasswordUrl: "user/password",
termAddUrl: 'admin/structure/taxonomy/manage/terms/add',
termEditUrl: 'taxonomy/term/{tid}/edit',
termDeleteUrl: 'taxonomy/term/{tid}/delete',
termListUrl: 'admin/structure/taxonomy/manage/terms/overview',
termViewUrl: 'taxonomy/term/{tid}',
xmlSitemapUrl: 'admin/config/search/simplesitemap',
authDir: "tests/support",
dataDir: "tests/data",
supportDir: "tests/support",
testDir: "tests",
pantheon : {
isTarget: runOnPantheon,
site: "__PANTHEON_SITE__",
environment: currentEnv
}
}
Loading