Repository navigation
Release BetterWright 2.9.0: bounded Pi evidence history and browser fixes - #239
Conversation
…errors Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
[Medium impact] No findings remain that require code changes before merging; required release checks remain prerequisites for merge. SummaryReleases BetterWright 2.9.0 with bounded default Pi evidence responses and explicit history reads.
No blocking findings remain in this review. Required cross-platform and full managed-browser/binary checks should pass before merging. Reviews (1) · Last reviewed commit: "Record native Pi evidence measurements a..." · Reviewed by Greptile |
E2E review — PR #239 @
|
| Journey | Result |
|---|---|
lint, typecheck, test:types, check:versions, check:build, setup, doctor |
✅ pass |
test:unit |
hashlib.file_digest missing in scripts/chromium/package-runtime.py) |
browser.test.ts, full managed-browser run (--timeout 120000) |
navigate and read the title: https://example.com is outside this sandbox's network allowlist (ERR_CONNECTION_CLOSED). The Pi visibility/restart/quota-eviction, cookie, MCP browser/download and vault tests all pass |
Pi browser_evidence, real BetterChromium plus a local HTTP fixture (video 1) |
✅ 3 journeys. initialize 908→908→908 bytes, audit 1507→1507→1507; archivedCount replaces archivedChecklists. list paging/nextOffset and rejection of limit 0/101, offset -1/1.5/"5" work. get by exact name works, unknown/empty names are rejected, and reads are side-effect free. Reproof clears the audit. Proof copies are 0700 dir / 0600 file. A visibility restart resets current proofs while archived proofs stay readable. Shutdown removes the copies, and archived get then reports available=false. After the step budget runs out, audit/list/get still work and prove is refused |
| Pi edge cases | ✅ duplicate names (archived/current/whitespace) are rejected; offset ≥ total returns empty; get trims whitespace |
CLI/daemon/Cookie Sync, real dist/bin/betterwright.js (video 2) |
✅ An invalid HEADLESS/DOWNLOAD_POLICY/PROFILE gives an error naming the variable (exit 1). sessions still works with invalid env. Invalid env plus explicit --headed --no-ad-block --profile e2e starts the daemon headed. A relative --source-path resolves from the caller directory (sync from dirB succeeds; the same path from dirA fails while the daemon runs in dirA). An explicit path with --include-session gets the specific rejection |
| UI desktop/mobile/light/dark | N/A (no UI change) |
| Native macOS/Windows artifacts | Blocked locally (Linux x64 only); covered only by CI |
CI: On the first attempt, Cross-platform unit tests (macos-latest, 22) timed out (120 s) in onboard.test.ts › init wires detected hosts, verifies, and reports ready. The test is fully harness-injected and this PR leaves onboard alone. It passed 3/3 locally in about 10 ms and passed on the other five matrix jobs and on main. The re-run job passed, so this is a runner stall, not a PR defect. All required checks are now green.
Not a defect, possible hardening: proof evidence text is accepted as written by the model (a weak proof was accepted). This behaviour predates the PR and is out of its scope.
Gaps: I did not run a stdio MCP client session separately (MCP was covered only by the suite's MCP tests). I did not run release:check/package smoke as a single command, because the native packaging step is blocked as described above. I could not check prior automation sessions for deduplication (session search returned 403); this PR has no earlier E2E comments.
Videos (real desktop captures at this SHA; the webp previews below are cut from the MP4s):
- Video 1, Pi evidence E2E, ALL PASS:
MP4: https://app.devin.ai/api/presigned_proxy?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjpudWxsLCJidWNrZXRfbmFtZSI6ImRldmluYXR0YWNobWVudHMiLCJidWNrZXRfa2V5IjoiYXR0YWNobWVudHNfcHJpdmF0ZS9vcmctYmQ0NTliYTUyOTRiNDEyMGJiYWRhNmE1Y2JmODIyZTUvM2VmZmYwYzAtNGE1Mi00ZWQ2LWFiNTMtMDZjM2FmM2ViYjMyIiwiaWF0IjoxNzkyNDMzNTY3LCJvcmdfaWQiOiJvcmctYmQ0NTliYTUyOTRiNDEyMGJiYWRhNmE1Y2JmODIyZTUiLCJmaWxlbmFtZSI6InBpLWUyZS0yMzkubXA0In0.5mnXqXV-BhZHU671wX28S6-xclT2hL-T_HoEdQOXRpI - Video 2, CLI/daemon/Cookie Sync E2E:
MP4: https://app.devin.ai/api/presigned_proxy?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjpudWxsLCJidWNrZXRfbmFtZSI6ImRldmluYXR0YWNobWVudHMiLCJidWNrZXRfa2V5IjoiYXR0YWNobWVudHNfcHJpdmF0ZS9vcmctYmQ0NTliYTUyOTRiNDEyMGJiYWRhNmE1Y2JmODIyZTUvM2RmNDlkMmQtOTJhZS00ZGZjLTg3OTgtZThhMWI4NjRlZmVjIiwiaWF0IjoxNzkyNDMzNTY3LCJvcmdfaWQiOiJvcmctYmQ0NTliYTUyOTRiNDEyMGJiYWRhNmE1Y2JmODIyZTUiLCJmaWxlbmFtZSI6ImNsaS1lMmUtMjM5Lm1wNCJ9.4GoPai-oLas7JOR6_7zl3jqF8C0sqWcgfAngNggGilo
Limitation: GitHub has no supported direct MP4 upload here. The MP4 links are Devin attachments and require Devin sign-in; the webp previews above are public.
|
I found no confirmed defects in PR #239 at What I ran: the real build and CLI, the daemon, Cookie Sync, and the Pi
Failures and CI:
What I couldn't do:
This response includes attachments — view them in Devin |
Pi's evidence responses grew with every completed journey because each tool call repeated the full archive, and a proof call returned both pre-proof and post-proof state. This release keeps default responses independent of past evidence volume and adds explicit, read-only history retrieval.
browser_evidence listpages journey summaries (default 20, maximum 100);getretrieves full evidence by exact unique name. Default responses replacearchivedChecklistswitharchivedCountwhile preserving current constraints, pending items, warnings, status, and proof images.includeSessionvalidation. The other two fix: Cookie Sync path and launch-default issues found in E2E of #235 #237 fixes already shipped on main in 2.8.9 and are retained. The outdated maintenance-branch PRs will be closed as superseded after this lands.__daemonuses its resolved stdin configuration instead of revalidating overridden environment values. Management commands remain usable with invalid unrelated launch defaults. Regression coverage exercises an actual spawned daemon with invalid raw env and valid explicit stdin settings.Version 2.9.0 reflects the added history operations and model-facing response-format migration. SETUP.md documents activation, compatibility, artifact lifetime, cleanup, and the lack of cross-process recovery. Explicit cookie paths are documented as plaintext-capable; encrypted Chromium profiles require a registered browser source for credential selection.
Validation:
0c4127cb84c3ab9431f8084b9bbab978054a3924: no blocking correctness, security, or regression findings. All first-review findings independently verified as fixed.release:checkpassed against final0c4127c: 1,361 passing tests, 3 intentional live-service skips, 0 failures; lint, implementation/example/tool typechecks, build integrity, public declaration checks, version alignment, and package smoke test all passed..audit/pi-evidence-2.9.0.tsvand repeatableresearch/pi-evidence-history-probe.ts; default initialization stays at 937 UTF-8 bytes through three journeys instead of 885 → 1,341 → 1,795. Deterministic coverage checks 120 archived journeys with 2,048-byte evidence.Closes #238.
CI attempt 1: full managed-browser and binary E2E, package integrity, dependency audit, both Linux and Windows cells, and one macOS cell passed. The other identical macOS cell timed out in the unchanged onboarding test
init wires detected hosts, verifies, and reports ready. That complete test file passes locally (26/26); only the failed job was rerun and passed. All nine required CI jobs and the Greptile review are green on the reviewed head. No checks were bypassed.