Skip to content

Release BetterWright 2.9.0: bounded Pi evidence history and browser fixes - #239

Merged
SSHdotCodes merged 5 commits into
mainfrom
release/2.9.0
Oct 19, 2026
Merged

SSHdotCodes merged 5 commits into
mainfrom
release/2.9.0

Conversation

@SSHdotCodes

@SSHdotCodes SSHdotCodes commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Pi's evidence responses grew with every completed journey because each tool call repeated the full archive, and a proof call returned both pre-proof and post-proof state. This release keeps default responses independent of past evidence volume and adds explicit, read-only history retrieval.

  • browser_evidence list pages journey summaries (default 20, maximum 100); get retrieves full evidence by exact unique name. Default responses replace archivedChecklists with archivedCount while preserving current constraints, pending items, warnings, status, and proof images.
  • Accepted proofs receive private session copies outside the worker quota, linked to the original artifacts. Reads check copy availability, shutdown removes copies, and visibility restarts invalidate current proofs while keeping archived history accessible.
  • Missing, unreadable, empty, oversized, or unretainable proof screenshots cannot mark pending requirements complete. Reproof clears the previous audit. Evidence audit/history remains accessible after the browser step budget.
  • Incorporates fix(cli): name the launch variable in invalid launch-default startup errors #236 and the missing first two commits from fix: Cookie Sync path and launch-default issues found in E2E of #235 #237, preserving their author attribution: correct launch-variable errors, caller-relative cookie paths, and specific includeSession validation. The other two fix: Cookie Sync path and launch-default issues found in E2E of #235 #237 fixes already shipped on main in 2.8.9 and are retained. The outdated maintenance-branch PRs will be closed as superseded after this lands.
  • Daemon startup preserves explicit overrides: __daemon uses its resolved stdin configuration instead of revalidating overridden environment values. Management commands remain usable with invalid unrelated launch defaults. Regression coverage exercises an actual spawned daemon with invalid raw env and valid explicit stdin settings.

Version 2.9.0 reflects the added history operations and model-facing response-format migration. SETUP.md documents activation, compatibility, artifact lifetime, cleanup, and the lack of cross-process recovery. Explicit cookie paths are documented as plaintext-capable; encrypted Chromium profiles require a registered browser source for credential selection.

Validation:

  • Independent GPT-6-Astra final review of 0c4127cb84c3ab9431f8084b9bbab978054a3924: no blocking correctness, security, or regression findings. All first-review findings independently verified as fixed.
  • Pinned Bun 1.4.0 release:check passed against final 0c4127c: 1,361 passing tests, 3 intentional live-service skips, 0 failures; lint, implementation/example/tool typechecks, build integrity, public declaration checks, version alignment, and package smoke test all passed.
  • Native managed-browser Pi visibility/restart/quota tests: 3 passed, 0 failed. Real worker quota eviction removed the original proof while retained archive evidence stayed readable.
  • Recorded native before/after calls in .audit/pi-evidence-2.9.0.tsv and repeatable research/pi-evidence-history-probe.ts; default initialization stays at 937 UTF-8 bytes through three journeys instead of 885 → 1,341 → 1,795. Deterministic coverage checks 120 archived journeys with 2,048-byte evidence.
  • Full managed-browser/binary E2E and all required cross-platform CI checks must pass before merge.

Closes #238.

CI attempt 1: full managed-browser and binary E2E, package integrity, dependency audit, both Linux and Windows cells, and one macOS cell passed. The other identical macOS cell timed out in the unchanged onboarding test init wires detected hosts, verifies, and reports ready. That complete test file passes locally (26/26); only the failed job was rerun and passed. All nine required CI jobs and the Greptile review are green on the reviewed head. No checks were bypassed.

devin-ai-integration Bot and others added 5 commits October 19, 2026 13:42
…errors

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@greptile-apps

greptile-apps Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] No findings remain that require code changes before merging; required release checks remain prerequisites for merge.

Summary

Releases BetterWright 2.9.0 with bounded default Pi evidence responses and explicit history reads.

  • Pi agents can read journey history without growing every reply.
  • Cookie Sync resolves caller paths and rejects unsupported session imports.
  • CLI startup uses resolved settings and names invalid launch defaults.

No blocking findings remain in this review. Required cross-platform and full managed-browser/binary checks should pass before merging.

Reviews (1) · Last reviewed commit: "Record native Pi evidence measurements a..." · Reviewed by Greptile

@SSHdotCodes
SSHdotCodes merged commit bed7d46 into main Oct 19, 2026
18 of 19 checks passed
@devin-ai-integration

Copy link
Copy Markdown
Contributor

E2E review — PR #239 @ 0c4127cb84c3ab9431f8084b9bbab978054a3924

Note: this PR was merged while the review was running. The findings below cover the merged head 0c4127c.

Verdict: I found no confirmed defects at this SHA, so I opened no fix PR. Two items stay blocked by the environment (listed below), and one required CI job failed on a test this PR does not touch.

Environment: Ubuntu x86_64 (kernel 6.8), Bun 1.4.0, BetterChromium 153.0.8010.36, playwright-core/patchright-core 1.61.1, Python 3.10.12. Base 4ddb510. Disposable local fixtures only, with no credentials and no saved passwords.

Journey Result
lint, typecheck, test:types, check:versions, check:build, setup, doctor ✅ pass
test:unit ⚠️ 1356 pass / 6 skip / 2 fail. Both failures are native archive packaging tests, blocked by Python 3.10 (hashlib.file_digest missing in scripts/chromium/package-runtime.py)
browser.test.ts, full managed-browser run (--timeout 120000) ⚠️ 131/132. The one failure is navigate and read the title: https://example.com is outside this sandbox's network allowlist (ERR_CONNECTION_CLOSED). The Pi visibility/restart/quota-eviction, cookie, MCP browser/download and vault tests all pass
Pi browser_evidence, real BetterChromium plus a local HTTP fixture (video 1) ✅ 3 journeys. initialize 908→908→908 bytes, audit 1507→1507→1507; archivedCount replaces archivedChecklists. list paging/nextOffset and rejection of limit 0/101, offset -1/1.5/"5" work. get by exact name works, unknown/empty names are rejected, and reads are side-effect free. Reproof clears the audit. Proof copies are 0700 dir / 0600 file. A visibility restart resets current proofs while archived proofs stay readable. Shutdown removes the copies, and archived get then reports available=false. After the step budget runs out, audit/list/get still work and prove is refused
Pi edge cases ✅ duplicate names (archived/current/whitespace) are rejected; offset ≥ total returns empty; get trims whitespace
CLI/daemon/Cookie Sync, real dist/bin/betterwright.js (video 2) ✅ An invalid HEADLESS/DOWNLOAD_POLICY/PROFILE gives an error naming the variable (exit 1). sessions still works with invalid env. Invalid env plus explicit --headed --no-ad-block --profile e2e starts the daemon headed. A relative --source-path resolves from the caller directory (sync from dirB succeeds; the same path from dirA fails while the daemon runs in dirA). An explicit path with --include-session gets the specific rejection
UI desktop/mobile/light/dark N/A (no UI change)
Native macOS/Windows artifacts Blocked locally (Linux x64 only); covered only by CI

CI: On the first attempt, Cross-platform unit tests (macos-latest, 22) timed out (120 s) in onboard.test.ts › init wires detected hosts, verifies, and reports ready. The test is fully harness-injected and this PR leaves onboard alone. It passed 3/3 locally in about 10 ms and passed on the other five matrix jobs and on main. The re-run job passed, so this is a runner stall, not a PR defect. All required checks are now green.

Not a defect, possible hardening: proof evidence text is accepted as written by the model (a weak proof was accepted). This behaviour predates the PR and is out of its scope.

Gaps: I did not run a stdio MCP client session separately (MCP was covered only by the suite's MCP tests). I did not run release:check/package smoke as a single command, because the native packaging step is blocked as described above. I could not check prior automation sessions for deduplication (session search returned 403); this PR has no earlier E2E comments.

Videos (real desktop captures at this SHA; the webp previews below are cut from the MP4s):

Limitation: GitHub has no supported direct MP4 upload here. The MP4 links are Devin attachments and require Devin sign-in; the webp previews above are public.

Written by Devin

@devin-ai-integration

Copy link
Copy Markdown
Contributor

I found no confirmed defects in PR #239 at 0c4127c, so I opened no fix PR. Note that the PR was merged while the review was running, but the merged head is still 0c4127c, so the results apply to what landed. Report: #239 (comment)

What I ran: the real build and CLI, the daemon, Cookie Sync, and the Pi browser_evidence flow on BetterChromium against disposable local fixtures, with no credentials. Two short MP4 recordings are attached.

  • Static checks: lint, typecheck, type tests, version checks and build checks all passed.
  • Unit suite: 1356 pass, 6 skip, 2 fail.
  • Full managed-browser suite: 131 of 132 pass.
  • Real flows: both recorded flows passed every check. That covers bounded response sizes, list/get paging and errors, proof file permissions, visibility restart, shutdown cleanup, step budget, launch-variable errors, caller-relative cookie paths and the includeSession rejection.

Failures and CI:

  • Unit suite failures: both are native packaging tests. They need hashlib.file_digest, which this box's Python 3.10.12 doesn't have.
  • Browser suite failure: navigate and read the title loads https://example.com, which isn't on this session's network allowlist. I didn't request access because that code path isn't part of this PR.
  • CI: the first-attempt macOS 22 timeout was in an onboarding test the PR doesn't touch. It passed 3 of 3 locally, passed on main and the other matrix jobs, and passed on re-run; all checks are green now.

What I couldn't do:

  • I didn't run a standalone MCP client; MCP was covered only by the suite's MCP tests.
  • I didn't run release:check as one command, because its packaging step hits the same Python issue.
  • Native macOS and Windows builds weren't checked here (this box is Linux x64 only).
  • Checking earlier automation runs for duplicates was blocked: session search returned 403. No earlier review comment exists on the PR.
  • I couldn't upload the MP4s to GitHub directly. The comment embeds public webp previews and links the MP4s as Devin attachments, which need Devin sign-in.
  • I confirmed the PR page and that the comment exists. I couldn't open the media links from here because app.devin.ai isn't on the allowlist.

This response includes attachments — view them in Devin

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[feature] Pi: bound default evidence responses and retrieve archived checklists on demand

1 participant