Skip to content

feat(computer): WebsiteBrowserEnvironment, the base class for website browsers - #449

Merged
earakely-scale merged 6 commits into
mainfrom
edgararakelyan/fd-4937-m58-websitebrowserenvironment-the-base-class-for-website
Nov 2, 2026
Merged

earakely-scale merged 6 commits into
mainfrom
edgararakelyan/fd-4937-m58-websitebrowserenvironment-the-base-class-for-website

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

What and why

This adds the base class every website browser subclasses, agentenv_protocol.website_browser.WebsiteBrowserEnvironment. It is M5.8 of the website browser design (FD-4937), built to the signed-off design note. A subclass says how its build installs and starts. agentenv owns the rest, the same inside a computer and alone:

@computer_app(supports=[{"os": {"type": "linux"}, "arch": ["amd64", "arm64"]}])
@linux(packages=..., downloads=..., launch=Launch(command=["chromium/chrome/chrome", "--no-sandbox"],
                                                  window={"class": "Chromium"}, ready=Ready(window=True)))
class Chromium(WebsiteBrowserEnvironment):
    version = "156.0.8078.4"
    policy_dir = Path("/etc/opt/chrome_for_testing/policies/managed")

What the base class adds:

  • Flags after the subclass's command, then start_urls: the profile (--user-data-dir=~/<profile>), --remote-debugging-pipe, --no-first-run, --no-default-browser-check and --lang.

  • The site proxy, when AGENTENV_SITE_PROXY names one, with WebRTC's non-proxied UDP off:

    • Alone, loopback goes through the proxy too (<-loopback>), and the browser refuses to start without a proxy.
    • In a computer, loopback is bypassed, so the apps beside it stay reachable, while *.localhost names still go through.
  • Alone (AGENTENV_BROWSER_ALONE=1), it also takes --window-size and --headless=new, except on the open web (AGENTENV_BROWSER_OPEN_WEB=1).

  • Owned flags are refused. declaration() refuses a subclass command that carries any of these, --remote-allow-origins, or any other way to open a debugging port.

  • BrowserSettings:

    Field Default Notes
    start_urls ["about:blank"] None may start with -, so env_params can't smuggle in a flag
    window 1280 × 800 Alone only
    locale en-US Passed as --lang
    time_zone UTC Passed as TZ in the launch's env
  • write_policies, an install step: agentenv's unattended-use policies, then the subclass's policies over them, written to policy_dir.

  • A debugging pipe, not a port. The app server launches Chromium with a new pipe's ends as its fds 3 and 4, and keeps the other ends as self.devtools, a DevToolsPipe. await self.devtools.call(method, params) returns the result, or raises DevToolsError.

    • No port listens and no DevToolsActivePort is written. So nothing else in a computer, such as an agent that opens a terminal, can drive the browser over raw CDP around the way in its task allows, the action log, or context scoping.
    • The ready check, devtools_answers, asks Browser.getVersion over the pipe.
    • Headless Chromium takes only one start URL on its command line, so the rest open over the pipe, behind the first.
    • This was decision 7, changed after the design note was first signed off (the note is updated). Agents still get a token-bearing address from the front door (M8.6), whose relay will multiplex their connections onto the pipe. On the VM, a prototype of that relay served two Playwright connect_over_cdp clients at once and refused a wrong token.
  • browser/v1 on the card, params only: chromium, clock: "real" and, alone, window.

  • The skill: sites by their real names, ~/Uploads and ~/Downloads.

Changes to the app SDK (computer/app.py, computer/app_runtime.py, computer/session.py, and the new computer/_with_fds.py). Every other app is unchanged.

  • ComputerAppEnvironment.__init__ sets self.launch = self._launch_for(...), so the launch op, the ready timeout, the card and the runtime all see one launch. _on_desktop() tells the runtime whether the app has a desktop. Both hooks are private.
  • Launch.env, optional and None by default. Declarations are stored with exclude_none, so existing apps store exactly what they did, and older readers still parse them.
  • AppRuntime.start waits for X only when DISPLAY is set, and writes the menu launcher only on a desktop.
  • session.launch takes fds, moved into place by computer/_with_fds.py, run by path with -I -S so it loads only the standard library. The server runs no Python between fork and exec. AppRuntime asks the app for them before each launch (_launch_fds) and closes its copies after.
  • With relaunch_on_start, a restarted server stops a boot app it finds running and launches it again. That's for an app only the launching server can talk to, as a browser on its pipe.

Not in this PR:

  • the control connection (M5.10);
  • browser/v1's operations (M5.11);
  • start tabs (M5.12);
  • the live data plane (M5.13);
  • the browser provider (M8.10).

For M5.9: the shipped chromium launches through chromium.py as a wrapper script, and it passes its own --user-data-dir. It has to launch Chrome directly and drop those flags before moving onto this class, or declaration() refuses it.

The CUA session reviewed the design note against main (4 points folded in) and has nothing in progress in the SDK files this touches.

How it was tested

  • ruff check: clean.
  • Protocol tests: 971 passed (packages/agentenv-protocol/tests). That includes 21 new ones in test_website_browser.py, which cover:
    • the flags in each mode;
    • owned flags refused at declaration;
    • settings, TZ, and a start URL that is a flag;
    • no display wait and no launcher alone;
    • the alone launch keeping its port and command checks and timeout;
    • a fake browser that answers CDP on fds 3 and 4, launched through the real session.launch and _with_fds: a new pipe at each launch, the ready check going false when it stops and true when relaunched, and the headless start tabs opened over the pipe;
    • a restarted server relaunching a browser and leaving any other app alone;
    • policies, card params and skill;
    • an existing app's launch and stored declaration unchanged.
  • CI on the previous head (c4393b7), before the pipe: all green. CodeBuild needed one retry for a flaky conformance unit test in the computer driver, which this PR doesn't touch.
  • Integration, in a computer (tst/integration/computer/website_browser_test.py, slow tier), with a fixture browser bundle on the shipped Chrome for Testing 156 build. Run on an agent-environments-admin VM:
    • gate 3 makes the arch ready, with the check over the pipe part of it;
    • Chromium's argv is the fixture's command, then agentenv's flags with --remote-debugging-pipe and --lang=fr-FR, then the start tabs;
    • no TCP socket listens in Chromium's session, and there is no DevToolsActivePort;
    • the first start tab reports Europe/Paris;
    • a loopback page loads through the computer's own keys;
    • the policies are where Chrome for Testing reads them;
    • the card's browser/v1 params are right;
    • data/reset relaunches it with the profile template, on a new pipe the app's readiness answers over.
    • On the same VM, the existing chromium_app_test.py (2 tests) and computer_app_test.py (3) still pass.
  • E2E alone, in plain containers on the same VM: 14 of 14 checks passed, with the real build and a stub proxy that logs each request:
    • Alone:
      • ready over the pipe, and the card's params are right;
      • headless at 1280 × 800, with TZ set and no DISPLAY;
      • no listening socket in the browser's session, no DevToolsActivePort, and no launcher;
      • both start tabs (http://site.localhost/ and http://127.0.0.1:8765/) go through the proxy, and nothing reaches the loopback server directly.
    • Server restart: the app server is killed and started again. It stops the old browser, launches a new one, and is ready over the new pipe.
    • As in a computer (Xvfb):
      • headful on the pipe, and the launcher is written;
      • site.localhost goes through the proxy and loopback is reached directly.
    • Without a proxy: the server refuses to start and names AGENTENV_SITE_PROXY.
    • The in-computer proxy case is checked here rather than in a composed computer. Nothing sets AGENTENV_SITE_PROXY there until M5.5/M5.7.
    • The harness stays out of this PR.
  • Port versus pipe: on the VM, the same socket check found one listener and a DevToolsActivePort in port mode, and none of either on the pipe.
  • Policies are applied: chrome://policy, read over the pipe, lists agentenv's policies.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The latest changes appear safe to merge; both numbered previous findings are fixed.

Summary

Adds WebsiteBrowserEnvironment with browser settings, managed policies, proxy flags, and a debugging pipe.

  • Website browser builds share settings and launch behavior inside a computer or alone.
  • App servers can launch processes with custom environments and inherited file handles.

Diagram

sequenceDiagram
    participant Server as App server
    participant Runtime as AppRuntime
    participant Session as session.launch
    participant Browser as Chromium
    Server->>Runtime: Start or recover browser
    Runtime->>Server: Request new debugging pipe
    Runtime->>Session: Launch with pipe ends
    Session->>Browser: Execute with fds 3 and 4
    Server->>Browser: Browser.getVersion over pipe
    Browser-->>Server: Version response
Loading

Reviews (5) · Last reviewed commit: "fix(computer): a restarted server relaun..." · Reviewed by Greptile

earakely-scale and others added 2 commits November 2, 2026 09:58
… browsers

A website browser subclasses agentenv_protocol.website_browser's
WebsiteBrowserEnvironment, a computer app, and agentenv owns the rest,
inside a computer and alone:

- The flags after the subclass's command: its profile, the debugging port
  on loopback, no first run or default-browser check, --lang, and the site
  proxy when AGENTENV_SITE_PROXY names one. Alone, loopback goes through the
  proxy too, and the browser refuses to start without one. In a computer,
  loopback is bypassed so the apps beside it stay reachable. Alone it also
  runs headless (headful on the open web) at its window size.
  declaration() refuses a subclass command that carries any of these, or
  --remote-allow-origins.
- BrowserSettings (start_urls, window, locale, time_zone, as TZ), the
  write_policies install step, @cdp with Chromium's DevToolsActivePort
  default and a ready check on that address, browser/v1 params on the card,
  and the skill.

The app SDK gains what a browser alone needs, with every other app as
before: ComputerAppEnvironment's _launch_for and _on_desktop hooks; an
optional Launch.env, left out of stored declarations when unset; and an
AppRuntime that waits for X only when DISPLAY is set and adds the menu
launcher only on a desktop.

A fixture browser bundle tests it in a computer on the shipped Chrome for
Testing build.

FD-4937

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ns on the desktop

- Its argv is split on whitespace, since Chromium rewrites its process title
  as one line.
- The time zone is read from a page, Intl's, since that rewrite also
  clobbers /proc/<pid>/environ.
- The loopback pages are served by the script that opens them, and its tab
  is brought to the front: on the desktop a background tab doesn't finish
  loading.

FD-4937

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale requested a review from a team as a code owner November 2, 2026 19:11
earakely-scale and others added 2 commits November 2, 2026 11:21
… ready waits on

Alone, the launch dropped all of its ready, not just the window wait, so a
subclass that also waits on a port or a command lost those checks. It now
drops only the window, and the runtime takes the ready timeout from the
launch as declared, so the timeout holds even when nothing else is left to
wait on. The ready check's wait on the debugging address is
devtools_timeout_s.

FD-4937

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Chromium now runs with --remote-debugging-pipe instead of
--remote-debugging-port=0. Its app server launches it with a new pipe's
ends as fds 3 and 4 and keeps the other ends as self.devtools, a
DevToolsPipe. No port listens and no DevToolsActivePort is written, so
nothing else in a computer, such as an agent with a terminal, can drive the
browser over raw CDP and bypass the way in its task allows. The ready check
asks Browser.getVersion over the pipe. @cdp and devtools_address() are
gone, since there is no address. Agents get theirs from the front door
(M8.6).

- session.launch takes fds, moved into place by
  agentenv_protocol.computer._with_fds, so the server runs no Python between
  fork and exec. AppRuntime asks the app for them before each launch
  (_launch_fds) and closes its copies after.
- relaunch_on_start: a restarted server stops a browser it finds running
  and launches it again, since only the launching server holds the pipe.
- Headless Chromium takes one start URL on its command line, so it opens
  the rest over the pipe, behind the first, and is ready once they're open.

FD-4937

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
earakely-scale and others added 2 commits November 2, 2026 12:23
…rd library

python -m agentenv_protocol.computer._with_fds imported the protocol
package and the whole computer SDK, pydantic included, in the child before
it exec'd the app, with the pipe fds open. Each launch was slower, and a
failed import there broke it. session.launch now runs _with_fds.py by path
with -I -S: about 64 ms here, against about 236 ms for importing the
computer package. The helper also tolerates no moves.

FD-4937

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mand too

With relaunch_on_start, an app that was launched since boot, at boot or on
demand, is stopped and launched again when its server restarts, so a
browser launched on demand gets a new pipe instead of being stranded. One
that was never launched stays unlaunched. DevToolsPipe's default wait is
the named call_timeout_s.

FD-4937

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale merged commit 549cbd5 into main Nov 2, 2026
25 checks passed
@earakely-scale
earakely-scale deleted the edgararakelyan/fd-4937-m58-websitebrowserenvironment-the-base-class-for-website branch November 2, 2026 21:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant