From afac0836a4742a4b5abd72e4b04a918201bc95f4 Mon Sep 17 00:00:00 2001 From: Edgar Arakelyan Date: Mon, 2 Nov 2026 23:46:49 -0800 Subject: [PATCH 1/5] feat(computer): the front door of a website browser running alone python -m agentenv_protocol.computer.front_door : [--site-proxy COMMAND] runs from the app's venv as its container's command. It runs the app's server and the site proxy, starting each again whenever it ends (0.5 s, then longer while it keeps ending soon, up to 30 s). On :8000 it serves the app's card, with no interface for agents and with computer-ready/v1, whose status is the app's readiness and the site proxy's, and failed for good once the app's launch log has an exit, since nothing alone launches it again. Every other path goes on to the app's server, its answer streamed back, behind the control token, which no forwarded request and none of its programs is given. computer-ready/v1's URN moves to operations, beside ReadyCheck, so the front door runs on the app extra; card still exports it. The browser's record reads each tab's URL and title from the browser when it's read and when its page loads, since Chrome sends no event when a page's title changes. Co-Authored-By: Claude Opus 5.5 --- packages/agentenv-protocol/README.md | 25 +- .../agentenv_protocol/computer/__init__.py | 4 +- .../src/agentenv_protocol/computer/apps.py | 6 +- .../src/agentenv_protocol/computer/card.py | 2 +- .../computer/conformance/checks.py | 2 +- .../agentenv_protocol/computer/extensions.py | 3 +- .../agentenv_protocol/computer/front_door.py | 250 +++++++++++++ .../agentenv_protocol/computer/operations.py | 4 + .../src/agentenv_protocol/website_browser.py | 21 +- .../tests/test_computer_front_door.py | 344 ++++++++++++++++++ .../tests/test_website_browser.py | 25 +- tst/integration/computer/front_door_test.py | 256 +++++++++++++ 12 files changed, 926 insertions(+), 16 deletions(-) create mode 100644 packages/agentenv-protocol/src/agentenv_protocol/computer/front_door.py create mode 100644 packages/agentenv-protocol/tests/test_computer_front_door.py create mode 100644 tst/integration/computer/front_door_test.py diff --git a/packages/agentenv-protocol/README.md b/packages/agentenv-protocol/README.md index 93779fc3a..4cbda4ff8 100644 --- a/packages/agentenv-protocol/README.md +++ b/packages/agentenv-protocol/README.md @@ -511,9 +511,9 @@ driver has them, through the `served_tools`, `served_extensions` and `served_ope class's `notes`, markdown for what its card can't say. A driver whose software ships a skill of its own returns it from `bundled_skill()` (its description, body and other files), and the computer skill leads with it, the computer's part following. -- **Framework extensions.** `computer-ready/v1` (`status`: real checks of the display and a +- **Framework extensions.** `computer-ready/v1` (`status`: `{ready, checks}`, real checks of the display and a capture, the driver's own `ready_checks()`, and one per app on the computer, asked of the app's - own server (see Computer apps); tools refuse `device_unavailable` until they pass once), `computer-display/v1`, `computer-spaces/v1` (the default space, written into the tool + own server (see Computer apps), with `failed` too from an app's front door once it can't be ready again; tools refuse `device_unavailable` until they pass once), `computer-display/v1`, `computer-spaces/v1` (the default space, written into the tool descriptions), `computer-keyboard/v1`, `computer-exec/v1` (`run {argv, cwd?, env?, timeout_s?, request_id}`: output capped per stream, a repeated `request_id` answered with the first result) and, with `recording`, `computer-recording/v1` (below). @@ -741,6 +741,21 @@ hook runs in a thread, which the reset and relaunch after it wait for. The drive its card lists each one's card as a child environment, rooted at `/svc//`, where it forwards requests to the app's server. +An app that runs alone, with no computer around it, as a website browser does, has a front door in the +driver's place: `python -m agentenv_protocol.computer.front_door : +[--site-proxy COMMAND] [--port 8000]`, run from the app's venv as its container's command. It runs the +app's server in ``, and the site proxy command when it's given one, and starts each again +whenever it ends, after 0.5 s, then longer each time it ends within a minute of starting, up to 30 s. +On its port it serves the app's card, with no interface for agents, since nothing alone serves them +yet, and with `computer-ready/v1`, whose `status` (GET) is `{ready, checks, failed?}`: one check for +the app, its own readiness or why its server isn't answering, and one for whether the site proxy runs. +Once the app's launch log has an `exit`, the app ended on its own, which nothing that runs it alone +launches it again after, so `status` says `failed`, with when and the code or signal, for good, even +once its server restarts and launches it again. Every other request goes on to the app's server at its +path, its answer streamed back. Everything but the card takes the control token (`CONTROL_TOKEN_ENV`), +which the front door takes out of each request it sends on and out of the environment of each program +it runs. + `MailApp.declaration()` returns what the class declares, an `AppDeclaration`, which a `computer_app` env stores, and `python -m agentenv_protocol.computer._declare ` prints it as JSON, or a driver's `DriverDeclaration`, without the class's own dependencies installed. @@ -791,7 +806,9 @@ are the profile's template. The base class adds: `devtools_answers` asks `Browser.getVersion` over the pipe. - **Its control connection**, `self.control`, a `BrowserControl` over the pipe. At each launch it attaches to every page with flattened sessions and keeps a record, `since` its server started: open tabs in - opened order, the one whose page has focus `active`; `closed_tabs`, each with `closed_at` and + opened order, each with its URL and title as the browser has them when the record is read (Chrome sends no + event for a title), the one whose page has focus `active`; `closed_tabs`, each as of its page's last load, + with `closed_at` and `closed_by` (`agentenv`, over this connection; `unknown`, anyone else; `exit` or `stop`, when the browser ended on its own or its server stopped it); `downloads`, into `~/Downloads`, each with its state and, completed, its size and sha256; and `dialogs`, each with its type, message, answer and who gave it. @@ -801,7 +818,7 @@ are the profile's template. The base class adds: launch (with its cause: `boot`, `server_restart`, `launch`, `data/add` or `data/reset`), each stop it made and each exit it didn't, with its exit `code` or the `signal` that ended it, since the computer booted. Nothing relaunches a browser that exits on its own: in a computer the agent can, and alone it's an infra - failure, its ready check false. + failure, its ready check false and its front door's `computer-ready/v1` status `failed`. - **Its skill**: sites by their real names, files to upload in `~/Uploads` and downloads in `~/Downloads`, folders it makes. diff --git a/packages/agentenv-protocol/src/agentenv_protocol/computer/__init__.py b/packages/agentenv-protocol/src/agentenv_protocol/computer/__init__.py index 586e962cc..fdcc2cec2 100644 --- a/packages/agentenv-protocol/src/agentenv_protocol/computer/__init__.py +++ b/packages/agentenv-protocol/src/agentenv_protocol/computer/__init__.py @@ -44,6 +44,7 @@ Unsupported, ) from .operations import ( + COMPUTER_READY, Display, Effect, ExecResult, @@ -80,7 +81,6 @@ COMPUTER_EXEC, COMPUTER_KEYBOARD, COMPUTER_POINTER, - COMPUTER_READY, COMPUTER_RECORDING, COMPUTER_SPACES, ) @@ -90,7 +90,7 @@ # The names whose modules need the computer extra (mcp, av and Pillow), by module. _ON_USE = { "driver": ("ComputerDriverEnvironment", "computer_driver", "DEFAULT_IDENTITY", "DriverDeclaration"), - "card": ("COMPUTER", "COMPUTER_READY", "COMPUTER_DISPLAY", "COMPUTER_SPACES", "COMPUTER_POINTER", + "card": ("COMPUTER", "COMPUTER_DISPLAY", "COMPUTER_SPACES", "COMPUTER_POINTER", "COMPUTER_KEYBOARD", "COMPUTER_EXEC", "COMPUTER_RECORDING"), "recording": ("Recording", "RecordHints", "RecordedFrame", "RecordedSegment", "frames_in_thread"), } diff --git a/packages/agentenv-protocol/src/agentenv_protocol/computer/apps.py b/packages/agentenv-protocol/src/agentenv_protocol/computer/apps.py index 9de569ed5..fd6275617 100644 --- a/packages/agentenv-protocol/src/agentenv_protocol/computer/apps.py +++ b/packages/agentenv-protocol/src/agentenv_protocol/computer/apps.py @@ -54,7 +54,7 @@ async def check(folder: Path) -> ReadyCheck: async def children(folders: list[Path]) -> list[EnvironmentCard]: """Each app in ``folders`` as a child environment of its computer: its server's card, rooted at ``/svc//``, or, while its server isn't answering, its name and route alone.""" - cards = await asyncio.gather(*(_card(folder) for folder in folders)) + cards = await asyncio.gather(*(served_card(folder) for folder in folders)) return [EnvironmentCard.model_validate(child_card(folder.name, card)) if card is not None else EnvironmentCard(name=folder.name, url=f"/svc/{folder.name}{RPC_PATH}") for folder, card in zip(folders, cards)] @@ -85,7 +85,7 @@ def rooted(path: object) -> object: async def owned_paths(folders: list[Path]) -> list[str]: """The paths under the session user's home that the apps in ``folders`` keep their data in, as their cards say.""" - cards = await asyncio.gather(*(_card(folder) for folder in folders)) + cards = await asyncio.gather(*(served_card(folder) for folder in folders)) return [path for card in cards if card is not None for path in client.extension_params(card, COMPUTER_APP).get("paths") or []] @@ -104,7 +104,7 @@ def endpoint(card: dict, method: str) -> str: return client.find_extension_method(card, COMPUTER_APP, method)["endpoint"] -async def _card(folder: Path) -> Optional[dict]: +async def served_card(folder: Path) -> Optional[dict]: """The card the app in ``folder`` serves, or None while its server isn't answering.""" try: async with _client(folder / SOCKET) as http: diff --git a/packages/agentenv-protocol/src/agentenv_protocol/computer/card.py b/packages/agentenv-protocol/src/agentenv_protocol/computer/card.py index a8453b7e2..027b853f8 100644 --- a/packages/agentenv-protocol/src/agentenv_protocol/computer/card.py +++ b/packages/agentenv-protocol/src/agentenv_protocol/computer/card.py @@ -6,6 +6,7 @@ from typing import TYPE_CHECKING from .dataplane import DEFAULT_DATA_PATHS +from .operations import COMPUTER_READY # noqa: F401 from .recording import recording_params from ..types import METHOD_ADD, EnvironmentExtension, EnvironmentTool, IntakeDeclaration, IntakeFormat, intake_extension @@ -13,7 +14,6 @@ from .driver import ComputerDriverEnvironment COMPUTER = "urn:agentenv:computer/v1" -COMPUTER_READY = "urn:agentenv:computer-ready/v1" COMPUTER_DISPLAY = "urn:agentenv:computer-display/v1" COMPUTER_SPACES = "urn:agentenv:computer-spaces/v1" COMPUTER_POINTER = "urn:agentenv:computer-pointer/v1" diff --git a/packages/agentenv-protocol/src/agentenv_protocol/computer/conformance/checks.py b/packages/agentenv-protocol/src/agentenv_protocol/computer/conformance/checks.py index ebcf64843..5b0d849b8 100644 --- a/packages/agentenv-protocol/src/agentenv_protocol/computer/conformance/checks.py +++ b/packages/agentenv-protocol/src/agentenv_protocol/computer/conformance/checks.py @@ -28,13 +28,13 @@ COMPUTER_EXEC, COMPUTER_KEYBOARD, COMPUTER_POINTER, - COMPUTER_READY, COMPUTER_RECORDING, COMPUTER_SPACES, GROUP_TOOLS, POINT_TOOLS, ) from ..manifest import RecordingManifest +from ..operations import COMPUTER_READY from ..recording import RECORDINGS_DIR from .client import Driver, image from .mp4 import read_mp4 diff --git a/packages/agentenv-protocol/src/agentenv_protocol/computer/extensions.py b/packages/agentenv-protocol/src/agentenv_protocol/computer/extensions.py index ff12c3b36..0c7ca98b5 100644 --- a/packages/agentenv-protocol/src/agentenv_protocol/computer/extensions.py +++ b/packages/agentenv-protocol/src/agentenv_protocol/computer/extensions.py @@ -10,14 +10,13 @@ COMPUTER_DISPLAY, COMPUTER_EXEC, COMPUTER_KEYBOARD, - COMPUTER_READY, COMPUTER_RECORDING, COMPUTER_SPACES, KEYBOARD_RANGES, MAX_EXEC_TIMEOUT_S, ) from .errors import Internal, InvalidArgument, Unsupported -from .operations import Display, ExecResult +from .operations import COMPUTER_READY, Display, ExecResult from .skill import skill from .spaces import check_space diff --git a/packages/agentenv-protocol/src/agentenv_protocol/computer/front_door.py b/packages/agentenv-protocol/src/agentenv_protocol/computer/front_door.py new file mode 100644 index 000000000..f1198770e --- /dev/null +++ b/packages/agentenv-protocol/src/agentenv_protocol/computer/front_door.py @@ -0,0 +1,250 @@ +"""The front door of a computer app that runs alone, with no computer around it, as a website browser does: +``python -m agentenv_protocol.computer.front_door : [--site-proxy COMMAND]``, run from the app's +venv as its container's command. It runs the app's own server (``serve_app``) in ````, and the site proxy +command when it's given one, and starts each again whenever it ends, after a delay that grows while it keeps ending +soon after it starts. On ``PORT`` it serves the app's card, with ``computer-ready/v1`` added and no interface for +agents, since nothing here serves them yet; ``computer-ready/v1``'s ``status``, which says whether the app is ready, or +``failed`` for good once its launch log says it ended on its own, since nothing launches it again; and every other +request, sent on to the app's server at its path, its answer streamed back. Everything but the card takes the env's +control token (``CONTROL_TOKEN_ENV``), which no request it sends on and none of the programs it runs is given. Needs +the ``app`` extra.""" +from __future__ import annotations + +import argparse +import asyncio +import contextlib +import json +import logging +import os +import shlex +import signal +import sys +import time +from dataclasses import asdict +from pathlib import Path +from typing import Any, AsyncIterator, Optional + +import httpx +import uvicorn +from starlette.applications import Starlette +from starlette.requests import Request +from starlette.responses import Response, StreamingResponse +from starlette.routing import Route + +from ..agent_env_environment import _error_response, _json_response, control_guard +from ..types import CONTROL_TOKEN_ENV, RPC_PATH, WELL_KNOWN_PATH, EnvironmentCard +from . import apps +from .app import SOCKET +from .app_runtime import LAUNCHES +from .errors import DeliveryFailed +from .operations import COMPUTER_READY, ReadyCheck + +logger = logging.getLogger(__name__) + +#: The env's one port, where the front door serves. +PORT = 8000 +#: Where it serves ``computer-ready/v1``'s status, as a driver does. +STATUS_PATH = f"{RPC_PATH}/ext/computer-ready/status" +#: The delays before a program that ended starts again: each in turn while it keeps ending within ``STABLE_S`` of +#: starting, then the last, and the first again once a run has lasted that long. +RESTART_DELAYS_S = (0.5, 1.0, 2.0, 5.0, 10.0, 30.0) +STABLE_S = 60.0 +#: How long a program asked to stop may take before it's killed. +STOP_TIMEOUT_S = 10.0 +#: The headers of one hop, and the control token's, which no request or answer it sends on carries. +UNFORWARDED = frozenset({"authorization", "connection", "host", "keep-alive", "proxy-authenticate", + "proxy-authorization", "te", "trailer", "transfer-encoding", "upgrade"}) +METHODS = ["GET", "POST", "PUT", "PATCH", "DELETE"] +READY_EXTENSION = { + "uri": COMPUTER_READY, + "description": "Whether the app running alone is ready, by its own checks and its front door's, or failed for " + "good, and why.", + "required": False, + "params": {"methods": {"status": {"method": "GET", "endpoint": STATUS_PATH, + "request": {"type": "object", "properties": {}}}}}, +} +USAGE = "python -m agentenv_protocol.computer.front_door : [--site-proxy COMMAND] [--port PORT]" + + +class Program: + """A command the front door runs in ``cwd``, started again whenever it ends, until it's stopped.""" + + def __init__(self, name: str, argv: list[str], cwd: Path) -> None: + self.name, self.argv, self.cwd = name, argv, cwd + self.process: Optional[asyncio.subprocess.Process] = None + self.stopping = False + + @property + def running(self) -> bool: + return self.process is not None and self.process.returncode is None + + async def run(self) -> None: + """Run it, and start it again each time it ends, with the environment the front door has.""" + ending_soon = 0 + while not self.stopping: + started = time.monotonic() + try: + self.process = await asyncio.create_subprocess_exec(*self.argv, cwd=self.cwd, + stdin=asyncio.subprocess.DEVNULL) + how = _how(await self.process.wait()) + except OSError as e: + how = f"couldn't start: {e}" + except asyncio.CancelledError: + if self.running: + self.process.kill() + raise + if self.stopping: + return + ran = time.monotonic() - started + if ran >= STABLE_S: + ending_soon = 0 + delay = RESTART_DELAYS_S[min(ending_soon, len(RESTART_DELAYS_S) - 1)] + ending_soon += 1 + logger.warning(f"{self.name} ended after {ran:.1f} s ({how}); starting it again in {delay:g} s") + await asyncio.sleep(delay) + + async def stop(self) -> None: + """Ask it to end, and kill it after ``STOP_TIMEOUT_S``; it isn't started again.""" + self.stopping = True + if not self.running: + return + self.process.terminate() + try: + await asyncio.wait_for(self.process.wait(), STOP_TIMEOUT_S) + except TimeoutError: + self.process.kill() + await self.process.wait() + + +class FrontDoor: + """A computer app alone: the app's server in ``folder``, serving the class ``served`` names, and the site proxy + command, the programs it runs; and what it serves.""" + + def __init__(self, folder: Path, served: str, site_proxy: Optional[str] = None) -> None: + self.folder = folder + self.name = os.environ.get("ENVIRONMENT_NAME", folder.name) + self.server = Program("the app's server", [sys.executable, "-m", "agentenv_protocol.computer.serve_app", served], + folder) + self.site_proxy = Program("the site proxy", shlex.split(site_proxy), folder) if site_proxy else None + self.programs = [self.server, *([self.site_proxy] if self.site_proxy else [])] + self.http: Optional[httpx.AsyncClient] = None + + def app(self) -> Starlette: + """What it serves; its guard takes the control token the environment has now.""" + guard = control_guard() + return Starlette(routes=[Route(WELL_KNOWN_PATH, self.card, methods=["GET"]), + Route(STATUS_PATH, guard(self.status), methods=["GET"]), + Route("/{path:path}", guard(self.forward), methods=METHODS)], + lifespan=self.lifespan) + + @contextlib.asynccontextmanager + async def lifespan(self, app: Starlette) -> AsyncIterator[None]: + self.http = httpx.AsyncClient(transport=httpx.AsyncHTTPTransport(uds=str(self.folder / SOCKET)), + base_url="http://app", timeout=None) + runs = [asyncio.ensure_future(program.run()) for program in self.programs] + try: + yield + finally: + await asyncio.gather(*(program.stop() for program in self.programs)) + for run in runs: + run.cancel() + await self.http.aclose() + + async def card(self, request: Request) -> Response: + """The app's card as its server serves it, with no interface for agents and with ``computer-ready/v1``; while + its server isn't answering, its name and ``computer-ready/v1`` alone.""" + card = await apps.served_card(self.folder) or EnvironmentCard(name=self.name, url=RPC_PATH).model_dump() + card["additionalInterfaces"] = [] + capabilities = card.get("capabilities") or {} + capabilities["extensions"] = [*(capabilities.get("extensions") or []), READY_EXTENSION] + card["capabilities"] = capabilities + return _json_response(card) + + async def status(self, request: Request) -> Response: + return _json_response(await self.readiness()) + + async def readiness(self) -> dict[str, Any]: + """``computer-ready/v1``'s status: the app's readiness as its server says, or why its server isn't answering, + and whether the site proxy runs; ``failed`` too, once the app can't be ready again.""" + checks = [await apps.check(self.folder)] + if self.site_proxy is not None: + checks.append(ReadyCheck("site proxy", self.site_proxy.running, + "running" if self.site_proxy.running else "not running; it starts again")) + failed = self.failure() + if failed is not None: + checks[0] = ReadyCheck(checks[0].name, False, failed) + return {"ready": failed is None and all(check.ok for check in checks), + "checks": [asdict(check) for check in checks], **({"failed": failed} if failed else {})} + + def failure(self) -> Optional[str]: + """Why the app can't be ready again: its launch log says it ended on its own, and alone nothing launches it + again, an infrastructure failure.""" + for entry in _launch_log(self.folder): + if entry.get("event") == "exit": + how = (f"by signal {entry['signal']}" if "signal" in entry else + f"with code {entry['code']}" if "code" in entry else "in a way that wasn't recorded") + return (f"it ended on its own at {entry.get('at')}, {how}, and nothing launches it again: an " + "infrastructure failure") + return None + + async def forward(self, request: Request) -> Response: + """``request`` sent on to the app's server at its path, but for its hop and token headers, and its answer + streamed back; it takes as long as the app does.""" + sent = self.http.build_request( + request.method, httpx.URL(path=request.url.path, query=request.url.query.encode()), + headers=_forwarded(request.headers.items()), content=await request.body()) + try: + answer = await self.http.send(sent, stream=True) + except (httpx.HTTPError, OSError) as e: + return _error_response(DeliveryFailed(f"the app's server isn't answering: {type(e).__name__}: {e}")) + async def body() -> AsyncIterator[bytes]: + try: + async for chunk in answer.aiter_raw(): + yield chunk + finally: + await answer.aclose() + + response = StreamingResponse(body(), status_code=answer.status_code) + response.raw_headers = [(name.encode("latin-1"), value.encode("latin-1")) + for name, value in _forwarded(answer.headers.multi_items())] + return response + + +def _forwarded(headers: Any) -> list[tuple[str, str]]: + return [(name, value) for name, value in headers if name.lower() not in UNFORWARDED] + + +def _launch_log(folder: Path) -> list[dict]: + """Each entry of the app's launch log that's whole, in order.""" + try: + lines = (folder / LAUNCHES).read_text(encoding="utf-8").splitlines() + except FileNotFoundError: + return [] + entries = [] + for line in lines: + with contextlib.suppress(ValueError): + entries.append(json.loads(line)) + return entries + + +def _how(code: int) -> str: + return f"signal {signal.Signals(-code).name}" if code < 0 else f"code {code}" + + +def main(argv: list[str]) -> None: + parser = argparse.ArgumentParser(prog="front_door", usage=USAGE) + parser.add_argument("folder", type=Path) + parser.add_argument("served") + parser.add_argument("--site-proxy") + parser.add_argument("--port", type=int, default=PORT) + args = parser.parse_args(argv) + logging.basicConfig(level=logging.WARNING, format="%(asctime)s front door: %(message)s") + folder = args.folder.resolve() + os.environ.setdefault("ENVIRONMENT_NAME", folder.name) + served = FrontDoor(folder, args.served, args.site_proxy).app() + os.environ.pop(CONTROL_TOKEN_ENV, None) + uvicorn.run(served, host="0.0.0.0", port=args.port, log_level="warning") + + +if __name__ == "__main__": + main(sys.argv[1:]) diff --git a/packages/agentenv-protocol/src/agentenv_protocol/computer/operations.py b/packages/agentenv-protocol/src/agentenv_protocol/computer/operations.py index 1e9936ec1..d4583f43c 100644 --- a/packages/agentenv-protocol/src/agentenv_protocol/computer/operations.py +++ b/packages/agentenv-protocol/src/agentenv_protocol/computer/operations.py @@ -69,6 +69,10 @@ class ExecResult: truncated: bool = False +#: The extension whose status says, by these checks, whether a computer, or an app running alone, is ready. +COMPUTER_READY = "urn:agentenv:computer-ready/v1" + + @dataclass(frozen=True) class ReadyCheck: name: str diff --git a/packages/agentenv-protocol/src/agentenv_protocol/website_browser.py b/packages/agentenv-protocol/src/agentenv_protocol/website_browser.py index 083cfb251..3c377b9ef 100644 --- a/packages/agentenv-protocol/src/agentenv_protocol/website_browser.py +++ b/packages/agentenv-protocol/src/agentenv_protocol/website_browser.py @@ -260,13 +260,26 @@ async def active_tab(self) -> Optional[str]: return visible async def record(self) -> dict: - """The record ``since`` it started: open ``tabs``, each with whether it's ``active``, ``closed_tabs``, - ``downloads`` and ``dialogs``.""" + """The record ``since`` it started: open ``tabs``, each with its URL and title now and whether it's ``active``, + ``closed_tabs``, ``downloads`` and ``dialogs``.""" + await self._refresh() active = await self.active_tab() return {"since": self.since, "tabs": [{**tab, "active": tab["id"] == active} for tab in self.tabs.values()], "closed_tabs": list(self.closed_tabs), "downloads": list(self.downloads.values()), "dialogs": list(self.dialog_log)} + async def _refresh(self, target: Optional[str] = None) -> None: + """The URL and title of the tab ``target``, or of every open tab, as the browser has them now: it sends no + event when a page's title changes.""" + try: + infos = (await self.devtools.call("Target.getTargets", {"filter": PAGES}, + timeout_s=ACTIVE_CHECK_TIMEOUT_S))["targetInfos"] + except (ConnectionError, DevToolsError, asyncio.TimeoutError, KeyError): + return + for info in infos: + if target in (None, info["targetId"]) and (tab := self.tabs.get(info["targetId"])) is not None: + tab.update(url=info.get("url", tab["url"]), title=info.get("title", tab["title"])) + def _event(self, message: dict) -> None: handler = getattr(self, "_on_" + message.get("method", "").replace(".", "_"), None) if handler is not None: @@ -299,6 +312,10 @@ def _on_Target_attachedToTarget(self, params: dict, session: Optional[str]) -> N def _on_Target_detachedFromTarget(self, params: dict, session: Optional[str]) -> None: self._sessions.pop(params["sessionId"], None) + def _on_Page_loadEventFired(self, params: dict, session: Optional[str]) -> None: + if (target := self._sessions.get(session)) is not None: + self._spawn(self._refresh(target)) + def _on_Page_javascriptDialogOpening(self, params: dict, session: Optional[str]) -> None: dialog = {"tab": self._sessions.get(session), "type": params.get("type"), "message": params.get("message", ""), "url": params.get("url", ""), "opened_at": _now(), "answer": None, "by": None} diff --git a/packages/agentenv-protocol/tests/test_computer_front_door.py b/packages/agentenv-protocol/tests/test_computer_front_door.py new file mode 100644 index 000000000..1352a2106 --- /dev/null +++ b/packages/agentenv-protocol/tests/test_computer_front_door.py @@ -0,0 +1,344 @@ +"""A computer app alone behind its front door, run as its container runs it, with a fake browser that answers CDP on its +pipe: the app's card with computer-ready/v1 and no interface for agents; every route but the card taking the control +token, which no program it runs is given; requests sent on to the app's server, a large answer streamed whole; a killed +app server started again, which launches the browser again; a browser that ends on its own failing the env for good, +with its signal; a killed site proxy started again; an app server that can't start started again later and later, +saying why; requests while the app's server restarts answering or failing, never hanging; stopping it stopping +everything it runs; and its needing only the app extra, as the app's venv has.""" +from __future__ import annotations + +import asyncio +import contextlib +import json +import os +import shutil +import signal +import socket +import subprocess +import sys +import tempfile +import time +from pathlib import Path + +import pytest + +pytest.importorskip("mcp.server.fastmcp") +pytest.importorskip("uvicorn") + +import httpx # noqa: E402 + +from agentenv_protocol import client # noqa: E402 +from agentenv_protocol.computer import front_door # noqa: E402 +from agentenv_protocol.computer.operations import COMPUTER_READY # noqa: E402 +from agentenv_protocol.types import CONTROL_TOKEN_ENV, WELL_KNOWN_PATH # noqa: E402 +from agentenv_protocol.website_browser import ALONE_ENV, BROWSER, SITE_PROXY_ENV # noqa: E402 +from test_website_browser import FAKE_BROWSER # noqa: E402 + +TOKEN = "front-door-token" +BEARER = {"authorization": f"Bearer {TOKEN}"} +#: How long it may take to start, or to recover; how often it's asked meanwhile. +WAIT_S, POLL_S = 30.0, 0.2 +#: How large an answer streams through whole. +LARGE_BYTES = 20_000_000 +SLEEPER = f"{sys.executable} -c 'import time; time.sleep(600)'" +# The app, as its folder holds it: a website browser whose launch runs the fake browser, on Linux, its launches marked +# in a folder of the test's own, with an extension that says whether its server was given the control token and one +# that answers with a large body. +APP = f"""import os, sys +from pathlib import Path +from agentenv_protocol.agent_env_environment import extension +from agentenv_protocol.computer import Launch, Ready, app, app_runtime, linux +from agentenv_protocol.computer.platform import Platform +from agentenv_protocol.website_browser import WebsiteBrowserEnvironment + +app.detect = lambda: Platform(os="linux") +app_runtime.LAUNCHED = Path(os.environ["TEST_LAUNCHED"]) + + +@linux(launch=Launch(command=[sys.executable, str(Path(__file__).with_name("fake_browser.py"))], + ready=Ready(command=["true"]))) +class Browser(WebsiteBrowserEnvironment): + @extension("urn:test:probe/v1", op="token", method="GET") + async def token(self) -> dict: + return {{"given": "{CONTROL_TOKEN_ENV}" in os.environ}} + + @extension("urn:test:probe/v1", op="large", method="GET") + async def large(self) -> dict: + return {{"data": "x" * {LARGE_BYTES}}} +""" + + +def _free_port() -> int: + with socket.socket() as sock: + sock.bind(("127.0.0.1", 0)) + return sock.getsockname()[1] + + +class Door: + """A front door running as its container's command, on a port of its own, with the app in a short folder, since a + Unix socket's path is short.""" + + def __init__(self, served: str = "browser.py:Browser", site_proxy: str | None = None) -> None: + self.folder = Path(tempfile.mkdtemp(prefix="fd-", dir="/tmp")) + (self.folder / "browser.py").write_text(APP) + (self.folder / "fake_browser.py").write_text(FAKE_BROWSER) + (self.folder / "home").mkdir() + (self.folder / ".agentenv").mkdir() + self.port = _free_port() + self.url = f"http://127.0.0.1:{self.port}" + env = {key: value for key, value in os.environ.items() if key != "DISPLAY"} + env.update({"HOME": str(self.folder / "home"), "ENVIRONMENT_NAME": "browser", ALONE_ENV: "1", + SITE_PROXY_ENV: "http://127.0.0.1:9", "TEST_LAUNCHED": str(self.folder / "launched"), + CONTROL_TOKEN_ENV: TOKEN}) + argv = [sys.executable, "-m", "agentenv_protocol.computer.front_door", str(self.folder), served, + "--port", str(self.port), *(["--site-proxy", site_proxy] if site_proxy else [])] + self.log = (self.folder / "front-door.log").open("w") + self.process = subprocess.Popen(argv, env=env, stdout=self.log, stderr=subprocess.STDOUT) + + def children(self, matching: str) -> list[int]: + found = subprocess.run(["pgrep", "-P", str(self.process.pid), "-f", matching], capture_output=True, text=True) + return [int(pid) for pid in found.stdout.split()] + + def launches(self) -> list[dict]: + try: + lines = (self.folder / ".agentenv" / "launches").read_text().splitlines() + except FileNotFoundError: + return [] + return [json.loads(line) for line in lines if line] + + def logged(self) -> str: + return (self.folder / "front-door.log").read_text() + + async def status(self) -> dict: + async with httpx.AsyncClient(base_url=self.url, timeout=10) as http: + return (await http.get(front_door.STATUS_PATH, headers=BEARER)).json() + + async def until(self, holds, what: str): + """``holds``' answer once it's truthy, within ``WAIT_S``.""" + last = None + deadline = time.monotonic() + WAIT_S + while time.monotonic() < deadline: + with contextlib.suppress(httpx.HTTPError, ValueError, OSError): + last = await holds() + if last: + return last + await asyncio.sleep(POLL_S) + raise AssertionError(f"{what} within {WAIT_S:g} s; last {last!r}; its log:\n{self.logged()}") + + async def ready(self) -> dict: + return await self.until(lambda: self._ready(), "ready") + + async def _ready(self): + status = await self.status() + return status if status["ready"] else None + + def close(self) -> None: + if self.process.poll() is None: + self.process.terminate() + with contextlib.suppress(subprocess.TimeoutExpired): + self.process.wait(15) + if self.process.poll() is None: + self.process.kill() + self.process.wait() + for entry in self.launches(): + with contextlib.suppress(ProcessLookupError, PermissionError): + os.killpg(entry["pid"], signal.SIGKILL) + self.log.close() + shutil.rmtree(self.folder, ignore_errors=True) + + +@pytest.fixture +def doors(): + opened: list[Door] = [] + + def open_door(**kwargs) -> Door: + door = Door(**kwargs) + opened.append(door) + return door + + yield open_door + for door in opened: + door.close() + + +def _browser(door: Door) -> int: + """The pid of the browser the app's server launched last.""" + return [entry for entry in door.launches() if entry["event"] == "launch"][-1]["pid"] + + +@pytest.mark.asyncio +async def test_its_card_is_the_apps_with_computer_ready_and_no_interface_for_agents_and_only_the_card_is_open(doors): + door = doors(site_proxy=SLEEPER) + status = await door.ready() + async with httpx.AsyncClient(base_url=door.url, timeout=10) as http: + card = (await http.get(WELL_KNOWN_PATH)).json() + ready = client.find_extension_method(card, "urn:agentenv:computer-app/v1", "ready")["endpoint"] + unguarded = [(await http.get(path)).status_code for path in (front_door.STATUS_PATH, ready)] + wrong = (await http.get(ready, headers={"authorization": "Bearer nope"})).status_code + app_ready = (await http.get(ready, headers=BEARER)).json() + token = (await http.get(client.find_extension_method(card, "urn:test:probe/v1", "token")["endpoint"], + headers=BEARER)).json() + + assert card["name"] == "browser" and card["additionalInterfaces"] == [] + assert client.find_extension(card, BROWSER) is not None + assert client.find_extension_method(card, COMPUTER_READY, "status") == { + "method": "GET", "endpoint": front_door.STATUS_PATH, "request": {"type": "object", "properties": {}}} + assert status == {"ready": True, "checks": [{"name": "app browser", "ok": True, "detail": "ready"}, + {"name": "site proxy", "ok": True, "detail": "running"}]} + assert unguarded == [401, 401] and wrong == 401 + assert app_ready["ready"] is True and token == {"given": False} + assert [entry["event"] for entry in door.launches()] == ["launch"] + + +@pytest.mark.asyncio +async def test_a_large_answer_streams_through_whole(doors): + door = doors() + await door.ready() + async with httpx.AsyncClient(base_url=door.url, timeout=60) as http: + card = (await http.get(WELL_KNOWN_PATH)).json() + answer = await http.get(client.find_extension_method(card, "urn:test:probe/v1", "large")["endpoint"], + headers=BEARER) + + assert answer.status_code == 200 and len(answer.json()["data"]) == LARGE_BYTES + + +@pytest.mark.asyncio +async def test_a_killed_app_server_starts_again_and_launches_the_browser_again_on_a_new_pipe(doors): + door = doors() + await door.ready() + first_browser = _browser(door) + (server,) = door.children("serve_app") + os.kill(server, signal.SIGKILL) + + async def relaunched(): + return door.launches()[-1].get("cause") == "server_restart" and (await door.status())["ready"] + + await door.until(relaunched, "the browser launched again and ready") + (restarted,) = door.children("serve_app") + + # The browser ends with its pipe, so the server finds nothing to stop, and no one logs it as its own exit. + assert restarted != server and [entry["event"] for entry in door.launches()] in ( + ["launch", "launch"], ["launch", "stop", "launch"]) + assert door.launches()[0]["cause"] == "boot" and _browser(door) != first_browser + assert "the app's server ended" in door.logged() and "signal SIGKILL" in door.logged() + + +@pytest.mark.asyncio +async def test_a_browser_that_ends_on_its_own_fails_the_env_for_good_with_its_signal(doors): + door = doors() + await door.ready() + os.killpg(_browser(door), signal.SIGKILL) + + async def failed(): + status = await door.status() + return status if "failed" in status else None + + status = await door.until(failed, "failed") + (server,) = door.children("serve_app") + os.kill(server, signal.SIGKILL) + + async def relaunched(): + return [entry["event"] for entry in door.launches()][-1] == "launch" and len(door.launches()) >= 3 + + await door.until(relaunched, "the browser launched again by the restarted server") + await asyncio.sleep(1) + after = await door.status() + + assert status["ready"] is False and "by signal SIGKILL" in status["failed"] + assert "an infrastructure failure" in status["failed"] + assert status["checks"][0] == {"name": "app browser", "ok": False, "detail": status["failed"]} + assert [entry["event"] for entry in door.launches()][:2] == ["launch", "exit"] + assert door.launches()[1]["signal"] == "SIGKILL" + assert after["ready"] is False and after["failed"] == status["failed"] + + +@pytest.mark.asyncio +async def test_a_killed_site_proxy_starts_again(doors): + door = doors(site_proxy=SLEEPER) + await door.ready() + (proxy,) = door.children("time.sleep") + os.kill(proxy, signal.SIGKILL) + + async def restarted(): + return [pid for pid in door.children("time.sleep") if pid != proxy] + + (again,) = await door.until(restarted, "the site proxy started again") + status = await door.ready() + + assert again != proxy and status["checks"][1] == {"name": "site proxy", "ok": True, "detail": "running"} + assert "the site proxy ended" in door.logged() + + +@pytest.mark.asyncio +async def test_an_app_server_that_cant_start_is_started_again_later_and_later_and_says_why(doors): + door = doors(served="browser.py:Missing") + + async def said_why(): + status = await door.status() + return status if "failed to start" in status["checks"][0]["detail"] else None + + status = await door.until(said_why, "the server's failure") + await door.until(lambda: asyncio.sleep(0, "starting it again in 2 s" in door.logged()), "a 2 s delay") + + assert status["ready"] is False and "Missing" in status["checks"][0]["detail"] + assert "failed" not in status + delays = [line.rsplit(" in ", 1)[1] for line in door.logged().splitlines() if "starting it again in" in line] + assert delays[:3] == ["0.5 s", "1 s", "2 s"] + + +@pytest.mark.asyncio +async def test_requests_while_its_app_server_restarts_answer_or_fail_and_never_hang(doors): + door = doors() + await door.ready() + (server,) = door.children("serve_app") + async with httpx.AsyncClient(base_url=door.url, timeout=WAIT_S) as http: + card = (await http.get(WELL_KNOWN_PATH)).json() + ready = client.find_extension_method(card, "urn:agentenv:computer-app/v1", "ready")["endpoint"] + os.kill(server, signal.SIGKILL) + answers = await asyncio.gather(*(http.get(ready, headers=BEARER) for _ in range(30)), + *(http.get(WELL_KNOWN_PATH) for _ in range(10))) + await door.ready() + + assert {answer.status_code for answer in answers} <= {200, 502} + assert all(answer.json()["name"] == "browser" for answer in answers[30:]) + + +@pytest.mark.asyncio +async def test_stopping_it_stops_every_program_it_runs(doors): + door = doors(site_proxy=SLEEPER) + await door.ready() + programs = door.children("serve_app") + door.children("time.sleep") + door.process.send_signal(signal.SIGTERM) + code = await asyncio.to_thread(door.process.wait, WAIT_S) + + def gone(pid: int) -> bool: + try: + os.kill(pid, 0) + except ProcessLookupError: + return True + return False + + # uvicorn ends with the signal it shut down on, once it has. + assert code in (0, -signal.SIGTERM) and len(programs) == 2 and all(gone(pid) for pid in programs) + + +def test_it_needs_only_the_app_extra(): + blocked = """ +import importlib.abc, sys +class Blocked(importlib.abc.MetaPathFinder): + def find_spec(self, name, path, target=None): + if name.split(".")[0] in ("av", "PIL"): + raise ImportError(name) +sys.meta_path.insert(0, Blocked()) +import agentenv_protocol.computer.front_door +""" + done = subprocess.run([sys.executable, "-c", blocked], capture_output=True, text=True) + + assert done.returncode == 0, done.stderr + + +def test_it_sends_on_no_hop_or_token_header(): + headers = [("authorization", "Bearer x"), ("Host", "a"), ("content-type", "application/json"), + ("Transfer-Encoding", "chunked"), ("x-forwarded-host", "b"), ("Connection", "close")] + + assert front_door._forwarded(headers) == [("content-type", "application/json"), ("x-forwarded-host", "b")] diff --git a/packages/agentenv-protocol/tests/test_website_browser.py b/packages/agentenv-protocol/tests/test_website_browser.py index d21dd495c..bfb0664df 100644 --- a/packages/agentenv-protocol/tests/test_website_browser.py +++ b/packages/agentenv-protocol/tests/test_website_browser.py @@ -3,7 +3,8 @@ its launch run with no desktop when alone; a CDP connection over a new debugging pipe at each launch, which its ready check asks, and a browser launched again when its server restarts; its shared memory in files only on a small /dev/shm; an empty cache and the deploy's prompt answers in its profile at each launch; its policies, card params and -skill; and an app that isn't a browser launched and declared as before.""" +skill; its record's titles read from the browser, which sends no event for a page's; and an app that isn't a browser +launched and declared as before.""" from __future__ import annotations import asyncio @@ -492,6 +493,28 @@ async def test_its_control_records_tabs_in_opened_order_and_who_closed_each_and_ assert devtools.calls[2][1] == {"behavior": "default", "eventsEnabled": True} +@pytest.mark.asyncio +async def test_its_record_reads_each_tabs_title_when_read_and_when_its_page_loads_since_no_event_says(tmp_path): + now = {"a": "Inbox", "b": "Draft"} + devtools = StubDevTools({"Target.getTargets": lambda params, session: {"targetInfos": [ + _page(target, f"https://mail.test/{target}", title)["targetInfo"] for target, title in now.items()]}}) + control = BrowserControl(devtools, "agent") + await control.attach() + for target in ("a", "b"): + devtools.send("Target.targetCreated", _page(target, f"https://mail.test/{target}", f"mail.test/{target}")) + devtools.send("Target.attachedToTarget", {"sessionId": f"s-{target}", **_page(target)}) + devtools.send("Page.loadEventFired", {"timestamp": 1}, session="s-b") + await asyncio.sleep(0) + del now["b"] + devtools.send("Target.targetDestroyed", {"targetId": "b"}) + now["a"] = "Inbox (1)" + record = await control.record() + + assert [(tab["id"], tab["title"]) for tab in record["tabs"]] == [("a", "Inbox (1)")] + assert [(tab["id"], tab["title"]) for tab in record["closed_tabs"]] == [("b", "Draft")] + assert ("Target.getTargets", {"filter": [{"type": "page"}]}, None) in devtools.calls + + @pytest.mark.asyncio @pytest.mark.parametrize("dialogs, kind, sent, answer, by", [ ("accept", "prompt", {"accept": True, "promptText": "yes"}, "accepted", "agentenv"), diff --git a/tst/integration/computer/front_door_test.py b/tst/integration/computer/front_door_test.py new file mode 100644 index 000000000..d06e7a8c0 --- /dev/null +++ b/tst/integration/computer/front_door_test.py @@ -0,0 +1,256 @@ +"""The shipped chromium alone behind its front door, in the chromium test bundle's composed image with the front door as +its command in place of the desktop's, as the browser provider will run it: headless, out only through a stand-in site +proxy; its card and browser/v1's record on the front door's port, with the control token, which no program it runs is +given, and no TCP port listening but the front door's and the proxy's; its killed app server started again, which +launches Chromium again on a new pipe; Chromium killed failing the env for good, with its signal, which a restarted app +server doesn't undo; and its killed site proxy started again. Needs Docker and the local registry on localhost:5000.""" + +import asyncio +import json +import platform +import shlex +import time +import uuid + +import subprocess + +import httpx +import pytest + +from agentenv_protocol import client as protocol_v1 +from agentenv_protocol.computer import front_door +from agentenv_protocol.computer.operations import COMPUTER_READY +from agentenv_protocol.types import CONTROL_TOKEN_ENV, WELL_KNOWN_PATH +from agentenv_protocol.website_browser import ALONE_ENV, BROWSER, SITE_PROXY_ENV + +pytestmark = [pytest.mark.integration, pytest.mark.int_test_slow] + +ARCH = {"x86_64": "amd64", "amd64": "amd64", "aarch64": "arm64", "arm64": "arm64"}[platform.machine().lower()] +APP = "/opt/agentenv/apps/chromium" +PROXY_PORT = 3128 +SITE = "http://site.test/" +TOKEN = uuid.uuid4().hex +BEARER = {"authorization": f"Bearer {TOKEN}"} +#: How long the browser may take to be ready, or to recover; how often it's asked meanwhile. +WAIT_S, POLL_S = 120.0, 1.0 +# A stand-in for the site proxy: any page asked for through it is one that names its host. +SITE_PROXY = f"""import http.server, urllib.parse +class Proxy(http.server.BaseHTTPRequestHandler): + def do_GET(self): + host = urllib.parse.urlsplit(self.path).hostname or "" + body = f"proxied {{host}}".encode() + self.send_response(200) + self.send_header("Content-Type", "text/html") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + def log_message(self, *args): + pass +http.server.ThreadingHTTPServer(("127.0.0.1", {PROXY_PORT}), Proxy).serve_forever() +""" +# Every TCP port something in the container listens on. +LISTENING = """ports = set() +for table in ("/proc/net/tcp", "/proc/net/tcp6"): + for line in open(table).read().splitlines()[1:]: + fields = line.split() + if fields[3] == "0A": + ports.add(int(fields[1].rsplit(":", 1)[1], 16)) +print(sorted(ports)) +""" + + +def docker(*args: str) -> subprocess.CompletedProcess: + return subprocess.run(["docker", *args], check=True, capture_output=True) + + +class Alone: + """One container of the composed image running chromium alone behind its front door.""" + + def __init__(self, image: str) -> None: + conf = docker("run", "--rm", "--entrypoint", "cat", image, + "/etc/supervisor/conf.d/agentenv-app-chromium.conf").stdout.decode() + served = next(line for line in conf.splitlines() if "serve_app" in line).rsplit(" ", 1)[1] + settings = json.dumps({"chromium": {"start_urls": [SITE]}}) + self.name = f"front-door-{uuid.uuid4().hex[:8]}" + docker("run", "-d", "--name", self.name, "-p", "127.0.0.1::8000", "-e", "DISPLAY=", + "-e", f"{ALONE_ENV}=1", "-e", f"{SITE_PROXY_ENV}=http://127.0.0.1:{PROXY_PORT}", + "-e", "ENVIRONMENT_NAME=chromium", "-e", f"{CONTROL_TOKEN_ENV}={TOKEN}", + "-e", f"AGENTENV_APP_SETTINGS={settings}", image, + f"{APP}/venv/bin/python", "-m", "agentenv_protocol.computer.front_door", APP, served, + "--site-proxy", shlex.join(["python3", "-c", SITE_PROXY])) + port = docker("port", self.name, "8000").stdout.decode().split()[0].rsplit(":", 1)[1] + self.url = f"http://127.0.0.1:{port}" + + def run(self, *argv: str) -> str: + return docker("exec", self.name, *argv).stdout.decode() + + def pids(self, pattern: str) -> list[int]: + """The front door's programs whose command line has ``pattern``.""" + door = "$(pgrep -n -f '[a]gentenv_protocol.computer.front_door')" + found = self.run("sh", "-c", f"pgrep -P {door} -f -- {shlex.quote(pattern)} || true") + return [int(pid) for pid in found.split()] + + def launches(self) -> list[dict]: + log = self.run("sh", "-c", f"cat {APP}/.agentenv/launches 2>/dev/null || true") + return [json.loads(line) for line in log.splitlines() if line] + + async def get(self, path: str, **kwargs) -> httpx.Response: + async with httpx.AsyncClient(base_url=self.url, timeout=30) as http: + return await http.get(path, **kwargs) + + async def status(self) -> dict: + return (await self.get(front_door.STATUS_PATH, headers=BEARER)).json() + + async def record(self) -> dict: + card = (await self.get(WELL_KNOWN_PATH)).json() + path = protocol_v1.find_extension_method(card, BROWSER, "record")["endpoint"] + return (await self.get(path, headers=BEARER)).json() + + def logs(self) -> str: + logged = docker("logs", self.name) + return (logged.stdout + logged.stderr).decode()[-4000:] + + async def until(self, holds, what: str): + """``holds``' answer once it's truthy, within ``WAIT_S``, while the container runs.""" + last, deadline = None, time.monotonic() + WAIT_S + while time.monotonic() < deadline: + if docker("inspect", "-f", "{{.State.Running}}", self.name).stdout.decode().strip() != "true": + raise AssertionError(f"the container ended before {what}; its logs:\n{self.logs()}") + try: + last = await holds() + if last: + return last + except (httpx.HTTPError, ValueError, KeyError): + pass + await asyncio.sleep(POLL_S) + raise AssertionError(f"{what} within {WAIT_S:g} s; last {last!r}; its logs:\n{self.logs()}") + + async def ready(self) -> dict: + async def ready(): + status = await self.status() + return status if status["ready"] else None + return await self.until(ready, "ready") + + async def on_the_site(self) -> dict: + async def loaded(): + record = await self.record() + return record if [tab["title"] for tab in record["tabs"]] == ["proxied site.test"] else None + return await self.until(loaded, "its start tab loaded through the site proxy") + + def remove(self) -> None: + docker("rm", "-f", self.name) + + +@pytest.fixture +def alone(chromium_desktop): + image = chromium_desktop.metadata["composed"]["container"][ARCH]["image"] + started = [] + + def start() -> Alone: + started.append(Alone(image)) + return started[-1] + + yield start + for container in started: + container.remove() + + +def _browser(container: Alone) -> int: + return [entry for entry in container.launches() if entry["event"] == "launch"][-1]["pid"] + + +@pytest.mark.asyncio +async def test_alone_it_serves_its_card_status_and_record_on_its_port_with_the_token_and_nothing_else_listens(alone): + container = alone() + status = await container.ready() + record = await container.on_the_site() + card = (await container.get(WELL_KNOWN_PATH)).json() + record_path = protocol_v1.find_extension_method(card, BROWSER, "record")["endpoint"] + refused = [(await container.get(path)).status_code for path in (front_door.STATUS_PATH, record_path)] + wrong = (await container.get(record_path, headers={"authorization": "Bearer nope"})).status_code + listening = json.loads(container.run("python3", "-c", LISTENING)) + (server,) = container.pids("serve_app") + environments = [container.run("sh", "-c", f"tr '\\0' '\\n' < /proc/{pid}/environ") + for pid in (server, _browser(container))] + flags = container.run("sh", "-c", f"tr '\\0' ' ' < /proc/{_browser(container)}/cmdline") + + assert card["name"] == "chromium" and card["additionalInterfaces"] == [] + assert protocol_v1.find_extension_method(card, COMPUTER_READY, "status")["endpoint"] == front_door.STATUS_PATH + assert [check["name"] for check in status["checks"]] == ["app chromium", "site proxy"] + assert [(tab["url"], tab["active"]) for tab in record["tabs"]] == [(SITE, True)] + assert [(entry["event"], entry["cause"]) for entry in record["launches"]] == [("launch", "boot")] + assert refused == [401, 401] and wrong == 401 + assert listening == sorted([8000, PROXY_PORT]) + assert all(f"{CONTROL_TOKEN_ENV}=" not in environment for environment in environments) + assert "--headless=new" in flags and "--remote-debugging-pipe" in flags + assert f"--proxy-server=http://127.0.0.1:{PROXY_PORT}" in flags + + +@pytest.mark.asyncio +async def test_its_killed_app_server_starts_again_and_launches_chromium_again_on_a_new_pipe(alone): + container = alone() + await container.ready() + await container.on_the_site() + first = _browser(container) + (server,) = container.pids("serve_app") + container.run("kill", "-9", str(server)) + + async def relaunched(): + return container.launches()[-1].get("cause") == "server_restart" and (await container.status())["ready"] + + await container.until(relaunched, "Chromium launched again and ready") + record = await container.on_the_site() + (restarted,) = container.pids("serve_app") + + assert restarted != server and _browser(container) != first + assert [entry["event"] for entry in container.launches()] in (["launch", "launch"], ["launch", "stop", "launch"]) + assert "exit" not in [entry["event"] for entry in record["launches"]] + + +@pytest.mark.asyncio +async def test_chromium_killed_fails_the_env_for_good_with_its_signal_even_once_its_server_restarts(alone): + container = alone() + await container.ready() + container.run("kill", "-9", str(_browser(container))) + + async def failed(): + status = await container.status() + return status if "failed" in status else None + + status = await container.until(failed, "failed") + record = await container.record() + (server,) = container.pids("serve_app") + container.run("kill", "-9", str(server)) + + async def relaunched(): + return container.launches()[-1].get("cause") == "server_restart" + + await container.until(relaunched, "Chromium launched again by the restarted server") + after = await container.status() + + assert status["ready"] is False and "by signal SIGKILL" in status["failed"] + assert status["checks"][0]["ok"] is False and status["checks"][0]["detail"] == status["failed"] + assert [entry["event"] for entry in record["launches"]] == ["launch", "exit"] + assert record["launches"][1]["signal"] == "SIGKILL" + assert [tab["closed_by"] for tab in record["closed_tabs"]] == ["exit"] + assert after["ready"] is False and after["failed"] == status["failed"] + + +@pytest.mark.asyncio +async def test_its_killed_site_proxy_starts_again(alone): + container = alone() + await container.ready() + (proxy,) = container.pids("ThreadingHTTPServer") + container.run("kill", "-9", str(proxy)) + + async def restarted(): + return [pid for pid in container.pids("ThreadingHTTPServer") if pid != proxy] + + (again,) = await container.until(restarted, "the site proxy started again") + status = await container.ready() + fetched = container.run("python3", "-c", "import urllib.request; print(urllib.request.build_opener(" + f"urllib.request.ProxyHandler({{'http': 'http://127.0.0.1:{PROXY_PORT}'}}))" + f".open('{SITE}').read().decode())") + + assert again != proxy and status["checks"][1] == {"name": "site proxy", "ok": True, "detail": "running"} + assert fetched.strip() == "proxied site.test" From 044c2ffb2ee303b78dc2f6e875f8cb4357ff7f72 Mon Sep 17 00:00:00 2001 From: Edgar Arakelyan Date: Tue, 3 Nov 2026 00:04:56 -0800 Subject: [PATCH 2/5] fix(computer): the front door drops headers a Connection header names, and kills a program that won't stop on 3.10 A header that Connection names belongs to that hop, so neither a request nor an answer sends it on. asyncio.wait_for raises asyncio.TimeoutError, not the builtin before 3.11, which the protocol still supports. Co-Authored-By: Claude Opus 5.5 --- .../agentenv_protocol/computer/front_door.py | 8 +++++-- .../tests/test_computer_front_door.py | 24 ++++++++++++++++--- 2 files changed, 27 insertions(+), 5 deletions(-) diff --git a/packages/agentenv-protocol/src/agentenv_protocol/computer/front_door.py b/packages/agentenv-protocol/src/agentenv_protocol/computer/front_door.py index f1198770e..57a1c7207 100644 --- a/packages/agentenv-protocol/src/agentenv_protocol/computer/front_door.py +++ b/packages/agentenv-protocol/src/agentenv_protocol/computer/front_door.py @@ -111,7 +111,7 @@ async def stop(self) -> None: self.process.terminate() try: await asyncio.wait_for(self.process.wait(), STOP_TIMEOUT_S) - except TimeoutError: + except asyncio.TimeoutError: self.process.kill() await self.process.wait() @@ -211,7 +211,11 @@ async def body() -> AsyncIterator[bytes]: def _forwarded(headers: Any) -> list[tuple[str, str]]: - return [(name, value) for name, value in headers if name.lower() not in UNFORWARDED] + """``headers`` but those of one hop: the standard ones, the control token's and any its ``Connection`` names.""" + headers = list(headers) + named = {token.strip().lower() for name, value in headers if name.lower() == "connection" + for token in value.split(",")} + return [(name, value) for name, value in headers if name.lower() not in UNFORWARDED | named] def _launch_log(folder: Path) -> list[dict]: diff --git a/packages/agentenv-protocol/tests/test_computer_front_door.py b/packages/agentenv-protocol/tests/test_computer_front_door.py index 1352a2106..d3f6fc15f 100644 --- a/packages/agentenv-protocol/tests/test_computer_front_door.py +++ b/packages/agentenv-protocol/tests/test_computer_front_door.py @@ -337,8 +337,26 @@ def find_spec(self, name, path, target=None): assert done.returncode == 0, done.stderr -def test_it_sends_on_no_hop_or_token_header(): +def test_it_sends_on_no_hop_or_token_header_nor_one_its_connection_header_names(): headers = [("authorization", "Bearer x"), ("Host", "a"), ("content-type", "application/json"), - ("Transfer-Encoding", "chunked"), ("x-forwarded-host", "b"), ("Connection", "close")] + ("Transfer-Encoding", "chunked"), ("x-forwarded-host", "b"), ("Connection", "close, X-Internal"), + ("x-internal", "1"), ("x-kept", "2")] - assert front_door._forwarded(headers) == [("content-type", "application/json"), ("x-forwarded-host", "b")] + assert front_door._forwarded(headers) == [("content-type", "application/json"), ("x-forwarded-host", "b"), + ("x-kept", "2")] + + +@pytest.mark.asyncio +async def test_a_program_that_wont_stop_is_killed_once_its_stop_runs_out(monkeypatch, tmp_path): + monkeypatch.setattr(front_door, "STOP_TIMEOUT_S", 0.5) + stubborn = front_door.Program("stubborn", [sys.executable, "-c", ( + "import signal, time; signal.signal(signal.SIGTERM, signal.SIG_IGN); open('up', 'w').close(); " + "time.sleep(60)")], tmp_path) + running = asyncio.ensure_future(stubborn.run()) + deadline = time.monotonic() + WAIT_S + while not (tmp_path / "up").exists() and time.monotonic() < deadline: + await asyncio.sleep(POLL_S) + await stubborn.stop() + await asyncio.wait_for(running, WAIT_S) + + assert stubborn.process.returncode == -signal.SIGKILL From e5c460d31ea23c8c60b6f6da4c4afb2f29206c50 Mon Sep 17 00:00:00 2001 From: Edgar Arakelyan Date: Tue, 3 Nov 2026 07:41:52 -0800 Subject: [PATCH 3/5] fix(computer): a popup the browser's control connection attached to as it opened navigates The control connection auto-attaches to every page, and Chrome holds a page it attached that way, a popup that window.open() opens say, until the attached client says to run. It never did, so every scripted popup stayed at no URL and never painted, in a computer as alone. Each attached page is told to run (Runtime.runIfWaitingForDebugger). Co-Authored-By: Claude Opus 5.5 --- .../src/agentenv_protocol/website_browser.py | 2 + .../tests/test_website_browser.py | 12 +++++ tst/integration/computer/chromium_app_test.py | 47 +++++++++++++++++++ 3 files changed, 61 insertions(+) diff --git a/packages/agentenv-protocol/src/agentenv_protocol/website_browser.py b/packages/agentenv-protocol/src/agentenv_protocol/website_browser.py index 3c377b9ef..a5a435caf 100644 --- a/packages/agentenv-protocol/src/agentenv_protocol/website_browser.py +++ b/packages/agentenv-protocol/src/agentenv_protocol/website_browser.py @@ -308,6 +308,8 @@ def _on_Target_attachedToTarget(self, params: dict, session: Optional[str]) -> N if params["targetInfo"].get("type") == "page": self._sessions[params["sessionId"]] = params["targetInfo"]["targetId"] self._spawn(self.devtools.call("Page.enable", session_id=params["sessionId"])) + # A page attached as it opens, a popup say, waits for this before it navigates. + self._spawn(self.devtools.call("Runtime.runIfWaitingForDebugger", session_id=params["sessionId"])) def _on_Target_detachedFromTarget(self, params: dict, session: Optional[str]) -> None: self._sessions.pop(params["sessionId"], None) diff --git a/packages/agentenv-protocol/tests/test_website_browser.py b/packages/agentenv-protocol/tests/test_website_browser.py index bfb0664df..c4ce5bbe8 100644 --- a/packages/agentenv-protocol/tests/test_website_browser.py +++ b/packages/agentenv-protocol/tests/test_website_browser.py @@ -493,6 +493,18 @@ async def test_its_control_records_tabs_in_opened_order_and_who_closed_each_and_ assert devtools.calls[2][1] == {"behavior": "default", "eventsEnabled": True} +@pytest.mark.asyncio +async def test_each_page_it_attaches_to_is_told_to_run_so_a_popup_it_attached_to_as_it_opened_navigates(tmp_path): + devtools = StubDevTools() + control = BrowserControl(devtools, "agent") + await control.attach() + devtools.send("Target.attachedToTarget", {"sessionId": "s-popup", "waitingForDebugger": True, **_page("popup")}) + await asyncio.sleep(0) + + assert ("Page.enable", {}, "s-popup") in devtools.calls + assert ("Runtime.runIfWaitingForDebugger", {}, "s-popup") in devtools.calls + + @pytest.mark.asyncio async def test_its_record_reads_each_tabs_title_when_read_and_when_its_page_loads_since_no_event_says(tmp_path): now = {"a": "Inbox", "b": "Draft"} diff --git a/tst/integration/computer/chromium_app_test.py b/tst/integration/computer/chromium_app_test.py index 4ca5b9701..89f671ea5 100644 --- a/tst/integration/computer/chromium_app_test.py +++ b/tst/integration/computer/chromium_app_test.py @@ -59,6 +59,27 @@ def log_message(self, *args): threading.Thread(target=server.serve_forever, daemon=True).start() time.sleep(float(sys.argv[1])) """ +POPPING = "http://127.0.0.1:8768/" +# Serves, on loopback for the seconds it's given, a page that is all one button, which opens /popped in a popup once. +SERVE_POPPING = """import http.server, sys, threading, time +class Page(http.server.BaseHTTPRequestHandler): + def do_GET(self): + if self.path == "/popped": + body = b"Popped page

Popped

" + else: + body = (b"Opener page") + self.send_response(200) + self.send_header("Content-Type", "text/html") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + def log_message(self, *args): + pass +server = http.server.ThreadingHTTPServer(("127.0.0.1", 8768), Page) +threading.Thread(target=server.serve_forever, daemon=True).start() +time.sleep(float(sys.argv[1])) +""" RECORDED = "http://127.0.0.1:8767/" # Serves, on loopback for the seconds it's given, a page that asks to confirm and then downloads report.bin, whose # content differs at each request: the nth is _report(n). @@ -184,6 +205,32 @@ async def test_a_desktop_that_gives_it_defaults_deploys_with_them_and_a_deploys_ assert not report.failed +@pytest.mark.asyncio +async def test_a_popup_a_page_opens_loads_with_its_control_attached(chromium_desktop): + with namespace_routing(): + deployed = await Env.get(chromium_desktop.id).deploy( + env_params={"app_settings": {"chromium": {"start_urls": [POPPING]}}}) + context = TaskStepContext(deployed_envs=[deployed]) + try: + card = await _app(deployed, WELL_KNOWN_PATH, "chromium") + serving = asyncio.ensure_future(inside.run(deployed, ["python3", "-c", SERVE_POPPING, "60"], + timeout_s=90)) + await asyncio.sleep(SERVE_START_S) + await _keys(deployed, ["F5"]) + await _chromium_windows(deployed, "Opener page") + await _keys(deployed) + popped = await _record(deployed, card, lambda r: any(tab["title"] == "Popped page" for tab in r["tabs"])) + windows = await _chromium_windows(deployed, "Popped page", count=2) + serving.cancel() + finally: + report = await teardown_run(context) + + assert [(tab["url"], tab["title"]) for tab in popped["tabs"]] == [ + (POPPING, "Opener page"), (f"{POPPING}popped", "Popped page")], popped + assert any(title.startswith("Popped page") for title in windows), windows + assert not report.failed + + @pytest.mark.asyncio async def test_its_control_records_tabs_a_download_a_dialog_and_each_launch_with_its_cause(chromium_desktop): settings = {"start_urls": [RECORDED], "dialogs": "accept"} From 38cb86db4201faab44f5f69da1143badd8b39ab0 Mon Sep 17 00:00:00 2001 From: Edgar Arakelyan Date: Tue, 3 Nov 2026 07:52:28 -0800 Subject: [PATCH 4/5] refactor(computer): drop the front door, since a browser alone will be a computer A website browser running alone becomes a computer with no display and a page driver whose screen is its page (decided Nov 3), so the computer's own driver fronts it and nothing will run agentenv_protocol.computer.front_door. Its module, tests and README section go, with what only it needed: apps.served_card and computer-ready/v1's URN in operations. The popup fix and the record's current titles stay. Co-Authored-By: Claude Opus 5.5 --- packages/agentenv-protocol/README.md | 29 +- .../agentenv_protocol/computer/__init__.py | 4 +- .../src/agentenv_protocol/computer/apps.py | 6 +- .../src/agentenv_protocol/computer/card.py | 2 +- .../computer/conformance/checks.py | 2 +- .../agentenv_protocol/computer/extensions.py | 3 +- .../agentenv_protocol/computer/front_door.py | 254 ------------ .../agentenv_protocol/computer/operations.py | 4 - .../tests/test_computer_front_door.py | 362 ------------------ tst/integration/computer/front_door_test.py | 256 ------------- 10 files changed, 16 insertions(+), 906 deletions(-) delete mode 100644 packages/agentenv-protocol/src/agentenv_protocol/computer/front_door.py delete mode 100644 packages/agentenv-protocol/tests/test_computer_front_door.py delete mode 100644 tst/integration/computer/front_door_test.py diff --git a/packages/agentenv-protocol/README.md b/packages/agentenv-protocol/README.md index 4cbda4ff8..b6cc8c649 100644 --- a/packages/agentenv-protocol/README.md +++ b/packages/agentenv-protocol/README.md @@ -511,9 +511,9 @@ driver has them, through the `served_tools`, `served_extensions` and `served_ope class's `notes`, markdown for what its card can't say. A driver whose software ships a skill of its own returns it from `bundled_skill()` (its description, body and other files), and the computer skill leads with it, the computer's part following. -- **Framework extensions.** `computer-ready/v1` (`status`: `{ready, checks}`, real checks of the display and a +- **Framework extensions.** `computer-ready/v1` (`status`: real checks of the display and a capture, the driver's own `ready_checks()`, and one per app on the computer, asked of the app's - own server (see Computer apps), with `failed` too from an app's front door once it can't be ready again; tools refuse `device_unavailable` until they pass once), `computer-display/v1`, `computer-spaces/v1` (the default space, written into the tool + own server (see Computer apps); tools refuse `device_unavailable` until they pass once), `computer-display/v1`, `computer-spaces/v1` (the default space, written into the tool descriptions), `computer-keyboard/v1`, `computer-exec/v1` (`run {argv, cwd?, env?, timeout_s?, request_id}`: output capped per stream, a repeated `request_id` answered with the first result) and, with `recording`, `computer-recording/v1` (below). @@ -741,21 +741,6 @@ hook runs in a thread, which the reset and relaunch after it wait for. The drive its card lists each one's card as a child environment, rooted at `/svc//`, where it forwards requests to the app's server. -An app that runs alone, with no computer around it, as a website browser does, has a front door in the -driver's place: `python -m agentenv_protocol.computer.front_door : -[--site-proxy COMMAND] [--port 8000]`, run from the app's venv as its container's command. It runs the -app's server in ``, and the site proxy command when it's given one, and starts each again -whenever it ends, after 0.5 s, then longer each time it ends within a minute of starting, up to 30 s. -On its port it serves the app's card, with no interface for agents, since nothing alone serves them -yet, and with `computer-ready/v1`, whose `status` (GET) is `{ready, checks, failed?}`: one check for -the app, its own readiness or why its server isn't answering, and one for whether the site proxy runs. -Once the app's launch log has an `exit`, the app ended on its own, which nothing that runs it alone -launches it again after, so `status` says `failed`, with when and the code or signal, for good, even -once its server restarts and launches it again. Every other request goes on to the app's server at its -path, its answer streamed back. Everything but the card takes the control token (`CONTROL_TOKEN_ENV`), -which the front door takes out of each request it sends on and out of the environment of each program -it runs. - `MailApp.declaration()` returns what the class declares, an `AppDeclaration`, which a `computer_app` env stores, and `python -m agentenv_protocol.computer._declare ` prints it as JSON, or a driver's `DriverDeclaration`, without the class's own dependencies installed. @@ -805,10 +790,10 @@ are the profile's template. The base class adds: the pipe rather than a second copy, which would hand its window to the first and exit. The ready check `devtools_answers` asks `Browser.getVersion` over the pipe. - **Its control connection**, `self.control`, a `BrowserControl` over the pipe. At each launch it attaches - to every page with flattened sessions and keeps a record, `since` its server started: open tabs in - opened order, each with its URL and title as the browser has them when the record is read (Chrome sends no - event for a title), the one whose page has focus `active`; `closed_tabs`, each as of its page's last load, - with `closed_at` and + to every page with flattened sessions, telling each page it attached to as it opened, a popup say, to run, and keeps a + record, `since` its server started: open tabs in opened order, each with its URL and title as the browser has them + when the record is read (Chrome sends no event for a title), the one whose page has focus `active`; `closed_tabs`, + each as of its page's last load, with `closed_at` and `closed_by` (`agentenv`, over this connection; `unknown`, anyone else; `exit` or `stop`, when the browser ended on its own or its server stopped it); `downloads`, into `~/Downloads`, each with its state and, completed, its size and sha256; and `dialogs`, each with its type, message, answer and who gave it. @@ -818,7 +803,7 @@ are the profile's template. The base class adds: launch (with its cause: `boot`, `server_restart`, `launch`, `data/add` or `data/reset`), each stop it made and each exit it didn't, with its exit `code` or the `signal` that ended it, since the computer booted. Nothing relaunches a browser that exits on its own: in a computer the agent can, and alone it's an infra - failure, its ready check false and its front door's `computer-ready/v1` status `failed`. + failure, its ready check false. - **Its skill**: sites by their real names, files to upload in `~/Uploads` and downloads in `~/Downloads`, folders it makes. diff --git a/packages/agentenv-protocol/src/agentenv_protocol/computer/__init__.py b/packages/agentenv-protocol/src/agentenv_protocol/computer/__init__.py index fdcc2cec2..586e962cc 100644 --- a/packages/agentenv-protocol/src/agentenv_protocol/computer/__init__.py +++ b/packages/agentenv-protocol/src/agentenv_protocol/computer/__init__.py @@ -44,7 +44,6 @@ Unsupported, ) from .operations import ( - COMPUTER_READY, Display, Effect, ExecResult, @@ -81,6 +80,7 @@ COMPUTER_EXEC, COMPUTER_KEYBOARD, COMPUTER_POINTER, + COMPUTER_READY, COMPUTER_RECORDING, COMPUTER_SPACES, ) @@ -90,7 +90,7 @@ # The names whose modules need the computer extra (mcp, av and Pillow), by module. _ON_USE = { "driver": ("ComputerDriverEnvironment", "computer_driver", "DEFAULT_IDENTITY", "DriverDeclaration"), - "card": ("COMPUTER", "COMPUTER_DISPLAY", "COMPUTER_SPACES", "COMPUTER_POINTER", + "card": ("COMPUTER", "COMPUTER_READY", "COMPUTER_DISPLAY", "COMPUTER_SPACES", "COMPUTER_POINTER", "COMPUTER_KEYBOARD", "COMPUTER_EXEC", "COMPUTER_RECORDING"), "recording": ("Recording", "RecordHints", "RecordedFrame", "RecordedSegment", "frames_in_thread"), } diff --git a/packages/agentenv-protocol/src/agentenv_protocol/computer/apps.py b/packages/agentenv-protocol/src/agentenv_protocol/computer/apps.py index fd6275617..9de569ed5 100644 --- a/packages/agentenv-protocol/src/agentenv_protocol/computer/apps.py +++ b/packages/agentenv-protocol/src/agentenv_protocol/computer/apps.py @@ -54,7 +54,7 @@ async def check(folder: Path) -> ReadyCheck: async def children(folders: list[Path]) -> list[EnvironmentCard]: """Each app in ``folders`` as a child environment of its computer: its server's card, rooted at ``/svc//``, or, while its server isn't answering, its name and route alone.""" - cards = await asyncio.gather(*(served_card(folder) for folder in folders)) + cards = await asyncio.gather(*(_card(folder) for folder in folders)) return [EnvironmentCard.model_validate(child_card(folder.name, card)) if card is not None else EnvironmentCard(name=folder.name, url=f"/svc/{folder.name}{RPC_PATH}") for folder, card in zip(folders, cards)] @@ -85,7 +85,7 @@ def rooted(path: object) -> object: async def owned_paths(folders: list[Path]) -> list[str]: """The paths under the session user's home that the apps in ``folders`` keep their data in, as their cards say.""" - cards = await asyncio.gather(*(served_card(folder) for folder in folders)) + cards = await asyncio.gather(*(_card(folder) for folder in folders)) return [path for card in cards if card is not None for path in client.extension_params(card, COMPUTER_APP).get("paths") or []] @@ -104,7 +104,7 @@ def endpoint(card: dict, method: str) -> str: return client.find_extension_method(card, COMPUTER_APP, method)["endpoint"] -async def served_card(folder: Path) -> Optional[dict]: +async def _card(folder: Path) -> Optional[dict]: """The card the app in ``folder`` serves, or None while its server isn't answering.""" try: async with _client(folder / SOCKET) as http: diff --git a/packages/agentenv-protocol/src/agentenv_protocol/computer/card.py b/packages/agentenv-protocol/src/agentenv_protocol/computer/card.py index 027b853f8..a8453b7e2 100644 --- a/packages/agentenv-protocol/src/agentenv_protocol/computer/card.py +++ b/packages/agentenv-protocol/src/agentenv_protocol/computer/card.py @@ -6,7 +6,6 @@ from typing import TYPE_CHECKING from .dataplane import DEFAULT_DATA_PATHS -from .operations import COMPUTER_READY # noqa: F401 from .recording import recording_params from ..types import METHOD_ADD, EnvironmentExtension, EnvironmentTool, IntakeDeclaration, IntakeFormat, intake_extension @@ -14,6 +13,7 @@ from .driver import ComputerDriverEnvironment COMPUTER = "urn:agentenv:computer/v1" +COMPUTER_READY = "urn:agentenv:computer-ready/v1" COMPUTER_DISPLAY = "urn:agentenv:computer-display/v1" COMPUTER_SPACES = "urn:agentenv:computer-spaces/v1" COMPUTER_POINTER = "urn:agentenv:computer-pointer/v1" diff --git a/packages/agentenv-protocol/src/agentenv_protocol/computer/conformance/checks.py b/packages/agentenv-protocol/src/agentenv_protocol/computer/conformance/checks.py index 5b0d849b8..ebcf64843 100644 --- a/packages/agentenv-protocol/src/agentenv_protocol/computer/conformance/checks.py +++ b/packages/agentenv-protocol/src/agentenv_protocol/computer/conformance/checks.py @@ -28,13 +28,13 @@ COMPUTER_EXEC, COMPUTER_KEYBOARD, COMPUTER_POINTER, + COMPUTER_READY, COMPUTER_RECORDING, COMPUTER_SPACES, GROUP_TOOLS, POINT_TOOLS, ) from ..manifest import RecordingManifest -from ..operations import COMPUTER_READY from ..recording import RECORDINGS_DIR from .client import Driver, image from .mp4 import read_mp4 diff --git a/packages/agentenv-protocol/src/agentenv_protocol/computer/extensions.py b/packages/agentenv-protocol/src/agentenv_protocol/computer/extensions.py index 0c7ca98b5..ff12c3b36 100644 --- a/packages/agentenv-protocol/src/agentenv_protocol/computer/extensions.py +++ b/packages/agentenv-protocol/src/agentenv_protocol/computer/extensions.py @@ -10,13 +10,14 @@ COMPUTER_DISPLAY, COMPUTER_EXEC, COMPUTER_KEYBOARD, + COMPUTER_READY, COMPUTER_RECORDING, COMPUTER_SPACES, KEYBOARD_RANGES, MAX_EXEC_TIMEOUT_S, ) from .errors import Internal, InvalidArgument, Unsupported -from .operations import COMPUTER_READY, Display, ExecResult +from .operations import Display, ExecResult from .skill import skill from .spaces import check_space diff --git a/packages/agentenv-protocol/src/agentenv_protocol/computer/front_door.py b/packages/agentenv-protocol/src/agentenv_protocol/computer/front_door.py deleted file mode 100644 index 57a1c7207..000000000 --- a/packages/agentenv-protocol/src/agentenv_protocol/computer/front_door.py +++ /dev/null @@ -1,254 +0,0 @@ -"""The front door of a computer app that runs alone, with no computer around it, as a website browser does: -``python -m agentenv_protocol.computer.front_door : [--site-proxy COMMAND]``, run from the app's -venv as its container's command. It runs the app's own server (``serve_app``) in ````, and the site proxy -command when it's given one, and starts each again whenever it ends, after a delay that grows while it keeps ending -soon after it starts. On ``PORT`` it serves the app's card, with ``computer-ready/v1`` added and no interface for -agents, since nothing here serves them yet; ``computer-ready/v1``'s ``status``, which says whether the app is ready, or -``failed`` for good once its launch log says it ended on its own, since nothing launches it again; and every other -request, sent on to the app's server at its path, its answer streamed back. Everything but the card takes the env's -control token (``CONTROL_TOKEN_ENV``), which no request it sends on and none of the programs it runs is given. Needs -the ``app`` extra.""" -from __future__ import annotations - -import argparse -import asyncio -import contextlib -import json -import logging -import os -import shlex -import signal -import sys -import time -from dataclasses import asdict -from pathlib import Path -from typing import Any, AsyncIterator, Optional - -import httpx -import uvicorn -from starlette.applications import Starlette -from starlette.requests import Request -from starlette.responses import Response, StreamingResponse -from starlette.routing import Route - -from ..agent_env_environment import _error_response, _json_response, control_guard -from ..types import CONTROL_TOKEN_ENV, RPC_PATH, WELL_KNOWN_PATH, EnvironmentCard -from . import apps -from .app import SOCKET -from .app_runtime import LAUNCHES -from .errors import DeliveryFailed -from .operations import COMPUTER_READY, ReadyCheck - -logger = logging.getLogger(__name__) - -#: The env's one port, where the front door serves. -PORT = 8000 -#: Where it serves ``computer-ready/v1``'s status, as a driver does. -STATUS_PATH = f"{RPC_PATH}/ext/computer-ready/status" -#: The delays before a program that ended starts again: each in turn while it keeps ending within ``STABLE_S`` of -#: starting, then the last, and the first again once a run has lasted that long. -RESTART_DELAYS_S = (0.5, 1.0, 2.0, 5.0, 10.0, 30.0) -STABLE_S = 60.0 -#: How long a program asked to stop may take before it's killed. -STOP_TIMEOUT_S = 10.0 -#: The headers of one hop, and the control token's, which no request or answer it sends on carries. -UNFORWARDED = frozenset({"authorization", "connection", "host", "keep-alive", "proxy-authenticate", - "proxy-authorization", "te", "trailer", "transfer-encoding", "upgrade"}) -METHODS = ["GET", "POST", "PUT", "PATCH", "DELETE"] -READY_EXTENSION = { - "uri": COMPUTER_READY, - "description": "Whether the app running alone is ready, by its own checks and its front door's, or failed for " - "good, and why.", - "required": False, - "params": {"methods": {"status": {"method": "GET", "endpoint": STATUS_PATH, - "request": {"type": "object", "properties": {}}}}}, -} -USAGE = "python -m agentenv_protocol.computer.front_door : [--site-proxy COMMAND] [--port PORT]" - - -class Program: - """A command the front door runs in ``cwd``, started again whenever it ends, until it's stopped.""" - - def __init__(self, name: str, argv: list[str], cwd: Path) -> None: - self.name, self.argv, self.cwd = name, argv, cwd - self.process: Optional[asyncio.subprocess.Process] = None - self.stopping = False - - @property - def running(self) -> bool: - return self.process is not None and self.process.returncode is None - - async def run(self) -> None: - """Run it, and start it again each time it ends, with the environment the front door has.""" - ending_soon = 0 - while not self.stopping: - started = time.monotonic() - try: - self.process = await asyncio.create_subprocess_exec(*self.argv, cwd=self.cwd, - stdin=asyncio.subprocess.DEVNULL) - how = _how(await self.process.wait()) - except OSError as e: - how = f"couldn't start: {e}" - except asyncio.CancelledError: - if self.running: - self.process.kill() - raise - if self.stopping: - return - ran = time.monotonic() - started - if ran >= STABLE_S: - ending_soon = 0 - delay = RESTART_DELAYS_S[min(ending_soon, len(RESTART_DELAYS_S) - 1)] - ending_soon += 1 - logger.warning(f"{self.name} ended after {ran:.1f} s ({how}); starting it again in {delay:g} s") - await asyncio.sleep(delay) - - async def stop(self) -> None: - """Ask it to end, and kill it after ``STOP_TIMEOUT_S``; it isn't started again.""" - self.stopping = True - if not self.running: - return - self.process.terminate() - try: - await asyncio.wait_for(self.process.wait(), STOP_TIMEOUT_S) - except asyncio.TimeoutError: - self.process.kill() - await self.process.wait() - - -class FrontDoor: - """A computer app alone: the app's server in ``folder``, serving the class ``served`` names, and the site proxy - command, the programs it runs; and what it serves.""" - - def __init__(self, folder: Path, served: str, site_proxy: Optional[str] = None) -> None: - self.folder = folder - self.name = os.environ.get("ENVIRONMENT_NAME", folder.name) - self.server = Program("the app's server", [sys.executable, "-m", "agentenv_protocol.computer.serve_app", served], - folder) - self.site_proxy = Program("the site proxy", shlex.split(site_proxy), folder) if site_proxy else None - self.programs = [self.server, *([self.site_proxy] if self.site_proxy else [])] - self.http: Optional[httpx.AsyncClient] = None - - def app(self) -> Starlette: - """What it serves; its guard takes the control token the environment has now.""" - guard = control_guard() - return Starlette(routes=[Route(WELL_KNOWN_PATH, self.card, methods=["GET"]), - Route(STATUS_PATH, guard(self.status), methods=["GET"]), - Route("/{path:path}", guard(self.forward), methods=METHODS)], - lifespan=self.lifespan) - - @contextlib.asynccontextmanager - async def lifespan(self, app: Starlette) -> AsyncIterator[None]: - self.http = httpx.AsyncClient(transport=httpx.AsyncHTTPTransport(uds=str(self.folder / SOCKET)), - base_url="http://app", timeout=None) - runs = [asyncio.ensure_future(program.run()) for program in self.programs] - try: - yield - finally: - await asyncio.gather(*(program.stop() for program in self.programs)) - for run in runs: - run.cancel() - await self.http.aclose() - - async def card(self, request: Request) -> Response: - """The app's card as its server serves it, with no interface for agents and with ``computer-ready/v1``; while - its server isn't answering, its name and ``computer-ready/v1`` alone.""" - card = await apps.served_card(self.folder) or EnvironmentCard(name=self.name, url=RPC_PATH).model_dump() - card["additionalInterfaces"] = [] - capabilities = card.get("capabilities") or {} - capabilities["extensions"] = [*(capabilities.get("extensions") or []), READY_EXTENSION] - card["capabilities"] = capabilities - return _json_response(card) - - async def status(self, request: Request) -> Response: - return _json_response(await self.readiness()) - - async def readiness(self) -> dict[str, Any]: - """``computer-ready/v1``'s status: the app's readiness as its server says, or why its server isn't answering, - and whether the site proxy runs; ``failed`` too, once the app can't be ready again.""" - checks = [await apps.check(self.folder)] - if self.site_proxy is not None: - checks.append(ReadyCheck("site proxy", self.site_proxy.running, - "running" if self.site_proxy.running else "not running; it starts again")) - failed = self.failure() - if failed is not None: - checks[0] = ReadyCheck(checks[0].name, False, failed) - return {"ready": failed is None and all(check.ok for check in checks), - "checks": [asdict(check) for check in checks], **({"failed": failed} if failed else {})} - - def failure(self) -> Optional[str]: - """Why the app can't be ready again: its launch log says it ended on its own, and alone nothing launches it - again, an infrastructure failure.""" - for entry in _launch_log(self.folder): - if entry.get("event") == "exit": - how = (f"by signal {entry['signal']}" if "signal" in entry else - f"with code {entry['code']}" if "code" in entry else "in a way that wasn't recorded") - return (f"it ended on its own at {entry.get('at')}, {how}, and nothing launches it again: an " - "infrastructure failure") - return None - - async def forward(self, request: Request) -> Response: - """``request`` sent on to the app's server at its path, but for its hop and token headers, and its answer - streamed back; it takes as long as the app does.""" - sent = self.http.build_request( - request.method, httpx.URL(path=request.url.path, query=request.url.query.encode()), - headers=_forwarded(request.headers.items()), content=await request.body()) - try: - answer = await self.http.send(sent, stream=True) - except (httpx.HTTPError, OSError) as e: - return _error_response(DeliveryFailed(f"the app's server isn't answering: {type(e).__name__}: {e}")) - async def body() -> AsyncIterator[bytes]: - try: - async for chunk in answer.aiter_raw(): - yield chunk - finally: - await answer.aclose() - - response = StreamingResponse(body(), status_code=answer.status_code) - response.raw_headers = [(name.encode("latin-1"), value.encode("latin-1")) - for name, value in _forwarded(answer.headers.multi_items())] - return response - - -def _forwarded(headers: Any) -> list[tuple[str, str]]: - """``headers`` but those of one hop: the standard ones, the control token's and any its ``Connection`` names.""" - headers = list(headers) - named = {token.strip().lower() for name, value in headers if name.lower() == "connection" - for token in value.split(",")} - return [(name, value) for name, value in headers if name.lower() not in UNFORWARDED | named] - - -def _launch_log(folder: Path) -> list[dict]: - """Each entry of the app's launch log that's whole, in order.""" - try: - lines = (folder / LAUNCHES).read_text(encoding="utf-8").splitlines() - except FileNotFoundError: - return [] - entries = [] - for line in lines: - with contextlib.suppress(ValueError): - entries.append(json.loads(line)) - return entries - - -def _how(code: int) -> str: - return f"signal {signal.Signals(-code).name}" if code < 0 else f"code {code}" - - -def main(argv: list[str]) -> None: - parser = argparse.ArgumentParser(prog="front_door", usage=USAGE) - parser.add_argument("folder", type=Path) - parser.add_argument("served") - parser.add_argument("--site-proxy") - parser.add_argument("--port", type=int, default=PORT) - args = parser.parse_args(argv) - logging.basicConfig(level=logging.WARNING, format="%(asctime)s front door: %(message)s") - folder = args.folder.resolve() - os.environ.setdefault("ENVIRONMENT_NAME", folder.name) - served = FrontDoor(folder, args.served, args.site_proxy).app() - os.environ.pop(CONTROL_TOKEN_ENV, None) - uvicorn.run(served, host="0.0.0.0", port=args.port, log_level="warning") - - -if __name__ == "__main__": - main(sys.argv[1:]) diff --git a/packages/agentenv-protocol/src/agentenv_protocol/computer/operations.py b/packages/agentenv-protocol/src/agentenv_protocol/computer/operations.py index d4583f43c..1e9936ec1 100644 --- a/packages/agentenv-protocol/src/agentenv_protocol/computer/operations.py +++ b/packages/agentenv-protocol/src/agentenv_protocol/computer/operations.py @@ -69,10 +69,6 @@ class ExecResult: truncated: bool = False -#: The extension whose status says, by these checks, whether a computer, or an app running alone, is ready. -COMPUTER_READY = "urn:agentenv:computer-ready/v1" - - @dataclass(frozen=True) class ReadyCheck: name: str diff --git a/packages/agentenv-protocol/tests/test_computer_front_door.py b/packages/agentenv-protocol/tests/test_computer_front_door.py deleted file mode 100644 index d3f6fc15f..000000000 --- a/packages/agentenv-protocol/tests/test_computer_front_door.py +++ /dev/null @@ -1,362 +0,0 @@ -"""A computer app alone behind its front door, run as its container runs it, with a fake browser that answers CDP on its -pipe: the app's card with computer-ready/v1 and no interface for agents; every route but the card taking the control -token, which no program it runs is given; requests sent on to the app's server, a large answer streamed whole; a killed -app server started again, which launches the browser again; a browser that ends on its own failing the env for good, -with its signal; a killed site proxy started again; an app server that can't start started again later and later, -saying why; requests while the app's server restarts answering or failing, never hanging; stopping it stopping -everything it runs; and its needing only the app extra, as the app's venv has.""" -from __future__ import annotations - -import asyncio -import contextlib -import json -import os -import shutil -import signal -import socket -import subprocess -import sys -import tempfile -import time -from pathlib import Path - -import pytest - -pytest.importorskip("mcp.server.fastmcp") -pytest.importorskip("uvicorn") - -import httpx # noqa: E402 - -from agentenv_protocol import client # noqa: E402 -from agentenv_protocol.computer import front_door # noqa: E402 -from agentenv_protocol.computer.operations import COMPUTER_READY # noqa: E402 -from agentenv_protocol.types import CONTROL_TOKEN_ENV, WELL_KNOWN_PATH # noqa: E402 -from agentenv_protocol.website_browser import ALONE_ENV, BROWSER, SITE_PROXY_ENV # noqa: E402 -from test_website_browser import FAKE_BROWSER # noqa: E402 - -TOKEN = "front-door-token" -BEARER = {"authorization": f"Bearer {TOKEN}"} -#: How long it may take to start, or to recover; how often it's asked meanwhile. -WAIT_S, POLL_S = 30.0, 0.2 -#: How large an answer streams through whole. -LARGE_BYTES = 20_000_000 -SLEEPER = f"{sys.executable} -c 'import time; time.sleep(600)'" -# The app, as its folder holds it: a website browser whose launch runs the fake browser, on Linux, its launches marked -# in a folder of the test's own, with an extension that says whether its server was given the control token and one -# that answers with a large body. -APP = f"""import os, sys -from pathlib import Path -from agentenv_protocol.agent_env_environment import extension -from agentenv_protocol.computer import Launch, Ready, app, app_runtime, linux -from agentenv_protocol.computer.platform import Platform -from agentenv_protocol.website_browser import WebsiteBrowserEnvironment - -app.detect = lambda: Platform(os="linux") -app_runtime.LAUNCHED = Path(os.environ["TEST_LAUNCHED"]) - - -@linux(launch=Launch(command=[sys.executable, str(Path(__file__).with_name("fake_browser.py"))], - ready=Ready(command=["true"]))) -class Browser(WebsiteBrowserEnvironment): - @extension("urn:test:probe/v1", op="token", method="GET") - async def token(self) -> dict: - return {{"given": "{CONTROL_TOKEN_ENV}" in os.environ}} - - @extension("urn:test:probe/v1", op="large", method="GET") - async def large(self) -> dict: - return {{"data": "x" * {LARGE_BYTES}}} -""" - - -def _free_port() -> int: - with socket.socket() as sock: - sock.bind(("127.0.0.1", 0)) - return sock.getsockname()[1] - - -class Door: - """A front door running as its container's command, on a port of its own, with the app in a short folder, since a - Unix socket's path is short.""" - - def __init__(self, served: str = "browser.py:Browser", site_proxy: str | None = None) -> None: - self.folder = Path(tempfile.mkdtemp(prefix="fd-", dir="/tmp")) - (self.folder / "browser.py").write_text(APP) - (self.folder / "fake_browser.py").write_text(FAKE_BROWSER) - (self.folder / "home").mkdir() - (self.folder / ".agentenv").mkdir() - self.port = _free_port() - self.url = f"http://127.0.0.1:{self.port}" - env = {key: value for key, value in os.environ.items() if key != "DISPLAY"} - env.update({"HOME": str(self.folder / "home"), "ENVIRONMENT_NAME": "browser", ALONE_ENV: "1", - SITE_PROXY_ENV: "http://127.0.0.1:9", "TEST_LAUNCHED": str(self.folder / "launched"), - CONTROL_TOKEN_ENV: TOKEN}) - argv = [sys.executable, "-m", "agentenv_protocol.computer.front_door", str(self.folder), served, - "--port", str(self.port), *(["--site-proxy", site_proxy] if site_proxy else [])] - self.log = (self.folder / "front-door.log").open("w") - self.process = subprocess.Popen(argv, env=env, stdout=self.log, stderr=subprocess.STDOUT) - - def children(self, matching: str) -> list[int]: - found = subprocess.run(["pgrep", "-P", str(self.process.pid), "-f", matching], capture_output=True, text=True) - return [int(pid) for pid in found.stdout.split()] - - def launches(self) -> list[dict]: - try: - lines = (self.folder / ".agentenv" / "launches").read_text().splitlines() - except FileNotFoundError: - return [] - return [json.loads(line) for line in lines if line] - - def logged(self) -> str: - return (self.folder / "front-door.log").read_text() - - async def status(self) -> dict: - async with httpx.AsyncClient(base_url=self.url, timeout=10) as http: - return (await http.get(front_door.STATUS_PATH, headers=BEARER)).json() - - async def until(self, holds, what: str): - """``holds``' answer once it's truthy, within ``WAIT_S``.""" - last = None - deadline = time.monotonic() + WAIT_S - while time.monotonic() < deadline: - with contextlib.suppress(httpx.HTTPError, ValueError, OSError): - last = await holds() - if last: - return last - await asyncio.sleep(POLL_S) - raise AssertionError(f"{what} within {WAIT_S:g} s; last {last!r}; its log:\n{self.logged()}") - - async def ready(self) -> dict: - return await self.until(lambda: self._ready(), "ready") - - async def _ready(self): - status = await self.status() - return status if status["ready"] else None - - def close(self) -> None: - if self.process.poll() is None: - self.process.terminate() - with contextlib.suppress(subprocess.TimeoutExpired): - self.process.wait(15) - if self.process.poll() is None: - self.process.kill() - self.process.wait() - for entry in self.launches(): - with contextlib.suppress(ProcessLookupError, PermissionError): - os.killpg(entry["pid"], signal.SIGKILL) - self.log.close() - shutil.rmtree(self.folder, ignore_errors=True) - - -@pytest.fixture -def doors(): - opened: list[Door] = [] - - def open_door(**kwargs) -> Door: - door = Door(**kwargs) - opened.append(door) - return door - - yield open_door - for door in opened: - door.close() - - -def _browser(door: Door) -> int: - """The pid of the browser the app's server launched last.""" - return [entry for entry in door.launches() if entry["event"] == "launch"][-1]["pid"] - - -@pytest.mark.asyncio -async def test_its_card_is_the_apps_with_computer_ready_and_no_interface_for_agents_and_only_the_card_is_open(doors): - door = doors(site_proxy=SLEEPER) - status = await door.ready() - async with httpx.AsyncClient(base_url=door.url, timeout=10) as http: - card = (await http.get(WELL_KNOWN_PATH)).json() - ready = client.find_extension_method(card, "urn:agentenv:computer-app/v1", "ready")["endpoint"] - unguarded = [(await http.get(path)).status_code for path in (front_door.STATUS_PATH, ready)] - wrong = (await http.get(ready, headers={"authorization": "Bearer nope"})).status_code - app_ready = (await http.get(ready, headers=BEARER)).json() - token = (await http.get(client.find_extension_method(card, "urn:test:probe/v1", "token")["endpoint"], - headers=BEARER)).json() - - assert card["name"] == "browser" and card["additionalInterfaces"] == [] - assert client.find_extension(card, BROWSER) is not None - assert client.find_extension_method(card, COMPUTER_READY, "status") == { - "method": "GET", "endpoint": front_door.STATUS_PATH, "request": {"type": "object", "properties": {}}} - assert status == {"ready": True, "checks": [{"name": "app browser", "ok": True, "detail": "ready"}, - {"name": "site proxy", "ok": True, "detail": "running"}]} - assert unguarded == [401, 401] and wrong == 401 - assert app_ready["ready"] is True and token == {"given": False} - assert [entry["event"] for entry in door.launches()] == ["launch"] - - -@pytest.mark.asyncio -async def test_a_large_answer_streams_through_whole(doors): - door = doors() - await door.ready() - async with httpx.AsyncClient(base_url=door.url, timeout=60) as http: - card = (await http.get(WELL_KNOWN_PATH)).json() - answer = await http.get(client.find_extension_method(card, "urn:test:probe/v1", "large")["endpoint"], - headers=BEARER) - - assert answer.status_code == 200 and len(answer.json()["data"]) == LARGE_BYTES - - -@pytest.mark.asyncio -async def test_a_killed_app_server_starts_again_and_launches_the_browser_again_on_a_new_pipe(doors): - door = doors() - await door.ready() - first_browser = _browser(door) - (server,) = door.children("serve_app") - os.kill(server, signal.SIGKILL) - - async def relaunched(): - return door.launches()[-1].get("cause") == "server_restart" and (await door.status())["ready"] - - await door.until(relaunched, "the browser launched again and ready") - (restarted,) = door.children("serve_app") - - # The browser ends with its pipe, so the server finds nothing to stop, and no one logs it as its own exit. - assert restarted != server and [entry["event"] for entry in door.launches()] in ( - ["launch", "launch"], ["launch", "stop", "launch"]) - assert door.launches()[0]["cause"] == "boot" and _browser(door) != first_browser - assert "the app's server ended" in door.logged() and "signal SIGKILL" in door.logged() - - -@pytest.mark.asyncio -async def test_a_browser_that_ends_on_its_own_fails_the_env_for_good_with_its_signal(doors): - door = doors() - await door.ready() - os.killpg(_browser(door), signal.SIGKILL) - - async def failed(): - status = await door.status() - return status if "failed" in status else None - - status = await door.until(failed, "failed") - (server,) = door.children("serve_app") - os.kill(server, signal.SIGKILL) - - async def relaunched(): - return [entry["event"] for entry in door.launches()][-1] == "launch" and len(door.launches()) >= 3 - - await door.until(relaunched, "the browser launched again by the restarted server") - await asyncio.sleep(1) - after = await door.status() - - assert status["ready"] is False and "by signal SIGKILL" in status["failed"] - assert "an infrastructure failure" in status["failed"] - assert status["checks"][0] == {"name": "app browser", "ok": False, "detail": status["failed"]} - assert [entry["event"] for entry in door.launches()][:2] == ["launch", "exit"] - assert door.launches()[1]["signal"] == "SIGKILL" - assert after["ready"] is False and after["failed"] == status["failed"] - - -@pytest.mark.asyncio -async def test_a_killed_site_proxy_starts_again(doors): - door = doors(site_proxy=SLEEPER) - await door.ready() - (proxy,) = door.children("time.sleep") - os.kill(proxy, signal.SIGKILL) - - async def restarted(): - return [pid for pid in door.children("time.sleep") if pid != proxy] - - (again,) = await door.until(restarted, "the site proxy started again") - status = await door.ready() - - assert again != proxy and status["checks"][1] == {"name": "site proxy", "ok": True, "detail": "running"} - assert "the site proxy ended" in door.logged() - - -@pytest.mark.asyncio -async def test_an_app_server_that_cant_start_is_started_again_later_and_later_and_says_why(doors): - door = doors(served="browser.py:Missing") - - async def said_why(): - status = await door.status() - return status if "failed to start" in status["checks"][0]["detail"] else None - - status = await door.until(said_why, "the server's failure") - await door.until(lambda: asyncio.sleep(0, "starting it again in 2 s" in door.logged()), "a 2 s delay") - - assert status["ready"] is False and "Missing" in status["checks"][0]["detail"] - assert "failed" not in status - delays = [line.rsplit(" in ", 1)[1] for line in door.logged().splitlines() if "starting it again in" in line] - assert delays[:3] == ["0.5 s", "1 s", "2 s"] - - -@pytest.mark.asyncio -async def test_requests_while_its_app_server_restarts_answer_or_fail_and_never_hang(doors): - door = doors() - await door.ready() - (server,) = door.children("serve_app") - async with httpx.AsyncClient(base_url=door.url, timeout=WAIT_S) as http: - card = (await http.get(WELL_KNOWN_PATH)).json() - ready = client.find_extension_method(card, "urn:agentenv:computer-app/v1", "ready")["endpoint"] - os.kill(server, signal.SIGKILL) - answers = await asyncio.gather(*(http.get(ready, headers=BEARER) for _ in range(30)), - *(http.get(WELL_KNOWN_PATH) for _ in range(10))) - await door.ready() - - assert {answer.status_code for answer in answers} <= {200, 502} - assert all(answer.json()["name"] == "browser" for answer in answers[30:]) - - -@pytest.mark.asyncio -async def test_stopping_it_stops_every_program_it_runs(doors): - door = doors(site_proxy=SLEEPER) - await door.ready() - programs = door.children("serve_app") + door.children("time.sleep") - door.process.send_signal(signal.SIGTERM) - code = await asyncio.to_thread(door.process.wait, WAIT_S) - - def gone(pid: int) -> bool: - try: - os.kill(pid, 0) - except ProcessLookupError: - return True - return False - - # uvicorn ends with the signal it shut down on, once it has. - assert code in (0, -signal.SIGTERM) and len(programs) == 2 and all(gone(pid) for pid in programs) - - -def test_it_needs_only_the_app_extra(): - blocked = """ -import importlib.abc, sys -class Blocked(importlib.abc.MetaPathFinder): - def find_spec(self, name, path, target=None): - if name.split(".")[0] in ("av", "PIL"): - raise ImportError(name) -sys.meta_path.insert(0, Blocked()) -import agentenv_protocol.computer.front_door -""" - done = subprocess.run([sys.executable, "-c", blocked], capture_output=True, text=True) - - assert done.returncode == 0, done.stderr - - -def test_it_sends_on_no_hop_or_token_header_nor_one_its_connection_header_names(): - headers = [("authorization", "Bearer x"), ("Host", "a"), ("content-type", "application/json"), - ("Transfer-Encoding", "chunked"), ("x-forwarded-host", "b"), ("Connection", "close, X-Internal"), - ("x-internal", "1"), ("x-kept", "2")] - - assert front_door._forwarded(headers) == [("content-type", "application/json"), ("x-forwarded-host", "b"), - ("x-kept", "2")] - - -@pytest.mark.asyncio -async def test_a_program_that_wont_stop_is_killed_once_its_stop_runs_out(monkeypatch, tmp_path): - monkeypatch.setattr(front_door, "STOP_TIMEOUT_S", 0.5) - stubborn = front_door.Program("stubborn", [sys.executable, "-c", ( - "import signal, time; signal.signal(signal.SIGTERM, signal.SIG_IGN); open('up', 'w').close(); " - "time.sleep(60)")], tmp_path) - running = asyncio.ensure_future(stubborn.run()) - deadline = time.monotonic() + WAIT_S - while not (tmp_path / "up").exists() and time.monotonic() < deadline: - await asyncio.sleep(POLL_S) - await stubborn.stop() - await asyncio.wait_for(running, WAIT_S) - - assert stubborn.process.returncode == -signal.SIGKILL diff --git a/tst/integration/computer/front_door_test.py b/tst/integration/computer/front_door_test.py deleted file mode 100644 index d06e7a8c0..000000000 --- a/tst/integration/computer/front_door_test.py +++ /dev/null @@ -1,256 +0,0 @@ -"""The shipped chromium alone behind its front door, in the chromium test bundle's composed image with the front door as -its command in place of the desktop's, as the browser provider will run it: headless, out only through a stand-in site -proxy; its card and browser/v1's record on the front door's port, with the control token, which no program it runs is -given, and no TCP port listening but the front door's and the proxy's; its killed app server started again, which -launches Chromium again on a new pipe; Chromium killed failing the env for good, with its signal, which a restarted app -server doesn't undo; and its killed site proxy started again. Needs Docker and the local registry on localhost:5000.""" - -import asyncio -import json -import platform -import shlex -import time -import uuid - -import subprocess - -import httpx -import pytest - -from agentenv_protocol import client as protocol_v1 -from agentenv_protocol.computer import front_door -from agentenv_protocol.computer.operations import COMPUTER_READY -from agentenv_protocol.types import CONTROL_TOKEN_ENV, WELL_KNOWN_PATH -from agentenv_protocol.website_browser import ALONE_ENV, BROWSER, SITE_PROXY_ENV - -pytestmark = [pytest.mark.integration, pytest.mark.int_test_slow] - -ARCH = {"x86_64": "amd64", "amd64": "amd64", "aarch64": "arm64", "arm64": "arm64"}[platform.machine().lower()] -APP = "/opt/agentenv/apps/chromium" -PROXY_PORT = 3128 -SITE = "http://site.test/" -TOKEN = uuid.uuid4().hex -BEARER = {"authorization": f"Bearer {TOKEN}"} -#: How long the browser may take to be ready, or to recover; how often it's asked meanwhile. -WAIT_S, POLL_S = 120.0, 1.0 -# A stand-in for the site proxy: any page asked for through it is one that names its host. -SITE_PROXY = f"""import http.server, urllib.parse -class Proxy(http.server.BaseHTTPRequestHandler): - def do_GET(self): - host = urllib.parse.urlsplit(self.path).hostname or "" - body = f"proxied {{host}}".encode() - self.send_response(200) - self.send_header("Content-Type", "text/html") - self.send_header("Content-Length", str(len(body))) - self.end_headers() - self.wfile.write(body) - def log_message(self, *args): - pass -http.server.ThreadingHTTPServer(("127.0.0.1", {PROXY_PORT}), Proxy).serve_forever() -""" -# Every TCP port something in the container listens on. -LISTENING = """ports = set() -for table in ("/proc/net/tcp", "/proc/net/tcp6"): - for line in open(table).read().splitlines()[1:]: - fields = line.split() - if fields[3] == "0A": - ports.add(int(fields[1].rsplit(":", 1)[1], 16)) -print(sorted(ports)) -""" - - -def docker(*args: str) -> subprocess.CompletedProcess: - return subprocess.run(["docker", *args], check=True, capture_output=True) - - -class Alone: - """One container of the composed image running chromium alone behind its front door.""" - - def __init__(self, image: str) -> None: - conf = docker("run", "--rm", "--entrypoint", "cat", image, - "/etc/supervisor/conf.d/agentenv-app-chromium.conf").stdout.decode() - served = next(line for line in conf.splitlines() if "serve_app" in line).rsplit(" ", 1)[1] - settings = json.dumps({"chromium": {"start_urls": [SITE]}}) - self.name = f"front-door-{uuid.uuid4().hex[:8]}" - docker("run", "-d", "--name", self.name, "-p", "127.0.0.1::8000", "-e", "DISPLAY=", - "-e", f"{ALONE_ENV}=1", "-e", f"{SITE_PROXY_ENV}=http://127.0.0.1:{PROXY_PORT}", - "-e", "ENVIRONMENT_NAME=chromium", "-e", f"{CONTROL_TOKEN_ENV}={TOKEN}", - "-e", f"AGENTENV_APP_SETTINGS={settings}", image, - f"{APP}/venv/bin/python", "-m", "agentenv_protocol.computer.front_door", APP, served, - "--site-proxy", shlex.join(["python3", "-c", SITE_PROXY])) - port = docker("port", self.name, "8000").stdout.decode().split()[0].rsplit(":", 1)[1] - self.url = f"http://127.0.0.1:{port}" - - def run(self, *argv: str) -> str: - return docker("exec", self.name, *argv).stdout.decode() - - def pids(self, pattern: str) -> list[int]: - """The front door's programs whose command line has ``pattern``.""" - door = "$(pgrep -n -f '[a]gentenv_protocol.computer.front_door')" - found = self.run("sh", "-c", f"pgrep -P {door} -f -- {shlex.quote(pattern)} || true") - return [int(pid) for pid in found.split()] - - def launches(self) -> list[dict]: - log = self.run("sh", "-c", f"cat {APP}/.agentenv/launches 2>/dev/null || true") - return [json.loads(line) for line in log.splitlines() if line] - - async def get(self, path: str, **kwargs) -> httpx.Response: - async with httpx.AsyncClient(base_url=self.url, timeout=30) as http: - return await http.get(path, **kwargs) - - async def status(self) -> dict: - return (await self.get(front_door.STATUS_PATH, headers=BEARER)).json() - - async def record(self) -> dict: - card = (await self.get(WELL_KNOWN_PATH)).json() - path = protocol_v1.find_extension_method(card, BROWSER, "record")["endpoint"] - return (await self.get(path, headers=BEARER)).json() - - def logs(self) -> str: - logged = docker("logs", self.name) - return (logged.stdout + logged.stderr).decode()[-4000:] - - async def until(self, holds, what: str): - """``holds``' answer once it's truthy, within ``WAIT_S``, while the container runs.""" - last, deadline = None, time.monotonic() + WAIT_S - while time.monotonic() < deadline: - if docker("inspect", "-f", "{{.State.Running}}", self.name).stdout.decode().strip() != "true": - raise AssertionError(f"the container ended before {what}; its logs:\n{self.logs()}") - try: - last = await holds() - if last: - return last - except (httpx.HTTPError, ValueError, KeyError): - pass - await asyncio.sleep(POLL_S) - raise AssertionError(f"{what} within {WAIT_S:g} s; last {last!r}; its logs:\n{self.logs()}") - - async def ready(self) -> dict: - async def ready(): - status = await self.status() - return status if status["ready"] else None - return await self.until(ready, "ready") - - async def on_the_site(self) -> dict: - async def loaded(): - record = await self.record() - return record if [tab["title"] for tab in record["tabs"]] == ["proxied site.test"] else None - return await self.until(loaded, "its start tab loaded through the site proxy") - - def remove(self) -> None: - docker("rm", "-f", self.name) - - -@pytest.fixture -def alone(chromium_desktop): - image = chromium_desktop.metadata["composed"]["container"][ARCH]["image"] - started = [] - - def start() -> Alone: - started.append(Alone(image)) - return started[-1] - - yield start - for container in started: - container.remove() - - -def _browser(container: Alone) -> int: - return [entry for entry in container.launches() if entry["event"] == "launch"][-1]["pid"] - - -@pytest.mark.asyncio -async def test_alone_it_serves_its_card_status_and_record_on_its_port_with_the_token_and_nothing_else_listens(alone): - container = alone() - status = await container.ready() - record = await container.on_the_site() - card = (await container.get(WELL_KNOWN_PATH)).json() - record_path = protocol_v1.find_extension_method(card, BROWSER, "record")["endpoint"] - refused = [(await container.get(path)).status_code for path in (front_door.STATUS_PATH, record_path)] - wrong = (await container.get(record_path, headers={"authorization": "Bearer nope"})).status_code - listening = json.loads(container.run("python3", "-c", LISTENING)) - (server,) = container.pids("serve_app") - environments = [container.run("sh", "-c", f"tr '\\0' '\\n' < /proc/{pid}/environ") - for pid in (server, _browser(container))] - flags = container.run("sh", "-c", f"tr '\\0' ' ' < /proc/{_browser(container)}/cmdline") - - assert card["name"] == "chromium" and card["additionalInterfaces"] == [] - assert protocol_v1.find_extension_method(card, COMPUTER_READY, "status")["endpoint"] == front_door.STATUS_PATH - assert [check["name"] for check in status["checks"]] == ["app chromium", "site proxy"] - assert [(tab["url"], tab["active"]) for tab in record["tabs"]] == [(SITE, True)] - assert [(entry["event"], entry["cause"]) for entry in record["launches"]] == [("launch", "boot")] - assert refused == [401, 401] and wrong == 401 - assert listening == sorted([8000, PROXY_PORT]) - assert all(f"{CONTROL_TOKEN_ENV}=" not in environment for environment in environments) - assert "--headless=new" in flags and "--remote-debugging-pipe" in flags - assert f"--proxy-server=http://127.0.0.1:{PROXY_PORT}" in flags - - -@pytest.mark.asyncio -async def test_its_killed_app_server_starts_again_and_launches_chromium_again_on_a_new_pipe(alone): - container = alone() - await container.ready() - await container.on_the_site() - first = _browser(container) - (server,) = container.pids("serve_app") - container.run("kill", "-9", str(server)) - - async def relaunched(): - return container.launches()[-1].get("cause") == "server_restart" and (await container.status())["ready"] - - await container.until(relaunched, "Chromium launched again and ready") - record = await container.on_the_site() - (restarted,) = container.pids("serve_app") - - assert restarted != server and _browser(container) != first - assert [entry["event"] for entry in container.launches()] in (["launch", "launch"], ["launch", "stop", "launch"]) - assert "exit" not in [entry["event"] for entry in record["launches"]] - - -@pytest.mark.asyncio -async def test_chromium_killed_fails_the_env_for_good_with_its_signal_even_once_its_server_restarts(alone): - container = alone() - await container.ready() - container.run("kill", "-9", str(_browser(container))) - - async def failed(): - status = await container.status() - return status if "failed" in status else None - - status = await container.until(failed, "failed") - record = await container.record() - (server,) = container.pids("serve_app") - container.run("kill", "-9", str(server)) - - async def relaunched(): - return container.launches()[-1].get("cause") == "server_restart" - - await container.until(relaunched, "Chromium launched again by the restarted server") - after = await container.status() - - assert status["ready"] is False and "by signal SIGKILL" in status["failed"] - assert status["checks"][0]["ok"] is False and status["checks"][0]["detail"] == status["failed"] - assert [entry["event"] for entry in record["launches"]] == ["launch", "exit"] - assert record["launches"][1]["signal"] == "SIGKILL" - assert [tab["closed_by"] for tab in record["closed_tabs"]] == ["exit"] - assert after["ready"] is False and after["failed"] == status["failed"] - - -@pytest.mark.asyncio -async def test_its_killed_site_proxy_starts_again(alone): - container = alone() - await container.ready() - (proxy,) = container.pids("ThreadingHTTPServer") - container.run("kill", "-9", str(proxy)) - - async def restarted(): - return [pid for pid in container.pids("ThreadingHTTPServer") if pid != proxy] - - (again,) = await container.until(restarted, "the site proxy started again") - status = await container.ready() - fetched = container.run("python3", "-c", "import urllib.request; print(urllib.request.build_opener(" - f"urllib.request.ProxyHandler({{'http': 'http://127.0.0.1:{PROXY_PORT}'}}))" - f".open('{SITE}').read().decode())") - - assert again != proxy and status["checks"][1] == {"name": "site proxy", "ok": True, "detail": "running"} - assert fetched.strip() == "proxied site.test" From b54fce11e054b8903aed0cd890c37feca29ad155 Mon Sep 17 00:00:00 2001 From: Edgar Arakelyan Date: Tue, 3 Nov 2026 08:23:57 -0800 Subject: [PATCH 5/5] test(computer): the popup test's page opens its popup on Enter, so the click before a key press can't open it early Co-Authored-By: Claude Opus 5.5 --- tst/integration/computer/chromium_app_test.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tst/integration/computer/chromium_app_test.py b/tst/integration/computer/chromium_app_test.py index 89f671ea5..aac3fb21f 100644 --- a/tst/integration/computer/chromium_app_test.py +++ b/tst/integration/computer/chromium_app_test.py @@ -60,15 +60,15 @@ def log_message(self, *args): time.sleep(float(sys.argv[1])) """ POPPING = "http://127.0.0.1:8768/" -# Serves, on loopback for the seconds it's given, a page that is all one button, which opens /popped in a popup once. +# Serves, on loopback for the seconds it's given, a page that opens /popped in a popup once, when Enter is pressed. SERVE_POPPING = """import http.server, sys, threading, time class Page(http.server.BaseHTTPRequestHandler): def do_GET(self): if self.path == "/popped": body = b"Popped page

Popped

" else: - body = (b"Opener page") + body = (b"Opener page") self.send_response(200) self.send_header("Content-Type", "text/html") self.send_header("Content-Length", str(len(body))) @@ -218,9 +218,9 @@ async def test_a_popup_a_page_opens_loads_with_its_control_attached(chromium_des await asyncio.sleep(SERVE_START_S) await _keys(deployed, ["F5"]) await _chromium_windows(deployed, "Opener page") - await _keys(deployed) + await _keys(deployed, ["Enter"]) popped = await _record(deployed, card, lambda r: any(tab["title"] == "Popped page" for tab in r["tabs"])) - windows = await _chromium_windows(deployed, "Popped page", count=2) + windows = await _chromium_windows(deployed, "Popped page") serving.cancel() finally: report = await teardown_run(context)