Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions DECISIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -19804,3 +19804,25 @@ identity the lock-guarded `overlay_agent` call already gives).
**Known limits:** an agent that alternates between a coordinate tool and an
element tool can show two cursors with its color and label for up to 10 s;
on Windows and Linux only agent-desktop's cursor exists.

## 2026-11-03 — TD-5230: the overlay helper starts at launch and gets a start-up budget (Class B)

**Decision:** tst-cu-mcp starts its macOS overlay helper in a background
thread when the server launches, and the first command after any spawn is a
`hide` probe allowed 15 seconds. Every later command keeps the 1 second ack
budget, and any failure still degrades the overlay to a quiet no-op.

**Rationale:** the packaged helper is the frozen `tstd`, which unpacks itself
before Python runs; its first ack took 2.4 to 5.4 seconds on a Mac. With the
old 1 second budget the overlay gave up on almost every cold start, so the
packaged app often had no ring and no cursors, while the source build (whose
helper starts in well under a second) always did. Starting at launch keeps
that wait off the agent's first action; the long first budget covers an
action that arrives before the helper is up.

**Rejected:** raising every ack budget (a hung helper would then stall each
action for seconds); a non-blocking start that drops commands until the
helper answers (the rules controller assumes a lit ring is lit, and would
need a replay path for nothing the probe does not already give); starting
the helper only when the overlay is first needed (that is the path that
failed).
21 changes: 21 additions & 0 deletions docs/tst-desk-backlog.md
Original file line number Diff line number Diff line change
Expand Up @@ -13042,6 +13042,27 @@ step 1 and the Mac app since TD-5228; on Windows the binary comes from

---

### TD-5230 — The packaged app's ring and cursors survive the helper's slow start
**Size:** 1 · **Depends on:** TD-4834, TD-5226, TD-5227
**Status:** In review 2026-11-03.

In the packaged Mac app the overlay helper is the frozen `tstd --cu-overlay`,
which unpacks itself before Python starts. Its first ack took 2.4 to 5.4
seconds on a Mac, but the sidecar waited 1 second for it and then switched the
overlay off for the rest of the process. So the packaged app often showed no
ring and no agent cursor at all, which the source build never shows. Found in
the TD-5227 live check: an episode opened right after the sidecar started left
no helper running.

**Acceptance criteria:**
- [x] The first command after a spawn is a `hide` probe allowed 15 seconds; later commands keep the 1 second budget
- [x] The server starts the helper in the background at launch, so the agent's first action rarely waits for it; a failed start still degrades quietly
- [x] Test children of the server run with the overlay off, so no test puts windows on the screen
- [x] Unit tests: a slow first ack is waited out (fails with the old 1 second budget), prewarm spawns once, a failed prewarm degrades
- [x] Live: the new code drives the packaged helper through a 5.4 second cold start and lights the ring on the targeted display

---

### TD-5227 — Each agent has its own cursor on the real display
**Size:** 5 · **Depends on:** TD-3402, TD-5220, TD-5226
**Status:** In review 2026-11-03.
Expand Down
7 changes: 7 additions & 0 deletions mcp/tst-cu-mcp/src/tst_cu_mcp/overlay/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,10 @@ def pointer(self, x: float, y: float, *, click: bool = False) -> None:
"""The acting agent's cursor goes to this global point (TD-5227)."""
...

def prewarm(self) -> None:
"""Start the painter's helper ahead of the first action (TD-5230)."""
...

@contextmanager
def grab_hidden(self) -> Iterator[None]:
"""Hide the glow for the duration of one screenshot grab."""
Expand Down Expand Up @@ -166,6 +170,9 @@ def use_agent(self, key: str, label: str = "") -> Agent:
def pointer(self, x: float, y: float, *, click: bool = False) -> None:
return None

def prewarm(self) -> None:
return None

def cursor(self, agent: Agent, x: float, y: float, click: bool) -> None:
return None

Expand Down
7 changes: 7 additions & 0 deletions mcp/tst-cu-mcp/src/tst_cu_mcp/overlay/control.py
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,13 @@ def grab_hidden(self) -> Iterator[None]:
with self._lock:
grab.__exit__(None, None, None)

def prewarm(self) -> None:
"""Start a slow-starting painter now, off the tool path (TD-5230)."""
warm = getattr(self.painter, "prewarm", None)
if callable(warm):
with contextlib.suppress(Exception):
warm()

def shutdown(self) -> None:
self._halt.set()
watcher = self._watcher
Expand Down
61 changes: 50 additions & 11 deletions mcp/tst-cu-mcp/src/tst_cu_mcp/overlay/darwin.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,15 @@
* ``quit`` ends the helper; stdin EOF does too, so a crashed sidecar never
leaves an orphaned halo behind.

Start-up (TD-5230): in the packaged app the helper is the frozen ``tstd``,
which unpacks itself before Python starts, so its first ack takes seconds
(2-5 s measured), not milliseconds. The first command after a spawn is a
``hide`` probe allowed :data:`STARTUP_ACK_TIMEOUT`; the server starts the
helper in the background at launch (:meth:`DarwinOverlay.prewarm`) so the
agent's first action rarely waits for it. Before this, the 1 s ack budget
expired on almost every cold start and the overlay switched itself off for
the life of the process, so the packaged app often showed no ring at all.

Failure rule (see the package docstring): any spawn, write, or ack trouble
degrades this manager — permanently — to a no-op. A capture whose grab ack
timed out still proceeds; worst case one frame contains the ring, and the
Expand All @@ -25,6 +34,7 @@
import select
import subprocess
import sys
import threading
from collections.abc import Callable, Iterator
from contextlib import contextmanager
from typing import Protocol
Expand All @@ -44,8 +54,10 @@
#: ``show``/``hide`` are acked by the helper's reader thread on arrival.
FAST_ACK_TIMEOUT = 1.0
#: ``grab_*`` wait for the helper's main thread to have acted (a tick is
#: ~33ms; the budget covers helper startup and a busy runloop).
#: ~33ms; the budget covers a busy runloop).
GRAB_ACK_TIMEOUT = 3.0
#: The first ack after a spawn: a frozen helper unpacks itself first.
STARTUP_ACK_TIMEOUT = 15.0


class Transport(Protocol):
Expand Down Expand Up @@ -131,19 +143,46 @@ def __init__(self, spawn: Callable[[], Transport] | None = None) -> None:
self._spawn = spawn or default_spawn
self._transport: Transport | None = None
self.degraded = False
# Serializes the prewarm thread with commands from the tool thread.
self._lock = threading.Lock()

def prewarm(self) -> None:
"""Start the helper in the background so the first action finds it up."""
if self.degraded or self._transport is not None:
return
threading.Thread(target=self._prewarm, name="overlay-prewarm", daemon=True).start()

def _prewarm(self) -> None:
with self._lock:
if self.degraded or self._transport is not None:
return
try:
self._transport = self._connect()
except Exception:
self._degrade()

def _connect(self) -> Transport:
"""Spawn the helper and wait, generously, for its first ack."""
transport = self._spawn()
if not transport.send(CMD_HIDE, timeout=STARTUP_ACK_TIMEOUT):
with contextlib.suppress(Exception):
transport.close()
raise RuntimeError("overlay helper did not start")
return transport

def _send(self, command: str, *, timeout: float = FAST_ACK_TIMEOUT) -> bool:
if self.degraded:
with self._lock:
if self.degraded:
return False
try:
if self._transport is None:
self._transport = self._connect()
if self._transport.send(command, timeout=timeout):
return True
except Exception:
pass
self._degrade()
return False
try:
if self._transport is None:
self._transport = self._spawn()
if self._transport.send(command, timeout=timeout):
return True
except Exception:
pass
self._degrade()
return False

def _degrade(self) -> None:
"""Give up on the glow for the life of the process."""
Expand Down
3 changes: 3 additions & 0 deletions mcp/tst-cu-mcp/src/tst_cu_mcp/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -522,6 +522,9 @@ def run() -> None:

configure_logging()
safety.set_config(load_config())
# The packaged helper takes seconds to start; begin now, not on the
# agent's first action (TD-5230). A no-op where the overlay is off.
get_overlay().prewarm()
try:
build_server().run(transport="stdio")
finally:
Expand Down
1 change: 1 addition & 0 deletions mcp/tst-cu-mcp/tests/test_agent_cursors.py
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,7 @@ def close(self) -> None:
painter.cursor_hide("s1")
painter.cursor_hide(None)
assert sent == [
"hide", # the start-up probe (TD-5230)
"cursor s1 10.0 20.2 #2F6FDE 1 Fix login",
"cursor_hide s1",
"cursor_hide *",
Expand Down
79 changes: 71 additions & 8 deletions mcp/tst-cu-mcp/tests/test_overlay_gating.py
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,9 @@ def use_agent(self, key: str, label: str = "") -> Agent:
def pointer(self, x: float, y: float, *, click: bool = False) -> None:
self.calls.append("cursor_click" if click else "cursor_move")

def prewarm(self) -> None:
self.calls.append("prewarm")

def activity(self) -> None:
self.calls.append("activity")

Expand Down Expand Up @@ -187,55 +190,115 @@ def explode() -> Any:


class TestDarwinGating:
"""Every spawn starts with a ``hide`` probe that waits out the start-up."""

def test_activity_shows_when_alive(self) -> None:
transport = RecordingTransport()
overlay = DarwinOverlay(spawn=lambda: transport)
overlay.activity()
assert transport.commands == ["show"]
assert transport.commands == ["hide", "show"]

def test_session_stays_until_end(self) -> None:
transport = RecordingTransport()
overlay = DarwinOverlay(spawn=lambda: transport)
overlay.begin_session()
overlay.begin_session()
overlay.end_session()
assert transport.commands == ["show", "show", "hide"]
assert transport.commands == ["hide", "show", "show", "hide"]

def test_killswitch_forces_hide(self, tmp_path: Path) -> None:
stop = tmp_path / "STOP"
stop.write_text("", encoding="utf-8")
transport = RecordingTransport()
overlay = DarwinOverlay(spawn=lambda: transport)
overlay.activity()
assert transport.commands == ["hide"]
assert transport.commands == ["hide", "hide"]

def test_notify_blocked_hides(self) -> None:
transport = RecordingTransport()
overlay = DarwinOverlay(spawn=lambda: transport)
overlay.notify_blocked()
assert transport.commands == ["hide"]
assert transport.commands == ["hide", "hide"]

def test_grab_pair_brackets_and_survives_failed_begin(self) -> None:
ok = RecordingTransport()
overlay = DarwinOverlay(spawn=lambda: ok)
with overlay.grab_hidden():
ok.commands.append("<grab>")
assert ok.commands == ["grab_begin", "<grab>", "grab_end"]
assert ok.commands == ["hide", "grab_begin", "<grab>", "grab_end"]

dead = RecordingTransport(ack=False)
overlay2 = DarwinOverlay(spawn=lambda: dead)
with overlay2.grab_hidden():
pass # begin was never acked; end must not be sent either
assert dead.commands == ["grab_begin"]
pass # the helper never answered; no grab command may follow
assert dead.commands == ["hide"]
assert overlay2.degraded is True
assert dead.closed is True

def test_a_slow_start_is_waited_out_not_given_up_on(self) -> None:
"""TD-5230: the frozen helper's first ack takes seconds."""

class SlowStart(RecordingTransport):
def __init__(self) -> None:
super().__init__()
self.timeouts: list[float] = []

def send(self, command: str, *, timeout: float = 1.0) -> bool:
self.timeouts.append(timeout)
self.commands.append(command)
# Only a budget that covers a cold start sees the ack.
return len(self.commands) > 1 or timeout >= 5.0

transport = SlowStart()
overlay = DarwinOverlay(spawn=lambda: transport)
overlay.activity()
assert overlay.degraded is False
assert transport.commands == ["hide", "show"]
assert transport.timeouts[0] >= 5.0
assert transport.timeouts[1] <= 1.0

def test_prewarm_starts_the_helper_once_off_the_tool_path(self) -> None:
import threading

spawned: list[RecordingTransport] = []
started = threading.Event()

def spawn() -> RecordingTransport:
transport = RecordingTransport()
spawned.append(transport)
started.set()
return transport

overlay = DarwinOverlay(spawn=spawn)
overlay.prewarm()
assert started.wait(5.0)
overlay.activity()
overlay.prewarm()
assert len(spawned) == 1
assert spawned[0].commands == ["hide", "show"]

def test_a_failed_prewarm_degrades_quietly(self) -> None:
import time

def spawn() -> RecordingTransport:
raise OSError("no window server")

overlay = DarwinOverlay(spawn=spawn)
overlay.prewarm()
for _ in range(50):
if overlay.degraded:
break
time.sleep(0.02)
assert overlay.degraded is True
overlay.activity() # still a quiet no-op

def test_set_target_names_the_display(self) -> None:
transport = RecordingTransport()
overlay = DarwinOverlay(spawn=lambda: transport)
overlay.set_target(51)
overlay.set_target(None)
overlay.set_target(-1)
assert transport.commands == ["target 51"]
assert transport.commands == ["hide", "target 51"]

def test_shutdown_closes_transport(self) -> None:
transport = RecordingTransport()
Expand Down
3 changes: 3 additions & 0 deletions mcp/tst-cu-mcp/tests/test_stdio_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,9 @@ def child_env(*, internal: bool = False) -> dict[str, str]:
"""
env = dict(os.environ)
env.pop(INTERNAL_ENV, None)
# The server starts its overlay helper at launch (TD-5230); a test child
# must never put windows on the developer's screen.
env["TST_CU_MCP_OVERLAY"] = "0"
if internal:
env[INTERNAL_ENV] = "1"
return env
Expand Down
Loading