[Bug] macOS 26 real-world findings on 0.3.5: wrong-window targeting, silent set_value, docs/version mismatch, and more #83

Summary

Six findings from a real integration session on macOS (Apple Silicon). All were reproduced through both the opencode MCP server (open-computer-use mcp) and the ocu CLI (open-computer-use call ...). Four are defects with concrete reproduction steps; two are ergonomics/feature requests.

The nine-tool surface works well for ordinary app automation — the problems below are specific and reproducible.

Environment

• macOS: 26.6.2 (build 25G83), arm64 (Apple Silicon)
• open-computer-use: 0.3.5 (npm view open-computer-use version → 0.3.5, latest; published 2026-09-09)
• Node.js: v26.8.2, npm 11.19.1
• opencode: 1.18.32, MCP stdio server via open-computer-use mcp
• Google Chrome: 153.0.8010.54
• Permissions: Accessibility = granted, Screen Recording = granted (open-computer-use doctor)
• Reproduced via both the opencode MCP tools and the CLI

Priority

1. [Bug] Wrong-window targeting in multi-window apps; no window/tab selector

Summary

When the target app is not frontmost and has more than one window, get_app_state and subsequent actions can bind to a window other than the one the caller needs. There is no window/tab parameter in any tool, and element_index is not scoped to the window it was captured from.

Repro A — snapshot binds to the wrong window

1. Chrome open with two windows: window A's active tab is a local HTML form, window B's active tab is YouTube.
2. Focus another app so Chrome is not frontmost.
3. Run a single-process sequence:

[
  { "tool": "get_app_state", "args": { "app": "Google Chrome" } },
  { "tool": "click", "args": { "app": "Google Chrome", "element_index": "46", "click_method": "accessibility" } }
]

4. get_app_state returns window B, not the form window:

Window: "(15) YouTube - Sabitlendi - Google Chrome - Selçuk"
44 link [YouTube Premium Ana Sayfası](https://www.youtube.com/)
46 button Frame: x=16, y=121, w=125, h=1

5. The click then acts in that window: The focused UI element is 44 link [YouTube Premium Ana Sayfası].

Repro B — an index captured in one window is applied to another

1. Chrome frontmost, the App Store Connect window active. get_app_state → element_index 85 is a link inside that page.
2. click {"app":"Google Chrome","element_index":"85"}.
3. Result: the action landed in a different Chrome window (active tab "Panel | Zentio"). The reported window became Panel | Zentio - Google Chrome and text on that page was selected (Selected text: [GPT 5.6 SolCodex Core×1520:14]). The intended link was not activated.

Expected

• A way to select the target window/tab (title, index, or window id), and/or
• element_index scoped to the window it came from, with an explicit error when an index does not belong to the currently bound window, and/or
• deterministic binding so get_app_state and the following action always hit the same window.

Actual

Actions can silently execute against the wrong window. In any multi-window app this risks unintended clicks or typing in the wrong place, including destructive UI.

2. [Bug] set_value reports success but does not change the value (silent no-op)

Minimal page

<input type="text" aria-label="City">
<input type="date" aria-label="Birth date">
<select aria-label="Country">
  <option value="">Select</option>
  <option>Türkiye</option>
</select>

Steps to reproduce

1. get_app_state {"app":"Google Chrome"}. Chrome exposes:
   • stepper (settable, float) Gün ..., stepper (settable, float) Ay ..., stepper (settable, float) Yıl ...
   • pop-up düğmesi (settable, string) Ülke Value: Seçiniz
2. In one sequence, call set_value on the three date steppers ("15", "3", "1990") and on the select ("Türkiye").
3. Every call returns isError: false. In an 11-call batch, 11/11 calls returned isError=false.
4. Re-read state: the date segments are still Value: 0 and the select is still Value: Seçiniz. Nothing changed.

Expected

Either the value changes, or the call returns an error/warning. An element advertised as settable that silently ignores set_value leaves the agent unable to detect that the mutation failed.

Actual

set_value returns success with no effect. On the same page, set_value on the plain text input works correctly and fires input, so this is specific to these element types.

3. [Bug] Published npm release (0.3.5) is missing the code-first CLI that the skill/README document

Steps to reproduce

$ ocu capabilities
Unknown command: capabilities
$ ocu js '1+1'
Unknown command: js
$ ocu -h
Commands:
  mcp, doctor, list-apps, snapshot, call, turn-ended,
  install-claude-mcp, install-gemini-mcp, install-codex-mcp,
  install-opencode-mcp, install-codex-plugin, help, version

Meanwhile skills/open-computer-use/SKILL.md (installed from main) instructs:

For direct code-first orchestration, run ocu capabilities and then use one-shot ocu js or persistent ocu repl.

and skills/open-computer-use/references/usage.md states:

Its help always lists js / repl.

The README Quick Start also lists ocu capabilities, ocu js, and ocu repl.

Expected

Publish a release that contains the code-first CLI, or version-gate the skill/docs and have the installers verify the capability before advertising it.

Impact

Agents that install the skill (e.g. npx skills add iFurySt/open-codex-computer-use ...) and follow it will run commands that do not exist in the published package. It also removes the only scripting escape hatch (previously run_jxa) for cases the nine structured tools cannot cover — see finding 5.

4. [Bug] --calls-file / --args-file do not resolve relative paths against the process cwd

Steps to reproduce

$ cd /var/folders/.../cu-form-test
$ pwd
/var/folders/.../cu-form-test
$ ls -l calls.json
-rw-r--r--@ 1 selcuk staff 1185 calls.json
$ open-computer-use call --calls-file calls.json
Unable to read JSON file calls.json: The file "calls.json" couldn't be opened
because there is no such file.

Using an absolute path works:

$ open-computer-use call --calls-file /var/folders/.../cu-form-test/calls.json
# runs successfully

Expected

Relative paths resolve against the process working directory.

Actual

Relative paths fail even though the cwd is correct and the file exists. The error prints only the bare filename, which makes the cause hard to diagnose.

Suggested fix

Resolve against cwd (preferred). If an absolute path is intentionally required, reject relative paths early and include the resolved/expected absolute path in the error message.

5. [Bug] Native <input type="date"> cannot be filled with the nine-tool surface

Minimal page

<input type="date" aria-label="Birth date">

Attempts and results

1. set_value on the container (AX tarih seçici / date field):
   isError: true → Cannot set a value for an element that is not settable
2. set_value on the day/month/year steppers ("15", "3", "1990"):
   isError: false, no effect (see finding 2)
3. click on the Tarih seçiciyi göster ("show date picker") popup button with click_method: "accessibility":
   the calendar popup did not open — it is absent from the accessibility tree and from the returned screenshot
4. click on the field (accessibility) then type_text "15031990":
   with Chrome in the background → type_text requires a focused editable text element; with Chrome frontmost and the field focused → the value still did not change

Note: there is no run_jxa in this line, and ocu js / ocu repl are absent in 0.3.5 (finding 3), so there is no scripting fallback.

Expected

At least one path should work: the date segments should be settable, or the picker popup should open and accept clicks, or set_value should accept an ISO date on the container.

Actual

A standard HTML form control is unreachable, so any form containing a date field cannot be completed.

6. [Feature] Result size and ergonomics requests

6a. Allow omitting the screenshot from results

Every get_app_state and every action-result refresh embeds a base64 screenshot. A single 11-call --calls-file batch produced 1,671,757 bytes of output, while only the accessibility text was needed. Please add something like screenshot: false (or include_screenshot), and consider not attaching a screenshot to action-result refreshes by default. In an agent loop this is the dominant cost.

6b. Allow scrolling without a pre-captured element

scroll requires element_index, so an app or window cannot be scrolled before a snapshot exists. An app-level scroll (app + direction, optional coordinates) would remove a forced round trip.

6c. install-opencode-mcp does not reconcile an existing OpenCode setup

The installer writes:

"mcp": { "open-computer-use": { "type": "local", "command": ["open-computer-use", "mcp"] } }

Observations from reading the installer/helper (not verified live):

• No enabled/timeout is written. Please confirm whether OpenCode defaults to enabled when enabled is omitted.
• If a user already has a differently-named computer-use server (for example an existing computer_use entry), the installer adds a second server instead of reconciling, leaving two servers controlling the same desktop. A note or a --replace <name> flag would help.
• Only opencode.json / config.json are considered; opencode.jsonc is ignored.

6d. Missing tools compared with the previous surface

This line has no run_jxa, key_sequence, or select_text, and no clipboard tool. key_sequence and clipboard support were useful for multi-step text entry. If these are intentionally out of scope, documenting the gap next to the nine-tool list would help agents plan around it.

What works well

• Background-first app_post clicks: the physical cursor does not move and the target window is not raised.
• Bounded before/after accessibility diff after each mutating action.
• Clear error messages (for example type_text requires a focused editable text element).
• Idempotent installers, the packaged skill, the Codex plugin, and the cross-platform goal.
• set_value is Unicode-safe: Turkish characters (Şükrü Çağdaş Öztürk, ĞÜŞİÖÇ ğüşıöç) round-trip correctly on plain text inputs.