[功能汇总] fork 仓库的新增功能，后续持续追加，可随时合回主线（已兼容0.2.1-rc.3）

.agents/notes/implemented/architecture/2026-10-21-browser-input-layer.i18n.yaml

@@ -0,0 +1,4 @@
+# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
+# side as of the last confirmed-consistent state. Both languages carry equal authority.
+2026-10-21-browser-input-layer.md: 2f0d79cfc30baa9fd67cce7da53d3f7f33ded490
+2026-10-21-browser-input-layer.zh.md: 9212436671c106db0b58eeedeb17f5e853c2d0fd

.agents/notes/implemented/architecture/2026-10-21-browser-input-layer.md

@@ -0,0 +1,87 @@
+# Agent Note: Browser input layer
+
+Status: implemented
+
+**English** | [中文](2026-10-21-browser-input-layer.zh.md)
+
+## Problem
+
+The browser bridge could read a page and operate elements that the snapshot had
+numbered. Anything that did not expose a semantic role — canvas surfaces, SVG
+menus, rich-text editors, drag-and-drop panels — stayed out of reach: there was
+no pointer movement or press/release, no key chord, no way to put text into an
+editor that owns its document model, no clipboard transfer, and no file input.
+The gap was a missing input layer, not a missing app integration.
+
+## Decision
+
+**One general input layer, no application-specific code.** Keyboard, pointer,
+text, clipboard, and file are addressed by element identity or coordinates;
+every page uses the same parameters. The implementation never keys on a domain,
+a component class, or an editor implementation, and application recipes live in
+documentation only.
+
+**Two delivery channels with one model-facing contract.** Element targeting,
+plain text, paste events, and ordinary clicks stay in the content script and
+never need a debugger. Real pointer events, wheel gestures, key chords, caret
+insertion, system paste, and file attachment need the browser process, so they
+run over `chrome.debugger`. The tool result marks the channel (`(trusted)` or
+`(synthetic)`) and the platform's inability to deliver is a stable
+`input-unavailable` error instead of a silent no-op.
+
+**Input is stateful per tab.** `browser_press { action: down|up }` holds a
+modifier, `browser_click { action: down|up }` holds a mouse button, and the next
+gesture automatically carries them. This composes Shift+click, Cmd+drag, and
+press-drag-release without dedicated combination tools. Navigation, tab close,
+and attachment loss reset the state.
+
+**Coordinates are frame-local.** A pointer action addresses a point inside its
+target frame's viewport; the content script reports the chain of frame owners
+along `window.parent`, and the background sums it into the tab viewport. A
+cross-origin ancestor hides `frameElement`, so coordinate actions fail there
+while element targeting and paste events keep working.
+
+**One debugger per tab, shared.** A tab can only be attached once, so a small
+broker owns the per-tab session, counts the features holding it (developer tools
+and input), and detaches immediately for the developer tools or after an idle
+window for input. Real input is an ordinary action for consent purposes: it
+follows the same per-call approval, trusted-origin, and unrestricted-access
+rules as click/type, because it grants the page no authority a DOM click does
+not. The isolated consent surface stays reserved for the developer-tools group
+(arbitrary page JavaScript, DOM writes, network capture).
+
+**Clipboard has two distinct paths.** A paste event built from a `DataTransfer`
+is the default: it works in every supported browser, never touches the user's
+clipboard, and HTML is sanitized before it reaches the page. Writing the system
+clipboard uses a packaged offscreen document in Chrome (an MV3 worker has no
+DOM) or the event page's hidden editable host in Firefox; the platform's paste
+shortcut then covers apps that only respond to genuine pastes.
+
+**File attachment reads nothing in the extension.** The content script tags the
+file input with a one-shot marker, the background resolves it over CDP, and the
+browser process reads the caller-supplied paths. The extension never opens the
+files.
+
+## Consequences
+
+- Tool parameters grew on `browser_snapshot`, `browser_click`, `browser_type`,
+  `browser_press`, `browser_scroll`, and `browser_follow_tab`; two tools were
+  added (`browser_clipboard`, `browser_upload`).
+- Chrome gains `clipboardWrite` and `offscreen` permissions plus a packaged
+  offscreen page; Firefox keeps its manifest and degrades.
+- Input never activates a tab, focuses a window, or moves the system pointer.
+  `browser_follow_tab { activate: true }` is the only explicit foreground
+  switch, and `browser_screenshot` still requires the active tab.
+- A debugger attachment shows Chrome's infobar and blocks DevTools on that tab;
+  the idle detach window keeps the cost bounded to one input sequence.
+- Attaching can change the viewport height (the infobar): measure coordinates
+  after attachment, which the broker's idle window makes the default.
+- macOS editing shortcuts are application-menu commands: select-all, copy,
+  paste, cut, undo, and redo work only when the key-down names the command, so
+  Meta combinations carry `commands` on macOS.
+- Wheel input is compositor-driven; a background or frozen page never answers
+  it. The tool rejects that case up front with a `follow_tab { activate: true }`
+  instruction instead of waiting for a timeout, and works on the active page.
+- Screenshots come from the compositor: the tab must be active in a focused
+  window, or the capture can be a stale frame. The tool refuses an unfocused
+  window; an occluded but focused window remains best-effort.

.agents/notes/implemented/architecture/2026-10-21-browser-input-layer.zh.md

@@ -0,0 +1,36 @@
+# Agent Note: 浏览器通用输入层
+
+Status: implemented
+
+[English](2026-10-21-browser-input-layer.md) | **中文**
+
+## 问题
+
+浏览器桥此前能读页面、能操作快照里编过号的元素，但凡是没暴露语义角色的目标都到不了：画布、SVG 菜单、富文本编辑器、拖拽面板。缺少的是指针移动与按下/抬起、组合键、向自持文档模型的编辑器写入文本、剪贴板传输和文件输入。这是缺少输入层，不是缺少某个应用适配。
+
+## 决策
+
+**只做一层通用输入，不写应用特判。** 键盘、指针、文本、剪贴板、文件都按「元素身份或坐标」寻址，任何页面用同一组参数。实现里不出现域名、组件类名或编辑器实现的判断，应用配方只放在文档里。
+
+**两条投递通道，一个模型契约。** 元素寻址、纯文本、粘贴事件和普通点击留在内容脚本，不需要调试器。真实指针事件、滚轮手势、组合键、光标插入、系统粘贴和文件附加需要浏览器进程，走 `chrome.debugger`。结果标注通道（`(trusted)` 或 `(synthetic)`）；平台无法投递时返回稳定的 `input-unavailable`，而不是静默失败。
+
+**输入状态按标签页保存。** `browser_press { action: down|up }` 按住修饰键，`browser_click { action: down|up }` 按住鼠标键，后续手势自动携带它们。于是 Shift+点击、Cmd+拖拽、按下-拖动-抬起无需专门的组合工具。导航、关标签页和附加丢失都会重置状态。
+
+**坐标是 frame 局部的。** 指针动作寻址目标 frame 视口内的点；内容脚本沿 `window.parent` 上报每层 frame 的位置链，后台求和换算成标签页视口坐标。跨源祖先无法访问 `frameElement`，因此坐标类动作在那里失败，而元素寻址与粘贴事件照常可用。
+
+**每个标签页一个调试器，共享使用。** 一个标签页只能被附加一次，因此由 broker 持有按标签页的会话、统计持有它的功能（开发者工具与输入），开发者工具停止时立即释放、输入空闲一段时间后释放。真实输入在同意面上就是普通动作：与 click/type 一样按逐次审批、可信来源和完全访问规则处理，因为它并没有给页面 DOM 点击之外的新权限。独立同意面只保留给开发者工具组（任意页面 JavaScript、DOM 写入、网络抓包）。
+
+**剪贴板有两条不同的路径。** 默认是用 `DataTransfer` 构造粘贴事件：所有支持的浏览器都能用，不碰用户剪贴板，HTML 在进入页面前净化。写系统剪贴板在 Chrome 用打包的离屏文档（MV3 worker 没有 DOM），Firefox 用事件页的隐藏可编辑节点；随后用平台粘贴快捷键覆盖只认真实粘贴的应用。
+
+**文件附加不读文件内容。** 内容脚本给 file input 打一次性标记，后台经 CDP 解析该节点，浏览器进程读取调用方给出的路径。扩展从不打开文件。
+
+## 影响
+
+- `browser_snapshot`、`browser_click`、`browser_type`、`browser_press`、`browser_scroll`、`browser_follow_tab` 增加参数；新增两个工具（`browser_clipboard`、`browser_upload`）。
+- Chrome 增加 `clipboardWrite` 与 `offscreen` 权限以及一个打包的离屏页面；Firefox 清单不变并降级。
+- 输入不会激活标签页、聚焦窗口或移动系统鼠标。`browser_follow_tab { activate: true }` 是唯一显式切前台入口，截图仍要求活动标签页。
+- 附加调试器会显示 Chrome 提示条，并让该标签页暂时不能开 DevTools；空闲释放把代价限制在一次输入序列内。
+- 附加可能改变视口高度（提示条占位）：坐标应在附加之后测量，broker 的空闲窗口让这一点成为默认。
+- macOS 的编辑快捷键是应用菜单命令：全选、复制、粘贴、剪切、撤销、重做只有在 key-down 里指明命令名时才生效，因此 macOS 的 Meta 组合键会携带 `commands`。
+- 滚轮由合成器驱动；后台或被冻结的页面永远不会应答。工具会前置拒绝并给出 `follow_tab { activate: true }` 指引，而不是等到超时；活动页面上可正常滚动。
+- 截图来自合成器：标签页必须是活动页且窗口聚焦，否则可能拿到过期帧。工具会拒绝未聚焦窗口；被遮挡但已聚焦的窗口仍属尽力而为。

README.i18n.yaml

@@ -1,5 +1,5 @@
 # Bilingual consistency record for the repository README files.
 # README.md is the default English page; README.zh.md is its Chinese counterpart.
 # Values are git blob hashes from the last confirmed-consistent review.
-README.md: 1163cc9c9a7332a95f8b2448461243f0813ffdf2
-README.zh.md: 6ce7a8314f10dcd45316906f79991e0604b0814c
+README.md: 3a405c6d46e0450ac7699fb773b52d57d6d311d7
+README.zh.md: 0674eae8e00d6b6962d0ba242a4364ffa7658a53

README.md

@@ -49,20 +49,24 @@ The paired Playwright / extension duration ratio was **1.24** (95% CI **1.16–1
 
 | Capability | Tool | Notes |
 |---|---|---|
-| Read page | `browser_snapshot` | Structured text snapshot: title, URL, main text, numbered controls, and masked form fields; `delta: true` returns only changes |
-| Click element | `browser_click` | Click links, buttons, checkboxes, and other controls by inventory number |
-| Fill forms | `browser_type` | React/Vue-compatible input; `replace` clears the field first |
-| Press keys | `browser_press` | Keyboard events such as Enter, Tab, Escape, and arrow keys |
-| Scroll | `browser_scroll` | Viewport scrolling: up, down, top, and bottom |
+| Read page | `browser_snapshot` | Structured text snapshot: title, URL, main text, numbered controls, and masked form fields; `delta: true` returns only changes, `frame` reads one iframe, and `query` locates any element by text/selector/role with its selector and rect |
+| Pointer | `browser_click` | Click, double-click, hover, press, release, or drag by index or selector; canvas targets without an identity accept frame-local `point` coordinates |
+| Text | `browser_type` | React/Vue-compatible input; `replace` clears the field first, and `mode` selects caret insertion or per-character key events |
+| Press keys | `browser_press` | Single keys or chords (`modifiers`); `action: down/up` holds a key across calls and `repeat` repeats it |
+| Scroll | `browser_scroll` | Viewport scrolling; with `point` it sends a real wheel gesture at those coordinates (canvas zoom/pan) |
+| Clipboard | `browser_clipboard` | `write` puts text/HTML on the system clipboard; `paste` delivers it as a paste event or a real platform shortcut |
+| Files | `browser_upload` | Attach local files to an `input[type=file]` (Chrome) |
 | Navigate | `browser_navigate` / `browser_open_tab` / `browser_back` / `browser_forward` / `browser_reload` | Navigation inside the controlled tab, or open a URL in a new tab and follow it (`active:false` keeps the current tab in front) |
 | List tabs | `browser_list_tabs` | List accessible tabs with stable IDs, titles, URLs, window/index metadata, and active/controlled state |
-| Follow tab | `browser_follow_tab` | Bind later browser tools to a tab returned by `browser_list_tabs` without activating it |
+| Follow tab | `browser_follow_tab` | Bind later browser tools to a tab returned by `browser_list_tabs`; it stays in the background unless `activate: true` |
 | Close tab | `browser_close_tab` | Close a tab returned by `browser_list_tabs` |
 | Read region | `browser_get_text` | Lazy-loaded or partial page text |
 | Wait for stability | `browser_wait` | Page-load and render-settle detection |
 | Send images | `session.prompt` / `session.attachment` | Host-capability-gated image drafts, image-only prompts, and durable history previews |
 | Quote a selection | side panel composer | Text you highlight in the page appears in the composer and is sent with your next message as fenced, attributed page content |
 
+The input layer is a set of general primitives, not per-app handling: keyboard, pointer, text, clipboard, and files all address a target by element identity or coordinates, so any rich editor, canvas, code editor, or drag surface uses the same parameters. Input runs in the background by default — no tab activation, no window focus, no system pointer movement. When real events matter (`isTrusted`, browser default actions) the Chrome build delivers them through the debugger; Firefox degrades to synthetic events and marks the result `(synthetic)`. Wheel gestures and screenshots are the exceptions: a wheel needs the page to be the active tab in a focused window (Chrome freezes background pages, so the tool fails fast with a `follow_tab {activate:true}` hint), and a screenshot needs the same focus or it may return an older frame.
+
 ## Repository layout
 
 ```
@@ -148,7 +152,8 @@ Local Chrome use requires no configuration; Firefox requires the local bridge to
 
 - Make sure dsh web is running locally (default `http://127.0.0.1:3080`).
 - Verify the bridge is loaded: open `http://127.0.0.1:3080/ext/bridge-config`. It should return JSON such as `{"wsUrl":"ws://127.0.0.1:3080/ext/bridge"}`. If it returns a web page instead of JSON, the running dsh predates the bridge registration — restart dsh and refresh the page; the extension reconnects on its own.
-- The extension probes ports 3080, 3081, 3090, and 14389 automatically. If dsh runs on another port — or you use a remote `--host 0.0.0.0` deployment — set the address (and bridge token) in the panel settings. Firefox always requires the token.
+- Discovery runs once when the panel opens. A filled-in address is tried first; if a **loopback** address does not answer, the panel makes one automatic pass (last working URL → `dsh web` defaults → the plugin's fixed beacon ports 43189–43198). When that pass finds nothing the status reads **No service found** and nothing retries until you reopen the panel or press **Detect**.
+- DSH Desktop assigns a random local Web port on every launch, which is why the bridge plugin also listens on a fixed loopback beacon port. If the panel cannot find it, open **Settings → Detect** to list every local instance and click the one to use; the address can still be typed by hand. Remote `--host` deployments need the address and token filled in manually; Firefox always requires the token.
 
 ## Development
 
@@ -186,7 +191,7 @@ If you encounter `cache.hydratePrepared is not a function`, update the repositor
 - Local Chrome extension origins retain zero-configuration loopback access; Firefox origins are per-install UUIDs and must present the bearer token.
 - Privileged gateway methods such as `settings.*`, `credentials.*`, and `host.open*` reject non-loopback sources.
 - The browser-page pipeline is text-only and never captures screenshots; explicitly attached chat images use dsh's durable attachment service. Password and payment-card values never leave the page.
-- When work begins, the assistant binds to the active tab (at prompt submission, or at the first direct browser-tool call). If you switch tabs manually, later browser actions pause and the side panel asks whether the assistant should continue on the original tab or follow the new one. Choosing the original tab permits background operation; the extension never silently retargets or changes your visible tab. Closing the controlled tab also pauses tools until you explicitly select the current page.
+- When work begins, the assistant binds to the active tab (at prompt submission, or at the first direct browser-tool call). If you switch tabs manually, later browser actions pause and the side panel asks whether the assistant should stay on the original page or follow the current one, and offers "jump back to the assistant page" to move your own tab back to the assistant's page. Choosing the original page permits background operation; apart from that button the extension never silently retargets or changes your visible tab. Closing the controlled tab also pauses tools until you explicitly select the current page.
 - Text you highlight is captured only while a side panel is open and page sharing is not `off`, and never from password or payment-card fields. It stays inside the extension until you send the message, is dropped when you dismiss it or its page navigates or closes, and reaches the model inside the same untrusted-content boundary as page snapshots — including its source title and URL, which the page also controls.
 - Page-authored text is wrapped as untrusted input. The default `auto` mode reads only the controlled tab without an extra prompt; privacy-sensitive users can select `ask` for per-read confirmation or `off` to block reads entirely. In `ask` mode, the read dialog can allow one read or persistently switch back to `auto`; this can be reversed in Settings. Read page text is sent to the selected model.
 - Click, type, keypress, navigation, history, and reload calls fail closed until the user approves them. An origin may be trusted for the current side-panel session (cleared when the last panel closes or the service worker restarts), while permanent trust is managed explicitly in Settings. Explicit cross-origin `browser_navigate` calls and unknown history destinations always prompt again.