Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions docs/interactive-capabilities/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
<!-- generated by scripts/generate-interactive-capabilities.mjs; do not edit -->
# BitFun 功能与设置目录 / BitFun Features & Settings

BitFun Playbook 当前包含 **22 个功能**和 **16 个设置页**,共 **38 个**用户可理解的条目、**316 项**有源码证据的子能力。每个条目有独立 Markdown,并直接服务于说明书网站、BitFun 全局搜索和 `BitFunControl` Agent 工具。
BitFun Playbook 当前包含 **22 个功能**和 **16 个设置页**,共 **38 个**用户可理解的条目、**318 项**有源码证据的子能力。每个条目有独立 Markdown,并直接服务于说明书网站、BitFun 全局搜索和 `BitFunControl` Agent 工具。

BitFun Playbook currently contains **22 features**, **16 settings pages**, and **316** source-backed sub-capabilities across **38** user-facing entries. Every entry has its own Markdown page and directly powers the website, in-app global search, and the `BitFunControl` agent tool.
BitFun Playbook currently contains **22 features**, **16 settings pages**, and **318** source-backed sub-capabilities across **38** user-facing entries. Every entry has its own Markdown page and directly powers the website, in-app global search, and the `BitFunControl` agent tool.

## 唯一事实源 / Single source of truth

Expand All @@ -27,20 +27,20 @@ BitFun Playbook currently contains **22 features**, **16 settings pages**, and *
- Generated per-item interaction audit: `docs/interactive-capabilities/technical/product-control-open-audit.json`
- Generated low-level audit map: `docs/interactive-capabilities/technical/tauri-command-map.json`

说明书、网站、搜索和 Agent 只看“功能 + 设置 + 子能力”。每项子能力都必须引用已注册 Tauri Command 或可解析的源码标记;这些证据不会进入公开目录。当前 **642** 个 Tauri 命令只用于实现覆盖审计。产品 UI 交互源码会在生成和检查时扫描并校验,但不会保存成随普通 UI 改动频繁变化的版本化快照。
说明书、网站、搜索和 Agent 只看“功能 + 设置 + 子能力”。每项子能力都必须引用已注册 Tauri Command 或可解析的源码标记;这些证据不会进入公开目录。当前 **651** 个 Tauri 命令只用于实现覆盖审计。产品 UI 交互源码会在生成和检查时扫描并校验,但不会保存成随普通 UI 改动频繁变化的版本化快照。

Docs, website, search, and agents see only features, settings, and documented sub-capabilities. Every sub-capability must reference a registered Tauri command or a resolvable source marker; evidence is stripped from public projections. The **642** Tauri commands remain implementation-audit evidence only. Product UI interaction sources are scanned and validated during generation and checks, but are not stored as a versioned snapshot that churns with ordinary UI changes.
Docs, website, search, and agents see only features, settings, and documented sub-capabilities. Every sub-capability must reference a registered Tauri command or a resolvable source marker; evidence is stripped from public projections. The **651** Tauri commands remain implementation-audit evidence only. Product UI interaction sources are scanned and validated during generation and checks, but are not stored as a versioned snapshot that churns with ordinary UI changes.

## 控制边界 / Control boundary

- 每个子能力都明确标记为直接控制、委托给专用 Agent 工具、需交互打开或不支持;“打开页面”不会再被统计成“Agent 已控制”。当前覆盖:直接 **49**、委托 **62**、需交互 **205**、不支持 **0**。
- 每个子能力都明确标记为直接控制、委托给专用 Agent 工具、需交互打开或不支持;“打开页面”不会再被统计成“Agent 已控制”。当前覆盖:直接 **49**、委托 **62**、需交互 **207**、不支持 **0**。
- 稳定行为声明为带 JSON 输入契约的 `operations` 或 `options`,并绑定原生产品控制 Provider;Agent 不接触原始 Tauri Command。
- `BitFunControl list` 和 `search` 都返回带 `nextCursor` 的精简分页结果;目录可持续增长,不靠固定总量上限。完整目录和 316 项子能力都不会写入 system prompt。
- `BitFunControl list` 和 `search` 都返回带 `nextCursor` 的精简分页结果;目录可持续增长,不靠固定总量上限。完整目录和 318 项子能力都不会写入 system prompt。
- 目录发现与契约读取不依赖 React 或可见窗口。普通配置型 option 统一由 Product Assembly 的共享 ConfigService 执行器读、写并回读,因此 Desktop、CLI 与 Headless 表面走同一份实现;只有宿主原生 operation/provider option 和界面导航按表面注册适配器,缺失时必须明确返回不可用,禁止静默回退本机。只读 Agent 只能发现和读取目录。

- Every documented item is classified as direct control, delegated Agent control, interactive opening, or unsupported; opening a page is never counted as direct control. Current coverage is **49 direct**, **62 delegated**, **205 interactive**, and **0 unsupported**.
- Every documented item is classified as direct control, delegated Agent control, interactive opening, or unsupported; opening a page is never counted as direct control. Current coverage is **49 direct**, **62 delegated**, **207 interactive**, and **0 unsupported**.
- Stable behavior becomes a typed `operation` or `option` with a JSON input contract and a native product-control provider. Agents never receive raw Tauri commands.
- `BitFunControl list` and `search` return compact pages with a `nextCursor`; the catalog can grow without a fixed total-size ceiling. Neither the full catalog nor its 316 documented items enters the system prompt.
- `BitFunControl list` and `search` return compact pages with a `nextCursor`; the catalog can grow without a fixed total-size ceiling. Neither the full catalog nor its 318 documented items enters the system prompt.
- Discovery and contract lookup do not depend on React or a visible window. Ordinary config-backed options are read, written, and read back by one Product Assembly ConfigService executor shared by Desktop, CLI, and headless surfaces. Only host-native operations/provider options and presentation routes install surface adapters; missing adapters return explicit unavailability without local fallback. Read-only agents may only discover and inspect entries.

## 防腐化门禁 / Anti-drift gates
Expand Down
161 changes: 148 additions & 13 deletions docs/interactive-capabilities/capabilities.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"title": "BitFun Playbook",
"origin": "https://playbook.openbitfun.com",
"source": "src/shared/interactive-capabilities/catalog.json",
"digest": "b915ffcb329182de3e033d8e48186a1dbd5c683d975484567d9dfe9b9c595c13",
"digest": "b0794e2a65d5356a8fdb262f0acf26753ac1aebfedf9b1e252d4dce4a21bc9a1",
"ownerDigest": "a6a8c5337fb877e641b66f5774740d5213caed4800fe9b006eb53632f7eb1c2d",
"searchAcceptance": [
{
Expand Down Expand Up @@ -138,11 +138,11 @@
"features": 22,
"settings": 16,
"userFacing": 38,
"documentedItems": 316,
"documentedItems": 318,
"controlCoverage": {
"direct": 49,
"delegated": 62,
"interactive": 205,
"interactive": 207,
"unsupported": 0
}
},
Expand Down Expand Up @@ -9563,6 +9563,8 @@
"itemIds": [
"composer-recording",
"stream-audio",
"realtime-assistant-call",
"realtime-assistant-config",
"local-models",
"cancel-download",
"language-microphone",
Expand Down Expand Up @@ -9703,6 +9705,100 @@
"viewId": "voice"
}
},
{
"id": "feature.voice-input:open:realtime-assistant-call",
"capabilityId": "feature.voice-input",
"itemIds": [
"realtime-assistant-call"
],
"kind": "open",
"risk": "ui",
"executionHost": "presentationSurface",
"availability": {
"desktop": {
"available": true
},
"cli": {
"available": false,
"reason": "This delivery profile has no live presentation surface"
},
"peer": {
"available": true,
"requiredCapabilities": [
"product_control_v1",
"product_control_presentation_v1"
]
},
"remoteControl": {
"available": true
},
"detachedDispatch": {
"available": false,
"reason": "This delivery profile has no live presentation surface"
}
},
"inputSchema": {
"type": "object",
"additionalProperties": false
},
"outputSchema": {
"type": "object",
"additionalProperties": true
},
"openReason": "unstructuredInteraction",
"presentationTarget": {
"kind": "settings",
"pageId": "application.input",
"viewId": "voice"
}
},
{
"id": "feature.voice-input:open:realtime-assistant-config",
"capabilityId": "feature.voice-input",
"itemIds": [
"realtime-assistant-config"
],
"kind": "open",
"risk": "ui",
"executionHost": "presentationSurface",
"availability": {
"desktop": {
"available": true
},
"cli": {
"available": false,
"reason": "This delivery profile has no live presentation surface"
},
"peer": {
"available": true,
"requiredCapabilities": [
"product_control_v1",
"product_control_presentation_v1"
]
},
"remoteControl": {
"available": true
},
"detachedDispatch": {
"available": false,
"reason": "This delivery profile has no live presentation surface"
}
},
"inputSchema": {
"type": "object",
"additionalProperties": false
},
"outputSchema": {
"type": "object",
"additionalProperties": true
},
"openReason": "unstructuredInteraction",
"presentationTarget": {
"kind": "settings",
"pageId": "application.input",
"viewId": "voice"
}
},
{
"id": "feature.voice-input:open:local-models",
"capabilityId": "feature.voice-input",
Expand Down Expand Up @@ -23168,35 +23264,41 @@
"id": "feature.voice-input",
"kind": "feature",
"categoryId": "application",
"titleZh": "语音输入",
"titleEn": "Voice input",
"summaryZh": "在聊天输入框中录音并转成文字,可选择本地模型、语言、麦克风与最长录音时间。",
"summaryEn": "Record speech in the chat composer and transcribe it to text, with control over local models, language, microphone, and duration.",
"titleZh": "语音输入与实时通话",
"titleEn": "Voice input and realtime calls",
"summaryZh": "将录音转成文字,或在客户端持续进行实时语音通话,让助手进入项目执行任务并播报简要进展。",
"summaryEn": "Transcribe recordings or hold a client-wide realtime voice call that can enter projects, run tasks, and speak concise progress updates.",
"keywordsZh": [
"语音输入",
"录音",
"转写",
"麦克风",
"语音模型",
"听写"
"听写",
"实时通话",
"语音助手"
],
"keywordsEn": [
"voice input",
"recording",
"transcription",
"microphone",
"speech model",
"dictation"
"dictation",
"realtime voice",
"voice assistant"
],
"highlightsZh": [
"下载和校验本地语音模型",
"选择识别语言与麦克风",
"在发送前编辑转写文本"
"在发送前编辑转写文本",
"通过客户端级实时通话执行或停止项目任务并听取简要进展"
],
"highlightsEn": [
"Download and verify local speech models",
"Choose a recognition language and microphone",
"Edit transcribed text before sending"
"Edit transcribed text before sending",
"Run or stop project tasks from a client-wide realtime call and hear concise progress updates"
],
"items": [
{
Expand All @@ -23221,6 +23323,28 @@
"reasonEn": "“Stream audio chunks during recording and show recognition progress” spans multiple live-state-dependent steps and currently has no single structured Command that can deterministically complete the whole workflow; the Agent opens the exact entry and keeps the remaining interaction visible to the user."
}
},
{
"id": "realtime-assistant-call",
"titleZh": "在客户端启动或结束实时语音通话,听取回复,并让助手执行或停止项目任务",
"titleEn": "Start or end a client-wide realtime voice call, hear replies, and ask the assistant to run or stop project tasks",
"control": {
"kind": "open",
"reasonCode": "unstructuredInteraction",
"reasonZh": "“在客户端启动或结束实时语音通话,听取回复,并让助手执行或停止项目任务”依赖麦克风、流式模型响应和任务状态,无法由单个结构化 Command 确定完成;Agent 会打开精确入口,并把通话与任务进度保留在用户可见界面。",
"reasonEn": "“Start or end a client-wide realtime voice call, hear replies, and ask the assistant to run or stop project tasks” depends on microphone input, streaming model responses, and task state, so no single structured Command can deterministically complete the workflow; the Agent opens the exact entry and keeps the call and task progress visible."
}
},
{
"id": "realtime-assistant-config",
"titleZh": "读取或保存实时语音服务配置",
"titleEn": "Read or save realtime voice service settings",
"control": {
"kind": "open",
"reasonCode": "unstructuredInteraction",
"reasonZh": "“读取或保存实时语音服务配置”包含凭据和模型选项,需要由用户在设置界面中检查并保存;Agent 会打开精确入口,将敏感输入保留在用户可见界面。",
"reasonEn": "“Read or save realtime voice service settings” includes credentials and model choices that the user must review and save in Settings; the Agent opens the exact entry and keeps sensitive input in the visible interface."
}
},
{
"id": "local-models",
"titleZh": "查看本地语音模型及安装、下载、验证、修复和删除状态",
Expand Down Expand Up @@ -23304,31 +23428,42 @@
"options": [],
"searchTerms": [
"feature.voice-input",
"语音输入",
"Voice input",
"语音输入与实时通话",
"Voice input and realtime calls",
"应用设置",
"Application settings",
"语音输入",
"录音",
"转写",
"麦克风",
"语音模型",
"听写",
"实时通话",
"语音助手",
"voice input",
"recording",
"transcription",
"microphone",
"speech model",
"dictation",
"realtime voice",
"voice assistant",
"下载和校验本地语音模型",
"选择识别语言与麦克风",
"在发送前编辑转写文本",
"通过客户端级实时通话执行或停止项目任务并听取简要进展",
"Download and verify local speech models",
"Choose a recognition language and microphone",
"Edit transcribed text before sending",
"Run or stop project tasks from a client-wide realtime call and hear concise progress updates",
"从聊天输入框开始、结束或取消一次录音",
"Start, finish, or cancel a recording from the chat composer",
"录音过程中分块传输音频并显示识别进度",
"Stream audio chunks during recording and show recognition progress",
"在客户端启动或结束实时语音通话,听取回复,并让助手执行或停止项目任务",
"Start or end a client-wide realtime voice call, hear replies, and ask the assistant to run or stop project tasks",
"读取或保存实时语音服务配置",
"Read or save realtime voice service settings",
"查看本地语音模型及安装、下载、验证、修复和删除状态",
"Browse local speech models and their install, download, verify, repair, and delete states",
"取消正在进行的语音模型下载",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,24 +3,28 @@
id: feature.voice-input
kind: feature
category: application
title_zh: "语音输入"
title_en: "Voice input"
title_zh: "语音输入与实时通话"
title_en: "Voice input and realtime calls"
---

# 语音输入 / Voice input
# 语音输入与实时通话 / Voice input and realtime calls

> 功能 / Feature

在聊天输入框中录音并转成文字,可选择本地模型、语言、麦克风与最长录音时间
将录音转成文字,或在客户端持续进行实时语音通话,让助手进入项目执行任务并播报简要进展

Record speech in the chat composer and transcribe it to text, with control over local models, language, microphone, and duration.
Transcribe recordings or hold a client-wide realtime voice call that can enter projects, run tasks, and speak concise progress updates.

## 完整功能清单 / Everything included

- **Agent 可定位入口,需交互完成 / Agent opens; interaction required** · 从聊天输入框开始、结束或取消一次录音
- Start, finish, or cancel a recording from the chat composer
- **Agent 可定位入口,需交互完成 / Agent opens; interaction required** · 录音过程中分块传输音频并显示识别进度
- Stream audio chunks during recording and show recognition progress
- **Agent 可定位入口,需交互完成 / Agent opens; interaction required** · 在客户端启动或结束实时语音通话,听取回复,并让助手执行或停止项目任务
- Start or end a client-wide realtime voice call, hear replies, and ask the assistant to run or stop project tasks
- **Agent 可定位入口,需交互完成 / Agent opens; interaction required** · 读取或保存实时语音服务配置
- Read or save realtime voice service settings
- **Agent 可定位入口,需交互完成 / Agent opens; interaction required** · 查看本地语音模型及安装、下载、验证、修复和删除状态
- Browse local speech models and their install, download, verify, repair, and delete states
- **Agent 可定位入口,需交互完成 / Agent opens; interaction required** · 取消正在进行的语音模型下载
Expand Down
Loading
Loading