Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -376,13 +376,15 @@ The dictionary handles your proper nouns, product names, names of people, and ne
- **Learn from corrections (experimental).** Open Settings → Experiments & extensions → Learn from corrections to enable it and configure observation duration (10–60 seconds, default 60), suggestion duration (5–60 seconds, default 10), and maximum automatic phrase length (2–32 characters, default 12). It defaults to off and is not enabled by cursor context or cloud sync. On macOS, Windows and Android, supported editors can be observed after insertion. Suggestions require confirmation before expiry to enter the dictionary. Changing parameters stops the current observation and clears pending suggestions; new values apply to the next dictation. Android requires accessibility. When observation is unavailable, use **Remember a word** in history details; Android IME result editing also offers an unchecked dictionary option. Every path requires explicit confirmation and does not create global replacement rules.
- **Entries that earn their keep get priority.** The hotword budget sent to ASR providers is finite (a few hundred characters). Entries are ranked by hit count, with a few reserved seats for words you just added by hand, so the terms you actually use keep their place instead of being pushed out by whatever you added most recently.

### Cursor context (opt-in, macOS)
### Cursor context (opt-in, macOS / Android)

Settings → Privacy → Data storage → **Cursor context**. Off by default.

When on, each dictation reads a few hundred characters around your cursor **in the app you are writing in** and sends them with the polish request, so the model knows what you are writing about. Chinese homophones (接口/借口, 大鱼/大禹) are indistinguishable to an acoustic model but obvious from context. Local vocabulary learning has a separate switch and does not require cursor context. Observed text is not sent to a model; words explicitly added to the dictionary participate in future ASR and polish requests as described above.

Cursor context excludes password fields, macOS Secure Input, known password managers and terminals. Turning it off stops reading cursor context for polishing; other authorized accessibility features, including local vocabulary learning and insertion, work independently. Turning vocabulary learning off stops observation and clears pending suggestions.
On Android, the OpenLess keyboard reads the text before and after the caret straight from its own `InputConnection` at the moment recording starts — no Accessibility permission involved, one read, no retry. Dictation, quick notes and cloud notes all use it for polishing, but the context only informs how the spoken words are written: it is never copied into the note, the history, the cloud note webhook or the logs. With a cloud polish model it is sent to that provider as part of the polish request. The floating overlay does not read cursor context yet.

Cursor context excludes password fields, macOS Secure Input, Android editors that are password-typed or ask for no personalized learning, known password managers and terminals. Turning it off stops reading cursor context for polishing; other authorized accessibility features, including local vocabulary learning and insertion, work independently. Turning vocabulary learning off stops observation and clears pending suggestions.

The main window is organized as Home / History / Dictionary / Settings. The Dictionary tab opens a separate editor window when you click "New". The Home tab shows total dictation time, total characters, average characters per minute, estimated time saved, and dictionary participation statistics.

Expand Down
6 changes: 4 additions & 2 deletions README.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -383,13 +383,15 @@ OpenLess 的润色模型只重塑文本。它不回答问题、不执行任务
- **手改学词(实验)。** 在「设置 → 实验与扩展 → 手改学词」进入独立配置页,设置观察时长(10–60 秒,默认 60)、建议保留时长(5–60 秒,默认 10)及自动建议最大词长(2–32 个字符,默认 12)。功能默认关闭,不跟随光标上下文或云同步授权。macOS、Windows 和 Android 可在支持的编辑器中观察插入后的修改,候选需在有效期内逐条确认才加入词典。修改参数会结束当前观察并清空待确认建议,下次听写生效。Android 需要无障碍服务。无法观察时,可在历史详情点击「记住词汇」手动输入正确词;Android 输入法编辑结果也提供默认不勾选的加入词典选项。所有入口均需明确确认,不自动创建全局替换规则。
- **真正在用的词优先。** 发给 ASR 的热词预算是有限的(几百字符)。条目按命中次数排序,并给刚手动添加的词留几个保底席位——这样你天天在用的那些词不会被「最近刚加的」挤出去。

### 光标上下文(需手动开启,仅 macOS)
### 光标上下文(需手动开启,macOS / Android)

设置 → 隐私 → 数据存储 → **光标上下文**。默认关闭。

开启后,每次听写会读取**你正在写的那个应用里**光标附近的几百个字,随润色请求一起发出,让模型知道你在写什么。中文同音词(接口/借口、大鱼/大禹)声学模型分不出来,但上下文能分。手改学词是独立的本地功能,不需要开启此设置。观察文本本身不会发送给模型;确认加入词典的词会按词典规则参与后续 ASR/润色。

光标上下文排除密码输入框、macOS Secure Input、已知密码管理器和终端。关闭此开关后不会为润色读取光标上下文;其他已授权的辅助功能(如手改学词或插入)独立工作。手改学词关闭后会停止观察并清空待确认建议。
Android 上由 OpenLess 键盘在开始录音的那一刻,直接通过自身的 `InputConnection` 读取光标前后的文字——不需要无障碍权限,只读一次,不重试。听写、速记和云笔记润色时都会参考,但上下文只用来判断这次口述该怎么写:不会被抄进笔记、历史、云笔记 Webhook 或日志。使用云端润色模型时,上下文会随本次润色请求发送给该模型服务。悬浮球暂不读取光标上下文。

光标上下文排除密码输入框、macOS Secure Input、Android 上密码类型或声明不做个性化学习的输入框、已知密码管理器和终端。关闭此开关后不会为润色读取光标上下文;其他已授权的辅助功能(如手改学词或插入)独立工作。手改学词关闭后会停止观察并清空待确认建议。

主窗口组织为 首页 / 历史 / 词典 / 设置。点击“新建”时,词典页会打开一个独立的编辑窗口。首页展示总听写时长、总字数、平均每分钟字数、估算节省的时间,以及词典参与统计。

Expand Down
1 change: 1 addition & 0 deletions docs/android-ime.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
- 笔画输入:离线笔画字典、单字候选、确认后的词语联想、分词、简繁偏好、数字/符号面板、上滑数字和个人词频。
- 剪贴板:历史记录、收藏/删除/分类、选择/复制/粘贴、纠正词写入全局词典。
- 手改学词:独立授权默认关闭,无障碍服务在有界观察期内报告当前编辑器的文本变化,主进程 Core 判断建议与过期时间;确认卡或编辑结果的显式勾选才加入词典。跨进程共享截止时间,服务重连不延长观察。
- 光标上下文(默认关闭):开始录音时通过 `InputConnection` 读取光标前后各一小段文字(`ImePrivacyPolicy.kt` 负责门禁:密码框、`TYPE_NULL`、`IME_FLAG_NO_PERSONALIZED_LEARNING` 和敏感应用一律不读),随 `start` 命令结构化传给 Rust,由 Core 裁剪到 600 字并组装 Prompt。听写、速记、云笔记润色时都参考,但不写入笔记、历史、Webhook 或日志;悬浮球路径暂未接入。
- 英文键盘:字母/数字/符号三层布局、英文候选词、个人词频、自定义词长按删除、按键预览和上滑输入数字/符号。
- 跨应用插入:按可用性使用无障碍、Shizuku 或剪贴板回退;权限和输入法启用状态在 Android 侧单独管理。

Expand Down
58 changes: 58 additions & 0 deletions openless-all/app/android/kotlin/ImePrivacyPolicy.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
package com.openless.app

import android.text.InputType
import android.view.inputmethod.EditorInfo

/** Text around the caret, read once when a recording starts. */
internal data class AndroidCursorContext(
val before: String,
val after: String,
val packageName: String?,
)

/**
* Gates for reading text out of the host editor. Kept apart from
* [ImeLearningPolicy]: learning decides what stays on this device, while
* cursor context may be sent to the configured polish LLM provider.
*/
internal object ImePrivacyPolicy {
// Read a little wider than the 600-char budget; Core trims to the final window.
const val CURSOR_BEFORE_CHARS = 600
const val CURSOR_AFTER_CHARS = 200

private val sensitivePackageHints =
listOf("keepass", "bitwarden", "1password", "lastpass", "dashlane", "termux", "password")

/** Single list for cursor context and accessibility vocabulary observation alike. */
fun isSensitivePackage(packageName: String?): Boolean {
val name = packageName?.lowercase().orEmpty()
return sensitivePackageHints.any { name.contains(it) }
}

fun allowsCursorContext(inputType: Int, imeOptions: Int, packageName: String?): Boolean =
inputType != InputType.TYPE_NULL && !ImeLearningPolicy.isPassword(inputType) &&
imeOptions and EditorInfo.IME_FLAG_NO_PERSONALIZED_LEARNING == 0 &&
!isSensitivePackage(packageName)

/**
* One read, no retry: a null, a throw or an empty editor all mean "no
* context" and dictation carries on. The readers are never invoked while
* the switch is off or a gate rejects the field.
*/
fun captureCursorContext(
enabled: Boolean,
inputType: Int,
imeOptions: Int,
packageName: String?,
readBefore: (Int) -> CharSequence?,
readAfter: (Int) -> CharSequence?,
): AndroidCursorContext? {
if (!enabled || !allowsCursorContext(inputType, imeOptions, packageName)) return null
val before = runCatching { readBefore(CURSOR_BEFORE_CHARS)?.toString() }.getOrNull().orEmpty()
.let { if (it.firstOrNull()?.isLowSurrogate() == true) it.drop(1) else it }
val after = runCatching { readAfter(CURSOR_AFTER_CHARS)?.toString() }.getOrNull().orEmpty()
.let { if (it.lastOrNull()?.isHighSurrogate() == true) it.dropLast(1) else it }
if (before.isBlank() && after.isBlank()) return null
return AndroidCursorContext(before, after, packageName)
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ class OpenLessAccessibilityService : AccessibilityService() {
if (node == null) { stopVocabularyObservation(); return }
val packageName = node.packageName?.toString()?.lowercase().orEmpty()
if (!node.isEditable || node.isPassword || packageName.isEmpty() || packageName == this.packageName ||
listOf("keepass", "bitwarden", "1password", "lastpass", "dashlane", "termux", "password").any { packageName.contains(it) }) {
ImePrivacyPolicy.isSensitivePackage(packageName)) {
node.recycle()
stopVocabularyObservation()
return
Expand Down
3 changes: 3 additions & 0 deletions openless-all/app/android/kotlin/OpenLessAndroidPreferences.kt
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,9 @@ object OpenLessAndroidPreferences {
fun strokeUsageEnabled(context: Context): Boolean =
readPreferenceBoolean(context, "strokeUsageEnabled") ?: true

fun cursorContextEnabled(context: Context): Boolean =
readPreferenceBoolean(context, "cursorContextEnabled") ?: false

private fun readPreferenceString(context: Context, key: String): String? {
for (file in preferenceFiles(context).distinctBy { it.absolutePath }) {
if (!file.isFile) {
Expand Down
Loading
Loading