feat(android): add IME, software keyboard, and stroke input - #1067
Closed
DepengWang wants to merge 143 commits into
Closed
DepengWang wants to merge 143 commits into
DepengWang wants to merge 143 commits into
Conversation
Real-device crash-loop root cause: OpenLessBackendWarmupActivity's window Surface could be destroyed (not just backgrounded) via the default back-button behavior, racing HWUI's worker-pool teardown and aborting the whole process with a native destroyed-mutex signal. The in-memory retry throttle then reset on every crash-restart, so the IME immediately relaunched the same Activity and crashed again forever. - Persist the backend-warmup retry timestamp in SharedPreferences (30s cooldown) so it survives a process crash/restart. - Remove the "relaunch the focused app's launcher Activity" fallback, which could land the user on the wrong screen entirely. - Route settings through a single tracked warmup-Activity instance instead of ever starting a second, untracked MainActivity host. - Override onBackPressed() to always background (never finish) this Activity, since finishing it is what destroys the window Surface. Verified on-device: 3x cold-start + open-settings + back-button stress cycles with zero crashes across the crash log.
- Replace the Stroke/EN toggle text labels with fixed-size PNG icons (drawable-nodpi/toggle_stroke.png, toggle_en.png) so they no longer change size with the UI language, and enlarge them for legibility. - Fix header/toggle position drift: the four keyboard panels (voice, English, stroke, stroke-number) each used different root padding for their own body rows, which shifted the logo/toggle left-right and up-down when switching modes. Compensate with per-panel margins so all four land at the same canonical 16dp/8dp inset. - Enlarge stroke-panel labels (通配/分词/符号/繁/清除/123) and move the small per-key numbers down to match the "0" key's native TextView top-gravity position instead of sitting almost flush with the edge. - Remove the microphone key's "CHERRY◆" watermark text. - Long-pressing the stroke keyboard's mic/"0" key now also starts dictation immediately, instead of only switching to voice mode.
…, 3-state shift
Stroke keyboard:
- Implement real word segmentation (分词): pressing it marks the current
character as a segment in memory without committing it, shows a
leading "commit whole word" candidate, and folds any pending segments
into the next candidate tap so the last character never needs an
explicit segment press.
- Number/symbol panel: backspace/enter/voice/return/symbols labels are
now always Chinese regardless of UI language (also fixes enter having
shown "←" instead of "回车"); "符号" now switches the digit grid to a
paged symbol set (3 pages x 12, default page is common punctuation)
with ▲/▼ paging, and the toggle relabels to "数字" while in symbol mode.
Added a "0" key (phone dial-pad style, under "8").
- The main stroke panel's "符号" key now jumps straight into the number
panel's symbol mode instead of just typing "#".
- Left punctuation rail is swipeable: 5 groups of 5 symbols, one swipe
advances exactly one group.
- Redrew the 丿/丶/乙 (keys 3/4/5) stroke glyphs by pixel-tracing a
reference screenshot instead of eyeballing bezier curves.
Cross-panel swipe gestures:
- Left/right swipe on any panel's root steps InputMode toward
Voice/English (Voice-Stroke-English order, clamped at both ends),
matching the toggle switch.
- Fixed the vertical punctuation-rail swipe fighting Android's system
gesture navigation and visibly flickering the IME closed/open: defer
the view-rebuild until after the touch gesture unwinds (post{} instead
of calling inline mid-dispatch), add systemGestureExclusionRects, and
requestDisallowInterceptTouchEvent once a drag is claimed.
English keyboard:
- Shift is now 3-state (lowercase/shift-once/caps-lock) with 3 distinct
icons (hollow arrow / solid arrow / solid arrow + underline), cycled
by tapping. Letter key labels now show the case that will actually be
typed, including reverting to lowercase after a one-shot capital.
Voice panel:
- Removed the keyboard-switch ("⌨") button. Rearranged @ / return /
backspace to @-return-backspace, all 48dp tall, return centered.
Widened @ and backspace 1.5x and restyled all three as a flat pill
(matching the mic capsule's own fill, no elevation) instead of the
stroke panel's raised-keycap look. Backspace supports long-press
repeat.
- Recording waveform lengthened (17 bars) with a genuine left-flowing
traveling wave instead of a static shape with per-bar shimmer.
Thinking/processing indicator's orbit ring now breathes (radius
grows and shrinks) on top of its existing rotation, uniformly across
all dots rather than sizing each dot off its own angle.
Bring IME height/raise, QA Warmup host, JNI context without IME, log export, 拼音分词, 输入方式 tab, signed_debug CI, and related Android fixes.
Keep beta's secure Android signing (env-based keystore) and collect-apks CRC checks; re-apply signed_debug dispatch on top for overlay debug APKs.
beta left the CA fingerprint contract test calling disconnected_app / rendered_text / remote_ui after they were removed; restore them so Linux egui CI can compile again.
Replace production tokio::spawn in cloud_sync_e2ee_store with inline async awaits, and mark e2ee test modules so the seam scanner ignores their spawn usage.
Inner //! docs must precede items; the #[cfg(test)] marker module was breaking rustc E0753.
Mock HTTP server was panicking on client disconnect mid-chunk, which flaked all_text_entrypoints_use_the_selected_protocol_over_http in CI.
HKLHaoBin
previously approved these changes
Sep 26, 2026
Collaborator
|
实际使用下来没有问题，@chris233 请看一下代码质量是否达到要求。 |
Resolve content conflicts: - cloud_sync_e2ee_store: keep beta spawn_owned (TaskSpawner seam) - linux-egui main/host_contract: take beta (Open-Less#1060 parity + capsule liveness) - Cargo.toml: keep Android wry vendor patch; drop deleted wayland-scanner patch
Drop leftover Linux target deps, wayland-data-control, and wry [patch.crates-io] so tauri-linux-free-contract and --locked MSRV checks pass.
Collaborator
|
开始审查 |
Collaborator
|
辛苦了 |
Collaborator
新问题（真机启动）在 #1067 线的 Android 包上复测时，进程能起来，但 Core 初始化失败：
定位
修复随后会单独把 |
App-private storage returns EPERM on link(2), so durable_create never published device-id and startup stayed on local_storage_unavailable. Log the real IO error and exclusive-create when hard_link is unsupported.
Collaborator
|
已将修复单独推进本 PR（不含多模态 / #1118）：
|
6 tasks done
appergb
added a commit
that referenced
this pull request
Sep 30, 2026
Integrate ASR/Omni fixes, API Route, Android IME and confirmation-based vocabulary learning. Complete settings transitions and macOS single-item credential storage. Synchronize all application versions to 2.0.0-Beta.4+build.20260930. Integrated PRs: #1116, #1120, #1121, #1123, #1124, #1067 and #1128. Linux #1117 remains independent and is excluded from automatic release. Validated: full cross-platform CI, 18 UI motion cases, 112 frontend test files, 1245 Core tests (1 ignored) and 572 macOS native tests. Co-authored-by: HKLHaoBin <we3q@qq.com>
Contributor
Author
|
@HKLHaoBin 看到 #1126（Beta 4 整合）已经合并进 beta 了，查了一下 想确认一下：这个 PR (#1067) 现在是不是该关掉了？还是说你们那边还打算让它单独走完审查再合并一次（这样的话内容会不会在 beta 里重复）？麻烦告诉我一声，免得我这边一直傻等一个可能已经不需要再动的状态，谢谢！ |
Collaborator
是的，这个应该关掉了。不过一般来说，等到 Beta 4 整合合并的时候才会正式关掉这个 PR，在合并之前它还是打开着的。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1084
Closes #1088
Summary
Adds a native Android IME (system keyboard) input path to OpenLess, reaching usable/day-to-day quality:
InputConnection, with live status, mic swipe gestures (Raw / Quick notes / Cloud notes), and heartbeat link indicatorAlso includes follow-up polish for QA host (WarmupActivity), keyboard height/raise prefs with live preview, and CI fixes needed to land on
beta.Linked issues
Test plan