Hermesses

Hermesses is a standalone Android client that connects a Hermes Agent to Rokid Glasses through an Android phone. The phone uses the native hermes-webui transport contract and bridges sessions, streaming chat, approvals, voice, photos, and device controls to the glasses.

No server is preconfigured. Enter the exact HTTPS origin of the hermes-webui deployment you want to use.

Current v1 experience

• Long-press the glasses temple to start speech recognition.
• Stop speaking to stage the transcription on the HUD, then choose Send.
• Use Photo before Send to attach up to four Rokid camera images to the same Hermes turn.
• Read the answer as it streams onto the display; optional ElevenLabs TTS can read the final answer aloud.
• Browse, create, and switch Hermes sessions from the glasses.
• See compact tool progress without raw tool arguments or output.
• Resolve a Hermes approval on the glasses with Allow once or Deny.

Continuous listening and automatic photo capture for phrases such as “look at this” are intentionally deferred. The first version keeps every voice turn and photo capture explicit.

Architecture

Hermes Agent ←→ hermes-webui ←HTTPS/JSON+SSE→ Phone App ←Bluetooth/CXR→ Rokid Glasses
     │                │                              │                    │
 models/tools     auth + sessions              voice + uploads      HUD + camera
 skills/memory    chat stream + approvals      TTS + wake           gestures

The phone follows the native Hermex transport contract:

• password login at POST /api/auth/login, followed by the hermes_session cookie;
• sessions through /api/sessions, /api/session, and /api/session/new;
• turns through POST /api/chat/start and GET /api/chat/stream SSE;
• Rokid photos through multipart POST /api/upload;
• approval polling and one-shot responses through /api/approval/pending and /api/approval/respond.

Native requests intentionally omit browser Origin and Referer headers. Redirects, cleartext HTTP, credentials in URLs, automatic request retries, oversized REST responses, and oversized SSE events fail closed.

Gradle modules

Module	Purpose
phone-app/	Hermes WebUI client, secure password storage, voice, uploads, TTS, wake coordination, and Rokid CXR-M bridge.
glasses-app/	Rokid HUD, gestures, camera capture, staged input, session picker, tool state, and approval overlay.
shared/	Backend-neutral phone-to-glasses JSON messages.
contract-test/	Pure-JVM tests and a public/authenticated hermes-webui compatibility probe.

Prerequisites

• Android Studio and JDK 17
• Android SDK 34
• Rokid Glasses, or two Android emulators for the debug bridge
• Rokid CXR SDK credentials
• access to a password-authenticated hermes-webui deployment

Local configuration

Copy the tracked template, then fill in local values:

cp local.properties.example local.properties

local.properties is gitignored. Do not commit credentials. The template also contains the optional URL and password fields for the live JVM contract probe.

The Android app does not read the Hermes password from this file. Enter it on the phone under Settings → Server. The app encrypts it with an Android Keystore AES-GCM key; only the encrypted envelope is stored in private preferences. Android backup and device transfer are disabled.

Build and test

./scripts/check.sh
./gradlew :contract-test:run

check.sh resolves an installed JDK 17, runs all local unit suites, compiles Android instrumentation tests, runs debug lint, and assembles both apps. The live contract probe is separate because it needs an explicitly configured server.

The phone build bundles the glasses debug APK as phone-app/src/main/assets/glasses-app-release.apk, which is gitignored and can be installed from the phone settings screen.

For an installable local release of both apps, run:

./scripts/build-release.sh

The script resolves JDK 17 from HERMESSES_JAVA_HOME, JAVA_HOME, java on PATH, or common macOS locations. It builds and signs the optimized glasses APK first, embeds those exact bytes into the phone app, and writes both artifacts to build/release/. By default it uses the Android debug keystore; set HERMESSES_KEYSTORE to use another local keystore with the same debug-key credentials.

Install the release phone app with:

adb install -r build/release/Hermesses-phone-1.0.0.apk

For a debug build, use Android Studio or:

adb install -r phone-app/build/outputs/apk/debug/phone-app-debug.apk

Connect

1. Open Hermesses on the phone.
2. In Settings → Server, enter an exact HTTPS hermes-webui origin and its password, then tap Save and reconnect.
3. Fold the right glasses leg and triple-click the camera button to enter pairing mode.
4. Scan and connect from Settings → Glasses.
5. Use Install to glasses to transfer the bundled HUD app over Wi-Fi P2P.
6. Start Hermesses on the glasses. On the first connection the phone creates a dedicated glasses session; later connections resume it. The existing session picker can still switch to any other Hermes session.

Hermesses owns the package IDs com.hermesses.phone and com.hermesses.glasses. Pairing, credentials, and application storage are private to those identities.

Glasses brightness and volume

Open Settings → Glasses on the phone to set Brightness (1..15) and Media volume (0..15). A fresh installation shows Not set and sends no output-setting command until a value is chosen. Dragging commits after the gesture finishes; an accessibility adjustment commits its selected step directly. Committed values are stored on the phone, applied immediately over the native Rokid Bluetooth connection, and replayed once after each real reconnect. Wi-Fi P2P is not required.

The controls remain visible but disabled unless native Rokid Bluetooth is connected. A debug WebSocket client does not enable them. These controls use the phone-side CXR-M device APIs directly; they do not change the phone's own media volume or display brightness, microphone routing, the shared phone-to-glasses protocol, or the glasses HUD app.

Diagnostic log export

Open Settings → Diagnostics and tap Export diagnostic log to save a timestamped text snapshot with Android's document picker. The export includes bounded app lifecycle, connection, operation, permission, and device model metadata. It excludes user messages, transcripts, audio, photos, credentials, server URLs and payloads, raw identifiers, exception details, and stack traces.

Hermesses retains at most 2 MiB of diagnostic records for up to 7 days in app-private storage. Exporting takes one immutable snapshot without deleting or changing those retained records. The saved copy is user-controlled and is outside Hermesses retention once it is written to the selected local or cloud document provider.

Glasses controls

Gesture	Message history	Menu bar	Approval
Swipe forward	Scroll down	Previous item	Select Deny
Swipe backward	Scroll up	Next item	Select Allow once, when available
Tap	Scroll to bottom	Execute	Confirm selection
Double-tap	Jump to menu	Exit	—
Long-press	Start voice input	Start voice input	—

Menu items:

• Photo captures an image for the next staged turn, up to four.
• Session lists, creates, and switches Hermes sessions.
• Size changes HUD position.
• More contains font size, command shortcuts, and TTS toggle.

Command shortcuts are sent as ordinary Hermes chat input, matching the retained first-version behavior; hermes-webui currently exposes no command-catalog endpoint.

Voice and audio

OpenAI Realtime transcription is the primary voice recognizer when configured. Android SpeechRecognizer remains the fallback. ElevenLabs TTS is optional and receives only completed assistant text. Configure these providers separately in the phone settings.

Emulator bridge

Debug builds can replace Bluetooth/CXR with a local WebSocket bridge:

1. Create a 480×640, 5-inch glasses AVD.
2. Run the phone emulator; the debug server listens on port 8081.
3. Follow the ADB forwarding instruction shown in the phone settings.
4. Run the glasses emulator; it connects to 10.0.2.2:8081.

./gradlew :phone-app:installDebug
./gradlew :glasses-app:installDebug

The emulator exercises UI and bridge behavior and can confirm that the brightness and media-volume controls remain unavailable in debug mode. Physical Rokid hardware is still required to validate CXR pairing, actual output levels, brightness-aware wake behavior, camera capture, and APK installation. See the uncompleted physical acceptance checklist.

Troubleshooting

“Password is required” or “Authentication failed”

• Confirm that the password works in the browser at the same hermes-webui origin.
• Replace the saved password in Settings → Server. The app never displays the stored value.

“Cannot reach hermes-webui”

• Verify /health is reachable from the phone and the URL is the exact HTTPS origin with no path or trailing punctuation.
• Hermesses deliberately refuses HTTP and redirects so the password cannot be forwarded to another origin.

Glasses do not connect or install

• Verify rokid.clientSecret and rokid.accessKey in local.properties.
• Re-enter glasses pairing mode, reconnect Bluetooth, then retry the Wi-Fi installation.

Voice recognition is poor

• Add an OpenAI API key under Settings → Voice; otherwise Android recognition is used.

Contract details

See docs/hermes-webui-contract.md for the exact client boundary and probe status. Rokid integration notes and the physical acceptance checklist live in docs/ROKID.md.

License and attribution

Hermesses is licensed under GNU Affero General Public License v3.0. It originated from Clawsses by Pohlster BV; the required upstream notices remain in COPYRIGHT. Rokid CXR SDK components remain proprietary and separately licensed by Rokid Corporation.