|
| 1 | +# Camera Live View Protocol (V1) |
| 2 | + |
| 3 | +This document describes the reverse-engineered protocol for camera live streaming, WebRTC signaling, and media transport on camera-equipped Roborock V1 vacuum robots. |
| 4 | + |
| 5 | +## Scope & Evidence |
| 6 | + |
| 7 | +- **Protocol family**: Roborock V1 (`pv=1.0`). |
| 8 | +- **Feature flags**: Models supporting camera features report `is_video_monitor_supported` (bit mask `8`, bit 3 of `robotNewFeatures`) in device capabilities (see [V1_DEVICE_FEATURES.md](V1_DEVICE_FEATURES.md)). |
| 9 | +- **Tested hardware**: Roborock Qrevo Curv (model `a135`). |
| 10 | +- **Observation source**: Captures from the Roborock mobile app session negotiation and standalone aiortc test client verification. |
| 11 | +- **Note on model variance**: Roborock firmware and command parameters can vary across models. Unverified fields and behaviors are called out explicitly. |
| 12 | + |
| 13 | +--- |
| 14 | + |
| 15 | +## Architecture Overview |
| 16 | + |
| 17 | +The camera subsystem uses standard **Roborock V1 JSON-RPC for signaling** and **WebRTC for media transport**: |
| 18 | + |
| 19 | +- **Signaling channel**: Standard Roborock protocol RPC commands carried in `dps["101"]` / `dps["102"]`, transmitted via the cloud MQTT broker (`rr/m/...`) or local TCP channel. |
| 20 | +- **Media transport**: WebRTC PeerConnection with ICE (STUN / TURN) negotiation for low-latency video and audio streaming. |
| 21 | + |
| 22 | +```mermaid |
| 23 | +sequenceDiagram |
| 24 | + participant App as Client (App / Library) |
| 25 | + participant Robot as Vacuum Robot |
| 26 | + |
| 27 | + Note over App,Robot: 1. Authentication & Session Initialization |
| 28 | + App->>Robot: check_homesec_password(password=MD5) |
| 29 | + Robot-->>App: ['ok'] |
| 30 | + App->>Robot: switch_video_quality(quality="HD") |
| 31 | + Robot-->>App: ['ok'] |
| 32 | + App->>Robot: start_camera_preview(client_id, quality, password) |
| 33 | + Robot-->>App: ['ok'] |
| 34 | + |
| 35 | + Note over App,Robot: 2. Ephemeral TURN Server Credentials |
| 36 | + App->>Robot: get_turn_server() |
| 37 | + Robot-->>App: {url, user, pwd} |
| 38 | + |
| 39 | + Note over App,Robot: 3. WebRTC SDP & ICE Exchange |
| 40 | + App->>Robot: send_sdp_to_robot(app_sdp=base64(SDP offer)) |
| 41 | + Robot-->>App: ['ok'] |
| 42 | + loop Poll until SDP answer ready |
| 43 | + App->>Robot: get_device_sdp() |
| 44 | + Robot-->>App: {dev_sdp: "retry"} (pending) or {dev_sdp: base64(SDP answer)} |
| 45 | + end |
| 46 | + loop Poll remote candidates |
| 47 | + App->>Robot: get_device_ice() |
| 48 | + Robot-->>App: {dev_ice: [base64(candidate)...]} |
| 49 | + end |
| 50 | + App->>Robot: send_ice_to_robot(app_ice=base64(candidate)) |
| 51 | + Robot-->>App: ['ok'] |
| 52 | + |
| 53 | + Note over App,Robot: 4. WebRTC Media Stream Established |
| 54 | + Robot-->>App: Video Track (H.264) / Optional Audio (Opus) |
| 55 | + |
| 56 | + Note over App,Robot: 5. Teardown |
| 57 | + App->>Robot: stop_camera_preview() |
| 58 | + Robot-->>App: ['ok'] |
| 59 | +``` |
| 60 | + |
| 61 | +--- |
| 62 | + |
| 63 | +## Signaling Transport |
| 64 | + |
| 65 | +Camera commands are standard Roborock V1 RPC messages. They follow the library's transport structure documented in [DEVICES.md](DEVICES.md). |
| 66 | + |
| 67 | +### DPS Envelope Framing |
| 68 | + |
| 69 | +In V1 protocol messages: |
| 70 | +- **Requests**: The JSON-RPC request object is serialized to a JSON string and placed under `dps["101"]`. |
| 71 | +- **Responses**: The device replies with a JSON-serialized result string under `dps["102"]`. |
| 72 | +- **Transport framing**: Over MQTT, outer message protocol header is `101` and requests include a `security` block with nonce and endpoint. Over local TCP (port 58867), outer protocol is `4` and `security` is omitted. |
| 73 | + |
| 74 | +#### Complete DPS Request Example: |
| 75 | +```json |
| 76 | +{ |
| 77 | + "dps": { |
| 78 | + "101": "{\"id\":12345,\"method\":\"start_camera_preview\",\"params\":{\"client_id\":\"a1b2c3d4\",\"quality\":\"HD\",\"password\":\"81dc9bdb52d04dc20036dbd8313ed055\"},\"security\":{\"endpoint\":\"...\",\"nonce\":\"...\"}}" |
| 79 | + }, |
| 80 | + "t": 1760000000 |
| 81 | +} |
| 82 | +``` |
| 83 | + |
| 84 | +#### Complete DPS Response Example: |
| 85 | +```json |
| 86 | +{ |
| 87 | + "dps": { |
| 88 | + "102": "{\"id\":12345,\"result\":[\"ok\"]}" |
| 89 | + } |
| 90 | +} |
| 91 | +``` |
| 92 | + |
| 93 | +### MQTT Topics |
| 94 | + |
| 95 | +When signaling over the Roborock Cloud MQTT broker: |
| 96 | + |
| 97 | +```text |
| 98 | +Publish: rr/m/i/{rriot_u}/{mqtt_username}/{duid} |
| 99 | +Subscribe: rr/m/o/{rriot_u}/{mqtt_username}/{duid} |
| 100 | +``` |
| 101 | + |
| 102 | +Where: |
| 103 | +- `rriot_u`: User identifier from Roborock account credentials. |
| 104 | +- `duid`: Unique device identifier of the vacuum. |
| 105 | +- `mqtt_username`: Hashed MQTT username slice: `hashlib.md5(f"{rriot_u}:{rriot_k}".encode()).hexdigest()[2:10]`. |
| 106 | +- Broker hostname, port, and TLS configuration are obtained directly from `rriot.r.m` (see `create_mqtt_params` in `roborock/protocol.py`). |
| 107 | + |
| 108 | +> [!NOTE] |
| 109 | +> `{mqtt_username}` in the topic is distinct from the `client_id` parameter passed to `start_camera_preview`. While `{mqtt_username}` authenticates the MQTT transport channel, `client_id` identifies the individual camera streaming consumer session on the robot. Clients typically pass their hashed MQTT username (or device identifier / UUID) as `client_id`; whether the robot enforces specific format constraints on this identifier is unverified. |
| 110 | + |
| 111 | +--- |
| 112 | + |
| 113 | +## Authentication & Password Hashing |
| 114 | + |
| 115 | +Camera viewing is protected by a separate PIN / pattern lock configured in the mobile app: |
| 116 | + |
| 117 | +1. **Pattern conversion**: The 3x3 pattern lock points are represented as a sequence of numeric digits (e.g. `"1234"`). |
| 118 | +2. **Hash derivation**: The password parameter is the lowercase hexadecimal MD5 digest of the UTF-8 encoded digits: |
| 119 | + ```python |
| 120 | + import hashlib |
| 121 | + |
| 122 | + password_hash = hashlib.md5("1234".encode("utf-8")).hexdigest() |
| 123 | + # Result: "81dc9bdb52d04dc20036dbd8313ed055" |
| 124 | + ``` |
| 125 | +3. Both `check_homesec_password` and `start_camera_preview` expect this MD5 hash. |
| 126 | +4. An incorrect password returns an error response (e.g. `{"code": -1, "message": "password error"}`). Multiple consecutive invalid attempts may trigger a temporary security lockout on the robot. |
| 127 | + |
| 128 | +--- |
| 129 | + |
| 130 | +## Commands Reference |
| 131 | + |
| 132 | +See also [V1_API_COMMANDS.md](V1_API_COMMANDS.md) for general V1 device RPC references. |
| 133 | + |
| 134 | +> [!NOTE] |
| 135 | +> Device RPC responses in the tables below may be returned either as a single-element list (e.g. `[{"dev_sdp": "..."}]`) or as a bare dictionary (`{"dev_sdp": "..."}`), depending on firmware version. Parsers should unwrap single-element lists transparently. |
| 136 | + |
| 137 | +### Camera Session Control |
| 138 | + |
| 139 | +| Command | Parameters | Response Result | Description | |
| 140 | +|---------|------------|-----------------|-------------| |
| 141 | +| `check_homesec_password` | `{"password": "<md5_hash>"}` | `["ok"]` | Validates the camera security PIN/pattern hash. | |
| 142 | +| `start_camera_preview` | `{"client_id": "...", "quality": "HD", "password": "<md5_hash>"}` | `["ok"]` | Activates camera preview mode on the robot. | |
| 143 | +| `stop_camera_preview` | `[]` | `["ok"]` | Deactivates camera preview and powers down the camera sensor. | |
| 144 | +| `switch_video_quality` | `{"quality": "HD"}` or `{"quality": "SD"}` | `["ok"]` | Switches streaming resolution (`HD` or `SD`). | |
| 145 | + |
| 146 | +### WebRTC Signaling |
| 147 | + |
| 148 | +| Command | Parameters | Response Result | Description | |
| 149 | +|---------|------------|-----------------|-------------| |
| 150 | +| `get_turn_server` | `[]` | `[{"url": "turn:...", "user": "...", "pwd": "..."}]` | Retrieves ephemeral TURN relay credentials from the robot. | |
| 151 | +| `send_sdp_to_robot` | `{"app_sdp": "<base64_json>"}` | `["ok"]` | Sends the client's local SDP offer to the robot. | |
| 152 | +| `get_device_sdp` | `[]` | `[{"dev_sdp": "<base64_json>"}]` or `[{"dev_sdp": "retry"}]` | Polls for the robot's SDP answer. | |
| 153 | +| `send_ice_to_robot` | `{"app_ice": "<base64_json>"}` | `["ok"]` | Forwards a client ICE candidate to the robot. | |
| 154 | +| `get_device_ice` | `[]` | `[{"dev_ice": ["<base64_json>", ...]}]` | Polls for remote ICE candidates generated by the robot. | |
| 155 | + |
| 156 | +### Voice Intercom (Two-Way Audio) |
| 157 | + |
| 158 | +| Command | Parameters | Response Result | Description | |
| 159 | +|---------|------------|-----------------|-------------| |
| 160 | +| `start_voice_chat` | `[]` | `["ok"]` | Unblocks transmission of microphone audio packets from the robot. | |
| 161 | +| `stop_voice_chat` | `[]` | `["ok"]` | Mutes/stops voice chat audio streaming. | |
| 162 | + |
| 163 | +> [!NOTE] |
| 164 | +> Even if audio transceivers are negotiated in the WebRTC SDP offer, the robot does not transmit audio media from its onboard microphone until `start_voice_chat` is called. |
| 165 | + |
| 166 | +### Remote Driving During Live View |
| 167 | + |
| 168 | +| Command | Parameters | Response Result | Description | |
| 169 | +|---------|------------|-----------------|-------------| |
| 170 | +| `app_rc_start` | `[]` | `["ok"]` | Initiates manual remote control session. | |
| 171 | +| `app_rc_move` | `[{"velocity": 0.2, "omega": 0.0, "duration": 500, "seqnum": 1}]` | `["ok"]` | Sends motion vector (linear speed, angular speed, duration ms). | |
| 172 | +| `app_rc_end` | `[]` | `["ok"]` | Exits manual remote control session. | |
| 173 | + |
| 174 | +--- |
| 175 | + |
| 176 | +## SDP & ICE Payload Serialization |
| 177 | + |
| 178 | +All SDP descriptions and ICE candidate structures exchanged in signaling are packaged as base64-encoded UTF-8 JSON strings. |
| 179 | + |
| 180 | +### SDP Offer Packaging |
| 181 | + |
| 182 | +The client creates a standard WebRTC SDP offer string, wraps it in JSON with `"sdp"` and `"type"` keys, and base64-encodes it: |
| 183 | + |
| 184 | +```python |
| 185 | +import base64 |
| 186 | +import json |
| 187 | + |
| 188 | +sdp_obj = { |
| 189 | + "sdp": peer_connection.localDescription.sdp, |
| 190 | + "type": "offer", |
| 191 | +} |
| 192 | +app_sdp = base64.b64encode(json.dumps(sdp_obj).encode("utf-8")).decode("ascii") |
| 193 | +# Parameter payload sent: {"app_sdp": app_sdp} |
| 194 | +``` |
| 195 | + |
| 196 | +### SDP Answer Polling & Parsing |
| 197 | + |
| 198 | +The client polls `get_device_sdp` until the answer is available: |
| 199 | +- While pending, the robot returns `{"dev_sdp": "retry"}`. |
| 200 | +- When ready, `dev_sdp` contains a base64 string decoding to: |
| 201 | + ```json |
| 202 | + { |
| 203 | + "sdp": "v=0\r\no=- ...\r\ns=-\r\n...", |
| 204 | + "type": "answer" |
| 205 | + } |
| 206 | + ``` |
| 207 | +- Polling should be bounded (e.g. 20–30 attempts with 200–300 ms delays). |
| 208 | + |
| 209 | +### ICE Candidate Packaging |
| 210 | + |
| 211 | +When using aiortc with standard vanilla ICE, local candidates are embedded directly within the initial SDP offer, making separate calls to `send_ice_to_robot` optional. Clients supporting Trickle ICE (such as the official mobile app) send individual candidates dynamically via `send_ice_to_robot`: |
| 212 | + |
| 213 | +```python |
| 214 | +# In aiortc: candidate_str = f"candidate:{candidate_to_sdp(candidate)}" |
| 215 | +ice_obj = { |
| 216 | + "candidate": candidate_str, |
| 217 | + "sdpMid": candidate.sdpMid, |
| 218 | + "sdpMLineIndex": candidate.sdpMLineIndex, |
| 219 | +} |
| 220 | +app_ice = base64.b64encode(json.dumps(ice_obj).encode("utf-8")).decode("ascii") |
| 221 | +# Parameter payload sent: {"app_ice": app_ice} |
| 222 | +``` |
| 223 | + |
| 224 | +Remote candidates received from `get_device_ice`: |
| 225 | +- Returns a list under `"dev_ice"`, where each element is a base64-encoded JSON string matching the above candidate structure. |
| 226 | +- Candidates should be deduplicated by foundation/priority before injecting into the local peer connection. |
| 227 | + |
| 228 | +### TURN Server Mapping |
| 229 | + |
| 230 | +`get_turn_server` returns credentials mapped to `RTCIceServer`: |
| 231 | +- `url` → `urls` (e.g. `"turn:turn-us.roborock.com:3478"`) |
| 232 | +- `user` → `username` |
| 233 | +- `pwd` → `credential` |
| 234 | + |
| 235 | +--- |
| 236 | + |
| 237 | +## Media Transport & Codec Specifications |
| 238 | + |
| 239 | +### Video |
| 240 | +- **Codec**: H.264 (AVC) baseline / constrained baseline profile. |
| 241 | +- **Resolution**: Dynamically switched between 1080p/720p (`HD`) and 480p/360p (`SD`). |
| 242 | +- **Framerate**: Approximately 15–25 FPS depending on link quality. |
| 243 | + |
| 244 | +### Audio (Two-Way Intercom) |
| 245 | +- **Wire Codec**: Opus (negotiated via SDP offer/answer). |
| 246 | +- **Robot Microphone → Client Speaker**: |
| 247 | + - Sample rate: 48 kHz. |
| 248 | + - Channels: Negotiated as stereo (aiortc decodes to 16-bit interleaved PCM, 960 samples per 20 ms frame). Mono audio can be extracted by taking the left channel (`audio[::2]`). |
| 249 | +- **Client Microphone → Robot Speaker**: |
| 250 | + - Sample rate: 48 kHz mono Opus audio stream. |
| 251 | + - Frame duration: 20 ms frames (960 samples). |
| 252 | + |
| 253 | +--- |
| 254 | + |
| 255 | +## Operational & Reliability Considerations |
| 256 | + |
| 257 | +1. **Session Concurrency**: Roborock vacuums permit only a single active camera preview session at a time. Requests while another client (or the official app) is streaming will be rejected. |
| 258 | +2. **NAT Traversal**: Because vacuum robots are commonly located behind residential NATs/firewalls, direct peer-to-peer ICE host/srflx candidates frequently fail. The TURN server credentials from `get_turn_server` provide essential relay connectivity. |
| 259 | +3. **Session Teardown**: Always ensure `stop_camera_preview` is dispatched upon stream termination or error to power down the camera sensor and release the hardware session. |
0 commit comments