Skip to content

Commit f695bf9

Browse files
committed
docs: Document camera live view WebRTC signaling protocol for V1 vacuums
1 parent 08bc715 commit f695bf9

2 files changed

Lines changed: 266 additions & 0 deletions

File tree

‎docs/CAMERA_PROTOCOL.md‎

Lines changed: 259 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,259 @@
1+
# Camera Live View Protocol (V1)
2+
3+
This document describes the reverse-engineered protocol for camera live streaming, WebRTC signaling, and media transport on camera-equipped Roborock V1 vacuum robots.
4+
5+
## Scope & Evidence
6+
7+
- **Protocol family**: Roborock V1 (`pv=1.0`).
8+
- **Feature flags**: Models supporting camera features report `is_video_monitor_supported` (bit mask `8`, bit 3 of `robotNewFeatures`) in device capabilities (see [V1_DEVICE_FEATURES.md](V1_DEVICE_FEATURES.md)).
9+
- **Tested hardware**: Roborock Qrevo Curv (model `a135`).
10+
- **Observation source**: Captures from the Roborock mobile app session negotiation and standalone aiortc test client verification.
11+
- **Note on model variance**: Roborock firmware and command parameters can vary across models. Unverified fields and behaviors are called out explicitly.
12+
13+
---
14+
15+
## Architecture Overview
16+
17+
The camera subsystem uses standard **Roborock V1 JSON-RPC for signaling** and **WebRTC for media transport**:
18+
19+
- **Signaling channel**: Standard Roborock protocol RPC commands carried in `dps["101"]` / `dps["102"]`, transmitted via the cloud MQTT broker (`rr/m/...`) or local TCP channel.
20+
- **Media transport**: WebRTC PeerConnection with ICE (STUN / TURN) negotiation for low-latency video and audio streaming.
21+
22+
```mermaid
23+
sequenceDiagram
24+
participant App as Client (App / Library)
25+
participant Robot as Vacuum Robot
26+
27+
Note over App,Robot: 1. Authentication & Session Initialization
28+
App->>Robot: check_homesec_password(password=MD5)
29+
Robot-->>App: ['ok']
30+
App->>Robot: switch_video_quality(quality="HD")
31+
Robot-->>App: ['ok']
32+
App->>Robot: start_camera_preview(client_id, quality, password)
33+
Robot-->>App: ['ok']
34+
35+
Note over App,Robot: 2. Ephemeral TURN Server Credentials
36+
App->>Robot: get_turn_server()
37+
Robot-->>App: {url, user, pwd}
38+
39+
Note over App,Robot: 3. WebRTC SDP & ICE Exchange
40+
App->>Robot: send_sdp_to_robot(app_sdp=base64(SDP offer))
41+
Robot-->>App: ['ok']
42+
loop Poll until SDP answer ready
43+
App->>Robot: get_device_sdp()
44+
Robot-->>App: {dev_sdp: "retry"} (pending) or {dev_sdp: base64(SDP answer)}
45+
end
46+
loop Poll remote candidates
47+
App->>Robot: get_device_ice()
48+
Robot-->>App: {dev_ice: [base64(candidate)...]}
49+
end
50+
App->>Robot: send_ice_to_robot(app_ice=base64(candidate))
51+
Robot-->>App: ['ok']
52+
53+
Note over App,Robot: 4. WebRTC Media Stream Established
54+
Robot-->>App: Video Track (H.264) / Optional Audio (Opus)
55+
56+
Note over App,Robot: 5. Teardown
57+
App->>Robot: stop_camera_preview()
58+
Robot-->>App: ['ok']
59+
```
60+
61+
---
62+
63+
## Signaling Transport
64+
65+
Camera commands are standard Roborock V1 RPC messages. They follow the library's transport structure documented in [DEVICES.md](DEVICES.md).
66+
67+
### DPS Envelope Framing
68+
69+
In V1 protocol messages:
70+
- **Requests**: The JSON-RPC request object is serialized to a JSON string and placed under `dps["101"]`.
71+
- **Responses**: The device replies with a JSON-serialized result string under `dps["102"]`.
72+
- **Transport framing**: Over MQTT, outer message protocol header is `101` and requests include a `security` block with nonce and endpoint. Over local TCP (port 58867), outer protocol is `4` and `security` is omitted.
73+
74+
#### Complete DPS Request Example:
75+
```json
76+
{
77+
"dps": {
78+
"101": "{\"id\":12345,\"method\":\"start_camera_preview\",\"params\":{\"client_id\":\"a1b2c3d4\",\"quality\":\"HD\",\"password\":\"81dc9bdb52d04dc20036dbd8313ed055\"},\"security\":{\"endpoint\":\"...\",\"nonce\":\"...\"}}"
79+
},
80+
"t": 1760000000
81+
}
82+
```
83+
84+
#### Complete DPS Response Example:
85+
```json
86+
{
87+
"dps": {
88+
"102": "{\"id\":12345,\"result\":[\"ok\"]}"
89+
}
90+
}
91+
```
92+
93+
### MQTT Topics
94+
95+
When signaling over the Roborock Cloud MQTT broker:
96+
97+
```text
98+
Publish: rr/m/i/{rriot_u}/{mqtt_username}/{duid}
99+
Subscribe: rr/m/o/{rriot_u}/{mqtt_username}/{duid}
100+
```
101+
102+
Where:
103+
- `rriot_u`: User identifier from Roborock account credentials.
104+
- `duid`: Unique device identifier of the vacuum.
105+
- `mqtt_username`: Hashed MQTT username slice: `hashlib.md5(f"{rriot_u}:{rriot_k}".encode()).hexdigest()[2:10]`.
106+
- Broker hostname, port, and TLS configuration are obtained directly from `rriot.r.m` (see `create_mqtt_params` in `roborock/protocol.py`).
107+
108+
> [!NOTE]
109+
> `{mqtt_username}` in the topic is distinct from the `client_id` parameter passed to `start_camera_preview`. While `{mqtt_username}` authenticates the MQTT transport channel, `client_id` identifies the individual camera streaming consumer session on the robot. Clients typically pass their hashed MQTT username (or device identifier / UUID) as `client_id`; whether the robot enforces specific format constraints on this identifier is unverified.
110+
111+
---
112+
113+
## Authentication & Password Hashing
114+
115+
Camera viewing is protected by a separate PIN / pattern lock configured in the mobile app:
116+
117+
1. **Pattern conversion**: The 3x3 pattern lock points are represented as a sequence of numeric digits (e.g. `"1234"`).
118+
2. **Hash derivation**: The password parameter is the lowercase hexadecimal MD5 digest of the UTF-8 encoded digits:
119+
```python
120+
import hashlib
121+
122+
password_hash = hashlib.md5("1234".encode("utf-8")).hexdigest()
123+
# Result: "81dc9bdb52d04dc20036dbd8313ed055"
124+
```
125+
3. Both `check_homesec_password` and `start_camera_preview` expect this MD5 hash.
126+
4. An incorrect password returns an error response (e.g. `{"code": -1, "message": "password error"}`). Multiple consecutive invalid attempts may trigger a temporary security lockout on the robot.
127+
128+
---
129+
130+
## Commands Reference
131+
132+
See also [V1_API_COMMANDS.md](V1_API_COMMANDS.md) for general V1 device RPC references.
133+
134+
> [!NOTE]
135+
> Device RPC responses in the tables below may be returned either as a single-element list (e.g. `[{"dev_sdp": "..."}]`) or as a bare dictionary (`{"dev_sdp": "..."}`), depending on firmware version. Parsers should unwrap single-element lists transparently.
136+
137+
### Camera Session Control
138+
139+
| Command | Parameters | Response Result | Description |
140+
|---------|------------|-----------------|-------------|
141+
| `check_homesec_password` | `{"password": "<md5_hash>"}` | `["ok"]` | Validates the camera security PIN/pattern hash. |
142+
| `start_camera_preview` | `{"client_id": "...", "quality": "HD", "password": "<md5_hash>"}` | `["ok"]` | Activates camera preview mode on the robot. |
143+
| `stop_camera_preview` | `[]` | `["ok"]` | Deactivates camera preview and powers down the camera sensor. |
144+
| `switch_video_quality` | `{"quality": "HD"}` or `{"quality": "SD"}` | `["ok"]` | Switches streaming resolution (`HD` or `SD`). |
145+
146+
### WebRTC Signaling
147+
148+
| Command | Parameters | Response Result | Description |
149+
|---------|------------|-----------------|-------------|
150+
| `get_turn_server` | `[]` | `[{"url": "turn:...", "user": "...", "pwd": "..."}]` | Retrieves ephemeral TURN relay credentials from the robot. |
151+
| `send_sdp_to_robot` | `{"app_sdp": "<base64_json>"}` | `["ok"]` | Sends the client's local SDP offer to the robot. |
152+
| `get_device_sdp` | `[]` | `[{"dev_sdp": "<base64_json>"}]` or `[{"dev_sdp": "retry"}]` | Polls for the robot's SDP answer. |
153+
| `send_ice_to_robot` | `{"app_ice": "<base64_json>"}` | `["ok"]` | Forwards a client ICE candidate to the robot. |
154+
| `get_device_ice` | `[]` | `[{"dev_ice": ["<base64_json>", ...]}]` | Polls for remote ICE candidates generated by the robot. |
155+
156+
### Voice Intercom (Two-Way Audio)
157+
158+
| Command | Parameters | Response Result | Description |
159+
|---------|------------|-----------------|-------------|
160+
| `start_voice_chat` | `[]` | `["ok"]` | Unblocks transmission of microphone audio packets from the robot. |
161+
| `stop_voice_chat` | `[]` | `["ok"]` | Mutes/stops voice chat audio streaming. |
162+
163+
> [!NOTE]
164+
> Even if audio transceivers are negotiated in the WebRTC SDP offer, the robot does not transmit audio media from its onboard microphone until `start_voice_chat` is called.
165+
166+
### Remote Driving During Live View
167+
168+
| Command | Parameters | Response Result | Description |
169+
|---------|------------|-----------------|-------------|
170+
| `app_rc_start` | `[]` | `["ok"]` | Initiates manual remote control session. |
171+
| `app_rc_move` | `[{"velocity": 0.2, "omega": 0.0, "duration": 500, "seqnum": 1}]` | `["ok"]` | Sends motion vector (linear speed, angular speed, duration ms). |
172+
| `app_rc_end` | `[]` | `["ok"]` | Exits manual remote control session. |
173+
174+
---
175+
176+
## SDP & ICE Payload Serialization
177+
178+
All SDP descriptions and ICE candidate structures exchanged in signaling are packaged as base64-encoded UTF-8 JSON strings.
179+
180+
### SDP Offer Packaging
181+
182+
The client creates a standard WebRTC SDP offer string, wraps it in JSON with `"sdp"` and `"type"` keys, and base64-encodes it:
183+
184+
```python
185+
import base64
186+
import json
187+
188+
sdp_obj = {
189+
"sdp": peer_connection.localDescription.sdp,
190+
"type": "offer",
191+
}
192+
app_sdp = base64.b64encode(json.dumps(sdp_obj).encode("utf-8")).decode("ascii")
193+
# Parameter payload sent: {"app_sdp": app_sdp}
194+
```
195+
196+
### SDP Answer Polling & Parsing
197+
198+
The client polls `get_device_sdp` until the answer is available:
199+
- While pending, the robot returns `{"dev_sdp": "retry"}`.
200+
- When ready, `dev_sdp` contains a base64 string decoding to:
201+
```json
202+
{
203+
"sdp": "v=0\r\no=- ...\r\ns=-\r\n...",
204+
"type": "answer"
205+
}
206+
```
207+
- Polling should be bounded (e.g. 20–30 attempts with 200–300 ms delays).
208+
209+
### ICE Candidate Packaging
210+
211+
When using aiortc with standard vanilla ICE, local candidates are embedded directly within the initial SDP offer, making separate calls to `send_ice_to_robot` optional. Clients supporting Trickle ICE (such as the official mobile app) send individual candidates dynamically via `send_ice_to_robot`:
212+
213+
```python
214+
# In aiortc: candidate_str = f"candidate:{candidate_to_sdp(candidate)}"
215+
ice_obj = {
216+
"candidate": candidate_str,
217+
"sdpMid": candidate.sdpMid,
218+
"sdpMLineIndex": candidate.sdpMLineIndex,
219+
}
220+
app_ice = base64.b64encode(json.dumps(ice_obj).encode("utf-8")).decode("ascii")
221+
# Parameter payload sent: {"app_ice": app_ice}
222+
```
223+
224+
Remote candidates received from `get_device_ice`:
225+
- Returns a list under `"dev_ice"`, where each element is a base64-encoded JSON string matching the above candidate structure.
226+
- Candidates should be deduplicated by foundation/priority before injecting into the local peer connection.
227+
228+
### TURN Server Mapping
229+
230+
`get_turn_server` returns credentials mapped to `RTCIceServer`:
231+
- `url` → `urls` (e.g. `"turn:turn-us.roborock.com:3478"`)
232+
- `user` → `username`
233+
- `pwd` → `credential`
234+
235+
---
236+
237+
## Media Transport & Codec Specifications
238+
239+
### Video
240+
- **Codec**: H.264 (AVC) baseline / constrained baseline profile.
241+
- **Resolution**: Dynamically switched between 1080p/720p (`HD`) and 480p/360p (`SD`).
242+
- **Framerate**: Approximately 15–25 FPS depending on link quality.
243+
244+
### Audio (Two-Way Intercom)
245+
- **Wire Codec**: Opus (negotiated via SDP offer/answer).
246+
- **Robot Microphone → Client Speaker**:
247+
- Sample rate: 48 kHz.
248+
- Channels: Negotiated as stereo (aiortc decodes to 16-bit interleaved PCM, 960 samples per 20 ms frame). Mono audio can be extracted by taking the left channel (`audio[::2]`).
249+
- **Client Microphone → Robot Speaker**:
250+
- Sample rate: 48 kHz mono Opus audio stream.
251+
- Frame duration: 20 ms frames (960 samples).
252+
253+
---
254+
255+
## Operational & Reliability Considerations
256+
257+
1. **Session Concurrency**: Roborock vacuums permit only a single active camera preview session at a time. Requests while another client (or the official app) is streaming will be rejected.
258+
2. **NAT Traversal**: Because vacuum robots are commonly located behind residential NATs/firewalls, direct peer-to-peer ICE host/srflx candidates frequently fail. The TURN server credentials from `get_turn_server` provide essential relay connectivity.
259+
3. **Session Teardown**: Always ensure `stop_camera_preview` is dispatched upon stream termination or error to power down the camera sensor and release the hardware session.

‎docs/README.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,4 +6,11 @@ This directory contains lower level documentation for developers interested in:
66
- Undocumented or unsupported device commands or RPCs
77
- The internal design of components of the library
88

9+
Available developer documentation:
10+
11+
- [Camera Protocol](CAMERA_PROTOCOL.md) - WebRTC signaling, MQTT topics, and media streaming protocol for vacuum cameras
12+
- [Devices & Discovery](DEVICES.md) - Device management architecture, channels, and protocol versions
13+
- [V1 API Commands](V1_API_COMMANDS.md) - Reference for V1 device RPC commands
14+
- [V1 Device Features](V1_DEVICE_FEATURES.md) - Feature mapping for V1 vacuums
15+
916
For details about using the library see the higher level documentation at https://python-roborock.github.io/python-roborock/

0 commit comments

Comments
 (0)