Repository navigation
Conversation
Redact JSON documents stored as strings, such as the A01 product info data point, which exposed the serial number, Wi-Fi name, IP and MAC. Also redact ssid and iotName. Decode the A01 product info and voice pack data points, which the device reports as JSON strings and always decoded to None, and add the decoded A01 state to device diagnostics as traits.
allenporter
left a comment
There was a problem hiding this comment.
Thank you for this nice improvement.
| redacted[key] = redact_device_uid(value) | ||
| elif isinstance(value, dict): | ||
| else: | ||
| redacted[key] = redact_device_data(value, curr_path) |
There was a problem hiding this comment.
The list case was not dropped: redact_device_data handles lists itself at the top, so the else gave the same result. That was not clear from the diff, so I restored the dict and list branches and added str to the dict branch.
|
|
||
| data = api.as_dict() | ||
|
|
||
| json.dumps(data) |
There was a problem hiding this comment.
No, the assertions below already check each converted type. Removed.
| if not value.lstrip().startswith(("{", "[")): | ||
| return value | ||
| try: | ||
| decoded = json.loads(value) |
There was a problem hiding this comment.
If this function strictly just converted json data to the python type only (or returnd the original value), then redact_device_data in the calling function could just run through the normal routine (e.g. fall through to the dict or list redaction code or be a no-op).
There was a problem hiding this comment.
Done. _decode_json_string now only decodes a JSON object or array and otherwise returns the string unchanged, and the normal routine redacts the result. Two side effects: every JSON string in diagnostics now appears as structured data (for example extra and the schema property fields), which updated the V1, Q10 and device manager snapshots. A string that looks like JSON but does not parse is now kept as is, where before it was fully redacted.
Decode JSON documents stored as strings and redact the result through the normal dict and list handling, instead of redacting and re-encoding them. JSON strings now appear as structured data in diagnostics.
| RoborockDyadDataProtocol.SCHEDULE_TASK: _decode_dyad_schedule, | ||
| RoborockDyadDataProtocol.SND_STATE: lambda val: DyadSndState.from_dict(val), | ||
| RoborockDyadDataProtocol.PRODUCT_INFO: lambda val: DyadProductInfo.from_dict(val), | ||
| RoborockDyadDataProtocol.SND_STATE: lambda val: DyadSndState.from_dict(json.loads(val)), |
There was a problem hiding this comment.
i'm thinking about this more and now thinking this is not right.
In general we already decode values here:
So i think the problem is that this is using the initial status thing.. so if anything it jsut needs to be here
I don't think this is correct when used on the values coming from the backend.
But i'm not sure if i trust the current methods of testing.
There was a problem hiding this comment.
The A01 path does not decode values inside dps: a01_protocol.decode_rpc_response only parses the message envelope. The B01 Q7 code you linked unwraps the B01 response envelope, which B01 sends as a JSON string in a single data point, so it does not apply to A01 data points. The SND_STATE and PRODUCT_INFO converters have passed the raw value to from_dict since they were added, so they always returned None for both the home data snapshot and MQTT.
Over MQTT the F25 Ultra sends 10004 and 10005 as JSON strings, the same way it sends SCHEDULE_TASK (10002), which #1019 decodes with json.loads and tests through an MQTT push. I checked this against a real F25 Ultra: query_values([SND_STATE, PRODUCT_INFO]) returned DyadSndState and DyadProductInfo with this change. If the device sent dicts, json.loads would raise TypeError and both would be None.
I can add a test that sends these data points through an MQTT query response. Or, if you prefer decoding at the boundary, I can decode JSON string values once in decode_rpc_response and _parse_device_status and have the converters take dicts. Which would you prefer?
There was a problem hiding this comment.
I went with decoding at the boundary. decode_rpc_response now decodes data point values that hold a JSON object or array as a string, so query responses, pushes and listeners all get structured values, and _parse_device_status does the same for the home data snapshot. The SND_STATE and PRODUCT_INFO converters are back to plain from_dict, and the SCHEDULE_TASK converter takes the list instead of calling json.loads.
The tests now cover 10004 and 10005 through the snapshot, a push and a query response. I also checked it against a real F25 Ultra: 10002, 10004 and 10005 decode from both the snapshot and MQTT.
Decode data point values that hold a JSON object or array as a string in decode_rpc_response and in the home data status snapshot, instead of in the SND_STATE, PRODUCT_INFO and SCHEDULE_TASK converters. Query responses, pushes and listeners now all get structured values.
Drop the simulator's JSON string query branch, which decode_rpc_response made unreachable, and keep JSON values in the string form the device sends. Make the redact_device_data return type cover lists, type the schedule converter input as wire data, explain why diagnostics has its own JSON decoder, and run the A01 diagnostics test for both Dyad and Zeo.
Summary
A01 devices report some data points as JSON stored in a string.
redact_device_dataonly checked dictionary keys, so DP 10005 (product info) went into diagnostics unredacted, with the serial number, Wi-Fi name, IP and MAC. Home Assistant diagnostics build this section fromdevice.diagnostic_data(), so the values ended up in downloaded diagnostics files.Changes
redact_device_datanow decodes JSON objects and arrays stored as strings and redacts the result like any other dict or list. These values now appear as structured data in diagnostics, for exampleextraand the schemapropertyfields, which is why the V1, Q10 and device manager snapshots changed. Other strings, including invalid JSON, are kept unchanged.ssidandiotNameare now redacted.iotNameholds the same user-given room name as the already redactedrawName.SND_STATE(10004) andPRODUCT_INFO(10005) now decode. The device sends them as JSON strings, over MQTT and in the home data snapshot, but the converters passed the string tofrom_dict, so they always returnedNone. A01 data point values that hold a JSON object or array as a string are now decoded once when they arrive, indecode_rpc_responseand_parse_device_status, so the converters get structured values. TheSCHEDULE_TASKconverter no longer callsjson.loadsitself.traitsindiagnostic_data(), as V1 and Q10 devices do.Testing
a194) and a Saros 10R (a144). None of the identifiers remained in the output.This does not depend on #1021 and merges cleanly with it in either order.