Summary
events.send(name, data) escapes data correctly but not name. A Lua app
can publish a message that is not valid JSON, and it goes out on both MQTT and
local UDP multicast.
Root cause
ArduinoJson does not emit \u00XX. Its escape table
(ArduinoJson/Json/EscapeSequence.hpp, v7.4.3) is a fixed private string:
return &"//''\"\"\\\\b\bf\fn\nr\rt\t"[isSerializing ? 4 : 0];
That covers " \ \b \f \n \r \t and nothing else. Bytes below 0x20
outside that set are written raw, which RFC 8259 forbids inside a string.
There is no config knob.
Verified against v7.4.3:
input s"1\x<01>y<1F><09>/end
serialized {"s":"s\"1\\x<01>y<1F>\t/end"} <- 0x01 and 0x1F raw
python json.loads -> JSONDecodeError: Invalid control character
ArduinoJson's own parser accepts what it emits, so a device-to-device round trip
looks fine — this only fails at a strict consumer.
The asymmetry
Within a single published envelope, the two halves take different paths:
| Field |
Writer |
Control bytes |
data |
writeJsonString(), ResidentSandbox.cpp:1270 |
correct — emits \u%04x |
type |
doc["type"] = name, ResidentSandbox.cpp:878 (ArduinoJson) |
raw |
writeJsonString is already correct. The event name simply doesn't go through
it.
Reachability
EventsModule::send (ResidentSandbox.cpp:1476) takes the name straight from
Lua with no validation:
const char* name = luaL_checkstring(L, 1);
and hands it to publishEventEx, which does doc["type"] = name. So from any
app:
events.send("\1bad", { ok = true })
publishes invalid JSON to devices/{id}/event and to the room's UDP multicast
group.
Suggested fix
Build the envelope with the existing writeJsonString rather than ArduinoJson,
or validate/escape name before assigning it. Preferring the in-tree writer
also removes the two-writers-one-envelope split.
Worth auditing the other ArduinoJson serialize sites for the same pattern — the
KV store (ResidentStoreModule.h:137) persists Lua-supplied keys and values to
NVS the same way, so a stored blob can be invalid JSON for any external reader.
Not a courier bug
Courier's serializeJson calls just emit whatever document they're handed; it
carries this, it doesn't cause it. No change needed there.
Context
Found while reviewing hawthorn-firmware#132 (session telemetry), which hit the
same ArduinoJson limitation and routed around it.
Summary
events.send(name, data)escapesdatacorrectly but notname. A Lua appcan publish a message that is not valid JSON, and it goes out on both MQTT and
local UDP multicast.
Root cause
ArduinoJson does not emit
\u00XX. Its escape table(
ArduinoJson/Json/EscapeSequence.hpp, v7.4.3) is a fixed private string:That covers
"\\b\f\n\r\tand nothing else. Bytes below 0x20outside that set are written raw, which RFC 8259 forbids inside a string.
There is no config knob.
Verified against v7.4.3:
ArduinoJson's own parser accepts what it emits, so a device-to-device round trip
looks fine — this only fails at a strict consumer.
The asymmetry
Within a single published envelope, the two halves take different paths:
datawriteJsonString(),ResidentSandbox.cpp:1270\u%04xtypedoc["type"] = name,ResidentSandbox.cpp:878(ArduinoJson)writeJsonStringis already correct. The event name simply doesn't go throughit.
Reachability
EventsModule::send(ResidentSandbox.cpp:1476) takes the name straight fromLua with no validation:
and hands it to
publishEventEx, which doesdoc["type"] = name. So from anyapp:
publishes invalid JSON to
devices/{id}/eventand to the room's UDP multicastgroup.
Suggested fix
Build the envelope with the existing
writeJsonStringrather than ArduinoJson,or validate/escape
namebefore assigning it. Preferring the in-tree writeralso removes the two-writers-one-envelope split.
Worth auditing the other ArduinoJson serialize sites for the same pattern — the
KV store (
ResidentStoreModule.h:137) persists Lua-supplied keys and values toNVS the same way, so a stored blob can be invalid JSON for any external reader.
Not a courier bug
Courier's
serializeJsoncalls just emit whatever document they're handed; itcarries this, it doesn't cause it. No change needed there.
Context
Found while reviewing hawthorn-firmware#132 (session telemetry), which hit the
same ArduinoJson limitation and routed around it.