Skip to content

events.send escapes data but not the event name — control bytes produce invalid JSON on the wire #36

Description

@ACEFGI

Summary

events.send(name, data) escapes data correctly but not name. A Lua app
can publish a message that is not valid JSON, and it goes out on both MQTT and
local UDP multicast.

Root cause

ArduinoJson does not emit \u00XX. Its escape table
(ArduinoJson/Json/EscapeSequence.hpp, v7.4.3) is a fixed private string:

return &"//''\"\"\\\\b\bf\fn\nr\rt\t"[isSerializing ? 4 : 0];

That covers " \ \b \f \n \r \t and nothing else. Bytes below 0x20
outside that set are written raw, which RFC 8259 forbids inside a string.
There is no config knob.

Verified against v7.4.3:

input       s"1\x<01>y<1F><09>/end
serialized  {"s":"s\"1\\x<01>y<1F>\t/end"}      <- 0x01 and 0x1F raw
python json.loads -> JSONDecodeError: Invalid control character

ArduinoJson's own parser accepts what it emits, so a device-to-device round trip
looks fine — this only fails at a strict consumer.

The asymmetry

Within a single published envelope, the two halves take different paths:

Field Writer Control bytes
data writeJsonString(), ResidentSandbox.cpp:1270 correct — emits \u%04x
type doc["type"] = name, ResidentSandbox.cpp:878 (ArduinoJson) raw

writeJsonString is already correct. The event name simply doesn't go through
it.

Reachability

EventsModule::send (ResidentSandbox.cpp:1476) takes the name straight from
Lua with no validation:

const char* name = luaL_checkstring(L, 1);

and hands it to publishEventEx, which does doc["type"] = name. So from any
app:

events.send("\1bad", { ok = true })

publishes invalid JSON to devices/{id}/event and to the room's UDP multicast
group.

Suggested fix

Build the envelope with the existing writeJsonString rather than ArduinoJson,
or validate/escape name before assigning it. Preferring the in-tree writer
also removes the two-writers-one-envelope split.

Worth auditing the other ArduinoJson serialize sites for the same pattern — the
KV store (ResidentStoreModule.h:137) persists Lua-supplied keys and values to
NVS the same way, so a stored blob can be invalid JSON for any external reader.

Not a courier bug

Courier's serializeJson calls just emit whatever document they're handed; it
carries this, it doesn't cause it. No change needed there.

Context

Found while reviewing hawthorn-firmware#132 (session telemetry), which hit the
same ArduinoJson limitation and routed around it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions