Skip to content

About

MQTT bridge for DecisionNode, the decision engine: MQTT devices ask questions about text, JSON and camera images and get answers back on a topic

Topics

Resources

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

decision-node-mqtt

An MQTT bridge for DecisionNode. DecisionNode is a decision engine: you ask it a yes-or-no, choice, score, number, point or box question about text, JSON, images or video, and it answers with calibrated probabilities. The bridge lets any MQTT device ask those questions without an SDK of its own: an ESP32, an industrial gateway, a Node-RED flow or a camera that can only publish JPEG bytes.

A device publishes a request on a topic. The bridge sends it to the DecisionNode API and publishes the answer on a response topic.

device ── decisionnode/request/<id> ──▶ broker ──▶ decision-node-mqtt ──▶ DecisionNode API
device ◀── decisionnode/response/<id> ── broker ◀──────────┘

One static binary for Linux (x86-64, arm64, armv7, armv6), macOS and Windows, or a container image for linux/amd64, arm64 and arm/v7.

Install

  • Binaries: every release has an archive for Linux (x86-64, arm64, armv7, armv6), macOS (x86-64, arm64) and Windows (x86-64, arm64), with checksums.txt and a keyless signature of it.
  • Container image: docker pull ghcr.io/bynn-intelligence/decision-node-mqtt:latest, for linux/amd64, linux/arm64 and linux/arm/v7, also tagged with each version.

Verify the checksums' signature with cosign:

cosign verify-blob --certificate checksums.txt.pem --signature checksums.txt.sig \
  --certificate-identity-regexp 'https://github.com/Bynn-Intelligence/decision-node-mqtt/' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com checksums.txt

Quickstart with mosquitto

You need an API key from the DecisionNode console and Docker.

cd examples
DECISIONNODE_API_KEY=dn_live_... docker compose up --build

That starts a mosquitto broker on port 1883 and the bridge beside it. In another terminal, listen for answers and ask a question:

mosquitto_sub -h localhost -t 'decisionnode/response/#' -v &

mosquitto_pub -h localhost -t decisionnode/request/sensor-7 -m '{
  "id": "r-1",
  "state": {"temperature_c": 81, "trend": "rising", "limit_c": 85},
  "questions": {
    "overheating": {"type": "truth", "instructions": "Will this machine overheat within the hour?"}
  }
}'

The answer arrives on decisionnode/response/sensor-7:

{"id":"r-1","request_id":"req_...","decision":{"model":"decisionnode-moe-...","answers":{"overheating":{"type":"truth","truth":0.87}},"usage":{"input_tokens":96,"output_tokens":0,"cost":{"usd":0.000004,"model_usd":0.000004,"boosters_usd":0,"free_tier":false}}}}

Without Docker, run the binary against any broker:

export DECISIONNODE_API_KEY=dn_live_...
export DN_MQTT_URL=mqtt://localhost:1883
decision-node-mqtt -config config.yaml

Topics

  • Requests arrive on the topic filters in mqtt.subscribe, decisionnode/request/+ by default.
  • Answers go to <response_prefix>/<id>, decisionnode/response/<id> by default. <id> is the part of the request topic the subscription's wildcards matched: a request on decisionnode/request/sensor-7 is answered on decisionnode/response/sensor-7.
  • MQTT 5 clients may set the request's response topic and correlation data. The answer then goes to that response topic, with the same correlation data, when the topic matches one of mqtt.response_topics (by default decisionnode/response/#), so a device can never make the bridge publish anywhere else. A response topic the bridge itself subscribes to is never used, so an answer can never come back as a request. Otherwise the answer goes to the default topic, still with the correlation data.
  • MQTT 3.1.1 clients put an id in the JSON request. The answer carries the same id.

The bridge connects to the broker over MQTT 5 (mosquitto 1.6 or later, EMQX, HiveMQ, VerneMQ and others). Devices may use MQTT 3.1.1 or 5.

Requests

A request payload is either a JSON request or the bytes of an image.

JSON. Every field is optional, except that a request needs questions, its own or its topic's profile's.

Field What it is
id Any JSON value, echoed in the answer
state What is judged: a string, or any JSON object or array
questions The questions, by id, as the DecisionNode API takes them: truth, choice, score, number, point and box
images Up to 16 images: {"id": "gate", "url": "https://..."}, or {"id": "gate", "data": "<base64>", "media_type": "image/jpeg"}
model decisionnode-moe (the default) or decisionnode-flash
boosters Specialist readings of the images, added as evidence: names such as ["age"], or objects for boosters with settings, such as {"name": "bynn_segment", "options": {"prompt": "landing pad"}}. See Boosters.
fallback false to wait for the requested model rather than be answered by another
knowledge Options for the request's knowledge booster, such as {"scope": "X200"}: each replaces the option of that name in the profile's booster, or in the payload's own. See Knowledge.
profile The name of a profile to start from, instead of the topic's

An unknown field is refused, so a typo is never silently ignored.

An image. A payload that starts like a JPEG, PNG or WebP file is sent as the image of its topic's profile. A camera that can only publish JPEG bytes is answered with the profile's questions.

Profiles

A profile is a named request that a topic's payloads fill in. An image payload becomes its image; a JSON payload's fields replace its own.

profiles:
  gate:
    state: Camera 2 at the north gate of the loading yard.
    image_id: gate
    questions:
      vehicle:
        type: truth
        instructions: Is a vehicle waiting at the gate?
      people:
        type: number
        instructions: How many people are in the picture?
        min: 0
        max: 20

topics:
  - filter: decisionnode/request/gate-north
    profile: gate
mosquitto_pub -h localhost -t decisionnode/request/gate-north -f snapshot.jpg

Boosters

Boosters are specialist models that read the images (or, for knowledge, your own documents) before the model answers. Their readings come back in the answer's decision.evidence, one per image (one in all for knowledge). Up to 4 a request, each a name or an object {"name", "images", "require", "options", "timeout_ms"}:

Booster Reads Options
age The age of the person in a picture none
ai_generated Whether a picture was made by a generator none
face_detection The faces in a picture: how many, and a box and a confidence per face none
face_recognition The faces in a picture matched against a collection: per match a box, a score and the item's reference, with your label or a public figure's name collection (required: celebrities, or the id of one of your workspace's face collections), threshold 0 to 1 (0.5)
ocr The text a picture shows prompt
bynn_depth (Depth Vision) Distances in meters grid_size
bynn_segment (Segment Vision) Where the things named in prompt are prompt (required), max_instances, min_score, polygon_points, depth
knowledge (Knowledge) The request's state and questions, never a picture: it finds the passages of your own documents that apply collection (required: the id of one of your workspace's knowledge bases), scope, query, top_k, max_tokens, pinned, min_score, match, filter, include_text. See Knowledge.

Robots and drones

Depth Vision and Segment Vision are made for machines that move:

  • Depth Vision (bynn_depth): distances in meters. ahead is the nearest obstacle straight ahead (the floor left out, null when there is none), directions the same in 7 vertical strips from left to right, ground whether a floor was found and left out, and grid a grid of the whole picture (option grid_size, 2 to 16, default 8) with depth_min, depth_max and depth_median.
  • Segment Vision (bynn_segment): finds what its required prompt option names, one phrase or a list of 1 to 4. Each match in instances has its label, score, box, polygon, area, centroid, opening (the hole of a gate, doorway or ring, else null), and with depth on (the default) its depth and size_m in meters. Coordinates are fractions of the picture from its top left; descriptive phrases ("orange H landing pad") match better than one word. Other options: max_instances (1 to 16, default 8), min_score (0 to 1, default 0.25), polygon_points (3 to 64, default 24) and depth (default true).

Their distances and sizes are estimates from one picture, for decisions and planning, never a safety stop: a robot or a drone keeps stereo cameras or LiDAR for that. Each image a booster reads is one booster call.

A profile for a front camera that publishes JPEG frames:

profiles:
  drone:
    state: Front camera of the inspection drone.
    image_id: front
    boosters:
      - bynn_depth
      - name: bynn_segment
        options:
          prompt: [purple gate, orange H landing pad]
    questions:
      clear:
        type: truth
        instructions: Is the path straight ahead clear for at least 2 meters?
      gate:
        type: point
        instructions: The centre of the opening of the purple gate.
        units: normalized

topics:
  - filter: decisionnode/request/drone-front
    profile: drone

Every booster is listed in the boosters documentation.

Knowledge

A knowledge base holds your own documents: operating manuals, site procedures, safety rules. The knowledge booster finds the passages that apply to each request and gives them to the model as facts and rules to apply; rules you pin always apply. The answer cites each passage, so a device or an operator can see which manual and which section a decision rests on.

Create the knowledge base in the console, or with the dn command line tool. A scope key keeps each robot model's manual apart; a document without the key applies to every model:

dn collections create "Operating manuals" --type text --scope-key model
dn documents add col_3f6c1a9e x200-manual.pdf --metadata '{"model":"X200"}' --wait
dn documents add col_3f6c1a9e x100-manual.pdf --metadata '{"model":"X100"}' --wait
dn documents add col_3f6c1a9e safety-rules.md --pinned --wait

A PDF, DOCX, PPTX or XLSX file, HTML, text, Markdown or a picture of a page (PNG, JPEG, TIFF or WebP) is converted to text (up to 50 MB a file, 100 pages a PDF, 512 KiB of text once extracted), and only its text is kept. Documents are added with the console, dn or the SDKs; the bridge only reads them, so a device's Decisions-only key is enough. include_text needs a Full access key.

Name it in a profile. A robot's camera publishes JPEG frames on its model's topic; each model's profile names its scope:

profiles:
  x200:
    state: Front camera of a cleaning robot, model X200.
    image_id: front
    boosters:
      - name: knowledge
        require: true
        options:
          collection: col_3f6c1a9e
          scope: X200
          top_k: 4
    questions:
      stop:
        type: truth
        instructions: Must the robot stop here, by its operating manual?

topics:
  - filter: decisionnode/request/x200/+
    profile: x200

A JSON request can set the scope, or any other option, without repeating the booster:

mosquitto_pub -t decisionnode/request/robot-7 -m '{"profile":"x200","state":"Water on the floor ahead.","knowledge":{"scope":"X200"}}'

The passages come back in decision.evidence.knowledge, exactly as the API sent them, pinned rules first:

"knowledge": {"status": "ok", "results": [{"collection": "col_3f6c1a9e",
  "scope": {"key": "model", "value": "X200"},
  "passages": [
    {"n": 1, "pinned": true, "title": "Safety rules", "heading": "Stopping", "item": "itm_01", "chunk": 0, "byte_start": 0, "byte_end": 412, "score": null},
    {"n": 2, "pinned": false, "title": "X200 operating manual", "heading": "4.2 Wet floors", "item": "itm_02", "chunk": 17, "byte_start": 20480, "byte_end": 21904, "score": 0.8124}
  ], ...}]}

item and chunk name the document and its part; byte_start and byte_end are UTF-8 byte offsets, end exclusive, into the document's stored text, which dn documents get COLLECTION ITEM --text prints.

With require: true a request is not answered without its knowledge: a knowledge base with no document in it is the API's 409 knowledge_collection_empty, with details.items and details.failed; while its first documents are still being processed it is a 529 booster_unavailable: ask again shortly. Without require, the decision is made without the passages and the reading says why.

Answers

An answer is one JSON object:

  • id: the request's id, or null.
  • request_id: the API's request id, to quote to support.
  • decision: the API's response body, exactly as it sent it: answers, model and usage, with evidence when the request named boosters.

A request that could not be answered gets error instead of decision:

{"id":"r-1","error":{"status":null,"error_type":"rate_limited","message":"too many requests on this topic: try again later","request_id":null}}

status, error_type and message are the API's own, exactly as it sent them, with details when the API sent some (such as a knowledge base's items and failed). Where the API was not reached, status is null and error_type is one of the bridge's:

error_type Why
payload_too_large The payload is over limits.max_payload_bytes
rate_limited The request topic is over limits.rate_per_second and limits.burst
overloaded limits.max_in_flight requests are being answered already
invalid_payload The payload is neither a JSON request nor an image, or the JSON is not a request the bridge can send
no_profile An image payload on a topic without a profile, or a profile that does not exist
timeout_error, connection_error The API did not answer in time, or could not be reached, after the retries

Over a limit a request is answered at once with its error and dropped. Nothing is queued without bound.

An ESP32 example

With the Arduino PubSubClient library (MQTT 3.1.1), a device publishes its reading with an id and listens on its own response topic:

#include <WiFi.h>
#include <PubSubClient.h>

WiFiClient net;
PubSubClient mqtt(net);

void onAnswer(char* topic, byte* payload, unsigned int length) {
  // {"id":"r-42","request_id":"req_...","decision":{"answers":{"open":{"type":"truth","truth":0.91}}}}
  Serial.write(payload, length);
  Serial.println();
}

void setup() {
  Serial.begin(115200);
  WiFi.begin("my-network", "my-password");
  while (WiFi.status() != WL_CONNECTED) delay(250);
  mqtt.setServer("192.168.1.20", 1883);
  mqtt.setBufferSize(1024);
  mqtt.setCallback(onAnswer);
}

void loop() {
  if (!mqtt.connected()) {
    mqtt.connect("esp32-door-3");
    mqtt.subscribe("decisionnode/response/esp32-door-3");
  }
  mqtt.loop();

  static unsigned long last = 0;
  if (millis() - last > 60000) {
    last = millis();
    char body[256];
    snprintf(body, sizeof body,
      "{\"id\":\"r-%lu\",\"state\":{\"door_sensor\":%d,\"lux\":%d},"
      "\"questions\":{\"open\":{\"type\":\"truth\",\"instructions\":\"Is the door open?\"}}}",
      last / 1000, digitalRead(4), analogRead(34));
    mqtt.publish("decisionnode/request/esp32-door-3", body);
  }
}

Configuration

Settings come from a YAML file (-config, or DN_MQTT_CONFIG) and the environment, the environment winning. config.example.yaml lists every setting with its default.

Environment variable Setting
DECISIONNODE_API_KEY The API key (else api.key_file)
DECISIONNODE_BASE_URL api.base_url, for a self-hosted server
DECISIONNODE_MODEL api.model
DN_MQTT_URL mqtt.url: mqtt:// or tcp://, mqtts://, ssl:// or tls://, ws:// or wss://
DN_MQTT_CLIENT_ID, DN_MQTT_USERNAME mqtt.client_id, mqtt.username
DN_MQTT_PASSWORD The broker password (else mqtt.password_file)
DN_MQTT_HTTP_LISTEN http.listen

The API key and the broker password come only from the environment or a file. The YAML has no key for either, and an unknown key is an error, so a key pasted into the file is refused rather than used. Neither is ever read from a topic or written to a log.

TLS. An mqtts://, ssl://, tls:// or wss:// URL connects over TLS 1.2 or later, verifying the broker's certificate against the system's roots, or against mqtt.ca_file. mqtt.cert_file and mqtt.key_file add a client certificate. Certificate checks cannot be turned off.

Health and metrics

The bridge serves on http.listen (:8080):

  • /healthz: 200 {"status":"ok"} while it is connected to the broker and subscribed, else 503.
  • /metrics: Prometheus metrics: decisionnode_mqtt_requests_total{result}, decisionnode_mqtt_in_flight, decisionnode_mqtt_decision_seconds, decisionnode_mqtt_connected and decisionnode_mqtt_publish_errors_total, with the Go runtime's.

Logs are JSON lines on stderr: the request topic, the response topic, the result, the time taken and the API's request id. Payloads are never logged.

On SIGTERM or SIGINT the bridge answers the requests it has taken, for up to a minute, then disconnects.

Build from source

Go 1.26 or later:

go build -o decision-node-mqtt ./cmd/decision-node-mqtt
go test ./...

License

MIT. See LICENSE.

About

MQTT bridge for DecisionNode, the decision engine: MQTT devices ask questions about text, JSON and camera images and get answers back on a topic

Topics

Resources

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages