streams.md

Streams

Part of the protocol reference: audio-only live streams and the live media state.

Experimental audio-only live streams (on by default since 2026-10-17)

Status (implemented 2026-10-12; default on since 2026-10-17 by design decision after short phone checks, AudioLiveFeatureDefaults.ENABLED; the remaining T-AL* tests stay open): Settings > Features "Audio-only live" turns it off (DEBUG test_audio_live is the benchmark switch). Design and reasons: DECISIONS "Design AL". With the switch off, every byte on air and on the link is unchanged and an audio-only offer or header is never produced. The change is a new stream kind: the layers BMH2/BMH3, BMS1, BME1, BFG1 and the lifecycle records are exactly as for video streams, and a source, a forwarding viewer and a listener authenticate each other as they do for video. What is new is that the source signs every audio record (below), because a viewer-forwarder holds the media key and could otherwise forge audio that appears to come from the source.

Offer marker. In a stream offer (announcement flag STREAM_AVAILABLE, any version, including a forwarder's offer), capabilitySummary bit 6 (0x40, AUDIO_ONLY) means the stream carries audio only. Rules for a reader that understands it:

• The summary is exactly 0x40 (no codec bit, no other bit); any other value with bit 6 set, or 0x40 on anything that is not a stream offer, or 0x40 with a non-zero width, height or FPS, is ignored (StreamOfferKind.MALFORMED).
• Width, height and FPS are 0. The bitrate field is the nominal audio rate in kbps and is display only: a version 5 offer rounds it to 25 kbps units and the compact field floors it, so 16 and 24 kbps are not distinguishable. Relay admission and the relay trickle test use the preset in the authenticated BTL1 header, and an advertised rate of 0 must not skip admission.
• Marker and header must agree. A viewer that selected an offer marked 0x40 and receives a BTV3 header, or an offer without the marker and a BTL1 header, fails the link closed. A forwarder derives its own marker from the authenticated header it received, never from its parent's advertisement.
• Penalties are scoped to the attempted route (review 17). An offer is unauthenticated, so a forged mismatch must not poison an honest stream. The failure is recorded against the exact advertising route (advertising address, announced prefix, route token) for 2 minutes, in a table bounded to 64 entries (oldest dropped). No node ID is penalised, even when the mismatching header arrived over an authenticated link (corrected 2026-10-20, Design QV review 1, R1; the app has always recorded the route only). The marker that disagreed is in the unauthenticated advert, while the header is authenticated: a phone that advertises an honest source's stream (copying its ID and token, or forwarding it) with a false marker would otherwise get that honest source's node refused for 10 minutes by every viewer it fooled. The route penalty costs only the route that advertised the false marker. The logical stream (prefix or token alone) and the source's other routes are never penalised.
• The offer's route token, route cost, capacity generation and SEQUENCE_SOURCE_FULL bit, flags (paused, transport) and rotating-ID rules are unchanged. A forwarder advertises the same marker as its source. A stream offer and its relays group into one Watch row by the existing LogicalStreamTarget (token plus summary), so the marker also keeps an audio stream apart from a video stream with a coincident token.
• Compact (iPhone) stream offers: codec mask 0 together with resolution index 3 and FPS index 3 decodes to summary 0x40, and the encoder emits exactly that for summary 0x40 with zero geometry. A compact offer with mask 0 and any other geometry decodes as before (summary 0, listed as unsupported). Vector below.

An older reader finds no overlap between 0x40 and its decoder mask, lists the row dimmed with its "not supported" label (it still counts in the live total, and a tap only shows its codec notice) and cannot join it. It reaches the header below by no path of its own; a connection forced by other means ends at the header.

Stream header BTL1 (BME1 content type 1 STREAM_HEADER, packet flags 0). Fixed part, 11 bytes:

Field	Bytes	Value
Magic BTL1	4	42 54 4c 31
Audio codec	1	1 Opus
Sample rate	2	48000
Channels	1	1
Frame duration (ms)	1	20
Preset	1	1 Low data (16 kbps), 2 Voice (24 kbps), 3 Music (48 kbps); informational and for display
Reserved	1	0

followed by the signer block, three length-prefixed fields (u16 length, 1 to 128 bytes for keys, 1 to 80 for the certificate):

1. the stream key: a fresh per-stream ECDSA P-256 public key (SPKI, 91 bytes) that signs every record;
2. the identity key (SPKI) of the source;
3. the certificate: the identity key's DER ECDSA signature over "BluetoothVideoMesh/AudioLive/Cert/1" 00 || streamId (16) || fixed header (11) || stream key SPKI.

streamId is the mesh stream ID, or sixteen zero bytes for a direct (non-mesh) stream, as for signed labels. A reader checks NodeId.fromPublicKey(identity key) against the source node ID it already authenticates (the direct endpoint peer, or the source node ID in the parent's BFG1 grant), then the certificate, then that both keys are P-256. Any failure fails the stream closed at the header. The identity key signs once per stream (the Android Keystore is off the 5-per-second path); the stream key is software and signs every record. Per-stream pseudonyms (Design F) will replace the identity key in block 2 without a wire change.

Anything else (unknown magic or codec, other rate, channels or frame size, a non-zero reserved byte, a bad length, trailing bytes after the block) fails the stream closed. A reader dispatches on the magic: BTV3 is the video header, BTL1 this one, everything else is rejected with "Unsupported stream header". A reader that does not know BTL1 therefore stops with that error and closes the link: it never builds a video decoder or surface. Exactly one header is allowed per stream; a second one fails closed. In a mesh stream the header travels in the first mesh envelope like the BTV3 header, and a relay caches it with the signed codec configuration for a joining child.

Signed media records. After the header only audio media records are valid (BME1 type 2, inner BTV3 packet framing: length, flags, one presentation timestamp, payload), and every payload is

seq (u32, big endian) || body || signature (64 bytes, r || s)

where body is exactly what an unsigned audio record carries: the Opus configuration (flags AUDIO | CODEC_CONFIG; the existing OpusHead handling, repeated unchanged for a joining child by the relay bootstrap list, at most 4 times per stream), one Opus access unit (flags AUDIO), a BTA1 pair or a BTA2 batch (flags AUDIO | AUDIO_PAIR; 3 to 20 units, 200 ms by default). The signature is by the stream key over "BluetoothVideoMesh/AudioLive/Rec/1" 00 || streamId (16) || flags (1) || presentation timestamp (8) || seq (4) || body, so a record cannot move to another stream, change its flags or time, or be edited. The source assigns seq once per record when it signs (shared by every child; the cost is one signature per record, not per listener): 0, 1, 2, and so on, the configuration included. A reader requires, for every non-configuration record, a strictly greater seq than the last (gaps are legitimate after age shedding; a replayed, reordered or duplicated record fails). The cost is 68 bytes per record (about 2.7 kbps at 200 ms). A viewer-forwarder, which holds the media key and therefore can seal anything, cannot forge, edit, reorder or replay audio, because it does not hold the stream key. It can only withhold or delay. A relay queues a record (and the header) for its children only after its own reader verified it (the header's certificate, the record's signature and sequence), so a record that fails there is never forwarded; before 2026-10-27 the Android forwarder queued it first and only the listeners caught the forgery. What it can still do is present a different source identity: the identity key must hash to the source node ID that the listener authenticates from the parent's BFG1 grant, and a malicious forwarder can state its own (the existing "trusts the forwarder" limit in "Identity verification"; verified contacts still compare the node ID and refuse a mismatch). The signature therefore turns "a relay can alter the source's audio" into "a relay can only present itself as the source", which a verified contact detects.

After verification the reader hands the unsigned body to the existing pipeline unchanged. A video packet, a display-rotation packet, a debug-integrity flag, a keyframe, discontinuity or discardable flag, a batched configuration or a second configuration with different content fails the stream closed. The outer mesh flags carry AUDIO (and CODEC_CONFIG for the configuration), never KEY_FRAME; a relay child queue therefore never waits for a join keyframe, and an audio-only receiver or relay never sends a keyframe request (type 5). The lifecycle records (LIVE, PAUSED, ENDED), the terminal acknowledgement, route withdrawal, heartbeats and hop-local reports are unchanged. PAUSED stops media and keeps the link, as for video.

Opus packet rules (review 14). RFC 6716 allows up to 120 ms and several frames per packet, so a byte limit does not enforce this profile. Each access unit must be exactly one 20 ms mono frame, checked from its first byte before any decoder sees it: table-of-contents code 0 (one frame), stereo bit clear, and a configuration number whose frame duration is 20 ms (SILK 1, 5, 9; hybrid 13, 15; CELT 19, 23, 27, 31); 1 to 1275 bytes. The units of a record are contiguous (each exactly 20,000 us after the previous), a record carries 1 to 20 units, and the configuration must be an OpusHead (version 1, one channel, mapping family 0). Any other packet ends the stream.

Ingress budgets (review 13). Applied by the receiving reader before allocation or decoding, to each audio-only stream: the link reader refuses a record whose ciphertext exceeds 4 KiB until the header is read and 28 KiB afterwards (the largest valid record is 25.7 KB); the decrypted record is checked against the largest valid payload before it is parsed; records are metered at 60 per second with a burst of 600; payload bytes at 16,000 per second with a burst of 512 KiB; lifecycle records at 4 per second with a burst of 40; and signatures are only verified for records inside the budget. Exceeding any budget ends the stream. The decoded-sample bound follows from the 20 ms unit rule and the burst allowance below.

Clock rules (review 12). Three separate checks replace the earlier flat "1 s step, 2x rate" rule, which valid shedding and pause or resume would have violated:

• Gap. The step from the end of the previous record (its last unit plus 20 ms) to the start of the next must be at least 0 and at most 100 ms plus 400 ms for every record the sequence numbers say were dropped (a shed leaves a sequence gap), and never more than 60 s. A timestamp jump with consecutive sequence numbers is therefore rejected, and a shed of any size a real hop produces is accepted. Overlap is rejected.
• Resume. Any lifecycle record (PAUSED, LIVE, ENDED) restarts the baseline, so a source that was paused for minutes may resume at a later timestamp. Replay protection (seq) is not reset.
• Burst. The media position may lead the arrival clock by at most 32 s, measured against the slowest arrival offset seen since the baseline: a joining or recovering listener legitimately receives a backlog (at most 30 s, the withdrawal limit) faster than real time, and a source whose timestamps run ahead of its sending ends the stream. This is a cumulative allowance, not a per-second rate.

Age shedding (hop-local, optional, on in the feature). A sending hop may drop its oldest queued audio records whole when the oldest is older than 3 s, keeping the newest 1.5 s. Candidates are exactly the media records, decided by two separate exact-match predicates, because the namespaces differ (inner bit 4 is the debug-integrity flag, mesh bit 4 is "control"; review 18): a source sheds plaintext records whose inner flags equal AUDIO or AUDIO | AUDIO_PAIR; a relay sheds sealed envelopes of an audio-only stream whose mesh flags equal AUDIO (8) or AUDIO | 64 (72, the mirrored pair bit). Nothing else is ever shed: the header, the Opus configuration, lifecycle, control and terminal records never match, and neither predicate uses a mixed mask. A source signs each record once when it is batched; each child's queue then holds signed records, and shedding drops whole records from that queue and leaves a gap in seq that readers accept. A relay sheds sealed child-queue entries, which leaves a gap in the mesh, endpoint and seq sequences that readers accept. Opus is predictive, so the first frames after a gap may carry a short artefact that concealment (native decoder) or silence realignment (MediaCodec) limits. Each drop is counted and logged. With test_audio_live_age_bound=false nothing is shed.

Age guarantee. The 3 s and 1.5 s limits are local queue residence at one sending hop. They do not bound the age of a record at the listener: delay accumulates over hops and transport buffers, and each local queue can stay under its limit. The end-to-end age is a measurement (T-AL3 reports it), not a guarantee; a cumulative budget would need a clock-uncertainty model across phones and is not part of this version.

Health and withdrawal (hop-local). Because shedding keeps a child's backlog between 1.5 and 3 s, the video thresholds of the relay child health policy would never fire. For audio-only streams a relay or source withdraws a child (existing route-withdrawal record, so it reroutes) when more than 20% of the records queued for it were shed over 10 s, or, in the arm without shedding and also for a source, when its oldest queued record is older than 30 s. The byte limit of the sender queue stays as the last resort.

Mute (review 16). Mute gates capture atomically, discards every captured sample and every unsent partial batch (nothing recorded before the tap is transmitted afterwards), then sends silence in the same cadence, signed like any record. Audio already transmitted may still play at listeners; the panel says so.

Security. Layers unchanged and shared with video streams: authenticated link and endpoint sessions, the BFG1 grant that binds the source node ID and stream ID to the media key for a forwarded stream, ordered sealed records copied unchanged by a forwarding viewer, and the same checks that the original source equals the discovered source. The BTL1 header and all audio are inside BME1, so they are confidential on each hop; the end-to-end authenticity of audio comes from the source's signatures, not from BME1 (every viewer-forwarder holds the media key). What the stream kind adds: the offer marker is an unauthenticated hint like every announcement field (a forged marker only causes a connection that ends at the header or the handshake, and the penalty stays on that route); a stranger can run a stream with any title, so titles from unverified sources are shown as unconfirmed and a verified mark appears only for a full-fingerprint contact; a source can flood only audio records, bounded by the 1275-byte unit limit, the unit and record budgets, the age shedding and the existing memory withdrawal of a viewer that falls tens of seconds behind. Identity exposure. Until streams move to BMH3 with a per-stream pseudonym (Design F), a stream source answers BMH2 and every listener, including a relay, learns its real identity key; a broadcaster's real key is therefore visible to listeners and linkable across rotating discovery IDs. The disclosure text says so.

Vectors (generated by the JVM tests AudioLiveProtocolTest): fixed header Voice 42 54 4c 31 01 bb 80 01 14 02 00, Low data ... 14 01 00, Music ... 14 03 00; the full header is these 11 bytes, then 00 5b and 91 bytes of stream key, 00 5b and 91 bytes of identity key, then the certificate; summary 0x40 audio-only offer; compact vector for prefix 010203040506, GATT, route cost 1, token 0xdeadbeef, sequence 5, 25 kbps: C8D66010-2030-4050-6806-F56DF7782F81; rejected: summaries 0x41, 0x42, 0x60, 0xc0; BTL1 with rate 44100, with reserved 1, with a trailing byte after the signer block, with a zero-length key; a video packet or a keyframe flag after a BTL1 header; a 1276-byte unit; an Opus unit with TOC 0xF9 (two frames), 0xFC (stereo) or 0xF0 (10 ms); a record signed by another key; a replayed seq; a record that moves a timestamp by 5 s with consecutive sequence numbers.

Experimental live media state (off by default)

Status (implemented 2026-10-13 behind the default-off Settings switch "Show Muted and Camera off to viewers", live_media_state; DEBUG test_live_media_state is the benchmark switch; JVM-tested, not yet on phones, phone test T-LM1).

A video broadcaster can mute its microphone (audio stops, video goes on) and turn its camera off (the camera and encoder are released, audio goes on). The Mute and Camera buttons always work on the broadcaster's own phone. With the switch on, each change also sends one authenticated stream-state record (BME1 content type 4 STREAM_STATE, the same record as PAUSED/LIVE/ENDED, one payload byte) so viewers can say "Muted" or "Camera off". Nothing is added to a media packet.

Value	Name	Meaning
1, 2, 3	LIVE, PAUSED, ENDED	Unchanged. LIVE also means "microphone on and camera on".
4	LIVE_MUTED	Live; the microphone is muted.
5	LIVE_CAMERA_OFF	Live; the camera is off (no video frames are produced).
6	LIVE_MUTED_CAMERA_OFF	Live; both.

Rules:

• The record always describes the whole current state (not a delta), so the latest control record a relay keeps for a late joiner (ForwardedStreamSecurity.latestControl) and a source's own replay to a child that connects later (BluetoothVideoBroadcaster.mediaState) are enough. A source sends it in queue order after any change, never while paused; a resume (LIVE) is followed by the record again if anything is still off. PAUSED clears camera off (a pause stops the camera anyway) and keeps the mute.
• A reader treats records 4 to 6 like LIVE for liveness and cadence (a paused viewer resumes) and shows the notice; PAUSED and ENDED clear it. Relays forward the sealed record unchanged. A relay whose parent reports both off does not count the silent link as dead.
• Muted audio is dropped at the microphone encoder's output; the Opus codec configuration still goes out so audio decodes at once after Unmute. The unpaired Opus access unit waiting in the broadcaster is flushed on mute so it never pairs across the gap.
• Compatibility, honestly: an older build rejects an unknown state value as a protocol error and drops the link. There is no capability bit for streams to ask first, so a broadcaster sends records 4 to 6 only with the switch on, and the switch stays off until every phone and relay in a stream's chain has this version. With the switch off viewers just hear silence or see a still picture.
• Audio-only live is unchanged: its mute is local and its listeners get no record.