Skip to content

Fix E2EE decryption for H.264/H.265 frames with 3-byte start codes - #2144

Merged
1egoman merged 4 commits into
mainfrom
fix/e2ee-annexb-start-codes
Oct 11, 2026
Merged

1egoman merged 4 commits into
mainfrom
fix/e2ee-annexb-start-codes

Conversation

@1egoman

@1egoman 1egoman commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Warning

This pull request was LLM generated and has only been lightly reviewed by a human.

A more thorough review of this needs to occur before it could be merged.

Fixes #2140.

Problem

Some H.264/H.265 encoders, notably Intel Quick Sync on Windows Chrome, emit Annex B frames with 3-byte start codes (00 00 01). WebRTC's packetizer strips start codes and the receiving depacketizer always rebuilds NALUs with 4-byte start codes (00 00 00 01).

For H.264/H.265 the E2EE unencrypted header covers everything up to the first slice NALU (SPS/PPS/SEI plus the slice's own start code) and is authenticated as AES-GCM additionalData. With 3-byte start codes, the header the sender authenticated differs from the header the receiver reconstructs, so every frame from such an encoder fails with OperationError. This affects delta frames too, not only keyframes, because the slice's start code is inside the header.

Fix

Option 1 from the issue: in encryptFrame, when NALU processing applies, rewrite 3-byte start codes in the unencrypted header to 4-byte (normalizeAnnexBStartCodes in naluUtils.ts). The normalized header is used both as additionalData and as the header bytes sent in the frame, so it matches what any receiver's depacketizer produces.

  • Sender-only change, no wire-format change. Frames that already use 4-byte start codes are byte-identical on the wire (the helper returns the input untouched). Existing receivers on any SDK version decrypt frames from fixed senders.
  • Both start-code lengths are valid Annex B, so decoders are unaffected.
  • Inside a NALU, emulation prevention guarantees that 00 00 01 can only be a start code, so the rewrite is safe.

Tests

  • naluUtils.test.ts: exact input/output byte sequences for all-3-byte, mixed 3/4-byte, already-4-byte (same instance returned) and extra-leading-zero cases.
  • FrameCryptor.test.ts: encrypts H.264 keyframes and delta frames and asserts the exact unencrypted header bytes on the wire use 4-byte start codes, then decrypts that output. Includes a 4-byte case that checks the header stays unchanged. The two 3-byte tests fail without the fix.

🤖 Generated with Claude Code

Some H.264/H.265 encoders (e.g. Intel Quick Sync on Windows) emit 3-byte
start codes (00 00 01). WebRTC's depacketizer always reconstructs NALUs
with 4-byte start codes, so the unencrypted header authenticated as
AES-GCM additionalData on the sender no longer matched on the receiver
and every frame failed to decrypt.

The sender now rewrites 3-byte start codes to 4-byte in the unencrypted
header before encrypting and sends that header. Frames that already use
4-byte start codes are byte-identical on the wire, so receivers need no
change.

Fixes #2140

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 1c9819e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
livekit-client Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

size-limit report 📦

Path Size
dist/livekit-client.esm.mjs 112.58 KB (0%)
dist/livekit-client.umd.js 121.71 KB (0%)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comment thread src/e2ee/worker/naluUtils.ts Outdated
1egoman and others added 2 commits October 12, 2026 09:56
Co-authored-by: lukasIO <mail@lukasseiler.de>
Indent and prettier-format the applied review suggestion, use an indexed
for loop instead of forEach to match the rest of naluUtils.ts, and note
that the zero-filled result array supplies each inserted leading 00.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@1egoman
1egoman merged commit e3ae358 into main Oct 11, 2026
6 checks passed
@1egoman
1egoman deleted the fix/e2ee-annexb-start-codes branch October 11, 2026 21:11
@github-actions github-actions Bot mentioned this pull request Oct 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

E2EE: H.264 from Intel Quick Sync (Windows Chrome) never decrypts: AES-GCM additionalData includes 3-byte start codes that WebRTC rewrites to 4-byte

2 participants