-
Notifications
You must be signed in to change notification settings - Fork 0
feat: pluggable backend (captured/sunshine/vnc/rdp) + raw WebTransport + selfhosted web viewer #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
aca105f
005e9b5
69e6304
78683fd
c2c7d95
b47bc4b
fc4b9d4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| name: ci | ||
| on: | ||
| push: | ||
| pull_request: | ||
| permissions: | ||
| contents: read | ||
| jobs: | ||
| build: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| persist-credentials: false | ||
| - uses: actions/setup-go@v5 | ||
| with: | ||
| go-version: "1.26" | ||
| cache: true | ||
| - uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: "22" | ||
| cache: npm | ||
| cache-dependency-path: src/web/package-lock.json | ||
| - run: npm ci | ||
| working-directory: src/web | ||
| - run: npm run build | ||
| working-directory: src/web | ||
| - run: go vet ./... | ||
| - run: go build ./... | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,5 +7,7 @@ env/ | |
| # Build directory | ||
| build/ | ||
|
|
||
| # macOS | ||
| .DS_STORE | ||
| # macOS | ||
| .DS_Store | ||
| # web node modules | ||
| /src/web/node_modules/ | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -17,8 +17,8 @@ Reads BGRA from captured, GPU-encodes via ffmpeg, publishes over WebTransport/QU | |
|
|
||
| | Port | Transport | Purpose | | ||
| |------|-----------|---------| | ||
| | 52020 | UDP | WebTransport — `/wt` for JSON control, `/moq` for MoQ media (same QUIC conn) | | ||
| | 52022 | TCP | Web UI (plain HTTP, fingerprint display) | | ||
| | 52020 | UDP | WebTransport — `/wt` for JSON control + uni-stream media (QUIC over UDP) | | ||
| | 52022 | TCP | Web UI (HTTPS, fingerprint display) | | ||
|
|
||
| ### Protocol | ||
|
|
||
|
|
@@ -37,14 +37,22 @@ JSON control messages (bidirectional stream): | |
| {"type":"fingerprint-refresh","algorithm":"sha-256","fingerprint":"<hex>"} // sent on connect + cert rotation | ||
| ``` | ||
|
|
||
| Video: 64KB chunks of H.264/H.265/AV1 Annex B byte stream over the unidirectional stream. | ||
| `start` also accepts optional `codec` and `bitrate`. | ||
|
|
||
| **Media over QUIC (MoQ)**: `https://<server>:52020/moq` — separate WebTransport session using `@moq/lite`. | ||
| - gomoqt `WebTransportHandler` with `UpgradeFunc` wrapping via `okdaichi/webtransport-go` | ||
| - `PublishFunc("/video", ...)` registers each subscriber's `TrackWriter` | ||
| - Each ffmpeg chunk → one MoQ group → one frame | ||
| - Old uni-stream model kept for backwards compat; MoQ runs alongside it. | ||
| - gomoqt v0.15.0, falls back to IETF/moql mode (no ALPN h3/moq). | ||
| On connect the agent pushes `fingerprint-refresh` (only when it manages its own | ||
| cert) followed by an **unsolicited `displays`**, before the client asks for | ||
| anything. Clients must tolerate `displays` arriving unprompted. | ||
|
|
||
| Unrecognized message types are answered with | ||
| `{"type":"error","message":"unknown type: <t>"}`. There is currently no `input` | ||
| or `ping` handler. | ||
|
|
||
| ### Origin checks | ||
|
|
||
| WT upgrades accept an empty `Origin`, an `Origin` matching the request `Host`, | ||
| and the agent's own `https://<host>:<webPort>`. A viewer served from any other | ||
| origin (reverse proxy, separate web deployment) is rejected unless that origin is | ||
| passed with `--allow-origin`. | ||
|
|
||
| ### Cert system | ||
|
|
||
|
|
@@ -67,17 +75,7 @@ await transport.ready; | |
| const stream = await transport.createBidirectionalStream(); | ||
| ``` | ||
|
|
||
| MoQ video connection: | ||
| ```js | ||
| const moqTransport = new WebTransport(`https://${ip}:52020/moq`, { | ||
| serverCertificateHashes: [{ | ||
| algorithm: "sha-256", | ||
| value: new Uint8Array(fingerprintBytes) | ||
| }] | ||
| }); | ||
| await moqTransport.ready; | ||
| // Use @moq/lite to subscribe to "/video" | ||
| ``` | ||
| **Note**: MoQ integration has been removed. The agent now publishes video exclusively over WebTransport unidirectional streams (raw H.264 Annex B). | ||
|
|
||
| ### Web UI | ||
|
|
||
|
|
@@ -95,17 +93,26 @@ Embedded HTML at `http://<server>:52022/` showing: | |
| | `--fingerprint` | Print SHA-256 fingerprint and exit | | ||
| | `--cert cert.pem` | Custom TLS certificate (ECDSA P-256 PEM) | | ||
| | `--key key.pem` | Custom TLS private key (ECDSA P-256 PEM) | | ||
| | `--backend auto\|captured\|sunshine\|vnc\|rdp` | Video backend (auto probes in order captured → sunshine → vnc → rdp) | | ||
| | `--captured "source=...,device=..."` | captured backend opts (source/device for Spike B pipelines) | | ||
| | `--sunshine "addr=host:47989"` | Sunshine/Moonlight host address | | ||
| | `--vnc "addr=host:5901"` | VNC server address | | ||
| | `--rdp "addr=host:3389"` | RDP server address | | ||
| | `--allow-origin <origin>` | Additional allowed browser `Origin` for WT upgrades (repeatable; `*` allows any). Needed when the viewer is hosted somewhere other than the agent's own `:52022`, e.g. behind a reverse proxy. | | ||
| | `--dry-run` | List displays via the selected backend and exit | | ||
|
|
||
| ### Architecture | ||
|
|
||
| ``` | ||
| captured (Unix sockets) | ||
| └─ raw BGRA frames → agent | ||
| ├─ ffmpeg (GPU encode via VideoToolbox/NVENC/AMF/QSV/VAAPI/libx264) | ||
| │ └─ H.264/H.265/AV1 Annex B byte stream → stdout | ||
| └─ publishStream goroutine | ||
| ├─ writes 64KB chunks to each subscriber's unidirectional stream | ||
| └─ writes each chunk as a MoQ frame/group to each MoQ TrackWriter | ||
| ```text | ||
| backend.Backend { ListDisplays(ctx) ([]Display,error); StartStream(ctx, StartRequest) (Stream,error); Stream.Chunks() <-chan H264Chunk } (src/backend/backend.go) | ||
| ├─ captured — unix-socket daemon + ffmpeg encode (src/backend/captured.go) | ||
| ├─ sunshine — Moonlight RTSP passthrough H264 (STUB, src/backend/sunshine.go) | ||
| ├─ vnc — RFB frame polling (STUB, src/backend/vnc.go) | ||
| └─ rdp — MS-RDPBCGR (STUB, src/backend/rdp.go) | ||
|
|
||
| activeBackend (chosen via --backend at startup): | ||
| └─ StartStream → Stream.Chunks() channel | ||
| └─ publishStream goroutine writes each chunk to every subscriber's WT uni stream | ||
| ``` | ||
|
|
||
| ### Start sequence | ||
|
|
@@ -138,19 +145,21 @@ captured (Unix sockets) | |
| 4. Client caches additional fingerprint | ||
| 5. On next connection, includes both old and new hashes in `serverCertificateHashes` | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win 🔎 Supported by static analysis🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- applicable repository conventions ---'
head -5 /tmp/coderabbit-repo-knowledge/distancedesktop-agent-370d1915/*/*.md 2>/dev/null || true
printf '%s\n' '--- AGENTS.md relevant sections ---'
sed -n '130,165p' AGENTS.md
printf '%s\n' '--- transport definitions and certificate hash usage ---'
rg -n -A35 -B15 'connect|fingerprintHex|serverCertificateHashes|certificate' src/web/src/transport.ts
printf '%s\n' '--- viewer-side callers and certificate rotation flow ---'
rg -n -A25 -B15 'fingerprintHex|serverCertificateHashes|certificateHashes|certificate|rotation|old.*hash|new.*hash' src/web AGENTS.mdRepository: distancedesktop/agent Length of output: 45146 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- connection payload and recent-host state ---'
rg -n -A35 -B15 'interface Connect|type Connect|ConnectPayload|fingerprint|saveRecent|recent' src/web/src --glob '!transport.ts'
printf '%s\n' '--- certificate rotation and refresh emission ---'
rg -n -A35 -B20 'fingerprint-refresh|rotate|rotation|cert manager|certificate' --glob '*.go' --glob '*.ts' --glob '*.md' . | head -300
printf '%s\n' '--- repository conventions for the inspected scopes ---'
cat /tmp/coderabbit-repo-knowledge/distancedesktop-agent-370d1915/conventions/repo-wide.mdRepository: distancedesktop/agent Length of output: 45297 Preserve both certificate fingerprints across reconnects. 🤖 Prompt for AI Agents |
||
|
|
||
| ## MoQ integration | ||
| **Caveat:** the rotation broadcast only reaches sessions subscribed to a *live* | ||
| stream — `broadcastControlMsg` returns early when no stream is active, and | ||
| subscribers are only registered when a stream exists at connect time. A | ||
| connected-but-idle client is not notified. The connect-time push in | ||
| `handleSession` is unconditional. | ||
|
Comment on lines
+148
to
+152
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win Resolve the conflicting rotation audience. Line 62 says rotation notifications reach all connected clients, but this caveat says idle clients are not notified. Update one statement so the fingerprint-refresh contract is unambiguous. 🤖 Prompt for AI Agents |
||
|
|
||
| - `/moq` on same UDP port as `/wt` — separate WebTransport session using gomoqt. | ||
| - Each MoQ subscriber gets a `*moqt.TrackWriter` via `PublishFunc("/video", ...)`. | ||
| - `publishStream` writes each ffmpeg chunk to all `TrackWriter`s (one MoQ group + one frame per chunk). | ||
| - On teardown, `moqBroadcastCancel()` unregisters the publish handler. | ||
| - Server TLS `NextProtos` stays `["h3"]` — client MoQ WebTransport negotiates `h3`, `@moq/net` falls back to IETF/moql mode. | ||
| **With `--cert`/`--key`** there is no cert manager at all: no fingerprint push, | ||
| no rotation loop, and no web UI on `:52022`. That is the reverse-proxy / | ||
| publicly-trusted-cert mode, where clients connect without | ||
| `serverCertificateHashes`. | ||
|
Comment on lines
+154
to
+157
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift 🔎 Supported by static analysis🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- repository conventions ---'
head -5 /tmp/coderabbit-repo-knowledge/distancedesktop-agent-370d1915/*/*.md 2>/dev/null || true
printf '%s\n' '--- AGENTS.md relevant lines ---'
sed -n '140,165p' AGENTS.md
printf '%s\n' '--- transport symbol and nearby implementation ---'
rg -n -A45 -B12 'class Transport|connect\(' src/web/src/transport.ts
printf '%s\n' '--- transport call sites and certificate options ---'
rg -n -A8 -B8 'Transport|serverCertificateHashes|fingerprintHex|--cert|--key' src/web src AGENTS.mdRepository: distancedesktop/agent Length of output: 50379 Make custom-certificate mode compatible with the viewer.
🤖 Prompt for AI Agents |
||
|
|
||
| ## Dependencies | ||
|
|
||
| - `github.com/okdaichi/webtransport-go` — WebTransport over QUIC/HTTP-3 (fork used by gomoqt) | ||
| - `github.com/okdaichi/webtransport-go` — WebTransport over QUIC/HTTP-3 | ||
| - `github.com/quic-go/quic-go` — QUIC transport layer | ||
| - `github.com/qumo-dev/gomoqt` — Media over QUIC (MoQ) transport | ||
|
|
||
| ## Build | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,121 @@ | ||
| // Package backend defines the pluggable capture/stream backend interface | ||
| // for the distance agent. | ||
| // | ||
| // A Backend lists displays and produces an H.264 Annex B chunk stream that | ||
| // the WebTransport layer fans out to subscribers. Concrete backends live in | ||
| // this package (captured, sunshine, vnc, rdp) and self-register via init(). | ||
| package backend | ||
|
|
||
| import ( | ||
| "context" | ||
| "fmt" | ||
| "sort" | ||
| "sync" | ||
| ) | ||
|
|
||
| // Display describes one capturable output. | ||
| type Display struct { | ||
| ID uint32 `json:"id"` | ||
| Width int `json:"width"` | ||
| Height int `json:"height"` | ||
| X int `json:"x"` | ||
| Y int `json:"y"` | ||
| RefreshRate float64 `json:"refresh_rate"` | ||
| } | ||
|
|
||
| // StartRequest parameterizes a stream start. | ||
| type StartRequest struct { | ||
| DisplayID uint32 | ||
| FPS int | ||
| Codec string // h264 | hevc | av1 | vp9 | ||
| Bitrate int // bits/sec, 0 = backend default | ||
| } | ||
|
|
||
| // H264Chunk is a piece of H.264 Annex B data ready for transport. | ||
| type H264Chunk struct { | ||
| Data []byte | ||
| Keyframe bool | ||
| } | ||
|
|
||
| // Stream is a live video stream from a backend. | ||
| type Stream interface { | ||
| // Chunks yields encoded H.264 chunks until the stream ends, then closes. | ||
| Chunks() <-chan H264Chunk | ||
| Width() int | ||
| Height() int | ||
| FPS() int | ||
| Codec() string | ||
| // Close tears down capture + encode; idempotent. | ||
| Close() error | ||
| } | ||
|
|
||
| // Backend is a pluggable video source. | ||
| type Backend interface { | ||
| Name() string | ||
| ListDisplays(ctx context.Context) ([]Display, error) | ||
| StartStream(ctx context.Context, req StartRequest) (Stream, error) | ||
| } | ||
|
|
||
| var ( | ||
| regMu sync.Mutex | ||
| registry = map[string]Backend{} | ||
| AutoOrder = []string{"captured", "sunshine", "vnc", "rdp"} | ||
| ) | ||
|
|
||
| // Register adds a backend to the registry. Later registration of the same | ||
| // name replaces the earlier entry. | ||
| func Register(b Backend) { | ||
| regMu.Lock() | ||
| defer regMu.Unlock() | ||
| registry[b.Name()] = b | ||
| } | ||
|
|
||
| // Get returns the named backend. | ||
| func Get(name string) (Backend, error) { | ||
| regMu.Lock() | ||
| defer regMu.Unlock() | ||
| b, ok := registry[name] | ||
| if !ok { | ||
| return nil, fmt.Errorf("backend: unknown backend %q (available: %v)", name, namesSorted()) | ||
| } | ||
| return b, nil | ||
| } | ||
|
|
||
| // Names returns registered backend names sorted. | ||
| func Names() []string { | ||
| regMu.Lock() | ||
| defer regMu.Unlock() | ||
| return namesSorted() | ||
| } | ||
|
|
||
| // namesSorted returns the sorted list of registered backend names. | ||
| // Must be called with regMu held. | ||
| func namesSorted() []string { | ||
| out := make([]string, 0, len(registry)) | ||
| for n := range registry { | ||
| out = append(out, n) | ||
| } | ||
| sort.Strings(out) | ||
| return out | ||
| } | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
| // AutoCandidate is one AutoOrder step result used by callers implementing | ||
| // `--backend auto`. | ||
| type AutoCandidate struct { | ||
| Name string | ||
| Backend Backend | ||
| } | ||
|
|
||
| // Candidates returns backends in auto-probing order (registration order is | ||
| // irrelevant; AutoOrder wins). | ||
| func Candidates() []AutoCandidate { | ||
| regMu.Lock() | ||
| defer regMu.Unlock() | ||
| var out []AutoCandidate | ||
| for _, name := range AutoOrder { | ||
| if b, ok := registry[name]; ok { | ||
| out = append(out, AutoCandidate{Name: name, Backend: b}) | ||
| } | ||
| } | ||
| return out | ||
| } | ||
Uh oh!
There was an error while loading. Please reload this page.