Aiden Live screen sharing, audio polish, and 0.41.4 - #134
Conversation
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
This PR can leave screen capture running after the Live session has stopped and broadens the default session's permission policy; please address the inline findings before merging.
Reviewed changes
This review covers the four commits in PR #134, including the new gated screen-sharing flow, Live audio routing and cues, orb/Stop interaction changes, session-scoped Computer Use consent, release metadata, and their focused tests.
- Screen sharing — Adds document-bound native-picker authorization, bounded one-frame-per-second JPEG forwarding, resumption buffering, and visible sharing state.
- Live audio — Adds persisted input/output selection, explicit output routing, connection cues, and stable player ownership across capability refresh.
- Interaction and policy — Reworks orb transitions and two-click Stop, and changes Live Computer Use to session-scoped consent while retaining target checks.
- Release and contracts — Publishes
0.41.4and updates the parser, IPC, protocol, diagnostics, and regression contracts.
⚠️ Screen teardown paths are not covered by the current tests
The renderer tests cover successful selection, normal Stop, cancellation, source end, and audio-only start, but not a picker resolving after release or a rejected frame send. Add deterministic coverage for those paths because they decide whether capture and its authorization actually stop.
Technical details
# Add adversarial screen lifecycle coverage
## Affected sites
- `renderer/components/assistant/use-assistant-live.test.tsx:1563-1669` — covers the normal screen lifecycle but not late picker resolution or rejected frame sends.
- `renderer/components/assistant/use-assistant-live.ts:842-949` — contains the teardown and frame-pump paths that need regression coverage.
## Required outcome
- Prove that cancellation or Stop during an in-flight picker stops a late stream, and that a rejected `sendFrame` stops the pump, clears screen state, and releases display authority.GPT Luna ｜ 𝕏
| ); | ||
| electronSession.setPermissionRequestHandler( | ||
| (webContents, permission, callback, details) => { | ||
| if (String(permission) !== "display-capture") { |
There was a problem hiding this comment.
This handler is installed on session.defaultSession, but callback(true) grants every non-display permission rather than preserving the existing policy. Because the guard is never restored, opening the screen picker can change unrelated permissions such as clipboard reads, media, and notifications for the rest of the process.
Technical details
# Keep default-session permissions scoped
## Affected sites
- `main/services/gemini-live/display-media-contract.ts:133-158` — replaces both session permission handlers and grants non-display requests.
- `main/handlers/assistant-live.ts:53-62` — installs the handlers on `session.defaultSession` after a screen binding.
## Required outcome
- Restrict only the Live `display-capture` path without granting unrelated permission types or losing an existing default-session policy; restore or compose handler state when the guard is no longer active.| throw new Error("Screen sharing is unavailable for this window."); | ||
| } | ||
| const stream = await dependencies.getDisplayMedia(); | ||
| if (!mounted.current || sessionRef.current !== null) { |
There was a problem hiding this comment.
releaseScreen and setup dismissal clear screenStreamRef and sessionRef, so a picker that resolves after teardown passes this condition. The continuation then installs lifecycle listeners and keeps a live stream after the caller has dismissed setup; fence each picker operation with cancellation/generation state and stop late results.
Technical details
# Fence late picker results
## Affected sites
- `renderer/components/assistant/use-assistant-live.ts:842-879` — release can race the pending `getDisplayMedia()` continuation.
- `renderer/components/assistant/use-assistant-live.ts:1272-1281` — setup dismissal releases capture without invalidating the picker operation.
## Required outcome
- A picker result from a cancelled or replaced operation must be stopped and must not be installed, labelled, or retained after teardown.| if (frame && isCurrent()) { | ||
| await dependencies.api | ||
| .sendFrame?.(sessionId, frame) | ||
| .catch(() => false); |
There was a problem hiding this comment.
sendFrame returns false when the main service no longer owns an open screen session, and the service can close that session on a transport error without notifying this renderer. Ignoring the result leaves isCurrent() true, so the interval keeps capturing and retrying a dead session; stop the pump and release the stream/authority when a send is rejected.
Technical details
# Stop the frame pump after session rejection
## Affected sites
- `renderer/components/assistant/use-assistant-live.ts:924-945` — discards the boolean result from `sendFrame` and leaves the interval active.
- `main/services/gemini-live/service.ts:539-560` — returns `false` after a stale/closed session and closes the session on transport failure.
## Required outcome
- A rejected frame send must invalidate the renderer-side pump, stop its source, clear screen state, and release display authority so no frames are captured after the provider session is gone.| releaseDisplayMedia(owner: RendererDocumentOwner): void { | ||
| const entry = this.displayBindings.get(documentKey(owner)); | ||
| entry?.dispose(); | ||
| this.displayBindings.delete(documentKey(owner)); |
There was a problem hiding this comment.
releaseDisplayMedia removes whichever binding currently occupies this document key, not the binding that the caller originally acquired. During source replacement, an older asynchronous releaseDisplay can complete after the new bindDisplayMedia and delete the replacement, leaving a live stream with no matching picker authority; make release identity- or generation-fenced.
Technical details
# Fence display-binding release
## Affected sites
- `main/services/gemini-live/service.ts:278-303` — binds by document key and releases by key without checking binding identity.
- `renderer/components/assistant/use-assistant-live.ts:842-879` — permits overlapping release and replacement-picker operations.
## Required outcome
- Releasing an older capture operation must never remove a newer binding for the same document; release the exact binding or generation that issued the request.1 participant
Summary
Validation
npm testnpm run test:gemini-live(175 Live tests + 48 Computer Use tests)npm run type-checknpm run lintnpm run buildnpm run test:brandingnpm run release:check-consumersAcceptance boundary
The screen-sharing code remains behind
AIDEN_EXPERIMENTAL_GEMINI_LIVE_SCREEN=1. Keep that gate off by default until the exact signed packaged build completes the attended native picker/Screen Recording acceptance receipt. Hosted CI, signing, notarization, publication, and that operator acceptance are separate gates.