Skip to content

Commit f951ea1

Browse files
committed
fix(desktop): reuse Sim login for authenticated browser previews
1 parent 779b8af commit f951ea1

17 files changed

Lines changed: 842 additions & 192 deletions

‎apps/desktop/README.md‎

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,26 @@ const desktop = useDesktop()
156156

157157
Good fits for the bridge: OS notifications + dock badge on workflow completion, global shortcuts, "reveal in Finder", tray, secure OS-keychain storage. Anything that touches the server/DB still goes through normal APIs — the bridge is only for **native** capability. This same bridge is also the robust way to retire the web-app couplings in the table above: have the web app *tell* the shell (`signalLogout()`, `markAuthSurface()`) instead of the shell inferring from URLs.
158158

159+
### Browser authentication and Sim previews
160+
161+
Browser tabs for the exact configured Sim origin share the desktop app's existing
162+
Electron session. This includes dev: authenticated file previews and deployed chat
163+
pages use the current login without copying cookies or exposing tokens to the model.
164+
Normal resource permissions and any separate deployed-chat password still apply.
165+
External websites use the independent `persist:sim-browser-agent` partition.
166+
167+
Crossing between Sim and an external site opens a tab in the destination session and
168+
preserves the source tab's history; an unused blank tab adopts its first destination's
169+
session. A cross-session form POST is blocked instead of replayed as a GET. Restored
170+
tabs and popups select their session from the destination origin. Sign-out and account
171+
or server changes use the existing browser teardown. Browser views retain their own
172+
permission/download policy, SSRF guards, and minimal preload with no `simDesktop` API.
173+
174+
Generated HTML remains inside its `allow-scripts` sandbox. The driver can inspect and
175+
interact with inline frames through isolated Chromium worlds, including out-of-process
176+
frames; it does not grant those pages access to the parent app. These shell changes
177+
require a desktop update, not just a hosted web deployment.
178+
159179
### Local filesystem access
160180

161181
Copilot can inspect user-selected local directories through the ordinary VFS tools. Granted folders appear beneath the top-level `user-local/` namespace, and `glob`, `grep`, and `read` are routed to Electron only when their path/pattern is explicitly scoped there. This capability is:
@@ -188,8 +208,8 @@ Raw local file bytes are never exposed through the preload bridge and cannot be
188208

189209
## Known caveats
190210

191-
- The hosted Sim renderer may request microphone access for voice input from the configured app origin; camera access remains denied. On macOS the shell also requires the operating-system microphone grant. Separately, a page in the isolated agent browser may request microphone or camera only from its main frame after a recent native user gesture; Sim then requires an explicit document-scoped prompt and the operating-system grant where applicable.
192-
- The built-in agent browser is not a general-purpose download manager. Its dedicated partition applies the same bounded policy to every download, including one started by a direct user click: at most 2 GiB per file, two active downloads per task, six app-wide, and a 1 GiB free-disk reserve. A rejected download appears in the browser's downloads menu; use a normal browser for an intentionally larger transfer.
211+
- The hosted Sim renderer may request microphone access for voice input from the configured app origin; camera access remains denied. On macOS the shell also requires the operating-system microphone grant. Separately, a page in the agent browser may request microphone or camera only from its main frame after a recent native user gesture; Sim then requires an explicit document-scoped prompt and the operating-system grant where applicable.
212+
- The built-in agent browser is not a general-purpose download manager. Both browser session types apply the same bounded policy to every download, including one started by a direct user click: at most 2 GiB per file, two active downloads per task, six app-wide, and a 1 GiB free-disk reserve. A rejected download appears in the browser's downloads menu; use a normal browser for an intentionally larger transfer.
193213
- Default Electron ships H.264/AAC/MP3 — do not swap in the codec-free ffmpeg build.
194214
- Third-party web analytics (GTM/GA) are blocked at the network layer by default (`blockThirdPartyAnalytics`); first-party PostHog `/ingest` is untouched.
195215
- `Cmd+F` opens the native find overlay in built-in browser tabs. The hosted Sim workspace continues to use Monaco- and table-specific find surfaces.

‎apps/desktop/e2e/browser-tools.spec.ts‎

Lines changed: 142 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,44 @@ test.describe('browser tools', () => {
7171
response.end()
7272
return
7373
}
74+
if (path === '/enter-sim') {
75+
response.writeHead(302, { Location: `${origin}/private-chat` })
76+
response.end()
77+
return
78+
}
79+
if (path === '/api/auth/get-session') {
80+
response.writeHead(200, { 'Content-Type': 'application/json' })
81+
response.end(
82+
JSON.stringify(
83+
request.headers.cookie?.includes('better-auth.session_token=fixture')
84+
? { user: { id: 'browser-auth-fixture' }, session: { id: 'fixture-session' } }
85+
: null
86+
)
87+
)
88+
return
89+
}
90+
if (path === '/api/auth/sign-out') {
91+
response.writeHead(200, {
92+
'Content-Type': 'application/json',
93+
'Set-Cookie': 'better-auth.session_token=; HttpOnly; SameSite=Lax; Path=/; Max-Age=0',
94+
})
95+
response.end('{}')
96+
return
97+
}
98+
if (path === '/private-chat' || path === '/private-preview') {
99+
if (!request.headers.cookie?.includes('better-auth.session_token=fixture')) {
100+
response.writeHead(302, { Location: '/login' })
101+
response.end()
102+
return
103+
}
104+
response.writeHead(200, { 'Content-Type': 'text/html' })
105+
response.end(
106+
path === '/private-chat'
107+
? '<!doctype html><title>Private deployed chat</title><h1>Authenticated deployed chat</h1><label>Message <input id="message"></label><button onclick="document.getElementById(\'reply\').textContent = document.getElementById(\'message\').value">Send</button><p id="reply"></p>'
108+
: `<!doctype html><title>Private HTML preview</title><h1>Authenticated HTML preview</h1><iframe sandbox="allow-scripts" srcdoc="<button onclick='document.body.dataset.clicked = true'>Test quiz</button><script>try { parent.document.body.dataset.escaped = true } catch { document.body.dataset.isolated = true }</script>"></iframe>`
109+
)
110+
return
111+
}
74112
if (path === '/api/desktop/tool/authorize') {
75113
let body = ''
76114
for await (const chunk of request) body += chunk.toString()
@@ -80,7 +118,12 @@ test.describe('browser tools', () => {
80118
response.end(JSON.stringify(authorization ?? {}))
81119
return
82120
}
83-
response.writeHead(200, { 'Content-Type': 'text/html' })
121+
response.writeHead(200, {
122+
'Content-Type': 'text/html',
123+
...(path === '/workspace' || path === '/home' || path === '/'
124+
? { 'Set-Cookie': 'better-auth.session_token=fixture; HttpOnly; SameSite=Lax; Path=/' }
125+
: {}),
126+
})
84127
response.end(
85128
path === '/click'
86129
? CLICK_FIXTURE
@@ -168,6 +211,104 @@ test.describe('browser tools', () => {
168211
}
169212
}
170213

214+
test('shares desktop authentication for private HTML previews and deployed chats', async () => {
215+
for (const path of ['/private-preview', '/private-chat']) {
216+
const result = await execute('browser_open_url', { url: `${origin}${path}` })
217+
expect(result.ok, result.error).toBe(true)
218+
expect(result.result).toMatchObject({ url: `${origin}${path}` })
219+
const state = await app.evaluate(async ({ webContents, BrowserWindow }, url) => {
220+
const host = BrowserWindow.getAllWindows()[0].webContents
221+
const page = webContents.getAllWebContents().find((contents) => contents.getURL() === url)
222+
if (!page) throw new Error('Missing protected page')
223+
return {
224+
sharedSession: page.session === host.session,
225+
hasDesktopBridge: await page.executeJavaScript(
226+
'typeof window.simDesktop !== "undefined"'
227+
),
228+
heading: await page.executeJavaScript('document.querySelector("h1").textContent'),
229+
escaped: await page.executeJavaScript('document.body.dataset.escaped === "true"'),
230+
}
231+
}, `${origin}${path}`)
232+
expect(state.sharedSession).toBe(true)
233+
expect(state.hasDesktopBridge).toBe(false)
234+
expect(state.heading).toContain('Authenticated')
235+
expect(state.escaped).toBe(false)
236+
if (path === '/private-preview') {
237+
const outline = (result.result as { snapshot: { outline: string } }).snapshot.outline
238+
const button = outline
239+
.split('\n')
240+
.find((line) => line.includes('"Test quiz"') && /\[ref=\d+\]/.test(line))
241+
?.match(/\[ref=(\d+)\]/)?.[1]
242+
expect(button, JSON.stringify(result.result)).toBeTruthy()
243+
const click = await execute('browser_click', { elementId: Number(button) })
244+
expect(click.ok, click.error).toBe(true)
245+
const frameState = await app.evaluate(async ({ webContents }, url) => {
246+
const page = webContents.getAllWebContents().find((contents) => contents.getURL() === url)
247+
const frame = page?.mainFrame.frames.find((frame) => frame.url === 'about:srcdoc')
248+
if (!frame) throw new Error('Missing sandboxed preview')
249+
return frame.executeJavaScript(
250+
'({ clicked: document.body.dataset.clicked, isolated: document.body.dataset.isolated, bridge: typeof window.simDesktop })'
251+
)
252+
}, `${origin}${path}`)
253+
expect(frameState).toEqual({ clicked: 'true', isolated: 'true', bridge: 'undefined' })
254+
}
255+
}
256+
const typed = await execute('browser_open_url', { url: `${origin}/private-chat` })
257+
expect(typed.ok, typed.error).toBe(true)
258+
const result = typed.result as { snapshot: { outline: string } }
259+
const message = result.snapshot.outline
260+
.split('\n')
261+
.find((line) => line.includes('"Message"') && /\[ref=\d+\]/.test(line))
262+
?.match(/\[ref=(\d+)\]/)?.[1]
263+
const send = result.snapshot.outline
264+
.split('\n')
265+
.find((line) => line.includes('"Send"') && /\[ref=\d+\]/.test(line))
266+
?.match(/\[ref=(\d+)\]/)?.[1]
267+
expect(message, result.snapshot.outline).toBeTruthy()
268+
expect(send, result.snapshot.outline).toBeTruthy()
269+
expect(
270+
(await execute('browser_type', { elementId: Number(message), text: 'Session works' })).ok
271+
).toBe(true)
272+
expect((await execute('browser_click', { elementId: Number(send) })).ok).toBe(true)
273+
expect(JSON.stringify(await execute('browser_snapshot', {}))).toContain('Session works')
274+
})
275+
276+
test('keeps external navigation isolated and authenticates redirects back into Sim', async () => {
277+
expect((await execute('browser_open_url', { url: `${origin}/private-chat` })).ok).toBe(true)
278+
const external = origin.replace('127.0.0.1', 'localhost')
279+
const result = await execute('browser_open_url', { url: `${origin}/redirect` })
280+
expect(result.ok, result.error).toBe(true)
281+
expect(result.result).toMatchObject({ url: `${external}/landing` })
282+
expect(
283+
await app.evaluate(({ webContents, BrowserWindow }, url) => {
284+
const page = webContents.getAllWebContents().find((contents) => contents.getURL() === url)
285+
return page?.session === BrowserWindow.getAllWindows()[0].webContents.session
286+
}, `${external}/landing`)
287+
).toBe(false)
288+
const back = await execute('browser_open_url', { url: `${external}/enter-sim` })
289+
expect(back.ok, back.error).toBe(true)
290+
expect(back.result).toMatchObject({ url: `${origin}/private-chat` })
291+
})
292+
293+
test('sign-out clears authenticated browser pages with the desktop session', async () => {
294+
const opened = await execute('browser_open_url', { url: `${origin}/private-chat` })
295+
expect(opened.ok, opened.error).toBe(true)
296+
expect(opened.result).toMatchObject({ url: `${origin}/private-chat` })
297+
await window.evaluate(async () => {
298+
await fetch('/api/auth/sign-out', { method: 'POST' })
299+
})
300+
await expect
301+
.poll(() =>
302+
app.evaluate(
303+
({ webContents }, url) =>
304+
webContents.getAllWebContents().some((contents) => contents.getURL() === url),
305+
`${origin}/private-chat`
306+
)
307+
)
308+
.toBe(false)
309+
await expect(window).toHaveURL(`${origin}/login`)
310+
})
311+
171312
async function formState() {
172313
return app.evaluate(async ({ webContents }, origin) => {
173314
const page = webContents

‎apps/desktop/src/main/browser-agent/cdp.test.ts‎

Lines changed: 92 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ function createOopifFrameFixture() {
4545
{
4646
frame: {
4747
id: 'child',
48+
parentId: 'top',
4849
name: 'account-menu',
4950
url: 'https://accounts.example/menu',
5051
},
@@ -310,76 +311,99 @@ describe('browser-agent CDP instrumentation', () => {
310311
}
311312
})
312313

313-
it('routes OOPIF isolated-world creation and evaluation through its flattened session', async () => {
314-
const contents = new WebContentsView().webContents
315-
const { child, frameTree } = createOopifFrameFixture()
316-
await ensureInstrumented(contents, { onDialog: vi.fn() })
317-
const listener = vi
318-
.mocked(contents.debugger.on)
319-
.mock.calls.find(([event]) => event === 'message')?.[1] as
320-
| ((event: unknown, method: string, params: unknown, sessionId?: string) => void)
321-
| undefined
322-
expect(listener).toBeTypeOf('function')
323-
324-
listener?.(
325-
{},
326-
'Target.attachedToTarget',
327-
{
328-
sessionId: 'child-session',
329-
targetInfo: { targetId: 'child', type: 'iframe' },
330-
},
331-
undefined
332-
)
333-
expect(contents.debugger.sendCommand).toHaveBeenCalledWith(
334-
'Target.setAutoAttach',
335-
{ autoAttach: true, waitForDebuggerOnStart: false, flatten: true },
336-
'child-session'
337-
)
338-
vi.mocked(contents.debugger.sendCommand).mockClear()
339-
vi.mocked(contents.debugger.sendCommand).mockImplementation((method) => {
340-
if (method === 'Page.getFrameTree') {
341-
return Promise.resolve({ frameTree })
342-
}
343-
if (method === 'Page.createIsolatedWorld') {
344-
return Promise.resolve({ executionContextId: 42 })
345-
}
346-
if (method === 'Runtime.evaluate') {
347-
return Promise.resolve({ result: { type: 'number', value: 4 } })
348-
}
349-
return Promise.resolve({})
350-
})
351-
352-
await expect(evaluateInIsolatedFrame(contents, child, '2 + 2')).resolves.toBe(4)
353-
354-
expect(
355-
vi
356-
.mocked(contents.debugger.sendCommand)
357-
.mock.calls.filter(([method]) =>
358-
['Page.createIsolatedWorld', 'Runtime.evaluate'].includes(method)
359-
)
360-
).toEqual([
361-
[
362-
'Page.createIsolatedWorld',
363-
{
364-
frameId: 'child',
365-
worldName: 'sim-browser-agent',
366-
grantUniveralAccess: false,
367-
},
368-
'child-session',
369-
],
370-
[
371-
'Runtime.evaluate',
314+
it.each(['complete', 'split', 'ambiguous'])(
315+
'routes OOPIF evaluation through its session (%s tree)',
316+
async (treeKind) => {
317+
const contents = new WebContentsView().webContents
318+
const { child, frameTree } = createOopifFrameFixture()
319+
await ensureInstrumented(contents, { onDialog: vi.fn() })
320+
const listener = vi
321+
.mocked(contents.debugger.on)
322+
.mock.calls.find(([event]) => event === 'message')?.[1] as
323+
| ((event: unknown, method: string, params: unknown, sessionId?: string) => void)
324+
| undefined
325+
expect(listener).toBeTypeOf('function')
326+
327+
listener?.(
328+
{},
329+
'Target.attachedToTarget',
372330
{
373-
expression: '2 + 2',
374-
contextId: 42,
375-
returnByValue: true,
376-
awaitPromise: true,
377-
userGesture: false,
331+
sessionId: 'child-session',
332+
targetInfo: { targetId: 'child', type: 'iframe' },
378333
},
379-
'child-session',
380-
],
381-
])
382-
})
334+
undefined
335+
)
336+
expect(contents.debugger.sendCommand).toHaveBeenCalledWith(
337+
'Target.setAutoAttach',
338+
{ autoAttach: true, waitForDebuggerOnStart: false, flatten: true },
339+
'child-session'
340+
)
341+
vi.mocked(contents.debugger.sendCommand).mockClear()
342+
vi.mocked(contents.debugger.sendCommand).mockImplementation((method, _params, sessionId) => {
343+
if (method === 'Page.getFrameTree') {
344+
return Promise.resolve({
345+
frameTree:
346+
treeKind === 'complete'
347+
? frameTree
348+
: sessionId
349+
? frameTree.childFrames[0]
350+
: { frame: frameTree.frame },
351+
})
352+
}
353+
if (method === 'Page.createIsolatedWorld') {
354+
return Promise.resolve({ executionContextId: 42 })
355+
}
356+
if (method === 'Runtime.evaluate') {
357+
return Promise.resolve({ result: { type: 'number', value: 4 } })
358+
}
359+
return Promise.resolve({})
360+
})
361+
362+
if (treeKind === 'ambiguous') {
363+
/** An omitted twin must not be mistaken for the only frame in a partial tree. */
364+
child.parent?.frames.push(createOopifFrameFixture().child)
365+
await expect(evaluateInIsolatedFrame(contents, child, '2 + 2')).rejects.toThrow(
366+
'Could not map'
367+
)
368+
expect(
369+
vi
370+
.mocked(contents.debugger.sendCommand)
371+
.mock.calls.some(([method]) => method === 'Runtime.evaluate')
372+
).toBe(false)
373+
return
374+
}
375+
await expect(evaluateInIsolatedFrame(contents, child, '2 + 2')).resolves.toBe(4)
376+
377+
expect(
378+
vi
379+
.mocked(contents.debugger.sendCommand)
380+
.mock.calls.filter(([method]) =>
381+
['Page.createIsolatedWorld', 'Runtime.evaluate'].includes(method)
382+
)
383+
).toEqual([
384+
[
385+
'Page.createIsolatedWorld',
386+
{
387+
frameId: 'child',
388+
worldName: 'sim-browser-agent',
389+
grantUniveralAccess: false,
390+
},
391+
'child-session',
392+
],
393+
[
394+
'Runtime.evaluate',
395+
{
396+
expression: '2 + 2',
397+
contextId: 42,
398+
returnByValue: true,
399+
awaitPromise: true,
400+
userGesture: false,
401+
},
402+
'child-session',
403+
],
404+
])
405+
}
406+
)
383407

384408
it('falls back to the root target when OOPIF isolated-world creation fails', async () => {
385409
const contents = new WebContentsView().webContents

0 commit comments

Comments
 (0)