Skip to content

Keep a keystroke typed into a Bot's browser out of the app's own shortcuts - #602

Merged
davidmckayv merged 2 commits into
CopilotKit:mainfrom
kevin9327:driving-keys-stay-remote
Sep 18, 2026
Merged

davidmckayv merged 2 commits into
CopilotKit:mainfrom
kevin9327:driving-keys-stay-remote

Conversation

@kevin9327

@kevin9327 kevin9327 commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

What this changes

While somebody drives a Bot's browser, LiveScreen (app/src/components/computer/live-screen.tsx) forwards every keystroke from a keydown listener on window and calls preventDefault. The app's own shortcuts listen on the same window:

  • useHotkey (app/src/lib/hotkeys/use-hotkey.ts) handles Shift+N → New chat. AppHotkeys binds it when the signed-in app mounts in _authed.
  • SidebarProvider (app/src/components/ui/sidebar.tsx) handles Ctrl/Cmd+B → toggle sidebar.

Both are registered before LiveScreen, which only binds once driving starts, and listeners on one target run in the order they were added. So both saw every forwarded keystroke first. Neither checks defaultPrevented, and useHotkey's editable-field check does not help here, because while driving the keystroke's target is the body or the button that started driving. The result:

  • Typing a capital N into the Bot's browser starts a new chat. The first letter of "New York" in a search box goes to the remote page and also navigates to /channel/new. That unmounts the conversation and the live screen, so the person is taken away from the Bot in the middle of a word.
  • Ctrl+B there also toggles the sidebar here. In a document on the remote page, Ctrl+B makes text bold, and it also shows or hides the local sidebar.

After this change, LiveScreen listens for keydown in the capture phase and calls stopPropagation on a keystroke it forwards. A capture listener on window runs before any bubble listener, so a forwarded keystroke reaches the Bot's browser and nothing else on the page. Every existing exception still happens first and returns before the stop:

keyup is unchanged, because no shortcut listens for it. Nothing changes while nobody is driving: LiveScreen binds no key listener then.

This is a single fix in LiveScreen rather than a defaultPrevented check in each shortcut. That way a shortcut added later cannot bring the problem back.

Where it runs

  • New state that outlives a request? None. This is a keyboard listener in the browser.
  • What happens on the second replica? Nothing server-side is involved. What is sent over the live-screen socket is unchanged.
  • Anything serialised? No.
  • Anything fanned out to a browser? No.
  • New listener, port, or schedule? No new listener. The existing keydown listener moves to the capture phase.

Boundary and audit

  • Every acting call still goes through the gateway: resolve, decide, audit, then act. The keystrokes sent to the Bot's browser are the same messages as before.
  • New refusals and new failures each write a row. None are added.
  • Nothing new is trusted from the client that the server can resolve itself.

Changelog

  • A line in CHANGELOG.md under Unreleased.

Proof

app/tests/live-screen-app-shortcuts.test.tsx renders the signed-in shell's pieces with a memory router: SidebarProvider, AppHotkeys and a route showing LiveScreen (with the WebSocket double from live-screen-keyboard.test.tsx), plus a /channel/new route. Keystrokes are dispatched at document.body, where a browser sends them when no field has focus.

  • a capital N typed into the Bot's browser goes to it, and does not start a new chat: the path stays /, and the socket gets the Shift+N keydown.
  • Ctrl+B typed into the Bot's browser goes to it, and leaves the sidebar alone: the sidebar stays open, and the socket gets the Ctrl+B keydown.
  • the same shortcuts still work while nobody is driving: Ctrl+B closes the sidebar, and Shift+N navigates to /channel/new. This passes on main too and pins that the shortcuts are unaffected outside driving.

With live-screen.tsx from main (only the test added):

(fail) a capital N typed into the Bot's browser goes to it, and does not start a new chat
error: expect(received).toBe(expected)
Expected: "/"
Received: "/channel/new"

(fail) Ctrl+B typed into the Bot's browser goes to it, and leaves the sidebar alone
error: expect(received).toBe(expected)
Expected: "open"
Received: "closed"

 1 pass
 2 fail

With the change, all 3 pass, and so do the 9 existing tests in live-screen-keyboard.test.tsx: the paste shortcut on Latin, Russian, Greek and Dvorak layouts, its keyup, and the keys that are forwarded. Escape returns on the handler's first line, before the new call, and that line is unchanged. bun test --coverage shows all three changed lines executed (the stopPropagation call and the two listener registrations).

App suite (bun test app, Windows): 892 pass, 29 fail. The 29 failures are exactly the ones main has on this machine (serve port, WebSocket, path separator, locale). bun run typecheck in app/ is clean, and biome check passes for both files.

I checked the listener order in happy-dom, not in a real browser. The capture-before-bubble order it relies on is the standard DOM dispatch order: capture listeners on window run before bubble listeners on window for any event whose target is inside the document.

On Linux too (ubuntu-latest, Bun 1.3.14; a throwaway workflow on my fork, since fork PRs get no CI here): bun test app is 925 pass, 0 fail with all five of my open PRs merged together, and format:check, lint and typecheck are green.

🤖 Generated with Claude Code

…tcuts

While somebody drives a Bot's browser, LiveScreen forwards every keydown from a window listener and
prevents its default. The app's shortcuts listen on the same window: useHotkey for Shift+N, bound
by AppHotkeys when the signed-in app mounts, and SidebarProvider for Ctrl/Cmd+B. Both were bound
before LiveScreen, so both saw each keystroke first, and neither looks at defaultPrevented.

So a capital N typed into the Bot's browser, the first letter of "New York" in a search box,
navigated to /channel/new and took the person away from the Bot mid-word, and Ctrl+B, bold in a
document, toggled the local sidebar as well as reaching the remote page.

LiveScreen now listens for keydown in the capture phase and stops a keystroke it forwards, so
nothing else on this page acts on it. Escape and the paste shortcut return before that, as before:
Escape still closes the view and the paste event still fires locally. Keyup is unchanged; no
shortcut listens for it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@davidmckayv davidmckayv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code-verified clean; CI green on this sha.

@davidmckayv
davidmckayv merged commit a187355 into CopilotKit:main Sep 18, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants