Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CONTROLLERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ Every controller below has a committed `openmicro doctor` report. CI replays cap
| ----------------------------------------- | ------------------------------- | --------- | ---------- | --------- | -------------- | ------------- | --------------- |
| GameSir Cyclone 2 (DS4 mode) | DUALSHOCK 4 Wireless Controller | 054c:05c4 | bluetooth | ds4 | 17/17 | none | ✅ full |
| GameSir G7 Pro | GameSir-G7 Pro | 3537:1022 | bluetooth | gamesir | 17/17 | none | ✅ full |
| Microsoft Xbox 360 Controller | Controller | 045e:028e | usb | xbox | 17/17 | none | ✅ full |
| Microsoft Xbox Elite Series 2 | Controller | 045e:0b00 | usb | xbox | 17/17 | none | ✅ full |
| Microsoft Xbox Elite Series 2 (Bluetooth) | Xbox Wireless Controller | 045e:0b13 | bluetooth | xbox | 17/17 | none | ✅ full |
| Microsoft Xbox One S Controller | Controller | 045e:02ea | usb | xbox | 17/17 | none | ✅ full |
Expand Down
31 changes: 30 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,21 @@ Stick flicks fire after returning to center. Rotation gestures (`rstick_cw`/`rst

Voice and thinking-depth support varies by harness; see [OpenMicro feature parity](#openmicro-feature-parity).

#### Set up push-to-talk in Claude Code

The north button (△ / Y) starts and stops Claude Code dictation. It needs two settings:

1. Turn voice on with `/voice`.
2. Set the voice mode to `tap`. The default mode, `hold`, doesn't work with a controller. The `/config` screen doesn't show this setting, so add it to `~/.claude/settings.json` yourself:

```json
"voice": { "enabled": true, "mode": "tap" }
```

Then restart Claude Code.

In tap mode, press △ / Y once to start listening and again to stop. In `hold` mode, Claude Code detects a held Space by the repeated keystrokes a keyboard sends while the key is down. The controller sends one Space per press, so Claude Code types a space and doesn't start listening.

## See it in action

Animated demos (rendered, not filmed) of the six core interactions:
Expand All @@ -61,6 +76,16 @@ Animated demos (rendered, not filmed) of the six core interactions:
| ![Left-stick flick launches a workflow prompt](assets/demo/workflow-flick.gif)<br>**Workflow flick** — stick up = review PR | ![Right-stick flicks step thinking depth](assets/demo/thinking-dial.gif)<br>**Thinking dial** — stick flicks step `/effort` |
| ![L1 plus a face button switches control layers](assets/demo/layers.gif)<br>**Layers** — hold L1 to switch | ![Touchpad click moves focus between sessions](assets/demo/multi-session.gif)<br>**Multi-session** — touchpad switches focus |

### Xbox 360 Controller (wired)

The same interactions on the wired Xbox 360 Controller. Face buttons map by position (A = ✕, B = ○, Y = △, X = □), LB stands in for L1, and the Guide button takes the touchpad's role. OpenMicro can't drive the 360's Ring of Light, so session status appears only in the terminal and there is no status-lightbar demo.

| | |
| :-------------------------------------------------------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------------: |
| ![Xbox 360 face buttons submit, interrupt, dictate, and start a new chat](assets/demo/xbox360/command-keys.gif)<br>**Command keys** — A B Y X | ![Xbox 360 left-stick flick launches a workflow prompt](assets/demo/xbox360/workflow-flick.gif)<br>**Workflow flick** — stick up = review PR |
| ![Xbox 360 right-stick flicks step thinking depth](assets/demo/xbox360/thinking-dial.gif)<br>**Thinking dial** — stick flicks step `/effort` | ![Xbox 360 LB plus a face button switches control layers](assets/demo/xbox360/layers.gif)<br>**Layers** — hold LB to switch |
| ![Xbox 360 Guide button moves focus between sessions](assets/demo/xbox360/multi-session.gif)<br>**Multi-session** — Guide switches focus | |

## What it gives you

- Respond to agents without hunting through terminal tabs.
Expand All @@ -82,7 +107,7 @@ Animated demos (rendered, not filmed) of the six core interactions:
| Start a new chat | ✅ `/clear` | ✅ `/new` |
| D-pad TUI navigation | ✅ Arrow-key passthrough | ✅ Arrow-key passthrough |
| Stick-triggered workflow prompts | ✅ Supported | ✅ Supported |
| Push-to-talk | ✅ Enable with `/voice` | — No equivalent |
| Push-to-talk | ⚠️ `/voice`, `tap` mode | — No equivalent |
| Thinking-depth dial | ✅ `/effort`, low → max | — No deterministic binding |
| Six remappable control layers | ✅ Supported | ✅ Supported |
| Multi-session focus switching | ✅ Supported | ✅ Supported |
Expand Down Expand Up @@ -220,6 +245,10 @@ Each function returns `{ message, tone }` or `null`; agent statuses also return

## Troubleshooting

### △ / Y types a space instead of starting dictation

Claude Code's voice mode is set to `hold`. Change it to `tap`; see [Set up push-to-talk in Claude Code](#set-up-push-to-talk-in-claude-code).

### Controller is connected but OpenMicro cannot open it

Another process probably owns the device exclusively. Quit other controller tools, Steam, browser tabs using Gamepad/WebHID, and PS Remote Play, then retry.
Expand Down
Binary file modified assets/demo/command-keys.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/demo/xbox360/command-keys.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/demo/xbox360/layers.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/demo/xbox360/multi-session.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/demo/xbox360/thinking-dial.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/demo/xbox360/workflow-flick.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 2 additions & 1 deletion demo/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"type": "module",
"scripts": {
"studio": "remotion studio",
"render": "remotion render StatusLeds ../assets/demo/status-leds.gif --codec=gif --every-nth-frame=2 && remotion render CommandKeys ../assets/demo/command-keys.gif --codec=gif --every-nth-frame=2 && remotion render WorkflowFlick ../assets/demo/workflow-flick.gif --codec=gif --every-nth-frame=2 && remotion render ThinkingDial ../assets/demo/thinking-dial.gif --codec=gif --every-nth-frame=2 && remotion render Layers ../assets/demo/layers.gif --codec=gif --every-nth-frame=2 && remotion render MultiSession ../assets/demo/multi-session.gif --codec=gif --every-nth-frame=2"
"render": "remotion render StatusLeds ../assets/demo/status-leds.gif --codec=gif --every-nth-frame=2 && remotion render CommandKeys ../assets/demo/command-keys.gif --codec=gif --every-nth-frame=2 && remotion render WorkflowFlick ../assets/demo/workflow-flick.gif --codec=gif --every-nth-frame=2 && remotion render ThinkingDial ../assets/demo/thinking-dial.gif --codec=gif --every-nth-frame=2 && remotion render Layers ../assets/demo/layers.gif --codec=gif --every-nth-frame=2 && remotion render MultiSession ../assets/demo/multi-session.gif --codec=gif --every-nth-frame=2",
"render:xbox360": "remotion render Xbox360CommandKeys ../assets/demo/xbox360/command-keys.gif --codec=gif --every-nth-frame=2 && remotion render Xbox360WorkflowFlick ../assets/demo/xbox360/workflow-flick.gif --codec=gif --every-nth-frame=2 && remotion render Xbox360ThinkingDial ../assets/demo/xbox360/thinking-dial.gif --codec=gif --every-nth-frame=2 && remotion render Xbox360Layers ../assets/demo/xbox360/layers.gif --codec=gif --every-nth-frame=2 && remotion render Xbox360MultiSession ../assets/demo/xbox360/multi-session.gif --codec=gif --every-nth-frame=2"
},
"dependencies": {
"@remotion/cli": "^4.0.0",
Expand Down
78 changes: 73 additions & 5 deletions demo/src/Root.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,82 @@ import { Layers } from './scenes/Layers'
import { MultiSession } from './scenes/MultiSession'

const base = { width: 800, height: 450, fps: 30 } as const
const ds = { variant: 'dualsense' } as const
// No StatusLeds for the 360: OpenMicro cannot drive its Ring of Light.
const x360 = { variant: 'xbox360' } as const

export const Root: React.FC = () => (
<>
<Composition id="StatusLeds" component={StatusLeds} durationInFrames={330} {...base} />
<Composition id="CommandKeys" component={CommandKeys} durationInFrames={300} {...base} />
<Composition id="WorkflowFlick" component={WorkflowFlick} durationInFrames={270} {...base} />
<Composition id="ThinkingDial" component={ThinkingDial} durationInFrames={300} {...base} />
<Composition id="Layers" component={Layers} durationInFrames={270} {...base} />
<Composition id="MultiSession" component={MultiSession} durationInFrames={300} {...base} />
<Composition
id="CommandKeys"
component={CommandKeys}
durationInFrames={300}
defaultProps={ds}
{...base}
/>
<Composition
id="WorkflowFlick"
component={WorkflowFlick}
durationInFrames={270}
defaultProps={ds}
{...base}
/>
<Composition
id="ThinkingDial"
component={ThinkingDial}
durationInFrames={300}
defaultProps={ds}
{...base}
/>
<Composition
id="Layers"
component={Layers}
durationInFrames={270}
defaultProps={ds}
{...base}
/>
<Composition
id="MultiSession"
component={MultiSession}
durationInFrames={300}
defaultProps={ds}
{...base}
/>
<Composition
id="Xbox360CommandKeys"
component={CommandKeys}
durationInFrames={300}
defaultProps={x360}
{...base}
/>
<Composition
id="Xbox360WorkflowFlick"
component={WorkflowFlick}
durationInFrames={270}
defaultProps={x360}
{...base}
/>
<Composition
id="Xbox360ThinkingDial"
component={ThinkingDial}
durationInFrames={300}
defaultProps={x360}
{...base}
/>
<Composition
id="Xbox360Layers"
component={Layers}
durationInFrames={270}
defaultProps={x360}
{...base}
/>
<Composition
id="Xbox360MultiSession"
component={MultiSession}
durationInFrames={300}
defaultProps={x360}
{...base}
/>
</>
)
40 changes: 40 additions & 0 deletions demo/src/components/Pad.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import React from 'react'
import { DualSense, LEFT_STICK, RIGHT_STICK } from './DualSense'
import { Xbox360, XBOX_LEFT_STICK, XBOX_RIGHT_STICK } from './Xbox360'

export type Variant = 'dualsense' | 'xbox360'
export type SceneProps = { variant: Variant }

/** On-screen names for each pad's controls, keyed by the DualSense button id. */
export const LABELS = {
dualsense: {
cross: '✕',
circle: '○',
triangle: '△',
square: '□',
l1: 'L1',
touchpad: 'touchpad',
touchpadClick: 'touchpad click',
},
xbox360: {
cross: 'A',
circle: 'B',
triangle: 'Y',
square: 'X',
l1: 'LB',
touchpad: 'Guide',
touchpadClick: 'Guide press',
},
} as const

/** Stick centers, plus where a left-stick callout fits without covering other controls. */
export const STICKS = {
dualsense: { left: LEFT_STICK, right: RIGHT_STICK, callout: { x: 176, y: 258 } },
xbox360: { left: XBOX_LEFT_STICK, right: XBOX_RIGHT_STICK, callout: { x: 96, y: 246 } },
} as const

/** Renders the variant's controller; the Xbox 360 ignores lightbar and player-LED props. */
export const Pad: React.FC<React.ComponentProps<typeof DualSense> & { variant: Variant }> = ({
variant,
...props
}) => (variant === 'xbox360' ? <Xbox360 {...props} /> : <DualSense {...props} />)
26 changes: 14 additions & 12 deletions demo/src/components/SceneFrame.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -20,18 +20,20 @@ export const SceneFrame: React.FC<{
>
<div style={{ display: 'flex', flexDirection: 'column', alignItems: 'center', gap: 10 }}>
{pad}
{caption && (
<div
style={{
color: DIM,
fontFamily: 'ui-sans-serif, sans-serif',
fontSize: 14,
fontWeight: 600,
}}
>
{caption}
</div>
)}
{/* fixed-height line so the pad doesn't jump when a caption appears or clears */}
<div
style={{
color: DIM,
fontFamily: 'ui-sans-serif, sans-serif',
fontSize: 14,
fontWeight: 600,
height: 20,
lineHeight: '20px',
whiteSpace: 'pre',
}}
>
{caption}
</div>
</div>
{term}
</AbsoluteFill>
Expand Down
Loading