Skip to content

feat: send press and hold control events to surfaces - #4538

Draft
cbusillo wants to merge 8 commits into
bitfocus:mainfrom
cbusillo:feat/control-events
Draft

cbusillo wants to merge 8 commits into
bitfocus:mainfrom
cbusillo:feat/control-events

Conversation

@cbusillo

@cbusillo cbusillo commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

This lets a surface react when a press on one of its controls actually runs actions. It uses the optional controlEvent(controlId, event) API proposed in bitfocus/companion-surface-api#90. It cannot merge until that is released.

A physical press that runs an action group sends press to the pressed control. A run-while-held group sends hold when it fires. The surface decides what to do. The Loupedeck module plays a short haptic click.

  • Each action group has a "Surface cue" option in its group properties, on by default. A release does not send a second event.
  • Presses from HTTP, OSC, the web buttons and triggers send nothing. References pass the original surface through.
  • Events carry a per-open connection id, so a late event cannot reach a reconnected surface.
  • Satellite clients opt in per device with CONTROL_EVENTS=1 on ADD-DEVICE, like VARIABLE_ENCODING, and receive CONTROL-EVENT. It is listed under 1.17.0 in satellite-protocol.

Timed groups that run on release send nothing yet. I have not settled when that cue should fire.

Adapter draft: cbusillo/companion-surface-loupedeck feat/control-events. API on current main: cbusillo/companion-surface-api feat/control-events-on-main.

I tested this branch end to end on a Loupedeck Live S, with current main merged and a local build of the API change. I checked press and hold cues, the group and surface opt-outs, reconnect and rapid input. Lint and the related tests pass. Typecheck needs the API release. I signed the Bitfocus CLA in September.

This change was written with AI assistance and reviewed and tested by me.

Summary by CodeRabbit

  • New Features
    • Supported surfaces can now receive control events when press, release, or duration-based action groups run.
    • Configure surface cues for individual action groups, including duration groups, in the button editor.
    • Satellite surfaces can opt in to receive control events.
  • Documentation
    • Added guidance on when surface cues are sent and how to disable them for an action group.

When a physical press runs an action group, Companion sends a press
event to the control that was pressed. When a run-while-held group
fires, it sends hold. The surface decides how to show it, for example a
haptic click or a flash of the key. This uses the optional
controlEvent(controlId, event) surface API proposed in
bitfocus/companion-surface-api#90.

- Plugin surfaces receive controlEvent IPC with the per-open connection
  id, so a delayed event cannot reach a reconnected surface.
- Satellite API 1.15.0: a client that sets CONTROL_EVENTS on ADD-DEVICE
  receives CONTROL-EVENT with KEY or CONTROLID and EVENT. Advertised by
  CONTROL_EVENTS in CAPS and listed in the satellite-protocol feature
  table.
- Each action group has a "Surface cue" checkbox, on by default, stored
  as controlEventsDisabledSets on the step. A release does not send a
  second event after a press or hold already did.
- Synthetic presses (HTTP, OSC, web buttons, triggers) send nothing.
  Button references pass the original surface through. Cancel, lock and
  unload stop pending events.
- New internal action "Surface: Show cue on this button" sends a press
  event to the control showing the running button.

Timed groups that run on release send nothing yet; when that cue should
fire is still an open question.
@coderabbitai

coderabbitai Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Walkthrough

The change adds configurable surface cues for button action sets. Physical presses can carry event context through control execution, which sends press or hold events to supported surface panels and satellite clients. The change also updates the satellite protocol and button editor.

Changes: Surface Control Events

Layer / File(s) Summary
Action-set preferences and storage
shared-lib/lib/Model/*, companion/lib/Controls/ActionSetsTrpcRouter.ts, companion/lib/Controls/Entities/*, companion/lib/Instance/Connection/*, companion/test/Controls/*, companion/test/ImportExport/*, companion/test/Instance/Definitions.test.ts
Action-step options store disabled control-event action-set IDs. The backend validates and updates preferences. Defaults, export schemas, and tests include the new option.
Surface-cue editor controls
webui/src/Buttons/EditButton/*, webui/src/Services/Controls/ControlActionStepsAndSetsService.ts, docs/user-guide/3_config/program/buttons/creating/actions.md
The button editor exposes surface-cue settings for press, release, and duration groups. The service submits changes through the new mutation. The guide describes cue behavior.
Press context and event dispatch
companion/lib/Controls/*, companion/lib/Internal/Controls.ts, companion/lib/Cloud/Region.ts, companion/lib/Link/Controller.ts, companion/lib/Service/ServiceApi.ts, companion/lib/Surface/Handler.ts, companion/test/Controls/*, companion/test/Internal/Controls.test.ts, companion/test/Link/*, companion/test/integration/TestApp.ts
Control press APIs now pass a source object with a surface ID and optional event sender. Button execution requests cues for eligible action sets and finishes gesture context when the press ends or is invalidated.
Surface and satellite event delivery
companion/lib/Instance/Surface/*, companion/lib/Surface/*, companion/lib/Service/Satellite/SatelliteApi.ts, satellite-protocol/*, companion/test/Surface/*, companion/test/Service/Satellite/SatelliteApi.test.ts
Panels report event support and send events through IPC or satellite messages. Satellite clients opt in with CONTROL_EVENTS; the protocol version documents CONTROL-EVENT.
Merge Risk: Moderate · up to 027ce

Importing a button export made before this change can leave surface-cue settings missing, and pressing that button may then throw. The surface API dependency also needs updating once its release is published. Fix the import normalization and pin the dependency before merging.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title "feat: send press and hold control events to surfaces" matches the main change. The PR adds press and hold control events sent to surfaces when physical presses run action groups, along with…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title "feat: send press and hold control events to surfaces" matches the main change. The PR adds press and hold control events sent to surfaces when physical presses run action groups, along with the supporting API, settings, and UI. The title does not mention the opt-out 'Surface cue' setting or the satellite protocol bump, which is expected for a short title.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

A press sets the cue in motion
A held group sends its signal
A surface receives the event
A satellite can opt in
Each action set keeps its choice
Button and panel share the beat

@Julusian Julusian left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally I'm happy with this. I think all my comments are really polish, nothing major.

Sorry I put this off for so long, I partly got put off picking it up due to the reported line count and number of files, but this is naturally a bit sprawling and the line count is always bloated by tests.

Comment thread companion/lib/Controls/Controller.ts Resolved
Comment thread companion/lib/Internal/Surface.ts Resolved

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you may have already drafted it, but a reminder for perhaps myself to make sure the protocol docs are updated https://github.com/bitfocus/website/blob/main/for-developers/Satellite-API.md

Comment thread shared-lib/lib/Model/ActionModel.ts Resolved

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not related to this file, but I needed to write it somewhere while I rememebered;

It would be good to have something in the docs about this. Not sure where/what that should look like though

…eview

# Conflicts:
#	companion/lib/Controls/ControlDependencies.ts
#	companion/lib/Controls/Entities/EntityListPoolButton.ts
…ents

Document when a press's control events are finished.
Companion no longer advertises CONTROL_EVENTS in CAPS. A 1.17.0 client opts in per device with CONTROL_EVENTS on ADD-DEVICE, like VARIABLE_ENCODING.
…bled list

The disabled list now defaults to empty, so usages need no optional checks, and the input is trusted once zod has parsed it. The panel relies on the IPC promise alone for errors.
…ties modal

The checkbox was too prominent for a rarely used option. Document surface cues in the user guide.
@cbusillo

Copy link
Copy Markdown
Contributor Author

Thanks for the review, and no worries about the wait. I've pushed changes for all of it.

  • finish(): it ends a press's cue. The SurfaceHandler calls it right after the release has been dispatched, or when the surface locks or unloads. The button also calls it when a new press from the same surface replaces the hold state, when delayed actions are aborted, and on destroy. I kept it out of abortRunningHoldTimers because the release path calls that before the release group runs. Finishing there would drop the cue for a button that only has release actions. I added comments saying this.
  • Press arguments: now one source: { surfaceId, pressEvents } | null object, through all the layers. pressed and force stay as plain arguments.
  • Show cue action: removed. I can bring it back later with a picker that only lists surfaces that support it.
  • Satellite: CONTROL_EVENTS is gone from the CAPS Companion sends. A client still opts in per device with CONTROL_EVENTS=1 on ADD-DEVICE, the same way 1.16 does VARIABLE_ENCODING. Companion doesn't learn the client's API version, so gating on version alone would send an unknown command to older clients. Happy to change it if you had something else in mind.
  • Protocol docs: I have the Satellite-API.md change drafted and can open a PR on bitfocus/website once this settles.
  • setId check: one shared zod schema in shared-lib, used by the router, ActionStepOptionsSchema and the UI type.
  • controlEventsDisabledSets: uses .default(() => []), so the ! and the optional checks are gone.
  • Cleanup: dropped the extra try in PluginPanel and the defensive checks in EntityListPoolButton.
  • UI: the Surface cue checkbox is now in a small group properties modal.
  • Docs: added a short "Surface cues" section to the actions page in the user guide.

I also merged main. I tested this round on a Loupedeck Live S, against a local build of bitfocus/companion-surface-api#90: press and hold cues, the group and surface opt-outs, replugging and rapid presses all behave.

This still needs bitfocus/companion-surface-api#90 released before it typechecks. One heads up: API main's newer onVariableValue type no longer accepts undefined, so Entrypoint.ts:199 will need msg.value ?? null when Companion moves its API pin. That is not from this PR, so I left it out.

protected abstract entityListReportChange(options: ControlEntityListChangeProps): void

override abortDelayedActions(skip_up: boolean, exceptSignal: AbortSignal | null, _linkHops: number): void {
this.#finishPressEvents()

@Julusian Julusian Nov 21, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit worried about these this.#finishPressEvents() calls in both these abort methods.
Primarily that it ignores the relationship to the signal.

In abortDelayedActions, this means that when the user does an 'abort except this run' the events for the current still operating press will be finished.

In abortDelayedActionsSingle, that only aborts the one signal it is passing (badly named copy paste parameter, that should be called onlySignal). But this will finish all the active events, even though they keep running.

Or maybe this is unavoidable due to the relationship between these events and surfaces?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps a related thing; in the this.setPushed(false) path, should the 'up' event be passed to any just before finish is called?

Comment on lines +364 to 365
if (locked) this.#finishPressEvents()
if (this.#isSurfaceLocked) this.#displayed.clear()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (locked) this.#finishPressEvents()
if (this.#isSurfaceLocked) this.#displayed.clear()
if (this.#isSurfaceLocked) {
this.#finishPressEvents()
this.#displayed.clear()
}

Comment on lines +489 to +490
// The physical release ends its cue lifetime even if the current page has disappeared.
const panelCoordinate = `${y}/${x}/${pageOffset}`

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// The physical release ends its cue lifetime even if the current page has disappeared.
const panelCoordinate = `${y}/${x}/${pageOffset}`
// the key for saving button-press location, which has to refer to the physical button since that doesn't move between press and release
const panelCoordinate = `${y}/${x}/${pageOffset}`
// The physical release ends its cue lifetime even if the current page has disappeared.

params.PINCODE_LOCK !== undefined && (params.PINCODE_LOCK === 'FULL' || params.PINCODE_LOCK === 'PARTIAL')
const canChangePage =
typeof params.CAN_CHANGE_PAGE === 'string' && params.CAN_CHANGE_PAGE ? params.CAN_CHANGE_PAGE : undefined
const supportsControlEvents = params.CONTROL_EVENTS !== undefined && isTruthy(params.CONTROL_EVENTS)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be a truthy check, or match the array of events that the surface-api now has?

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Nitpick comments: 1

companion/test/Controls/ControlTypes/Button/PressEventsRuntime.test.ts: 1

14-14: Trivial · Maintainability & Code Quality · Quick win

Please make the new helper parameters required.

disabled = false and runMultipleActions = ... make these parameters optional. Please remove the defaults and pass the intended values at every call site. This keeps each test’s fixture behavior explicit without changing its assertions.

As per coding guidelines, “New function parameters are required, not optional.”

Also applies to: 63-63

Source: Coding guidelines

Review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 59a44c6a-0fcc-48e2-96e6-2a49a551f9e9
Commits

Reviewing files that changed from the base of the PR and between 7213ebd and 027cea2.

Files ignored due to path filters: 22

  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_fillWidth_50_-_fill_narrower_than_track_within_ring_width.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_in_non-square_element_-_stays_circular.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_marker_-_arc_bead_following_the_curve.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_multiColour_false_value_75_-_single_color_active.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_partial_arc_-_flat_track_ends_when_roundedEnds_false.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_partial_arc_-_gap_at_the_bottom_270_arc.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_partial_arc_-_rounded_track_ends_follow_roundedEnds.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_reverse_true_value_75_-_counter-clockwise.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_roundedEnds_false_value_75_-_flat_ends.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_thick_ringWidth_40.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_thin_ringWidth_8.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_trackWidth_50_-_track_narrower_than_fill_within_ring_width.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_value_0_-_inactive_arc_only_dark_bg_makes_it_visible.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_value_100_-_fully_active.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_value_33_-_one_color_within_first_segment.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_value_50_-_midway_through_first_segment.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_value_66_-_exactly_at_first_segment_boundary.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_value_75_-_crossing_into_yellow_segment.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_value_75_dimmed_inactive_-_both_halves_clearly_visible.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_gauge_element_ring_value_90_-_crossing_into_red_segment.png is excluded by !**/*.png
  • companion/test/Graphics/__snapshots__/GraphicsLayeredButtonRenderer_image_properties_image_with_rotation.png is excluded by !**/*.png
  • companion/test/Instance/__snapshots__/Definitions.test.ts.snap is excluded by !**/*.snap
Files selected for processing: 75

  • companion/lib/Cloud/Region.ts
  • companion/lib/Controls/ActionSetsTrpcRouter.ts
  • companion/lib/Controls/ButtonPressEvents.ts
  • companion/lib/Controls/ControlBase.ts
  • companion/lib/Controls/ControlDependencies.ts
  • companion/lib/Controls/ControlStore.ts
  • companion/lib/Controls/ControlTypes/Button/Base.ts
  • companion/lib/Controls/ControlTypes/Button/Link.ts
  • companion/lib/Controls/ControlTypes/Button/Reference.ts
  • companion/lib/Controls/ControlTypes/CustomAction.ts
  • companion/lib/Controls/ControlTypes/CustomFeedback.ts
  • companion/lib/Controls/ControlTypes/ExpressionVariable.ts
  • companion/lib/Controls/ControlTypes/Page.ts
  • companion/lib/Controls/ControlTypes/PageButton.ts
  • companion/lib/Controls/ControlTypes/PageDown.ts
  • companion/lib/Controls/ControlTypes/PageNumber.ts
  • companion/lib/Controls/ControlTypes/PageUp.ts
  • companion/lib/Controls/ControlTypes/Triggers/Trigger.ts
  • companion/lib/Controls/Controller.ts
  • companion/lib/Controls/ControlsTrpcRouter.ts
  • companion/lib/Controls/Entities/ControlActionSetAndStepsManager.ts
  • companion/lib/Controls/Entities/EntityListPoolButton.ts
  • companion/lib/Controls/Entities/EntityListPoolEditingMixin.ts
  • companion/lib/Controls/IControlStore.ts
  • companion/lib/Instance/Connection/PresetsLegacy.ts
  • companion/lib/Instance/Connection/Thread/Presets.ts
  • companion/lib/Instance/Surface/IpcTypes.ts
  • companion/lib/Instance/Surface/Thread/Entrypoint.ts
  • companion/lib/Internal/Controls.ts
  • companion/lib/Link/Controller.ts
  • companion/lib/Service/Satellite/SatelliteApi.ts
  • companion/lib/Service/ServiceApi.ts
  • companion/lib/Surface/Handler.ts
  • companion/lib/Surface/IP/Satellite.ts
  • companion/lib/Surface/PluginPanel.ts
  • companion/lib/Surface/Types.ts
  • companion/test/Controls/ActionSetsTrpcRouter.controlEvents.test.ts
  • companion/test/Controls/ButtonPressEvents.test.ts
  • companion/test/Controls/ControlStore.test.ts
  • companion/test/Controls/ControlTypes/Button/Layered.test.ts
  • companion/test/Controls/ControlTypes/Button/Link.test.ts
  • companion/test/Controls/ControlTypes/Button/Preset.test.ts
  • companion/test/Controls/ControlTypes/Button/PresetReference.test.ts
  • companion/test/Controls/ControlTypes/Button/PressEventsRuntime.test.ts
  • companion/test/Controls/ControlTypes/Button/Reference.test.ts
  • companion/test/Controls/ControlsTrpcRouter.test.ts
  • companion/test/Controls/Entities/EntityListPool.controlEvents.test.ts
  • companion/test/Controls/Entities/EntityListPool.test.ts
  • companion/test/ImportExport/RoundTrip.test.ts
  • companion/test/Instance/Definitions.test.ts
  • companion/test/Internal/Controls.test.ts
  • companion/test/Link/Controller.test.ts
  • companion/test/Link/EndToEnd.test.ts
  • companion/test/Service/Satellite/SatelliteApi.test.ts
  • companion/test/Surface/ControlEvents.test.ts
  • companion/test/Surface/ControllerLayouts.test.ts
  • companion/test/integration/TestApp.ts
  • docs/user-guide/3_config/program/buttons/creating/actions.md
  • satellite-protocol/assets/satellite-api-features.json
  • satellite-protocol/lib/constants.ts
  • satellite-protocol/lib/versions.ts
  • shared-lib/lib/Model/ActionModel.ts
  • shared-lib/lib/Model/ButtonModel.ts
  • shared-lib/lib/Model/Export/__tests__/snapshots/v17.json
  • shared-lib/lib/__tests__/ControlModels.test.ts
  • webui/src/Buttons/EditButton/ActionSetPropertiesButton.tsx
  • webui/src/Buttons/EditButton/ControlActionStepTab.tsx
  • webui/src/Buttons/EditButton/ControlEvents.stories.tsx
  • webui/src/Buttons/EditButton/EditActionsRelease.tsx
  • webui/src/Buttons/EditButton/EditDurationGroupPropertiesModal.tsx
  • webui/src/Buttons/EditButton/SurfaceCueText.ts
  • webui/src/Buttons/EditButton/__tests__/ActionSetPropertiesButton.test.tsx
  • webui/src/Buttons/EditButton/__tests__/ButtonEditorTabs.test.tsx
  • webui/src/Buttons/EditButton/__tests__/EditActionsRelease.test.tsx
  • webui/src/Services/Controls/ControlActionStepsAndSetsService.ts

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 6 remain after this review.

const disabledIndex = disabledSets.indexOf(setId)
if (enabled === (disabledIndex === -1)) return true

if (enabled) disabledSets.splice(disabledIndex, 1)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Clear all matching entries when enabling a cue.

The shared schema accepts controlEventsDisabledSets: ['down', 'down'], and loadStorage preserves that array. Enabling down removes only one entry and returns true. The runtime predicate still returns false, so the cue remains disabled.

Please remove all matching entries when enabling the cue, or normalize duplicates before storing the options. Removing all matches also preserves the ordinary single-entry case.

@Julusian

Copy link
Copy Markdown
Member

The lib is published as a prerelease, and that has been pulled into main, so if this can be rebased onto that it can be marked as ready. Hopefully not much more to get this merged

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants