Skip to content

fix(linux/wlr): map absolute mouse input on rotated outputs - #6020

Open
ahmedkall wants to merge 6 commits into
SunshineStream:masterfrom
ahmedkall:fix/wlr-rotated-abs-mouse
Open

ahmedkall wants to merge 6 commits into
SunshineStream:masterfrom
ahmedkall:fix/wlr-rotated-abs-mouse

Conversation

@ahmedkall

@ahmedkall ahmedkall commented Oct 30, 2026 •

Copy link
Copy Markdown

Description

With wlr capture, absolute mouse input lands in the wrong place when the streamed output is rotated. Moonlight on Android sends finger touch as absolute mouse, so taps on a portrait tablet miss.

A rotated output is still captured in its native orientation. For example, a 3000x1920 panel at scale 2 turned 90° streams as 3000x1920, but its logical size is 960x1500. Sunshine ignores the transform in two places:

  • make_port gets scalar_tpcoords = min(3000/960, 1920/1500) = 1.28 instead of 2.
  • The point is never turned into the output's orientation.

At 90° or 270°, a tap at stream (x, y) lands at (0.78x, 0.78y). The right part of the stream ends up on the neighbouring output. At 180° every tap is mirrored.

Changes

  • wl_geometry stores the wl_output transform. Physical geometry is still ignored.
  • touch_port_t and display_t carry it.
  • wlgrab copies it at init (through wl::apply_monitor_geometry()) and returns reinit when it changes, so the touch port follows a rotation without restarting the stream.
  • video::touch_port_scale() swaps the logical size for odd transforms. make_port uses it.
  • The absolute-mouse handler turns the point with a small rotate_to_output() before platf::abs_mouse.

Pen and touch packets are left alone. The compositor maps those devices to the output and applies the rotation itself, so rotating them here too would rotate them twice. With transform 0 the maths reduces to what it was.

Screenshot

Issues Fixed or Closed

Roadmap Issues

Testing

Added

  • MapsAbsoluteMouseOntoRotatedOutputs in test_input.cpp. It sends an absolute packet through passthrough to the fake libvirtualhid backend, via a new send_absolute_mouse_packet test hook, for transforms 0-3.
  • ScalesRotatedOutputsByTheirRotatedLogicalSize in test_video.cpp.
  • An extra transform assertion in ListenerIgnoresPhysicalGeometryScaleAndDone.
  • ReinitializesWhenTheOutputTurnsWithoutChangingSize in test_wayland.cpp. The reinit decision is now wl::capture_needs_reinit(). A 90° to 270° turn keeps both sizes, so only the transform check catches it.
  • RebuildsTheTouchPortForARotatedDisplay in test_input.cpp. make_port() on a display at transform 3 gives a port with that transform and scale 2. Fed into the fake backend, it maps (300,200) to (100,1350).
  • FollowsAWlrOutputThroughARotationOnlyReinit in test_input.cpp (Wayland builds). This walks the capture lifecycle through the same functions wlr_t::init() and acquire_frame() use:
    • A wl::monitor_t set up through its listeners at transform 1.
    • apply_monitor_geometry() and desktop_geometry_t, as in init(), then make_port(). A tap lands at (860,150).
    • A same-sized frame doesn't ask for a reinit.
    • A geometry event to transform 3 makes capture_needs_reinit() ask for one.
    • After re-initializing, the port has transform 3 and scale 2, and the same tap lands at (100,1350).

I checked that each one fails with its part of the fix taken out: the transform copy in apply_monitor_geometry, the transform term in capture_needs_reinit, the transform in make_port's port and scale, rotate_to_output, and wl_geometry storing it.

Checked

  • Rebased onto current master (37ee7d5), keeping fix(linux): target absolute input at the captured monitor #6055's output_name next to the new transform. I built with -D BUILD_TESTS=ON -D BUILD_WERROR=ON (Arch, GCC, no CUDA): test_sunshine gives 950 passed, 20 skipped, 0 failed.
  • In both runs platf_deinit asserts at the end, because the container has no display or audio session.
  • uv run --locked lb-update-clang-format leaves the diff unchanged. clang-format 23.1.0 with --dry-run --Werror and LineEnding: LF passes, as do git diff --check and the trailing-space and final-newline checks.
  • The dockle documentation build passes in strict mode.
  • On a device: Hyprland 0.56.2, a 3000x1920 headless output at scale 2, Moonlight Android over USB.
    • I injected taps with adb shell input touchscreen tap and read hyprctl cursorpos.
    • The expected positions come from screenshots of the client (adb exec-out screencap) with the cursor at known spots.
    • Pen taps (input stylus tap) were a second reference.
    • I changed the transform mid-stream:
Transform Stream points Expected Finger and pen
0 (300,200) (2700,1700) (1500,960) (150,100) (1350,850) (750,480) within 1 px
1 same (860,150) (110,1350) (480,750) within 1 px
3 same (100,1350) (850,150) (480,750) within 1 px

Before the fix, at transform 3 those taps landed at (233,155), (2109,1328) and (1171,749).

Not run / limits

  • I only tested on Hyprland, and transform 2 only in the unit test.
  • Automated coverage stops short of the real wlr_t::init() and acquire_frame(). Driving those needs a compositor (wl_output, xdg-output, screencopy), and there's no fixture for that. The lifecycle test runs the same functions they call, but it wouldn't notice if init() stopped calling apply_monitor_geometry() or acquire_frame() stopped calling capture_needs_reinit().
  • Hyprland 0.56.2 applies a rotation-only monitor change as a "soft" rule and sends no wl_output events for it. Sunshine picks up the new transform at the next stream start, or once the output is reconfigured another way. I worked around this locally with a brief refresh-rate change.
  • The device runs above were before rebasing onto fix(linux): target absolute input at the captured monitor #6055. Back then, on master with two outputs, the first absolute move after a touch port change sometimes landed on the other output, even though the coordinates Sunshine computed were right. fix(linux): target absolute input at the captured monitor #6055 targets that; I haven't re-run the device test on top of it.

Type of Change

  • feat: New feature (non-breaking change which adds functionality)
  • fix: Bug fix (non-breaking change which fixes an issue)
  • docs: Documentation only changes
  • style: Changes that do not affect the meaning of the code (white-space, formatting, missing semicolons, etc.)
  • refactor: Code change that neither fixes a bug nor adds a feature
  • perf: Code change that improves performance
  • test: Adding missing tests or correcting existing tests
  • build: Changes that affect the build system or external dependencies
  • ci: Changes to CI configuration files and scripts
  • chore: Other changes that don't modify src or test files
  • revert: Reverts a previous commit
  • BREAKING CHANGE: Introduces a breaking change (can be combined with any type above)

Checklist

  • Code follows the style guidelines of this project
  • Code has been self-reviewed
  • Code has been commented, particularly in hard-to-understand areas
  • Code docstring/documentation-blocks for new or existing methods/components have been added or updated
  • Tests have been added or updated for new or modified functionality, including regression tests for bug fixes

AI Usage

See our AI usage policy.

  • AI tools were used while preparing this PR

An AI agent found the cause, wrote the fix and tests, ran the builds and device tests, and drafted this description. I reviewed the diff and the results.

This pull request was generated by Claude Code with Claude Opus 5.5, medium thinking level

A rotated output is captured in its native orientation, so absolute mouse
input (touch from Moonlight Android) needs the output transform: store it
from wl_output, scale the touch port by the rotated logical size, and turn
the point into the output's orientation. Capture reinitializes when the
transform changes so the touch port follows a rotation mid-stream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ahmedkall

Copy link
Copy Markdown
Author

This pull request was generated by Claude Code with Claude Opus 5.5, medium thinking level

…wayland

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@ReenigneArcher ReenigneArcher left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed current head 119b25da0820eb73e2eb0b47c9b35220a823e888, including its clean merge with latest master (005ddb7c016ee754add3e16f1e9c452045a2cf7d). The original rotation/scaling problem remains on that target, and the mapping for transforms 0–3 looks consistent. Requesting changes for the missing regression coverage of rotation-only capture reinitialization described inline.

Local verification passed: git diff --check, clang-format 23.1.0 in dry-run/error mode for every changed file, macOS syntax-only compilation of src/input.cpp, src/video.cpp, tests/unit/test_input.cpp, and tests/unit/test_video.cpp using the existing project compile flags, plus a standalone harness using the PR's extracted mapping/scale functions (12 coordinate cases and 4 scale checks). I did not run the project test executable or a Linux/Wayland build.

Maintainer verification limit: CI, common lint, CodeQL, and Build GH-Pages await workflow approval. Sonar and the Sunshine documentation status passed. The separate GH Pages Read the Docs build ended after 900 seconds without a completed build command in its log; that status is not a code finding.

Review completed by Codex | Model: gpt-6.1-sol | Thinking: xhigh

Comment thread src/platform/linux/wlgrab.cpp Outdated
current_frame->sd.width != width ||
current_frame->sd.height != height
current_frame->sd.height != height ||
monitor->viewport.transform != transform // rotated: re-read the geometry for absolute input

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Cover rotation-only capture reinitialization with a regression test

The new tests exercise the listener, scale helper, and an input port whose transform and scale are supplied by the test, but none exercise wlr_t::init() / acquire_frame() or the resulting make_port() update. Removing this condition would leave those tests passing. For a 90°→270° change, both the native and logical dimensions stay the same, so the older size checks cannot refresh the port; absolute input would keep using the previous rotation until the capture is otherwise restarted.

Please add a regression case in tests/unit/platform/linux/test_wayland.cpp (with a capture test seam or fake protocol fixture as needed): initialize capture at transform 1 with a 3000×1920 native / 960×1500 logical output, deliver a geometry update to transform 3 and a same-sized ready frame, assert capture_e::reinit, then verify that the recreated display/port carries transform 3 and scale 2. Also assert that an unchanged transform returns ok. Feeding the refreshed port into the existing fake-input fixture should map (300,200) to (100,1350), rather than the old (860,150). This covers the new stream-lifetime behavior required by the repository's regression-test rule. The outlined capture test has not been run locally.

Review completed by Codex | Model: gpt-6.1-sol | Thinking: xhigh

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, good catch. Deleting that line did leave the tests passing.

Done in 6403dbf. There's no fake compositor that can drive wlr_t::init() / acquire_frame(), and adding one felt like more than this PR needs. So I did it the way #5883 tested its capture decisions, and split it into two tests:

  • The reinit decision is now wl::capture_needs_reinit(), and acquire_frame() calls it. ReinitializesWhenTheOutputTurnsWithoutChangingSize covers 90° → 270° at the same 3000x1920 / 960x1500 sizes (reinit), an unchanged transform (no reinit), and a size change (reinit).
  • make_port() is declared in video.h. RebuildsTheTouchPortForARotatedDisplay builds the port from a display at transform 3 and checks it carries transform 3 and scale 2. It then feeds that port to the fake input backend, where (300,200) lands at (100,1350).

I checked each one against the code it covers:

  • Dropping the transform term from capture_needs_reinit fails the first test.
  • Making make_port pass transform 0, or ignore the transform for the scale, fails the second.

With the real code, test_sunshine gives 854 passed / 0 failed (BUILD_WERROR=ON), and lb-update-clang-format changes nothing. Merged with current master (0ddc73d), it gives 902 / 0. The SHM fallback from #4946 goes through the same acquire_frame(), so it's covered too.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, these additions cover the production reinit predicate and make_port()/input mapping, and the mutation checks are useful.

On 946db93, the remaining gap from the original request is the lifecycle: initialize at transform 1 → receive a geometry update to transform 3 and a same-sized ready frame → return capture_e::reinit → recreate the display/port with transform 3 and scale 2. The new tests call the predicate directly and construct a display already at transform 3, so bypassing the predicate in acquire_frame() or failing to refresh the transform in init() could still leave them passing. Sharing acquire_frame() between SHM and DMA-BUF does not by itself exercise that wiring.

I understand there is no fake compositor fixture, and #5883 used the same helper-test approach. Could a focused capture test seam cover this transition? If that requires broader infrastructure, please explicitly note this lifecycle as an automated coverage limitation. I haven't found a new runtime defect here; this is the part of the original regression request that remains untested.

…build

Move the reinit decision into wl::capture_needs_reinit() so it can be
tested: a 90 to 270 degree turn keeps both sizes, so only the transform
check catches it. Declare video::make_port() and test that a display
rebuilt at transform 3 gives a port with that transform and scale 2,
which maps a tap at (300,200) to (100,1350).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…bs-mouse

# Conflicts:
#	tests/unit/test_input.cpp
@sonarqubecloud

Copy link
Copy Markdown

@ReenigneArcher ReenigneArcher left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed head 946db93f1df7800307179da91a6390475739513c, comparing the follow-up changes with 119b25da and inspecting the complete PR diff against current master (844af692). I found no new runtime defect. The tests added in 6403dbf4 cover the production reinit predicate and make_port()/fake-input mapping. The outstanding capture-lifecycle regression coverage remains as described in the existing thread: the tests still bypass initialization at transform 1, receipt of transform 3 with unchanged frame dimensions, the actual acquire_frame() reinit result, and recreation of the display/port. There have been no further commits since that follow-up discussion, so I have not duplicated the inline finding. Keeping changes requested for that unmet regression-test requirement.

Merge status: GitHub reports mergeable=false / dirty, and local git merge-tree confirms conflicts in src/platform/common.h and src/video.cpp. Please rebase onto current master and resolve these before merging. Preserve the connector identity/output_name propagation from #6055 (4062cb79) alongside this PR's transform metadata. That merged fix addresses input targeting; current target source still ignores the output transform and retains the rotated-size scaling problem, so it does not supersede this PR's rotation fix.

Local verification passed: git diff --check, clang-format 23.1.0 dry-run/error checks on all changed files, syntax-only compilation of src/input.cpp, src/video.cpp, tests/unit/test_input.cpp, and tests/unit/test_video.cpp using the existing macOS build flags/include environment, and a standalone harness using extracted PR functions (4 reinit-predicate cases and 48 port/coordinate cases including letterboxing and negative offsets). These are not a project test-suite run or a full capture-lifecycle test; I did not run test_sunshine or a Linux/Wayland build.

Hosted verification remains limited: CI, CodeQL, common lint, and Build GH-Pages await maintainer approval. Sonar and the Sunshine documentation status passed. The separate GH Pages documentation status failed, but its linked public log endpoint returns no matching build, so I cannot attribute that failure to this change.

Review completed by Codex | Model: gpt-6.1-sol | Thinking: xhigh

Merge master, keeping SunshineStream#6055's output_name next to the transform. wlr_t::init()
now copies the monitor geometry through wl::apply_monitor_geometry(), and a test
walks a rotation-only reinit through the functions init() and acquire_frame()
use, from transform 1 to transform 3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@codecov

codecov Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Bundle Report

Changes will increase total bundle size by 1.53kB (0.15%) ⬆️. This is within the configured threshold ✅

Detailed changes
Bundle name Size Change
sunshine-esm 1.0MB 1.53kB (0.15%) ⬆️

Affected Assets, Files, and Routes:

view changes for bundle: sunshine-esm

Assets Changed:

Asset Name Size Change Total Size Change (%)
images/logo-*.svg -630 bytes 1.51kB -29.47%
images/logo-*.svg (New) 2.17kB 2.17kB 100.0% 🚀

@codecov

codecov Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.45283% with 4 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (master@aed346b). Learn more about missing BASE report.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/platform/linux/wlgrab.cpp 75.00% 4 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff            @@
##             master    #6020   +/-   ##
=========================================
  Coverage          ?   51.27%           
=========================================
  Files             ?      148           
  Lines             ?    32585           
  Branches          ?    13743           
=========================================
  Hits              ?    16708           
  Misses            ?    14558           
  Partials          ?     1319           
Flag Coverage Δ
Alpine-aarch64 34.73% <91.66%> (?)
FreeBSD-amd64 35.61% <87.75%> (?)
Homebrew-macos-15 49.11% <100.00%> (?)
Homebrew-macos-26 49.48% <100.00%> (?)
Homebrew-ubuntu-24.04 38.09% <93.75%> (?)
Linux-AppImage-aarch64 40.79% <91.66%> (?)
Linux-AppImage-x86_64 41.04% <93.75%> (?)
PowerShell 100.00% <ø> (?)
Web-UI 57.76% <ø> (?)
Windows-AMD64 41.52% <100.00%> (?)
Windows-ARM64 35.44% <100.00%> (?)
macOS-arm64 41.87% <96.66%> (?)
macOS-x86_64 40.76% <96.66%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/input.cpp 77.30% <100.00%> (ø)
src/input.h 66.66% <ø> (ø)
src/platform/common.h 72.43% <100.00%> (ø)
src/platform/linux/wayland.h 81.63% <100.00%> (ø)
src/video.cpp 58.18% <100.00%> (ø)
src/video.h 76.55% <100.00%> (ø)
src/platform/linux/wlgrab.cpp 31.61% <75.00%> (ø)

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aed346b...a1fd9f7. Read the comment docs.

@github-actions

Copy link
Copy Markdown
Last Updated 2026-11-19 06:30:49 UTC
Source Run CI Run #6849
Commit a1fd9f75316929a257be6b4089ce1a9646e6ab2a

Screenshot Comparison

PR #6020 screenshots vs screenshots baseline.

Matrix: AppImage-aarch64

Image Baseline PR
sunshine_tray_initial.png
sunshine_tray_menu.png
sunshine_tray_pairing_request.png
sunshine_tray_paused.png
sunshine_tray_stopped.png
sunshine_tray_streaming.png
sunshine_tray_tooltip.png

Matrix: AppImage-x86_64

Image Baseline PR
sunshine_tray_initial.png
sunshine_tray_menu.png
sunshine_tray_pairing_request.png
sunshine_tray_paused.png
sunshine_tray_stopped.png
sunshine_tray_streaming.png
sunshine_tray_tooltip.png

Matrix: Windows-AMD64

Image Baseline PR
sunshine_tray_initial.png
sunshine_tray_menu.png
sunshine_tray_pairing_request.png
sunshine_tray_paused.png
sunshine_tray_stopped.png
sunshine_tray_streaming.png
sunshine_tray_tooltip.png

Matrix: Windows-ARM64

Image Baseline PR
sunshine_tray_initial.png
sunshine_tray_menu.png
sunshine_tray_pairing_request.png
sunshine_tray_paused.png
sunshine_tray_stopped.png
sunshine_tray_streaming.png
sunshine_tray_tooltip.png

Matrix: macOS-arm64

Image Baseline PR
sunshine_tray_initial.png
sunshine_tray_menu.png
sunshine_tray_pairing_request.png
sunshine_tray_paused.png
sunshine_tray_stopped.png
sunshine_tray_streaming.png
sunshine_tray_tooltip.png

Matrix: macOS-x86_64

Image Baseline PR
sunshine_tray_initial.png
sunshine_tray_menu.png
sunshine_tray_pairing_request.png
sunshine_tray_paused.png
sunshine_tray_stopped.png
sunshine_tray_streaming.png
sunshine_tray_tooltip.png

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants