Skip to content

feat(foveation): support external combined gaze over OSC - #3402

Merged
zmerp merged 4 commits into
alvr-org:masterfrom
JellyfishKnight:external-gaze-input
Sep 29, 2026
Merged

zmerp merged 4 commits into
alvr-org:masterfrom
JellyfishKnight:external-gaze-input

Conversation

@JellyfishKnight

Copy link
Copy Markdown
Contributor

Summary

this pr follows #3384 #3385 #3388 #3394

Add external combined-gaze input over OSC/UDP, reusing the existing eye-tracked foveation pipeline.

  • Add an External OSC input source with a configurable localhost UDP port (default: 9945).

  • Accept head-local XYZW quaternions at /alvr/eye/combined/v1, alongside sequence, validity and sample-age fields.

  • Validate incoming observations and bound receive processing to avoid blocking head/controller tracking.

  • Preserve the existing gaze filtering, per-eye projection, encoder alignment and per-frame center metadata. Missing input follows the existing hold and static fallback behavior.

  • This PR respects our AI policy: I completely understand and "own" the coding decisions made, and can discuss them without relying on AI copy-pasted responses.

  • This PR's description contains a human-written summary of the content of the code.

  • Testing:

    • formatting, Windows CI Clippy and all 4 session tests passed.
    • All 29 local OSC regression tests passed on Windows and Ubuntu WSL.
    • windows / ubuntu -> pico4 pro / quest pro / pico4 & self-built eye tracking stream with foveation tested

Comment thread alvr/session/src/settings.rs Outdated
Comment thread alvr/server_core/src/tracking/face.rs Outdated
Comment thread alvr/session/src/settings.rs Outdated

@zmerp zmerp left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last thing

Comment thread alvr/session/src/settings.rs Outdated
Comment on lines +910 to +912
pub enum FaceTrackingSocialPresenceSinkConfig {
#[schema(strings(display_name = "Disabled"))]
None,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This None variant is not needed anymore

@JellyfishKnight JellyfishKnight Sep 29, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But if we delete this now, we won’t be able to enable eye-tracking foveated rendering while disabling social eyes forwarding

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah true. But let's have social_presence: Option<FaceTrackingSocialPresenceSinkConfig> and remove the None variant.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@zmerp
zmerp added this pull request to the merge queue Sep 29, 2026
@zmerp

zmerp commented Sep 29, 2026

Copy link
Copy Markdown
Member

I realized that it should have been Switch instead of Option... I didn't want to make you correct again, I'll fix it post-merge

@JellyfishKnight

Copy link
Copy Markdown
Contributor Author

I realized that it should have been Switch instead of Option... I didn't want to make you correct again, I'll fix it post-merge

that's true, I forgot it too... anyways thanks to your detailed review for these prs, that really do helped me a lot cause it's actually my first time contribute to a open source project! have a good night.

Merged via the queue into alvr-org:master with commit fc80de2 Sep 29, 2026
12 checks passed
@SandroHc

This comment was marked as off-topic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants