Migrate drag interactions to AppKit gesture recognizers for Sidecar input

[ryanwaite]

Description

Migrate ZoomIt’s primary drag interactions from AppKit mouse-event overrides to gesture recognizers.

Beginning with macOS 27, touch and Apple Pencil input delivered through Sidecar is available exclusively through gesture recognizers. ZoomIt currently handles these interactions through mouseDown, mouseDragged, and mouseUp, so drawing and selection do not work with the new Sidecar input path.

Mouse behavior must continue to work through the same interaction logic on macOS 14 and later.

Current Behavior

ZoomIt’s primary drag interactions rely on AppKit mouse-event overrides. As a result, Sidecar touch and Apple Pencil input on macOS 27 cannot draw annotations or select regions.

Affected components:

First slice

• ZoomCanvasView
• SnipSelectionView

Follow-up slice

• VideoTimelineView
• DraggableWebcamView

The issue is complete when all four components use gesture-based primary dragging. The first slice may land independently.

Expected Behavior

Zoom canvas

Preserve the existing interaction state machine:

• The first click in static zoom arms drawing mode without creating a stroke.
• Subsequent drags create pen, highlighter, line, arrow, rectangle, or ellipse annotations.
• Modifier keys are evaluated at the start of a stroke:
  • Shift: line
  • Control: rectangle
  • Control+Shift: arrow
  • Tab: ellipse
• Region snipping within the canvas continues to use a three-point minimum selection.
• Typing placement, live-zoom behavior, right-click exit, cursor handling, and coordinate conversion remain unchanged.
• Freehand drawing begins at the initial contact point and does not acquire a delayed or missing first segment.

Snip selector

• Dragging creates the same normalized selection rectangle in any direction.
• Drags smaller than 3×3 points are treated as empty selections.
• Escape and gesture cancellation dismiss the selection without producing a capture.
• Snip, OCR, panorama, DemoMirror region selection, and region recording retain their existing behavior because they share SnipSelectionView.

Video timeline

• Scrubbing, trim grips, and pending-delete grips retain their existing hit-testing and nearest-handle behavior.
• Ending or cancelling a gesture always clears the active drag state.
• Existing right-button delete-range behavior remains available to mouse users.

Webcam overlay

• Dragging preserves the point grabbed within the overlay.
• The overlay window and recorded webcam frame remain synchronized while it moves.

Steps to Reproduce

1. Run ZoomIt on macOS 27 with an iPad connected through Sidecar.
2. Attempt to draw on ZoomCanvasView or drag a region in SnipSelectionView using Sidecar touch or Apple Pencil.
3. Observe that the interaction is not delivered through the existing mouseDown, mouseDragged, and mouseUp path.

Product Version

Current implementation on main.

macOS Version

macOS 27.

Additional Context

Implementation guidance

• Attach recognizers directly to the affected views, as Sidecar gestures are resolved through the view hierarchy rather than the responder chain.
• Route mouse and Sidecar input into shared begin/change/end/cancel handlers. Avoid maintaining independent business logic in recognizer callbacks and mouse overrides.
• Represent cancellation explicitly. In particular, AnnotationController needs a way to discard an in-progress annotation rather than committing a partial stroke.
• Ensure a single physical mouse interaction cannot be processed by both the legacy event path and the recognizer path.
• Gesture recognizers are available on the project’s macOS 14 deployment target; the behavior itself should not require a macOS 27 availability branch.

Out of scope

• Pressure-sensitive pen widths or other Apple Pencil-specific styling
• Changes to keyboard shortcuts, scroll-wheel zoom, or secondary-click commands
• New annotation tools or user-interface changes
• Instant Replay, capture-source selection, App Intents, or appearance modernization

Acceptance criteria

• Apple Pencil and touch input through Sidecar can draw and select regions on macOS 27.
• Mouse and trackpad behavior remains unchanged on supported macOS versions.
• Every interaction produces one begin/change/end sequence without duplicate strokes or callbacks.
• Modifier-key shapes, three-point selection thresholds, timeline hit testing, and webcam positioning match current behavior.
• Cancelled or interrupted gestures leave no partial annotation, selection, or stuck drag state.
• Existing self-tests pass, with new state-level coverage for begin, change, end, and cancellation.
• Manual coverage includes mouse, trackpad, Sidecar touch, and Apple Pencil.

Reference

Apple TN3212: Adopting gesture recognizers for Sidecar touch support