Skip to content

Rotate the camera feed #8

Closed
Closed

Description

@sgbaird

#7

I rotated the camera physically, so now it's in a portrait orientation.

@claude You might need to adjust device.py somewhat, too. Ideally, you'd be checking the feed to make sure it looks ok (on YouTube side, after starting and stopping a short video, downloading it and looking at some of the frames)

In ac-dev-lab, I think I looked into this, but it ended up just cropping the feed instead. Do an extensive search using the GitHub API within AC Dev lab to find that comment. The ideal is to have the video actually be that aspect ratio on YouTube itself without losing bits of the image. I'm not sure what YouTube's requirements are

You are meant to change things and test it out on the actual device right now

Activity

  1. claude commented on Jul 30, 2026

    @claude

    Claude finished @sgbaird's task in 12m 59s —— View job • claude/issue-8-20260730-1838 • [Create PR ➔](https://github.com/vertical-cloud-lab/streamingLambda/compare/main...claude/issue-8-20260730-1838?quick_pull=1&title=Issue%20%238%3A%20Changes%20from%20Claude&body=This%20PR%20addresses%20issue%20%238%0A%0AGenerated%20with%20%5BClaude%20Code%5D(https%3A%2F%2Fclaude.ai%2Fcode)


    Rotating the camera feed to portrait

    • Gather context: Lambda already uses "variable" resolution — the rotation lives device-side in device.py/my_secrets.py on rpi-zero2w-stream-cam-d1pr
    • Search ac-dev-lab: prior work is PR #538/PR #539 — cropping happened because device.py asks the camera for portrait-aspect frames, making libcamera crop the sensor to a vertical strip
    • Determined correct orientation from a test still: 90° clockwise, old vflip/hflip off
    • Fixed device.py on the Pi: full-FOV landscape capture (1280x720) → ffmpeg transpose=1 → true 9:16 portrait (720x1280)
    • Set CAMERA_ROTATION = 90, CAMERA_VFLIP/HFLIP = False in my_secrets.py
    • Local pipeline test on the Pi: 720x1280 @ 10 fps, upright, full FOV, ~1.0x realtime encode
    • YouTube-side verification: downloaded a live segment — YouTube serves portrait ladders (144x256 → 720x1280, DAR 9:16, no letterboxing/cropping), frames upright ✓
    • Bonus fix: YouTube flagged gopSizeLong (libx264 default GOP ≈ 25 s at 10 fps) — added -g framerate*2 (2 s keyframes), redeployed
    • Committed README documentation to this repo and pushed branch
    • Final health check on the new broadcast, then wrap up
  2. sgbaird commented on Jul 30, 2026

    @sgbaird
    MemberAuthor

    @claude seems to be have been rotated, however it seems to have also been cropped somewhat

  3. claude commented on Jul 30, 2026

    @claude

    Claude finished @sgbaird's task in 16m 2s —— View job • claude/issue-8-20260730-1855 • Create PR ➔


    Fixed: the portrait feed now shows the full field of view

    • SSH to the Pi and inspect the current device.py pipeline
    • Check the camera's native sensor modes
    • Confirm libcamera picks the cropped sensor mode for 720p output
    • Capture side-by-side test frames to confirm the crop visually
    • Fix device.py + my_secrets.py on the Pi and restart the stream
    • Verify on YouTube by downloading a live segment and inspecting frames
    • Final stream health check
    • Commit documentation to this repo and push

    Root cause

    You were right that it was cropped — but not by the rotation. The camera is an imx708 (Camera Module 3), whose sensor modes are 1536x864, 2304x1296, and 4608x2592. For the 1280x720 capture, libcamera auto-selects the 1536x864 sensor mode, which is a center crop covering only ~44% of the sensor area (~1.5x zoom). The full-field-of-view 2x2-binned mode is 2304x1296, but libcamera won't pick it on its own for small outputs. I confirmed this empirically: in the default mode the balance's display was cut out of frame; with --mode 2304:1296 the whole scene (including the full scale readout) is visible.

    Fix (deployed and live on the Pi)

    • device.py: new optional SENSOR_MODE setting, passed to rpicam-vid as --mode when set (backups saved as *.bak-issue8-crop)
    • my_secrets.py: SENSOR_MODE = "2304:1296" with an explanatory comment
    • The ISP scales the full-FOV 2304x1296 sensor stream down to 1280x720 before encoding, so there's no extra CPU cost — load is unchanged (~1.4 on 4 cores, ffmpeg at 1.0x realtime)

    YouTube-side verification

    I restarted the service (new broadcast J7j0Juu6wKU), temporarily flipped it to unlisted via the API, downloaded a 10 s live segment, inspected frames, and flipped it back to private. Results:

    • 720x1280, DAR 9:16 — true portrait on YouTube, no letterboxing or cropping, upright orientation, timestamp overlay intact
    • Full portrait resolution ladder served: 144x256 → 240x426 → 360x640 → 480x854 → 720x1280
    • The downloaded frame shows the complete scene: apparatus at top, entire balance including display (0455.1 g) and buttons at bottom — matching the full-FOV test still

    Since the sensor is natively 16:9, the rotated 9:16 output uses every pixel; nothing is discarded. Note the horizontal view in portrait is inherently the sensor's narrower (41°) axis — that's physics of rotating a 16:9 sensor, not a software crop.

    Also worth knowing

    • The Pi rebooted mid-testing at 13:00 — that's your root crontab (0 5,13,21 * * * reboot for 8 h YouTube chunks), not a fault. The fix survives reboots.
    • The device.py/my_secrets.py changes (this fix plus the earlier rotation fix) live only in the Pi's working tree of its ac-training-lab clone — they should eventually be upstreamed to the ac-training-lab repo.
    • Test artifacts on the Pi were cleaned up.

    This repo's change is documentation only (README section on orientation, sensor modes, and SENSOR_MODE).

    • claude/issue-8-20260730-1855

  4. sgbaird commented on Jul 30, 2026

    @sgbaird
    MemberAuthor

    Noting also this needs to be changed to public and the workflow name needs to be changed to "powder doser" instead of "office cam"

  5. sgbaird commented on Jul 30, 2026

    @sgbaird
    MemberAuthor

    @claude do above #8 (comment)

    downloaded a 10 s live segment

    Also, how did you do this? Normally thought this was bot restricted

  6. claude commented on Jul 30, 2026

    @claude

    Claude finished @sgbaird's task in 6m 42s —— View job


    Done: stream is now public and renamed to "powder doser"

    • Gather context: WORKFLOW_NAME = "office cam" and PRIVACY_STATUS = "private" live in my_secrets.py on the Pi (the lambda just relays what the device sends, so no repo code change was needed)
    • End the current "office cam" broadcast under its old name (the lambda's "end" matches broadcasts by title, so this had to happen before the rename)
    • Set WORKFLOW_NAME = "powder doser" and PRIVACY_STATUS = "public" on the Pi (backup saved as my_secrets.py.bak-issue8-rename) and restart device.service
    • Verify: new broadcast is live and public with the right title
    • Download a live segment and inspect a frame — portrait feed confirmed intact
    • Answer the segment-download question

    What's live now

    New broadcast: mRgVbT9TSpA — "powder doser stream picam-d1pr, 2026-07-30 UTC 19:21", confirmed via the YouTube API as lifeCycleStatus: live, privacyStatus: public. It was auto-added to a new "powder doser Livestreams Playlist", also public. I downloaded an 8 s segment and checked a frame: 720x1280 (9:16), upright, full field of view (whole balance including the 0455.1 g display visible), timestamp overlay working.

    Notes:

    • All future broadcasts from this device will be public and titled "powder doser …" since the device sends these values on every (re)start, including the 3x-daily reboot cycle.
    • The old "office cam" videos and playlist remain private. Say the word if you want those flipped to public or deleted.

    How the segment download works (your bot-restriction question)

    You're right that it's restricted — but the restriction is IP-reputation based, not account based. I verified this empirically today: running yt-dlp on the GitHub Actions runner fails with YouTube's "Sign in to confirm you're not a bot" error (datacenter IP), but the exact same command run on the Pi itself (over SSH via Tailscale, from its campus network IP) downloads without any cookies or credentials. For a public or unlisted video, no login is needed at all — yt-dlp just fetches the HLS manifest like a browser would; the bot gate only triggers on suspicious IPs. So the recipe is: (if private, temporarily flip to unlisted via the Data API →) SSH to the Pi → yt-dlp --downloader-args 'ffmpeg:-t 8' for a short segment → extract frames with ffmpeg → scp them back for inspection. Now that the stream is public, the privacy-flip step isn't needed anymore.

    No changes were made to this repository (device-side config only), so there's no PR — the branch has no commits.


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions