Skip to content

Fix Gemini 2 depth stream assembly on Forge 2 / Dora 1.0.1 - #3

Merged
mrgh97 merged 8 commits into
PhyAgentOS:masterfrom
mrgh97:fix/capture-readiness-and-stall
Nov 18, 2026
Merged

mrgh97 merged 8 commits into
PhyAgentOS:masterfrom
mrgh97:fix/capture-readiness-and-stall

Conversation

@mrgh97

@mrgh97 mrgh97 commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Fixes the malformed Gemini 2 bulk Y16 depth assembly tracked in #2. This PR targets Forge 2 / Dora 1.0.1 on master. See #4 for the other Forge branch.

The USB trace on the reporter's camera showed depth bulk fragments without a valid UVC header. LibUVC read those fragments as headers and forwarded short or oversized frames to the SDK. Linux package builds now compile the native SDK 2.10.6 from a pinned source revision with Gemini 2 bulk Y16 header, frame-boundary, and exact-size checks. Rejected headers and frames remain visible in the SDK shutdown summary. The binary includes the SDK and its license notices. The Python dependency remains pyorbbecsdk2==2.1.3.

The camera node also retains two encoded RGBD sets to absorb tick jitter. It publishes one set per tick and drops the oldest set if the two slots fill. For an explicitly selected USB serial, the isolated backend reports loss of the original USB address and stops the node.

Hardware regression

Tested the original Gemini 2, serial AY6Z363011Y, firmware 1.4.98, on the same VIA USB 3.0 controller at 5 Gbps. The original 1280×720 RGB at 30 fps, 640×400 Y16 at 30 fps, hardware D2C and frame sync configuration was used throughout.

  • 100/100 starts with the patched SDK, without startup retries, SDK buffer overflows, native unrecoverable exceptions, or SDK depth-size mismatch warnings. Latest first complete RGBD set at 2.480 s and latest native depth STREAMING at 0.215 s.
  • 2250 RGBD pairs at 29.935 fps through the Linux package and Dora 1.0.1. Session span 75.128 s, maximum receive gap 0.073 s, exit 0. Native logs verified that the bundled patch ran and that all three original SDK error signatures were absent.
  • A one-time firmware reboot initiated by the owning capture worker caused real USB re-enumeration. The isolated node reported the disconnected serial and USB address in about 1 s. The node exited with an error, as expected for this test.
  • The fixed depth profile was independently verified against the SDK's hardware D2C compatibility list. The native patch was built on Ubuntu 20.04 as well as the hardware build host.

The re-enumeration test used a firmware reboot with the USB cable in place. The 100-start run used the SDK patch with Python wheel bindings, while the continuous flow used the standalone Linux package.

@mrgh97 mrgh97 changed the title Fix Orbbec capture readiness, stall detection, and queue shutdown Mitigate Orbbec capture stalls on Forge 2 / Dora 1.0.1 Nov 18, 2026
@mrgh97 mrgh97 changed the title Mitigate Orbbec capture stalls on Forge 2 / Dora 1.0.1 Fix Gemini 2 depth stream assembly on Forge 2 / Dora 1.0.1 Nov 18, 2026

@mrgh97 mrgh97 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technical review for Forge 2 / Dora 1.0.1 and v2.0.1. No blocking findings. I checked capture readiness and recovery, the native Gemini 2 bulk Y16 bounds checks and diagnostic counters, the two-frame publication queue, and the embedded SDK build and license notices. The version build also checks the exact tag and binary version and removes a repeated toolchain package from the Ubuntu workflow.

The original USB 3.0 Gemini 2 on firmware 1.4.98 passed 100/100 starts with zero original SDK errors. The previous functional head's packaged dataflow delivered 29.935 fps for 75.13 s and exited cleanly. The node reported the USB disconnect in about one second during a firmware reboot. The version changes preserve the locked runtime dependencies. Local tests passed (136 tests and 81 subtests). The merged-tag package will be validated before publication as a Forge 2 prerelease.

@mrgh97
mrgh97 merged commit b3641ea into PhyAgentOS:master Nov 18, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants