Skip to content

Fix Gemini 2 depth stream assembly on Forge 1 / Dora 0.4.1 - #4

Merged
mrgh97 merged 8 commits into
PhyAgentOS:stable/1.xfrom
mrgh97:fix/forge1-capture-readiness-and-stall
Nov 18, 2026
Merged

mrgh97 merged 8 commits into
PhyAgentOS:stable/1.xfrom
mrgh97:fix/forge1-capture-readiness-and-stall

Conversation

@mrgh97

@mrgh97 mrgh97 commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Fixes the malformed Gemini 2 bulk Y16 depth assembly tracked in #2. This PR targets Forge 1 / Dora 0.4.1 on stable/1.x. See #3 for the other Forge branch.

The USB trace on the reporter's camera showed depth bulk fragments without a valid UVC header. LibUVC read those fragments as headers and forwarded short or oversized frames to the SDK. Linux package builds now compile the native SDK 2.10.6 from a pinned source revision with Gemini 2 bulk Y16 header, frame-boundary, and exact-size checks. Rejected headers and frames remain visible in the SDK shutdown summary. The binary includes the SDK and its license notices. The Python dependency remains pyorbbecsdk2==2.1.3.

The camera node also retains two encoded RGBD sets to absorb tick jitter. It publishes one set per tick and drops the oldest set if the two slots fill. For an explicitly selected USB serial, the isolated backend reports loss of the original USB address and stops the node.

Hardware regression

Tested the original Gemini 2, serial AY6Z363011Y, firmware 1.4.98, on the same VIA USB 3.0 controller at 5 Gbps. The original 1280×720 RGB at 30 fps, 640×400 Y16 at 30 fps, hardware D2C and frame sync configuration was used throughout.

  • 100/100 starts with the patched SDK, without startup retries, SDK buffer overflows, native unrecoverable exceptions, or SDK depth-size mismatch warnings. Latest first complete RGBD set at 2.480 s and latest native depth STREAMING at 0.215 s.
  • 2241 RGBD pairs at 29.920 fps through the Linux package and Dora 0.4.1. Session span 74.862 s, maximum receive gap 0.071 s, exit 0. Native logs verified that the bundled patch ran and that all three original SDK error signatures were absent.
  • A one-time firmware reboot initiated by the owning capture worker caused real USB re-enumeration. The isolated node reported the disconnected serial and USB address in about 1 s. The node exited with an error, as expected for this test.
  • The fixed depth profile was independently verified against the SDK's hardware D2C compatibility list. The native patch was built on Ubuntu 20.04 as well as the hardware build host.

The re-enumeration test used a firmware reboot with the USB cable in place. The 100-start run used the SDK patch with Python wheel bindings, while the continuous flow used the standalone Linux package.

@mrgh97 mrgh97 changed the title Fix Orbbec capture readiness and stalls on Forge 1 / Dora 0.4.1 Mitigate Orbbec capture stalls on Forge 1 / Dora 0.4.1 Nov 18, 2026
@mrgh97 mrgh97 changed the title Mitigate Orbbec capture stalls on Forge 1 / Dora 0.4.1 Fix Gemini 2 depth stream assembly on Forge 1 / Dora 0.4.1 Nov 18, 2026

@mrgh97 mrgh97 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technical review for Forge 1 / Dora 0.4.1 and v1.1.1. No blocking findings. I checked capture readiness and recovery, the native Gemini 2 bulk Y16 bounds checks and diagnostic counters, the two-frame publication queue, and the embedded SDK build and license notices.

The original USB 3.0 Gemini 2 on firmware 1.4.98 passed 100/100 starts with zero original SDK errors. The previous functional head's packaged dataflow delivered 29.920 fps for 74.86 s and exited cleanly. The node reported the USB disconnect in about one second during a firmware reboot. The version changes preserve the locked runtime dependencies. Local tests passed (115 tests and 81 subtests). The merged-tag package will be validated before publication as a Forge 1 stable release.

@mrgh97
mrgh97 merged commit 04c303d into PhyAgentOS:stable/1.x Nov 18, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant