Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ jobs:
uses: actions/checkout@v5

- name: Install runtime tools
run: sudo apt-get update && sudo apt-get install --yes binutils file libusb-1.0-0
run: sudo apt-get update && sudo apt-get install --yes binutils build-essential cmake file libusb-1.0-0 pkg-config

- name: Set up uv
uses: astral-sh/setup-uv@v10.0.1
Expand Down Expand Up @@ -99,8 +99,11 @@ jobs:
version: "0.12.0"
enable-cache: true

- name: Install audit interpreter
run: uv python install 3.12

- name: Export locked runtime dependencies
run: uv export --frozen --no-dev --no-emit-project --no-hashes --format requirements-txt --output-file requirements-audit.txt

- name: Audit dependencies
run: uvx pip-audit --requirement requirements-audit.txt
run: uvx --python 3.12 pip-audit --requirement requirements-audit.txt
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,19 @@ The project follows Semantic Versioning. Dates use the `YYYY-MM-DD` format.

## Unreleased

## 1.1.1 - 2026-11-18

### Fixed

- Validate Gemini 2 bulk Y16 payload headers and frame lengths in the bundled Orbbec SDK before passing depth frames to the capture pipeline.
- Require a complete first frame set, retry temporary capture failures with bounded backoff, and report the loss of a USB camera selected by serial.
- Retain two encoded frame sets to absorb Dora tick jitter and allow clean shutdown with unread frames.

### Changed

- Pin the Python camera bindings to `pyorbbecsdk2==2.1.3`. Linux executables build and bundle the patched native SDK 2.10.6.
- Keep this release on Dora 0.4.1 and the corresponding Forge dependency line on `stable/1.x`.

## 1.1.0 - 2026-08-24

### Added
Expand Down
32 changes: 32 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,38 @@ privileged helper 只原子安装固定内嵌规则、将 `PKEXEC_UID` 对应的
官方工具验证应先使用 Orbbec Viewer 或 SDK 官方示例确认设备、固件、流 profile
和深度工作模式,再运行本项目。

## 取流就绪与停滞处理

此 Forge 1 分支固定使用 `dora-rs==0.4.1`、`forge-common==1.0.1` 和
`forge-msgs==1.3.1`;相机节点应与 Dora 0.4.1 CLI/coordinator/daemon 一起运行。
`forge-msgs` 从 1.1.0 升至 1.3.1,以允许锁定 Pillow 13,修复依赖审计发现的
Pillow 12.3.0 安全问题。

`init_timeout_sec` 覆盖每次 SDK 初始化尝试、流配置、预热和首个完整帧。完整帧需包含彩色图像
(RGB 或 JPEG)及所有启用的 Depth/IR 流;可选点云转换不影响就绪判断。隔离进程
只有在首帧入队后才报告就绪。可恢复故障会以 1 秒间隔最多重开 Pipeline 两次;
父进程的启动期限覆盖三次尝试,并另留 5 秒用于启动与 IPC。

就绪后若连续 5 秒没有完整可用帧,会触发 Pipeline 重开。空帧或缺失流的帧
不能刷新计时;恢复期间读取端拒绝旧缓存,帧序号在重开后继续递增。短时间内
最多重试两次,连续正常出帧 60 秒后重置重试额度。设备断开或重试用尽仍报错退出。
停流期限内,短时 `wait_new_frame` 超时可能返回序号不变的当前帧,不代表获得新帧。

停止时会先关闭相机,再丢弃尚未读取的图像包,避免大 RGBD 消息阻塞队列退出。
强制结束会记录日志,无法停止的后端会抛出错误。

Linux 二进制使用 `pyorbbecsdk2==2.1.3` 的 Python 绑定,并从固定的 SDK 2.10.6
源码构建带补丁的原生库。补丁在 Gemini 2 的 bulk Y16 路径验证 UVC 包头及帧长度,
关闭流时记录被拒绝的包和帧数。指定 USB 序列号时,隔离后端会在设备地址消失时
立即报告断开,包括设备重新枚举的情况。
发布端每个 tick 最多发一组图像,最多缓存两组已编码
图像,缓存满时丢弃最旧的一组。

构建二进制需要 CMake、C/C++ 工具链、pkg-config 和 SDK 源码访问权限。Linux 源码
调试可运行 `bash scripts/build_native_sdk.sh build/native-sdk`,并将相机进程的
`LD_PRELOAD` 设为 `build/native-sdk/lib/libOrbbecSDK.so.2` 的绝对路径。
常规 `uv sync` 安装公开 wheel。

## 基础命令

```bash
Expand Down
11 changes: 11 additions & 0 deletions RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,14 @@ The public archive is named `orbbec_camera-v<version>-linux-x86_64.tar.gz` and c
The PyInstaller executable bundles Python and native Orbbec SDK components but remains tied to Linux x86_64 and a compatible glibc baseline. The target system still requires the appropriate kernel USB support, libusb runtime, udev permissions, and camera hardware.

Before upload, verify the CLI and a real snapshot, inspect ELF dependencies and minimum glibc symbols, scan for private paths and internal URLs, record SHA-256, and test on the oldest supported deployment image. Verify both `orbbec_camera --version` and `orbbec_camera licenses`; license information must also remain available in the repository and source release.

## GitHub Actions and publication

Push the annotated version tag to the upstream repository. From the default
branch, manually run **Build Ubuntu 20.04 binary** with that tag as `ref` and
the package version as `version`. This also builds maintenance tags on
`stable/1.x`. Download the archive and checksum from the successful Actions
run, validate them, and attach both to the GitHub Release for the tag.

The workflow uploads an Actions artifact. A branch merge, a tag push, or a
successful build does not publish a GitHub Release or publish to PyPI.
2 changes: 1 addition & 1 deletion config/sensor.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -257,7 +257,7 @@ frame_sync: false # 硬件帧同步,多台摄像头同步时
# ── 启动参数 ─────────────────────────────────────────────────────────────────
prewarm_frames: 0 # 启动后丢弃前N帧,消除首帧异常,范围: [0, 60]
connect_delay_ms: 0 # 连接前等待(ms),多节点时错开USB抢占
init_timeout_sec: 15.0 # 设备初始化超时(秒)
init_timeout_sec: 15.0 # 初始化、预热及完整首帧的总超时(秒)
capture_process: isolated # Dora 节点推荐隔离 SDK;单独工具调试可用 direct

# ── dora 输出 topic ──────────────────────────────────────────────────────────
Expand Down
7 changes: 4 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "forge-devices-orbbec-camera"
version = "1.1.0"
version = "1.1.1"
description = "Linux Orbbec Gemini 2 depth camera capture and Dora streaming node."
readme = "README.md"
requires-python = ">=3.11"
Expand All @@ -24,8 +24,9 @@ classifiers = [
dependencies = [
"dora-rs==0.4.1",
"forge-common==1.0.1",
"forge-msgs==1.1.0",
"pyorbbecsdk2>=2.0.18,<3",
"forge-msgs==1.3.1",
"pyorbbecsdk2==2.1.3",
"ipywidgets>=8.0.4,<9; python_version < '3.12'",
"numpy>=1.26,<3",
"opencv-python>=4.8,<6",
"pyyaml>=6.0,<7",
Expand Down
33 changes: 33 additions & 0 deletions scripts/build_native_sdk.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#!/usr/bin/env bash
set -euo pipefail

if [[ $# != 1 ]]; then
printf 'Usage: %s INSTALL_DIRECTORY\n' "$0" >&2
exit 2
fi

script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
sdk_revision=575347ca742d219ef61b1e214e97043f17f48d23
mkdir -p "$1"
install_dir="$(cd "$1" && pwd)"
scratch="$(mktemp -d)"
trap 'rm -rf -- "$scratch"' EXIT

git -C "$scratch" init -q
git -C "$scratch" fetch -q --depth 1 https://github.com/orbbec/OrbbecSDK_v2.git "$sdk_revision"
git -C "$scratch" checkout -q --detach FETCH_HEAD
test "$(git -C "$scratch" rev-parse HEAD)" = "$sdk_revision"
git -C "$scratch" apply --check "$script_dir/native-sdk/gemini2-y16-assembly.patch"
git -C "$scratch" apply "$script_dir/native-sdk/gemini2-y16-assembly.patch"

cmake -S "$scratch" -B "$scratch/build" \
-DCMAKE_BUILD_TYPE=Release \
-DOB_BUILD_EXAMPLES=OFF \
-DOB_BUILD_DOCS=OFF \
-DOB_BUILD_TOOLS=OFF \
-DOB_BUILD_TESTS=OFF \
-DOB_BUILD_WITH_EXTENSIONS_COMMIT_HASH=OFF \
-DOB_LIBUVC_TRANSFER_DIAGNOSTIC=ON
cmake --build "$scratch/build" --target OrbbecSDK --parallel "${CMAKE_BUILD_PARALLEL_LEVEL:-4}"
cmake --install "$scratch/build" --prefix "$install_dir"
test -f "$install_dir/lib/libOrbbecSDK.so.2.10.6"
22 changes: 22 additions & 0 deletions scripts/build_pyinstaller.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,22 @@ UV_PROJECT_ENVIRONMENT="${VENV_DIR}" uv sync \
--group build \
--python 3.12

NATIVE_SDK_DIR="${WORK_DIR}/native-sdk"
"${SCRIPT_DIR}/build_native_sdk.sh" "${NATIVE_SDK_DIR}"
WHEEL_SDK_DIR="$("${VENV_DIR}/bin/python" -c 'from importlib.util import find_spec; from pathlib import Path; print(Path(find_spec("pyorbbecsdk").origin).parent)')"
shopt -s nullglob
wheel_libraries=("${WHEEL_SDK_DIR}"/libOrbbecSDK.so*)
if [[ ${#wheel_libraries[@]} -eq 0 ]]; then
echo "ERROR: Orbbec wheel is missing its Linux SDK libraries." >&2
exit 1
fi
for library in "${wheel_libraries[@]}"; do
# Replace the environment's copy without writing through a uv cache hard link.
install -m 0755 "${NATIVE_SDK_DIR}/lib/libOrbbecSDK.so.2.10.6" "${library}.patched"
mv -f "${library}.patched" "${library}"
done
shopt -u nullglob

LICENSE_BUNDLE="${WORK_DIR}/THIRD_PARTY_LICENSES.txt"
LICENSE_JSON="${WORK_DIR}/third-party-licenses.json"
echo "==> [orbbec_camera] 正在生成内嵌许可证清单..."
Expand All @@ -47,6 +63,12 @@ fi
cat "${SCRIPT_DIR}/licenses/dora-rs-0.4.1.txt"
printf '\n\n'
cat "${SCRIPT_DIR}/licenses/pygame-2.6.1-LGPL-2.1.txt"
printf '\n\n%s\n\n' 'Native Orbbec SDK 2.10.6 and third-party notices'
cat "${NATIVE_SDK_DIR}/LICENSE.txt"
while IFS= read -r -d '' notice; do
printf '\n\n%s\n\n' "${notice##*/}"
cat "$notice"
done < <(find "${NATIVE_SDK_DIR}/licenses" -type f -print0 | sort -z)
} > "${LICENSE_BUNDLE}"
if grep -Eq '/home/|file:///|git\+ssh://' "${LICENSE_BUNDLE}"; then
echo "ERROR: 许可证清单包含私有 URL 或构建路径。" >&2
Expand Down
167 changes: 167 additions & 0 deletions scripts/native-sdk/gemini2-y16-assembly.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
diff --git a/3rdparty/libuvc/src/include/libuvc/libuvc_internal.h b/3rdparty/libuvc/src/include/libuvc/libuvc_internal.h
index 27a67a2..ca36c33 100644
--- a/3rdparty/libuvc/src/include/libuvc/libuvc_internal.h
+++ b/3rdparty/libuvc/src/include/libuvc/libuvc_internal.h
@@ -221,6 +221,13 @@ struct uvc_stream_handle {
uint64_t diagnostic_status[7];
uint64_t diagnostic_last_submit_ms, diagnostic_last_callback_ms, diagnostic_last_report_ms;
unsigned int diagnostic_active, diagnostic_inflight;
+ size_t gemini_depth_frame_bytes;
+ size_t gemini_depth_last_bad_size;
+ unsigned int gemini_depth_bad_headers;
+ unsigned int gemini_depth_bad_frames;
+ unsigned int gemini_depth_bad_after_first;
+ unsigned int gemini_depth_valid_frames;
+ uint8_t gemini_depth_invalid;
/* Transient submit-failure recovery: a parked slot keeps its transfer but holds no
* kernel request, so retries ride on the completions of slots still in flight.
* Guarded by cb_mutex. */
diff --git a/3rdparty/libuvc/src/src/stream.c b/3rdparty/libuvc/src/src/stream.c
index aa45864..0e56b92 100644
--- a/3rdparty/libuvc/src/src/stream.c
+++ b/3rdparty/libuvc/src/src/stream.c
@@ -575,12 +575,33 @@ uvc_error_t uvc_probe_still_ctrl(uvc_device_handle_t *devh, uvc_still_ctrl_t *st
return res;
}

+static void _uvc_reset_assembly(uvc_stream_handle_t *strmh) {
+ strmh->got_bytes = 0;
+ strmh->meta_got_bytes = 0;
+ strmh->last_scr = 0;
+ strmh->pts = 0;
+ strmh->payload_header_got_bytes = 0;
+ strmh->gemini_depth_invalid = 0;
+}
+
/** @internal
* @brief Swap the working buffer with the presented buffer and notify consumers
*/
void _uvc_swap_buffers(uvc_stream_handle_t *strmh) {
uint8_t *tmp_buf;

+ if(strmh->gemini_depth_frame_bytes) {
+ if(strmh->gemini_depth_invalid || strmh->got_bytes != strmh->gemini_depth_frame_bytes) {
+ if(!strmh->gemini_depth_invalid)
+ strmh->gemini_depth_last_bad_size = strmh->got_bytes;
+ strmh->gemini_depth_bad_frames++;
+ strmh->gemini_depth_bad_after_first += strmh->gemini_depth_valid_frames != 0;
+ _uvc_reset_assembly(strmh);
+ return;
+ }
+ strmh->gemini_depth_valid_frames++;
+ }
+
pthread_mutex_lock(&strmh->cb_mutex);

(void)clock_gettime(CLOCK_MONOTONIC, &strmh->capture_time_finished);
@@ -610,11 +631,7 @@ void _uvc_swap_buffers(uvc_stream_handle_t *strmh) {
pthread_mutex_unlock(&strmh->cb_mutex);

strmh->seq++;
- strmh->got_bytes = 0;
- strmh->meta_got_bytes = 0;
- strmh->last_scr = 0;
- strmh->pts = 0;
- strmh->payload_header_got_bytes = 0;
+ _uvc_reset_assembly(strmh);
}

/** @internal
@@ -657,6 +674,14 @@ void _uvc_process_payload(uvc_stream_handle_t *strmh, uint8_t *payload, size_t p
header_len = payload[0];

if(header_len > payload_len) {
+ if(strmh->gemini_depth_frame_bytes) {
+ strmh->gemini_depth_bad_headers++;
+ strmh->gemini_depth_bad_after_first += strmh->gemini_depth_valid_frames != 0;
+ if(strmh->got_bytes) {
+ strmh->gemini_depth_last_bad_size = strmh->got_bytes;
+ strmh->gemini_depth_invalid = 1;
+ }
+ }
UVC_DEBUG("bogus packet: actual_len=%zd, header_len=%zd\n", payload_len, header_len);
return;
}
@@ -667,6 +692,23 @@ void _uvc_process_payload(uvc_stream_handle_t *strmh, uint8_t *payload, size_t p
data_len = payload_len - header_len;
}

+ if(strmh->gemini_depth_frame_bytes) {
+ // Gemini 2 Y16 sends a UVC header with EOH, PTS and SCR on each payload.
+ // On some starts, bulk fragments without that header precede valid data.
+ if(header_len < 12 || (payload[1] & 0xcc) != 0x8c) {
+ strmh->gemini_depth_bad_headers++;
+ strmh->gemini_depth_bad_after_first += strmh->gemini_depth_valid_frames != 0;
+ if(strmh->got_bytes) {
+ strmh->gemini_depth_last_bad_size = strmh->got_bytes;
+ strmh->gemini_depth_invalid = 1;
+ }
+ return;
+ }
+ if(strmh->got_bytes && (strmh->fid != (payload[1] & 1) || strmh->pts != DW_TO_INT(payload + 2))) {
+ _uvc_swap_buffers(strmh);
+ }
+ }
+
if(strmh->payload_header_got_bytes == 0) {
strmh->payload_header_got_bytes = header_len;
memcpy(strmh->payload_header_outbuf, payload, header_len);
@@ -717,6 +759,20 @@ void _uvc_process_payload(uvc_stream_handle_t *strmh, uint8_t *payload, size_t p
}
}

+ if(strmh->gemini_depth_frame_bytes) {
+ if(!strmh->gemini_depth_invalid && data_len > strmh->gemini_depth_frame_bytes - strmh->got_bytes) {
+ strmh->gemini_depth_last_bad_size = strmh->got_bytes + data_len;
+ strmh->gemini_depth_invalid = 1;
+ }
+ if(!strmh->gemini_depth_invalid && data_len > 0) {
+ memcpy(strmh->outbuf + strmh->got_bytes, payload + header_len, data_len);
+ strmh->got_bytes += data_len;
+ }
+ if((header_info & (1 << 1)) && (strmh->got_bytes || strmh->gemini_depth_invalid)) {
+ _uvc_swap_buffers(strmh);
+ }
+ return;
+ }
if(data_len > 0) {
// bugfix:USB不稳定或带宽不足时会丢包,有概率会丢到EOF包,导致strmh->got_bytes过大,memcpy越界
if(strmh->got_bytes + data_len < LIBUVC_XFER_BUF_SIZE) {
@@ -1371,6 +1427,22 @@ uvc_error_t uvc_stream_start(uvc_stream_handle_t *strmh, uvc_frame_callback_t *c
* (UVC 1.5: 2.4.3. VideoStreaming Interface) */
isochronous = interface->num_altsetting > 1;

+ strmh->gemini_depth_frame_bytes = 0;
+ strmh->gemini_depth_last_bad_size = 0;
+ strmh->gemini_depth_bad_headers = 0;
+ strmh->gemini_depth_bad_frames = 0;
+ strmh->gemini_depth_bad_after_first = 0;
+ strmh->gemini_depth_valid_frames = 0;
+ _uvc_reset_assembly(strmh);
+ if(!isochronous && strmh->frame_format == UVC_FRAME_FORMAT_Y16) {
+ struct libusb_device_descriptor descriptor;
+ const size_t frame_bytes = (size_t)frame_desc->wWidth * frame_desc->wHeight * sizeof(uint16_t);
+ if(libusb_get_device_descriptor(strmh->devh->dev->usb_dev, &descriptor) == LIBUSB_SUCCESS
+ && descriptor.idVendor == 0x2bc5 && descriptor.idProduct == 0x0670 && frame_bytes <= LIBUVC_XFER_BUF_SIZE) {
+ strmh->gemini_depth_frame_bytes = frame_bytes;
+ }
+ }
+
if(isochronous) {
/* For isochronous streaming, we choose an appropriate altsetting for the endpoint
* and set up several transfers */
@@ -1807,6 +1879,14 @@ uvc_error_t uvc_stream_stop(uvc_stream_handle_t *strmh) {
pthread_cond_wait(&strmh->cb_cond, &strmh->cb_mutex);
} while(1);
_uvc_log_retry_summary(strmh, "stopped", 1);
+ if(strmh->gemini_depth_frame_bytes && strmh->diagnostic_cb) {
+ char message[384];
+ snprintf(message, sizeof(message),
+ "Gemini 2 Y16 assembly: valid=%u bad_headers=%u bad_frames=%u bad_after_first=%u expected=%zu last_bad_size=%zu",
+ strmh->gemini_depth_valid_frames, strmh->gemini_depth_bad_headers, strmh->gemini_depth_bad_frames,
+ strmh->gemini_depth_bad_after_first, strmh->gemini_depth_frame_bytes, strmh->gemini_depth_last_bad_size);
+ strmh->diagnostic_cb(UVC_DIAG_IMPORTANT, message, strmh->diagnostic_user);
+ }
// Kick the user thread awake
pthread_cond_broadcast(&strmh->cb_cond);
pthread_mutex_unlock(&strmh->cb_mutex);
2 changes: 1 addition & 1 deletion src/forge_devices_orbbec_camera/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

from typing import Any

__version__ = "1.1.0"
__version__ = "1.1.1"

__all__ = ["DeviceInfo", "OrbbeFrame", "OrbbecConfig", "__version__"]

Expand Down
3 changes: 3 additions & 0 deletions src/forge_devices_orbbec_camera/backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@
from .config import OrbbecConfig


_FRAME_STALL_TIMEOUT_SEC = 5.0


@dataclass(frozen=True)
class OrbbecPointCloud:
"""SDK-independent immutable columns for an organized point cloud."""
Expand Down
Loading
Loading