Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions .github/workflows/build-ubuntu20.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ jobs:
image: ubuntu:20.04
env:
RELEASE_VERSION: ${{ inputs.version }}
BUILD_REF: ${{ inputs.ref }}
steps:
- name: Install system tools
run: |
Expand All @@ -30,6 +31,7 @@ jobs:
binutils \
build-essential \
ca-certificates \
cmake \
curl \
file \
git \
Expand All @@ -46,7 +48,8 @@ jobs:
libxrandr2 \
libxss1 \
libxxf86vm1 \
linux-libc-dev
linux-libc-dev \
pkg-config
rm -rf /var/lib/apt/lists/*

- uses: actions/checkout@v6
Expand All @@ -66,9 +69,9 @@ jobs:

- name: Verify and package binary
run: |
test "${{ inputs.ref }}" = "v${RELEASE_VERSION}"
test "${BUILD_REF}" = "v${RELEASE_VERSION}"
./dist/orbbec_camera --version | tee /tmp/version.txt
grep -q "${RELEASE_VERSION}" /tmp/version.txt
grep -Fx "orbbec-camera ${RELEASE_VERSION}" /tmp/version.txt
file ./dist/orbbec_camera
ldd --version | sed -n '1p'
archive="orbbec_camera-v${RELEASE_VERSION}-ubuntu20.04-x86_64.tar.gz"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ jobs:
uses: actions/checkout@v5

- name: Install runtime tools
run: sudo apt-get update && sudo apt-get install --yes binutils file libusb-1.0-0
run: sudo apt-get update && sudo apt-get install --yes binutils build-essential cmake file libusb-1.0-0 pkg-config

- name: Set up uv
uses: astral-sh/setup-uv@v10.0.1
Expand Down
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,21 @@ The project follows Semantic Versioning. Dates use the `YYYY-MM-DD` format.

## Unreleased

## 2.0.1 - 2026-11-18

### Fixed

- Validate Gemini 2 bulk Y16 payload headers and frame lengths in the bundled Orbbec SDK before passing depth frames to the capture pipeline.
- Require a complete first frame set, retry temporary capture failures with bounded backoff, and report the loss of a USB camera selected by serial.
- Retain two encoded frame sets to absorb Dora tick jitter and allow clean shutdown with unread frames.

### Changed

- Pin the Python camera bindings to `pyorbbecsdk2==2.1.3`. Linux executables build and bundle the patched native SDK 2.10.6.
- Keep this release on Dora 1.0.1 and the corresponding Forge dependency line on `master`.

## 2.0.0 - 2026-11-12

### Added

- Added opt-in producer observability with `FORGE_OBSERVABILITY=1`: each actual output is published as an independent Forge Common origin with a stable output-specific low-cardinality ID, while `capture_timestamp_ns` continues to correlate outputs from one FrameSet and `frame_id` remains business metadata.
Expand Down
44 changes: 41 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,8 @@ Orbbec Gemini RGB, metric depth, infrared and optional organized point-cloud cap

## Migration status

Target node version: **2.0.0**. These changes are candidates pending the platform,
hardware and sustained-stream acceptance gates below. A version declaration does not
mean a stable release has been accepted.
Node version: **2.0.1**. This Forge 2 patch remains in the prerelease channel while
sustained-stream and full-system acceptance are pending.

Startup failures now close the capture backend and observer, including failed Dora registration and encoder-thread startup. Encoder failures terminate the node instead of leaving a stalled stream.

Expand Down Expand Up @@ -45,6 +44,39 @@ Use `orbbec-camera --help` (or `uv run --frozen orbbec-camera --help`) for suppo
The example configuration is `config/sensor.example.yaml`; choose the actual device and supported
stream profiles before starting. This command requires an appropriate Dora dataflow.

## Capture readiness and failures

`init_timeout_sec` bounds each SDK initialization attempt, stream setup,
`prewarm_frames` and receipt of the first usable frame. Readiness requires
color (RGB or JPEG) plus all enabled depth/IR streams; optional point-cloud
conversion does not block it.
In `capture_process: isolated` mode, the worker reports ready only after queuing
that first frame. It can reopen the Pipeline twice after recoverable failures,
with a 1-second pause. The parent's total startup deadline covers all three
attempts plus another 5 seconds for process startup/IPC.

After readiness, 5 seconds without a complete usable frame triggers a Pipeline
reopen. Empty or incomplete frames do not refresh this deadline. Readers reject
old cached frames while recovery is in progress, and frame sequences remain
monotonic across reopens. The worker can retry twice within a short failure
period; its retry budget resets after 60 seconds of healthy delivery. Device
disconnects and exhausted retries remain errors. A short `wait_new_frame`
timeout may still return the current frame with its unchanged sequence before
the stall deadline.

Shutdown closes the camera before discarding pending image packets, so an unread
large RGBD packet cannot hold the worker open on a requested stop. Forced process
termination is logged; a backend that still cannot stop raises an error.

The Linux binary build uses the Python bindings from `pyorbbecsdk2==2.1.3`
and compiles the pinned native SDK 2.10.6 with a Gemini 2 bulk Y16 patch.
The patch checks UVC headers and frame sizes before handing frames to the SDK.
The SDK logs rejected payload and frame counts on shutdown.
With an explicit USB serial, isolated mode reports a disconnect when the
original device address disappears, including during USB re-enumeration.
Each tick publishes at most one RGBD set. The publisher retains two encoded
sets and drops the oldest if the encoder gets further ahead.

## Message contract

RGB: Image rgb8 or JPEG CompressedImage; depth: Image 32FC1 in metres; IR: mono8/16UC1. Point cloud is optional.
Expand All @@ -60,6 +92,12 @@ uv run --frozen pytest tests -q
bash scripts/build_pyinstaller.sh
```

The binary build needs CMake, a C/C++ toolchain, pkg-config, and access to the
pinned Orbbec SDK source. To test the native patch from a Linux Python checkout,
run `bash scripts/build_native_sdk.sh build/native-sdk` and launch the camera
with `LD_PRELOAD` set to the absolute path to `build/native-sdk/lib/libOrbbecSDK.so.2`.
A regular `uv sync` installs the upstream wheel.

Use the committed `uv.lock` and frozen/locked installs. Python vendor SDK tests require
the supported SDK platform. For Linux headless MuJoCo rendering, install the required
EGL or OSMesa libraries and select the appropriate `MUJOCO_GL` backend.
Expand Down
30 changes: 29 additions & 1 deletion README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,34 @@ Python 独立示例见 `examples/python_list_devices` 和
当前不支持运行时配置更新。流 profile、设备、对齐和绝大多数控制参数需修改 YAML
后重启节点。

## 采集就绪与故障处理

`init_timeout_sec` 覆盖每次 SDK 初始化尝试、流配置、`prewarm_frames` 和首个有效帧。
就绪要求彩色帧(RGB 或 JPEG)以及所有已启用的深度/IR 流齐全;可选点云转换
不阻塞图像就绪。`capture_process: isolated` 模式在首帧入队后才报告就绪,
可恢复故障会以 1 秒间隔最多重开 Pipeline 两次。父进程启动期限覆盖三次尝试,
并额外保留 5 秒用于子进程启动和 IPC。

就绪后,连续 5 秒没有完整有效帧会触发 Pipeline 重开;空帧或不完整帧不会刷新期限。
恢复期间读取接口拒绝旧缓存,重开前后的帧序号保持递增。短时间内最多重试两次,
连续正常出帧 60 秒后重置重试额度。设备断开或重试用尽仍会报错退出。
停流期限内,较短的 `wait_new_frame` 超时仍可返回当前帧及未变化的序号。

主动关闭时先关闭相机,再丢弃待发送的图像包,避免未读取的大尺寸 RGBD 队列
阻塞子进程退出。强制终止会记录日志;仍无法停止的后端会抛出错误。

Linux 二进制使用 `pyorbbecsdk2==2.1.3` 的 Python 绑定,并从固定的 SDK 2.10.6
源码构建带补丁的原生库。补丁在 Gemini 2 的 bulk Y16 路径验证 UVC 包头及帧长度,
关闭流时记录被拒绝的包和帧数。指定 USB 序列号时,隔离后端会在设备地址消失时
立即报告断开,包括设备重新枚举的情况。
发布端每个 tick 最多发一组图像,最多缓存两组已编码
图像,缓存满时丢弃最旧的一组。

构建二进制需要 CMake、C/C++ 工具链、pkg-config 和 SDK 源码访问权限。Linux 源码
调试可运行 `bash scripts/build_native_sdk.sh build/native-sdk`,并将相机进程的
`LD_PRELOAD` 设为 `build/native-sdk/lib/libOrbbecSDK.so.2` 的绝对路径。
常规 `uv sync` 安装公开 wheel。

## 输出语义

| topic | 消息 | 编码/单位 |
Expand Down Expand Up @@ -211,7 +239,7 @@ USB 信息。

## Forge 2 / Dora 1 迁移

目标版本:`2.0.0`,当前为待验收改动。首轮验证组合使用 Dora 1.0.1、Forge Msgs 2.1.0、Common 2.1.0;依赖与构建工具以提交的锁文件为准。原有消息规范继续使用 `forge_msgs.v1`。
当前节点版本为 `2.0.1`,继续使用预发布渠道。首轮验证组合使用 Dora 1.0.1、Forge Msgs 2.1.0、Common 2.1.0;依赖与构建工具以提交的锁文件为准。原有消息规范继续使用 `forge_msgs.v1`。

本轮要求:检查所有示例最终 YAML,使用 `dora validate <dataflow.yaml>` 验证,并实际执行消息往返。相机取流、资源释放和异常退出须分别验收;高吞吐图像链预热后连续测试至少 4 小时,覆盖慢消费者、扇出、退出与重启,记录实际内存占用。无硬件单测不能替代设备验收。

Expand Down
11 changes: 11 additions & 0 deletions RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,3 +60,14 @@ The public archive is named `orbbec_camera-v<version>-linux-x86_64.tar.gz` and c
The PyInstaller executable bundles Python and native Orbbec SDK components but remains tied to Linux x86_64 and a compatible glibc baseline. The target system still requires the appropriate kernel USB support, libusb runtime, udev permissions, and camera hardware.

Before upload, verify the CLI and a real snapshot, inspect ELF dependencies and minimum glibc symbols, scan for private paths and internal URLs, record SHA-256, and test on the oldest supported deployment image. Verify both `orbbec_camera --version` and `orbbec_camera licenses`; license information must also remain available in the repository and source release.

## GitHub Actions and publication

Push the annotated version tag to the upstream repository. From the default
branch, manually run **Build Ubuntu 20.04 binary** with that tag as `ref` and
the package version as `version`. This also builds maintenance tags on
`stable/1.x`. Download the archive and checksum from the successful Actions
run, validate them, and attach both to the GitHub Release for the tag.

The workflow uploads an Actions artifact. A branch merge, a tag push, or a
successful build does not publish a GitHub Release or publish to PyPI.
2 changes: 1 addition & 1 deletion config/sensor.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -257,7 +257,7 @@ frame_sync: false # 硬件帧同步,多台摄像头同步时
# ── 启动参数 ─────────────────────────────────────────────────────────────────
prewarm_frames: 0 # 启动后丢弃前N帧,消除首帧异常,范围: [0, 60]
connect_delay_ms: 0 # 连接前等待(ms),多节点时错开USB抢占
init_timeout_sec: 15.0 # 设备初始化超时(秒)
init_timeout_sec: 15.0 # 初始化、预热及完整首帧的总超时(秒)
capture_process: isolated # Dora 节点推荐隔离 SDK;单独工具调试可用 direct

# ── dora 输出 topic ──────────────────────────────────────────────────────────
Expand Down
5 changes: 3 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "forge-devices-orbbec-camera"
version = "2.0.0"
version = "2.0.1"
description = "Linux Orbbec Gemini 2 depth camera capture and Dora streaming node."
readme = "README.md"
requires-python = ">=3.11"
Expand All @@ -25,7 +25,8 @@ dependencies = [
"dora-rs>=1.0.1,<2",
"forge-common>=2.1.0,<3",
"forge-msgs>=2.1.0,<3",
"pyorbbecsdk2>=2.0.18,<3",
"pyorbbecsdk2==2.1.3",
"ipywidgets>=8.0.4,<9; python_version < '3.12'",
"numpy>=1.26,<3",
"opencv-python>=4.8,<6",
"pyyaml>=6.0,<7",
Expand Down
33 changes: 33 additions & 0 deletions scripts/build_native_sdk.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#!/usr/bin/env bash
set -euo pipefail

if [[ $# != 1 ]]; then
printf 'Usage: %s INSTALL_DIRECTORY\n' "$0" >&2
exit 2
fi

script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
sdk_revision=575347ca742d219ef61b1e214e97043f17f48d23
mkdir -p "$1"
install_dir="$(cd "$1" && pwd)"
scratch="$(mktemp -d)"
trap 'rm -rf -- "$scratch"' EXIT

git -C "$scratch" init -q
git -C "$scratch" fetch -q --depth 1 https://github.com/orbbec/OrbbecSDK_v2.git "$sdk_revision"
git -C "$scratch" checkout -q --detach FETCH_HEAD
test "$(git -C "$scratch" rev-parse HEAD)" = "$sdk_revision"
git -C "$scratch" apply --check "$script_dir/native-sdk/gemini2-y16-assembly.patch"
git -C "$scratch" apply "$script_dir/native-sdk/gemini2-y16-assembly.patch"

cmake -S "$scratch" -B "$scratch/build" \
-DCMAKE_BUILD_TYPE=Release \
-DOB_BUILD_EXAMPLES=OFF \
-DOB_BUILD_DOCS=OFF \
-DOB_BUILD_TOOLS=OFF \
-DOB_BUILD_TESTS=OFF \
-DOB_BUILD_WITH_EXTENSIONS_COMMIT_HASH=OFF \
-DOB_LIBUVC_TRANSFER_DIAGNOSTIC=ON
cmake --build "$scratch/build" --target OrbbecSDK --parallel "${CMAKE_BUILD_PARALLEL_LEVEL:-4}"
cmake --install "$scratch/build" --prefix "$install_dir"
test -f "$install_dir/lib/libOrbbecSDK.so.2.10.6"
22 changes: 22 additions & 0 deletions scripts/build_pyinstaller.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,22 @@ UV_PROJECT_ENVIRONMENT="${VENV_DIR}" uv sync \
--group build \
--python 3.12

NATIVE_SDK_DIR="${WORK_DIR}/native-sdk"
"${SCRIPT_DIR}/build_native_sdk.sh" "${NATIVE_SDK_DIR}"
WHEEL_SDK_DIR="$("${VENV_DIR}/bin/python" -c 'from importlib.util import find_spec; from pathlib import Path; print(Path(find_spec("pyorbbecsdk").origin).parent)')"
shopt -s nullglob
wheel_libraries=("${WHEEL_SDK_DIR}"/libOrbbecSDK.so*)
if [[ ${#wheel_libraries[@]} -eq 0 ]]; then
echo "ERROR: Orbbec wheel is missing its Linux SDK libraries." >&2
exit 1
fi
for library in "${wheel_libraries[@]}"; do
# Replace the environment's copy without writing through a uv cache hard link.
install -m 0755 "${NATIVE_SDK_DIR}/lib/libOrbbecSDK.so.2.10.6" "${library}.patched"
mv -f "${library}.patched" "${library}"
done
shopt -u nullglob

LICENSE_BUNDLE="${WORK_DIR}/THIRD_PARTY_LICENSES.txt"
LICENSE_JSON="${WORK_DIR}/third-party-licenses.json"
echo "==> [orbbec_camera] 正在生成内嵌许可证清单..."
Expand All @@ -47,6 +63,12 @@ fi
cat "${SCRIPT_DIR}/licenses/dora-rs-1.0.1.txt"
printf '\n\n'
cat "${SCRIPT_DIR}/licenses/pygame-2.6.1-LGPL-2.1.txt"
printf '\n\n%s\n\n' 'Native Orbbec SDK 2.10.6 and third-party notices'
cat "${NATIVE_SDK_DIR}/LICENSE.txt"
while IFS= read -r -d '' notice; do
printf '\n\n%s\n\n' "${notice##*/}"
cat "$notice"
done < <(find "${NATIVE_SDK_DIR}/licenses" -type f -print0 | sort -z)
} > "${LICENSE_BUNDLE}"
if grep -Eq '/home/|file:///|git\+ssh://' "${LICENSE_BUNDLE}"; then
echo "ERROR: 许可证清单包含私有 URL 或构建路径。" >&2
Expand Down
Loading
Loading