UFO is an unsupervised reinforcement learning framework for humanoid control, built on mjlab. It supports Unitree G1 / RoboParty RP1.3, FB-CPR-Aux / TeCH, multi-GPU training, and tracking, reward and goal inference.
The RP1 TeCH policy has not been tested on the physical robot.
G1 / RP1 training data and the G1 FB policy tested on hardware are available.
2026-10-15 — Training performance and multi-robot update
192M-step FB / mjlab training: all runs below use 8 × NVIDIA H20, BF16 mixed precision, and 1024-wide networks with shared critics.
| Experiment | Global training FPS | Last EMD ↓ | Time to 192M steps |
|---|---|---|---|
| G1: original DR + original reward | 12,440 | 0.6655 | 4 h 45 min |
| G1: extended DR + original reward | 12,467 | 0.7406 | ≈4 h 51 min |
| G1: extended DR + updated reward | 12,217 | 0.7554 | 4 h 55 min |
| RP1: baseline | 12,485 | 0.7139 | 4 h 48 min |
Measurement settings and scope
All runs use 1,024 environments and batch size 1,024 per GPU, six residual layers, and 16 agent updates per rollout step. FPS counts environment transitions across all eight GPUs and is the median of the last 20 logged intervals, not whole-job throughput.
All three G1 runs use LaFAN-40 and 30% lying-down initialization. The original-DR run predates the reset-boundary fix; the extended-DR/original-reward run includes it. These historical runs span code changes and are not a controlled DR/reward ablation. The updated reward replaces the undesired-contact penalty with a body-impact penalty, sets the slip penalty weight to -4, and adds a foot-stumble penalty with weight -2. RP1 uses common LaFAN-40 and no lying-down initialization. BF16 is an experiment override for G1; the current G1 FB default remains FP32 + TF32, while RP1 FB defaults to BF16.
EMD is the mean over the last periodic tracking evaluation: all three G1 runs at 188,981,248 steps (862 clips), RP1 at 190,054,400 steps (902 clips), rather than a new evaluation of the final checkpoint. Wall time includes startup, evaluation and checkpoint saving. Times use launcher start/end records, except for G1 extended DR + original reward: its time is estimated from launch to the completed final save because its launcher completion timestamp is unavailable. All runs completed 191,995,904 steps under the nominal 192M budget.
The release also replaces the legacy humanoidverse layout with ufo, unifies G1 / RP1 and FB / TeCH commands, fixes reset-boundary observations and contact-force direction, and provides robot-aware ONNX exports, RP1 data and a G1 policy tested on hardware. See architecture and training/inference.
Requires Linux and an NVIDIA driver compatible with CUDA 12.8. The lockfile selects PyTorch 2.7.1 / CUDA 12.8 and NCCL 2.26.5, including Blackwell support. Run these commands, then work from the repository root:
git clone https://github.com/Roboparty/UFO.git
cd UFO
python3 -m pip install --user uv
export PATH="$HOME/.local/bin:$PATH"
uv sync --lockedThis guide requires the mjlab checkout containing ufo/ and run_inference.sh; it does not apply to the older humanoidverse/ release. Video export requires ffmpeg (sudo apt install ffmpeg on Ubuntu).
Download matching full motions and training clips; the script verifies SHA256:
bash scripts/download_data.sh g1_lafan
bash scripts/download_data.sh rp1_lafanG1 contains 40 motions. RP1 contains 55 motions at 50 Hz, converted for its 24 controlled joints, with 1,166 training clips. Each robot requires matching data; G1 data cannot be used directly for RP1.
For local files, see data paths. To convert your own RP1 NPZ files, see RP1 data import.
After preparing data, choose one command:
./run_train.sh --robot g1 --agent fb --gpu-ids all
./run_train.sh --robot rp1 --agent fb --gpu-ids all
./run_train.sh --robot g1 --agent tech --gpu-ids all
./run_train.sh --robot rp1 --agent tech --gpu-ids all--gpu-ids all uses all visible GPUs; use CUDA_VISIBLE_DEVICES to select them. --gpu-ids single --smoke checks startup and sampling only; it does not update weights. See a short training check with updates.
Runs default to runs/<robot>_<agent>. Use a new --work-dir for each experiment; an existing checkpoint in that directory is resumed. To enable W&B:
uv run wandb login
WANDB_PROJECT=UFO ./run_train.sh --robot g1 --agent fb --gpu-ids all \
--work-dir runs/g1_fb_exp --use-wandb --wandb-run-name g1_fb_expInference reads robot settings from the saved run. Use a checkpoint trained for the selected robot and matching motion data:
./run_inference.sh tracking --model-folder runs/g1_fb --motion-list 0 --save-mp4
./run_inference.sh reward --model-folder runs/g1_fb --save-mp4
./run_inference.sh goal --model-folder runs/g1_fb \
--data-path ufo/data/lafan_29dof.pkl --save-mp4- Tracking: FB and TeCH, G1 and RP1. Add
--data-pathto select motion data; videos and z are saved under the run directory. - Reward: FB only; requires saved rank-0 replay. RP1 currently supports root/locomotion tasks.
- Goal: FB and TeCH, G1 and RP1. Use full-motion data for the bundled G1 goals. For RP1 commands and a goal JSON example, see goal inference.
See ./run_inference.sh <tracking|reward|goal> --help for options.
The released G1 FB model includes the approximately 192M-step checkpoint, ONNX exports and matching tracking, goal and forward-walking z. Download it into model/:
uv run hf download xuewang/ufo-g1-policy --local-dir model
./run_inference.sh tracking --model-folder model/g1_policy \
--data-path ufo/data/lafan_29dof.pkl --motion-list 7 --save-mp4The download includes model weights, but no optimizer state or replay buffer. Computing new reward z requires your own saved replay; the provided reward z is ready for deployment.
- Training and inference: defaults, evaluation, checkpoints and inference options.
- Robot and data adaptation: add robots with
robot_setupand import retargeted motions. - Architecture: code layout and component responsibilities.
- G1 real-robot deployment.
- RP1 UFO deployment: Roboparty/roboparty_deploy.
Cite UFO
@misc{ufo2026,
author = {{RoboParty Lab Team}},
title = {UFO: An Unsupervised Reinforcement Learning Framework for Humanoid Control},
year = {2026},
howpublished = {\url{https://github.com/Roboparty/UFO}},
note = {Project page: \url{https://roboparty.github.io/UFO/}}
}UFO is released under CC BY-NC 4.0 for non-commercial research use.