Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
11 changes: 9 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
# Build output
**/bin/
**/dist/
*.tsbuildinfo
tmp/

# macOS
Expand All @@ -15,9 +16,15 @@ tmp/
# Local config overrides
*.local

# Go test output
# Test and coverage output
*.out
coverage.out
*.profraw
/coverage/
/apps/rlark-ui/coverage/

# Python cache
__pycache__/
*.py[cod]

# Dependencies
**/node_modules/
Expand Down
27 changes: 23 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@
# make # build everything (lint-go, go-vet, go-build)
# make docker-build # build rlark image and load into local docker

GREP = grep -P
UNAME_S := $(shell uname -s)
ifeq ($(UNAME_S),Darwin)
GREP = grep -E
endif

##@ Code style

.PHONY: lint-go lint-web fmt-go fmt-web
Expand All @@ -28,21 +34,34 @@ fmt-web: ## Format web UI (prettier)

##@ Go targets

.PHONY: build go-tidy
.PHONY: build test-go test-web go-tidy

build: ## Build all binaries (apps/rlark)
$(MAKE) -C apps/rlark build $(MAKEOVERRIDES)

test-go: ## Run Go unit tests with coverage
@mkdir -p coverage/go
@for module in api apps/rlark apps/embodied-runtime; do \
name=$$(echo $$module | tr '/' '-'); \
coverage=$$(cd $$module && realpath --relative-to=. $(CURDIR)/coverage/go/$$name.out); \
(cd $$module && go test ./... -v -count=1 -short -covermode=atomic -coverprofile=$$coverage) || exit $$?; \
done

test-web: ## Run web UI unit tests with coverage
npm ci --prefix apps/rlark-ui
npm run test:coverage --prefix apps/rlark-ui

go-tidy: ## Run go mod tidy on all workspace modules
@for dir in $$(grep '^\t\./' go.work | sed 's/^\t//'); do \
(cd $$dir && go mod tidy); \
@for dir in $$( $(GREP) '^\t\./' go.work | sed 's/^\t//' ); do \
echo "Running go mod tidy in $$dir"; \
(cd $$dir && go mod tidy -x); \
done

##@ Code generation

.PHONY: generate generate-crd generate-crd-schema-docs proto

generate: generate-crd generate-crd-schema-docs ## Generate CRD manifests, clients, and schema docs
generate: generate-crd-schema-docs ## Generate CRD manifests, clients, and schema docs

generate-crd: ## Generate CRD manifests and clients
$(MAKE) -C api generate-crd $(MAKEOVERRIDES)
Expand Down
1 change: 0 additions & 1 deletion NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ This product includes software developed by third parties:
- Bun ORM (https://github.com/uptrace/bun) - BSD 2-Clause License
- React (https://github.com/facebook/react) - MIT License
- gVisor (https://github.com/google/gvisor) - Apache License 2.0
- remotedialer (https://github.com/rancher/remotedialer) - Apache License 2.0
- RClone (https://github.com/rclone/rclone) - MIT License

For full license texts of third-party dependencies, see the respective
Expand Down
51 changes: 35 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
<div align="center">
<a href="README.md"><img src="https://img.shields.io/badge/lang-English-blue.svg" /></a>
<a href="README.zh-CN.md"><img src="https://img.shields.io/badge/语言-简体中文-red.svg" /></a>
<a href="https://rlark-ci-test.readthedocs.io/en/latest/"><img src="https://img.shields.io/badge/Documentation-Read%20the%20Docs-8A2BE2?logo=readthedocs&logoColor=white" alt="Documentation" /></a>
<a href="https://rlark-ci-test.readthedocs.io/zh-cn/latest/"><img src="https://img.shields.io/badge/中文文档-Read%20the%20Docs-red?logo=readthedocs&logoColor=white" alt="中文文档" /></a>
<a href="https://rlark.readthedocs.io/en/latest/"><img src="https://img.shields.io/badge/Documentation-Read%20the%20Docs-8A2BE2?logo=readthedocs&logoColor=white" alt="Documentation" /></a>
<a href="https://rlark.readthedocs.io/zh-cn/latest/"><img src="https://img.shields.io/badge/中文文档-Read%20the%20Docs-red?logo=readthedocs&logoColor=white" alt="中文文档" /></a>
<img src="https://img.shields.io/badge/Go-1.26-00ADD8?logo=go&style=flat-square" alt="Go Version" />
<img src="https://img.shields.io/badge/TypeScript-React-3178C6?logo=typescript&style=flat-square" alt="TypeScript" />
<img src="https://img.shields.io/badge/Kubernetes-kcp-326CE5?logo=kubernetes&style=flat-square" alt="Kubernetes" />
Expand All @@ -18,46 +18,65 @@

Manage cross-cluster embodied intelligence workloads through a unified cloud-native platform spanning cloud GPU training, cross-cluster collaboration, and edge device deployment across heterogeneous resources such as GPU clusters, robot arms, sensors, and cameras.

> **Explore the complete documentation on [Read the Docs](https://rlark-ci-test.readthedocs.io/en/latest/)** — start with the [Quick Start](https://rlark-ci-test.readthedocs.io/en/latest/quickstart/), then continue with the [platform user guide](https://rlark-ci-test.readthedocs.io/en/latest/user-guide/) or [administrator guide](https://rlark-ci-test.readthedocs.io/en/latest/admin-guide/).
> **Explore the complete documentation on [Read the Docs](https://rlark.readthedocs.io/en/latest/)** — start with the [Quick Start](https://rlark.readthedocs.io/en/latest/quickstart/), then continue with the [platform user guide](https://rlark.readthedocs.io/en/latest/user-guide/) or [administrator guide](https://rlark.readthedocs.io/en/latest/admin-guide/).

## What's NEW!

- [2026/08] RLark is now open-source.
- [2026/09] RLark is now open-source.

## Key Capabilities

- **Embodied AI Workload Orchestration**: From cloud GPU training (RL/LLM) to edge deployment (robot arm, sensor, camera), unified declarative Job/Workflow/Task abstraction across the full pipeline
- **Embodied AI Workload Orchestration**: From cloud GPU training (RL/LLM) to edge deployment (robot arm, sensor, camera), unified declarative Job/Task abstraction across the full pipeline
- **Multi-Runtime Data Plane**: Kubernetes provides unified management for cloud GPU clusters and edge devices across the complete training-to-deployment lifecycle; Docker and Raw runtime support will extend coverage to lightweight edge scenarios where Kubernetes is not suitable
- **Cross-Cluster Resource Abstraction**: Unify multi-site GPU clusters and edge devices via Domain (virtual network domain) and Node (compute node) CRDs, with the control plane running on kcp
- **Declarative Training Jobs**: Multi-layer abstraction (Job/Workflow/Task) with DAG-based training pipelines and declarative Ray cluster definition
- **Declarative Training Jobs**: Job/Task abstraction with declarative distributed training and Ray cluster definition
- **Cross-Cluster Pod Networking**: Virtual network based on TUN devices + gVisor netstack + SSH tunnels, enabling Pod-to-Pod communication without NAT traversal — cloud GPUs and edge robots communicate directly
- **Certificate System**: Dual-layer X.509 + SSH certificates for Agent access, Domain-scoped cross-cluster forwarding authentication, and user SSH authentication
- **Observability**: Prometheus metrics, real-time Pod log streaming, and web management UI

## Roadmap

### Available Today

- Kubernetes-based management of cloud GPU clusters and edge devices
- Declarative Job/Task orchestration for distributed training and Ray workloads
- Cross-cluster resource management through Domain and Node CRDs
- Cross-cluster Pod networking over TUN devices, gVisor netstack, and SSH tunnels
- X.509 and SSH certificate-based authentication
- Prometheus metrics, real-time Pod logs, and a web management UI

### Planned

- Docker runtime support for lightweight data planes
- Raw runtime support for hosts and edge devices without a container orchestrator
- More complete account, role, and permission management
- Continued web UI usability and workflow improvements
- Cross-cluster network throughput, latency, and resource-efficiency optimizations

## Architecture Overview

![System Architecture](apps/rlark/docs/images/architecture.png)

## Quick Start

Follow the [Quick Start Guide on Read the Docs](https://rlark-ci-test.readthedocs.io/en/latest/quickstart/) to choose one of the verified flows:
Follow the [Quick Start Guide on Read the Docs](https://rlark.readthedocs.io/en/latest/quickstart/) to choose one of the verified flows:

- **One-click CLI**: deploy the control plane and two kind data-plane clusters, then verify cross-cluster Pod networking.
- **UI-based flow**: create clusters and a Domain in the web console, deploy two kind data planes, schedule a Job across them, and verify connectivity.

## Documentation

The complete, searchable, and versioned documentation is published on **[Read the Docs](https://rlark-ci-test.readthedocs.io/en/latest/)**. Use these rendered guides as the primary entry points:
The complete, searchable, and versioned documentation is published on **[Read the Docs](https://rlark.readthedocs.io/en/latest/)**. Use these rendered guides as the primary entry points:

| Guide | Description |
|-------|-------------|
| [Quick Start](https://rlark-ci-test.readthedocs.io/en/latest/quickstart/) | Verified one-click and UI-based local deployment flows |
| [Core Concepts](https://rlark-ci-test.readthedocs.io/en/latest/concepts/) | Domain, Job, Task, Workflow, and other concepts |
| [Platform User Guide](https://rlark-ci-test.readthedocs.io/en/latest/user-guide/) | Web console, clusters, jobs, workflows, storage, and SSH keys |
| [Administrator Guide](https://rlark-ci-test.readthedocs.io/en/latest/admin-guide/) | Control plane, data plane, networking, security, and operations |
| [Developer Guide](https://rlark-ci-test.readthedocs.io/en/latest/developer-guide/) | Local development, project layout, debugging, and extensions |
| [API Reference](https://rlark-ci-test.readthedocs.io/en/latest/api/reference/) | Gateway REST API routes and behavior |
| [Architecture](https://rlark-ci-test.readthedocs.io/en/latest/architecture/) | Components, interactions, and data flows |
| [Quick Start](https://rlark.readthedocs.io/en/latest/quickstart/) | Verified one-click and UI-based local deployment flows |
| [Core Concepts](https://rlark.readthedocs.io/en/latest/concepts/) | Domain, Job, Task, and other concepts |
| [Platform User Guide](https://rlark.readthedocs.io/en/latest/user-guide/) | Web console, clusters, jobs, storage, and SSH keys |
| [Administrator Guide](https://rlark.readthedocs.io/en/latest/admin-guide/) | Control plane, data plane, networking, security, and operations |
| [Developer Guide](https://rlark.readthedocs.io/en/latest/developer-guide/) | Local development, project layout, debugging, and extensions |
| [API Reference](https://rlark.readthedocs.io/en/latest/api/reference/) | Gateway REST API routes and behavior |
| [Architecture](https://rlark.readthedocs.io/en/latest/architecture/) | Components, interactions, and data flows |

Repository-specific references remain available alongside the code:

Expand All @@ -69,7 +88,7 @@ Repository-specific references remain available alongside the code:
| [Go SDK](sdks/embodied-runtime-go/README.md) | Go client for embodied-runtime gRPC stubs |
| [Proto Definitions](proto/embodied-runtime/README.md) | gRPC service definitions for embodied-runtime |

> Prefer Chinese? Visit the **[中文 Read the Docs 站点](https://rlark-ci-test.readthedocs.io/zh-cn/latest/)**.
> Prefer Chinese? Visit the **[中文 Read the Docs 站点](https://rlark.readthedocs.io/zh-cn/latest/)**.

## Tech Stack

Expand Down
53 changes: 36 additions & 17 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
<div align="center">
<a href="README.md"><img src="https://img.shields.io/badge/lang-English-blue.svg" /></a>
<a href="README.zh-CN.md"><img src="https://img.shields.io/badge/语言-简体中文-red.svg" /></a>
<a href="https://rlark-ci-test.readthedocs.io/en/latest/"><img src="https://img.shields.io/badge/Documentation-Read%20the%20Docs-8A2BE2?logo=readthedocs&logoColor=white" alt="Documentation" /></a>
<a href="https://rlark-ci-test.readthedocs.io/zh-cn/latest/"><img src="https://img.shields.io/badge/中文文档-Read%20the%20Docs-red?logo=readthedocs&logoColor=white" alt="中文文档" /></a>
<a href="https://rlark.readthedocs.io/en/latest/"><img src="https://img.shields.io/badge/Documentation-Read%20the%20Docs-8A2BE2?logo=readthedocs&logoColor=white" alt="Documentation" /></a>
<a href="https://rlark.readthedocs.io/zh-cn/latest/"><img src="https://img.shields.io/badge/中文文档-Read%20the%20Docs-red?logo=readthedocs&logoColor=white" alt="中文文档" /></a>
<img src="https://img.shields.io/badge/Go-1.26-00ADD8?logo=go&style=flat-square" alt="Go Version" />
<img src="https://img.shields.io/badge/TypeScript-React-3178C6?logo=typescript&style=flat-square" alt="TypeScript" />
<img src="https://img.shields.io/badge/Kubernetes-kcp-326CE5?logo=kubernetes&style=flat-square" alt="Kubernetes" />
Expand All @@ -18,46 +18,65 @@

以云原生方式统一纳管跨集群具身智能工作负载,覆盖云端 GPU 训练、跨集群协同与端侧设备部署,连接 GPU 集群、机械臂、传感器和摄像头等异构资源。

> **前往 [Read the Docs 中文站](https://rlark-ci-test.readthedocs.io/zh-cn/latest/) 阅读完整文档** — 从[快速开始](https://rlark-ci-test.readthedocs.io/zh-cn/latest/quickstart/)入门,并继续查阅[平台使用指南](https://rlark-ci-test.readthedocs.io/zh-cn/latest/user-guide/)或[管理员指南](https://rlark-ci-test.readthedocs.io/zh-cn/latest/admin-guide/)。
> **前往 [Read the Docs 中文站](https://rlark.readthedocs.io/zh-cn/latest/) 阅读完整文档** — 从[快速开始](https://rlark.readthedocs.io/zh-cn/latest/quickstart/)入门,并继续查阅[平台使用指南](https://rlark.readthedocs.io/zh-cn/latest/user-guide/)或[管理员指南](https://rlark.readthedocs.io/zh-cn/latest/admin-guide/)。

## 最新动态

- [2026/08] RLark 现已开源。
- [2026/09] RLark 现已开源。

## 核心能力

- **具身智能工作负载编排**:从云端 GPU 训练(RL/LLM)到端侧部署(机械臂、传感器、摄像头),统一的声明式 Job/Workflow/Task 抽象覆盖全链路
- **具身智能工作负载编排**:从云端 GPU 训练(RL/LLM)到端侧部署(机械臂、传感器、摄像头),统一的声明式 Job/Task 抽象覆盖全链路
- **多运行时数据面**:基于 Kubernetes 统一纳管云端 GPU 集群与端侧设备,覆盖训练到具身设备部署的完整链路;面向不适合部署 Kubernetes 的轻量端侧场景,后续将扩展 Docker 和 Raw 运行时支持
- **跨集群资源抽象**:通过 Domain(虚拟网络域)和 Node(计算节点)CRD 统一管理多地 GPU 集群和端侧设备,控制面运行在 kcp 之上
- **声明式训练任务**:Job/Workflow/Task 多层抽象,支持 DAG 编排的训练流水线,声明式定义 Ray 集群
- **声明式训练任务**:Job/Task 抽象,支持声明式分布式训练与 Ray 集群定义
- **跨集群 Pod 网络**:基于 TUN 设备 + gVisor 协议栈 + SSH 隧道的虚拟网络,Pod 跨集群通信无需 NAT 穿透 — 云端 GPU 与端侧机器人直接通信
- **证书体系**:X.509 + SSH 双层证书,支持 Agent 接入、Domain 范围的跨集群转发鉴权、用户 SSH 登录鉴权
- **可观测性**:Prometheus 指标暴露、Pod 日志实时查询、Web 管理界面

## Roadmap

### 已实现

- 基于 Kubernetes 统一纳管云端 GPU 集群与端侧设备
- 面向分布式训练和 Ray 工作负载的声明式 Job/Task 编排
- 通过 Domain 和 Node CRD 管理跨集群资源
- 基于 TUN 设备、gVisor 协议栈和 SSH 隧道的跨集群 Pod 网络
- 基于 X.509 和 SSH 证书的身份认证
- Prometheus 指标、Pod 实时日志和 Web 管理界面

### 未来计划

- 面向轻量数据面的 Docker 运行时支持
- 面向无容器编排环境主机和端侧设备的 Raw 运行时支持
- 更完善的账号、角色与权限管理
- 持续优化 Web UI 易用性和操作流程
- 优化跨集群网络吞吐、时延与资源效率

## 架构概览

![系统架构](apps/rlark/docs/images/architecture.png)
![系统架构](apps/rlark/docs/images/architecture-zh.png)

## 快速开始

请前往 Read the Docs 的[快速开始指南](https://rlark-ci-test.readthedocs.io/zh-cn/latest/quickstart/),选择一种已经过验证的流程:
完整步骤请查看 Read the Docs 的[快速开始](https://rlark.readthedocs.io/zh-cn/latest/quickstart/):

- **一键 CLI 方式**:部署控制面和两个 kind 数据面集群,并验证 Pod 跨集群网络。
- **UI 交互方式**:在 Web 控制台创建集群和 Domain,部署两个 kind 数据面,将一个 Job 调度到两个集群并验证网络连通性。

## 文档索引

完整、可搜索且支持版本管理的文档发布在 **[Read the Docs 中文站](https://rlark-ci-test.readthedocs.io/zh-cn/latest/)**。面向用户的内容请优先从以下渲染后的指南进入:
完整文档见 **[Read the Docs 中文站](https://rlark.readthedocs.io/zh-cn/latest/)**:

| 指南 | 说明 |
|------|------|
| [快速开始](https://rlark-ci-test.readthedocs.io/zh-cn/latest/quickstart/) | 已验证的一键和 UI 本地部署流程 |
| [核心概念](https://rlark-ci-test.readthedocs.io/zh-cn/latest/concepts/) | Domain、Job、Task、Workflow 等概念解释 |
| [平台使用指南](https://rlark-ci-test.readthedocs.io/zh-cn/latest/user-guide/) | Web 控制台、集群、任务、工作流、存储和 SSH 密钥 |
| [管理员指南](https://rlark-ci-test.readthedocs.io/zh-cn/latest/admin-guide/) | 控制面、数据面、网络、安全和运维 |
| [开发者指南](https://rlark-ci-test.readthedocs.io/zh-cn/latest/developer-guide/) | 本地开发、项目结构、调试和扩展 |
| [API 参考](https://rlark-ci-test.readthedocs.io/zh-cn/latest/api/reference/) | Gateway REST API 路由与行为 |
| [架构设计](https://rlark-ci-test.readthedocs.io/zh-cn/latest/architecture/) | 组件、交互关系和数据流 |
| [快速开始](https://rlark.readthedocs.io/zh-cn/latest/quickstart/) | 已验证的一键和 UI 本地部署流程 |
| [核心概念](https://rlark.readthedocs.io/zh-cn/latest/concepts/) | Domain、Job、Task 等概念解释 |
| [平台使用指南](https://rlark.readthedocs.io/zh-cn/latest/user-guide/) | Web 控制台、集群、任务、存储和 SSH 密钥 |
| [管理员指南](https://rlark.readthedocs.io/zh-cn/latest/admin-guide/) | 控制面、数据面、网络、安全和运维 |
| [开发者指南](https://rlark.readthedocs.io/zh-cn/latest/developer-guide/) | 本地开发、项目结构、调试和扩展 |
| [API 参考](https://rlark.readthedocs.io/zh-cn/latest/api/reference/) | Gateway REST API 路由与行为 |
| [架构设计](https://rlark.readthedocs.io/zh-cn/latest/architecture/) | 组件、交互关系和数据流 |

与具体代码配套的说明继续保留仓库内入口:

Expand All @@ -69,7 +88,7 @@
| [Go SDK](sdks/embodied-runtime-go/README.md) | Go 语言 gRPC 客户端 |
| [Proto 定义](proto/embodied-runtime/README.md) | gRPC 服务接口定义 |

> English documentation is available on the **[English Read the Docs site](https://rlark-ci-test.readthedocs.io/en/latest/)**.
> English documentation is available on the **[English Read the Docs site](https://rlark.readthedocs.io/en/latest/)**.

## 技术栈

Expand Down
2 changes: 1 addition & 1 deletion api/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ clean-samples: ## Remove sample CRs
rm -rf $(SAMPLES_DIR)

$(CONTROLLER_GEN):
GOBIN=$(shell go env GOPATH)/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.16.5
GOBIN=$(shell go env GOPATH)/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.18.0

##@ Help

Expand Down
Loading
Loading