feat: improve networking, deployment, and roadmap

README.md

@@ -36,22 +36,7 @@ Manage cross-cluster embodied intelligence workloads through a unified cloud-nat
 
 ## Roadmap
 
-### Available Today
-
-- Kubernetes-based management of cloud GPU clusters and edge devices
-- Declarative Job/Task orchestration for distributed training and Ray workloads
-- Cross-cluster resource management through Domain and Node CRDs
-- Cross-cluster Pod networking over TUN devices, gVisor netstack, and SSH tunnels
-- X.509 and SSH certificate-based authentication
-- Prometheus metrics, real-time Pod logs, and a web management UI
-
-### Planned
-
-- Docker runtime support for lightweight data planes
-- Raw runtime support for hosts and edge devices without a container orchestrator
-- More complete account, role, and permission management
-- Continued web UI usability and workflow improvements
-- Cross-cluster network throughput, latency, and resource-efficiency optimizations
+See the [RLark Roadmap](ROADMAP.md) for planned runtime, account management, and custom workload resource improvements.
 
 ## Architecture Overview
 

README.zh-CN.md

@@ -36,22 +36,7 @@
 
 ## Roadmap
 
-### 已实现
-
-- 基于 Kubernetes 统一纳管云端 GPU 集群与端侧设备
-- 面向分布式训练和 Ray 工作负载的声明式 Job/Task 编排
-- 通过 Domain 和 Node CRD 管理跨集群资源
-- 基于 TUN 设备、gVisor 协议栈和 SSH 隧道的跨集群 Pod 网络
-- 基于 X.509 和 SSH 证书的身份认证
-- Prometheus 指标、Pod 实时日志和 Web 管理界面
-
-### 未来计划
-
-- 面向轻量数据面的 Docker 运行时支持
-- 面向无容器编排环境主机和端侧设备的 Raw 运行时支持
-- 更完善的账号、角色与权限管理
-- 持续优化 Web UI 易用性和操作流程
-- 优化跨集群网络吞吐、时延与资源效率
+计划中的运行时、账号管理与自定义任务资源改进，请参阅 [RLark 路线图](ROADMAP.zh-CN.md)。
 
 ## 架构概览
 

ROADMAP.md

@@ -0,0 +1,107 @@
+# RLark Roadmap
+
+[English](ROADMAP.md) | [简体中文](ROADMAP.zh-CN.md)
+
+This roadmap describes how the open-source RLark project will evolve from a Kubernetes-based workload platform into open, extensible infrastructure for cross-cluster embodied intelligence. RLark will provide stable resource models, APIs, and extension mechanisms for integrating external systems and adapting deployments to different environments. Priorities and implementation details may change as the project evolves.
+
+## Design Principles
+
+- Keep tenant isolation, identity, authorization, resource ownership, and lifecycle state in the RLark core rather than delegating security boundaries to extensions.
+- Integrate extensions as independently deployed services through versioned gRPC or HTTP APIs instead of loading code into RLark processes.
+- Define focused extension contracts for each domain instead of one unrestricted plugin API.
+- Support capability discovery so the control plane and API consumers do not assume every data plane is Kubernetes or supports the same operations.
+- Preserve declarative, idempotent reconciliation so operations can recover from retries, service restarts, and network failures.
+
+## Phase 1: Runtime and Workload Extensibility
+
+### Data Plane Runtimes
+
+The data plane currently supports Kubernetes only. Introduce a versioned Runtime Provider contract and add:
+
+- **Docker**: run and manage tasks directly with Docker for lightweight environments where Kubernetes is not suitable.
+- **Raw**: run and manage tasks directly on hosts or edge devices without a container runtime or orchestrator.
+- **Capability discovery**: advertise supported workload types, logs, exec, networking, storage, devices, and other runtime operations so unsupported features can be rejected before scheduling.
+
+The existing Kubernetes implementation should become the first implementation of the same runtime contract used by Docker and Raw providers.
+
+### Custom Workload Resources
+
+The Kubernetes data plane currently supports three built-in workload resource types: Deployment, StatefulSet, and DaemonSet.
+
+Introduce a service-based Workload Adapter contract for additional Kubernetes resources and external workload systems. The contract should cover the complete lifecycle, including validation, rendering or creation, status observation, updates, and deletion. It should support both:
+
+- **Declarative translation**: convert an RLark Task into resources such as RayJob, PyTorchJob, or a custom Kubernetes resource, while the Agent retains responsibility for applying and observing them.
+- **Delegated execution**: submit and reconcile work through an external service such as Slurm, a cloud training service, or a domain-specific robotics platform.
+
+The exact API remains under design. Requests must be versioned and idempotent, and generated resources must pass RLark authorization and policy checks before execution.
+
+### Device Capability Foundation
+
+RLark should elevate embodied devices from runtime implementation details into reusable platform primitives. This provides a common foundation for discovering, observing, allocating, and controlling cameras, robot arms, sensors, and accelerators without hard-coding every device type into RLark core.
+
+- **Device model and inventory**: stable identities, ownership and scope, location and topology, device class, vendor metadata, structured capabilities, endpoints, and lifecycle state.
+- **Discovery and synchronization**: service-based Device Providers discover physical or virtual devices and continuously reconcile their health, availability, capabilities, and connection state.
+- **Observation**: uniform APIs and events for status, telemetry, logs, alarms, and capability changes.
+- **Controlled operations**: capability-described commands such as opening a camera, capturing a frame, switching a robot mode, or resetting a device, with validation, timeout, idempotency, and result reporting.
+- **Data channels**: authorized access to real-time streams and large device data without forcing video frames or telemetry through the control-plane resource store.
+- **Allocation and leases**: exclusive or shared reservations, expiration, renewal, release, and conflict handling so interactive control and scheduled Tasks cannot use a device unsafely at the same time.
+- **Safety and authorization**: operation-level permissions, policy checks, emergency-stop and other safety boundaries where supported, complete auditing, and explicit distinction between read-only observation and physical control.
+
+The existing embodied-runtime robot and camera APIs can serve as initial use cases, while the platform contract should remain extensible to new device classes and vendor-specific operations. Generic metadata and capability schemas should be combined with typed, versioned operation interfaces where safety or interoperability requires stronger semantics.
+
+## Phase 2: Identity, Authorization, and Resource Scopes
+
+Improve account management and provide reusable security primitives without prescribing a single tenancy model:
+
+- Users, service accounts, machine identities, roles, and permissions.
+- Explicit ownership and scope for Domains, Nodes, Jobs, Workflows, Tasks, runtimes, credentials, policies, and quotas.
+- Delegated, short-lived, renewable, and revocable credentials for Agents, extension services, and external controllers.
+- End-to-end propagation of verified identity and scope context across the Gateway, controllers, database, Server, Agents, and extension services.
+- Scope-aware authorization for API access, list and watch operations, logs, exec, proxying, events, and secrets.
+- Audited privileged access and integration with external identity providers such as OIDC where appropriate.
+
+These primitives should allow operators to model users, projects, workspaces, organizations, or other security boundaries without requiring RLark to prescribe a single tenancy workflow. Core authorization and resource isolation must still be enforced by RLark rather than delegated entirely to schedulers or policy extensions.
+
+## Phase 3: Service-Based Extension Framework
+
+Provide a common operational framework for independently deployed extension services while keeping separate contracts for different extension domains.
+
+Initial extension types:
+
+- **Runtime Provider**: execute Tasks on Kubernetes, Docker, Raw hosts, or other runtimes.
+- **Workload Adapter**: translate or delegate custom workload lifecycles.
+- **Device Provider**: discover devices, publish structured capabilities and status, and expose authorized observation and control operations.
+- **Admission and Policy Service**: validate or mutate requests according to deployment-specific policy.
+- **Event Consumer**: integrate approvals, notifications, metering, auditing, and automation through webhooks or event streams.
+
+The framework should provide:
+
+- Declarative extension registration, health checks, API version negotiation, and capability discovery.
+- mTLS service identity, least-privilege authorization, request auditing, and secret-safe configuration.
+- Deadlines, retries, idempotency keys, circuit breaking, and reconciliation after lost events or partial failures.
+- `ExtensionDefinition` and scoped `ExtensionBinding` resources, separating installation and trust decisions from scope-specific configuration.
+- Protocol specifications, SDKs as convenience libraries, conformance tests, and reference implementations.
+
+Initially, platform administrators should approve extension endpoints. Allowing regular users to register arbitrary endpoints requires additional controls against data exfiltration, credential misuse, and server-side request forgery.
+
+## Phase 4: Open Integration and Customization
+
+Provide stable, composable interfaces so operators and developers can add domain capabilities without modifying RLark core:
+
+- **Extension resources and external controllers**: define resources such as `TrainingPipeline` or `RobotMission`, then reconcile them into RLark Jobs and Tasks through stable APIs.
+- **Stable public APIs and events**: prevent external systems and extensions from depending on internal database schemas or implementation details.
+- **Device APIs and data-plane access**: support integrations such as device inventories, status views, remote operations, and live camera access through stable discovery, control, event, and streaming contracts.
+- **Schema-driven UI contributions**: generate forms and resource views from schemas before considering arbitrary frontend code or micro-frontends.
+- **Quotas, policies, and metering events**: provide foundations for approvals, resource governance, usage accounting, and external system integration.
+
+RLark should expose composable primitives rather than hard-code a particular application workflow. Deployment-specific onboarding, user interfaces, approvals, and domain resources can be implemented outside RLark core.
+
+## Future Extension Areas
+
+After the foundational contracts are proven, evaluate focused service interfaces for:
+
+- Scheduler filtering and scoring based on topology, locality, latency, cost, quota, or domain-specific constraints.
+- Dataset, model, artifact, object storage, and secret providers.
+- Logs, metrics, traces, and alert integrations based on open observability standards.
+
+These interfaces should be introduced from concrete use cases rather than through a single generic plugin abstraction.

ROADMAP.zh-CN.md

@@ -0,0 +1,107 @@
+# RLark 路线图
+
+[English](ROADMAP.md) | [简体中文](ROADMAP.zh-CN.md)
+
+本文档描述开源项目 RLark 如何从基于 Kubernetes 的任务平台演进为开放、可扩展的跨集群具身智能基础设施。RLark 将提供稳定的资源模型、接口与扩展机制，便于在不同部署环境中集成外部系统并按需扩展能力。随着项目演进，优先级和具体实现方式可能会有所调整。
+
+## 设计原则
+
+- 租户隔离、身份、授权、资源归属和生命周期状态由 RLark 核心负责，不把安全边界委托给扩展服务。
+- 扩展以独立服务部署，通过版本化的 gRPC 或 HTTP API 接入，而不是向 RLark 进程内加载代码。
+- 按领域定义职责清晰的扩展协议，不设计一个无边界的通用插件接口。
+- 提供能力发现机制，使控制面和 API 使用方不再假设所有数据面都是 Kubernetes 或支持相同操作。
+- 保持声明式、幂等的协调机制，使操作能够从重试、服务重启和网络故障中恢复。
+
+## 第一阶段：运行时与任务扩展
+
+### 数据面运行时
+
+目前数据面仅支持 Kubernetes 模式。后续将定义版本化的 Runtime Provider 协议，并增加：
+
+- **Docker**：面向不适合部署 Kubernetes 的轻量环境，直接通过 Docker 运行和管理任务。
+- **Raw**：面向没有容器运行时或编排系统的主机及端侧设备，直接运行和管理任务。
+- **能力发现**：声明支持的任务类型、日志、Exec、网络、存储、设备和其他运行时操作，使平台能够在调度前拒绝不受支持的功能。
+
+现有 Kubernetes 实现应成为 Runtime Provider 协议的首个实现，并与 Docker、Raw Provider 使用相同的核心契约。
+
+### 自定义任务资源
+
+目前 Kubernetes 数据面仅支持 Deployment、StatefulSet 和 DaemonSet 三种内置任务资源类型。
+
+后续将定义服务化的 Workload Adapter 协议，用于接入更多 Kubernetes 资源和外部任务系统。协议应覆盖校验、渲染或创建、状态观察、更新和删除等完整生命周期，并支持两种模式：
+
+- **声明式转换**：将 RLark Task 转换为 RayJob、PyTorchJob 或自定义 Kubernetes 资源，由 Agent 负责应用资源并观察状态。
+- **委托执行**：通过 Slurm、云训练服务或机器人任务平台等外部服务提交和协调任务。
+
+具体 API 仍在设计中。调用必须支持版本化和幂等处理，生成的资源在执行前必须再次经过 RLark 的授权与策略校验。
+
+### 设备能力基座
+
+RLark 应将具身设备从运行时实现细节提升为可复用的平台原语，为摄像头、机械臂、传感器与加速器提供统一的发现、观察、分配和控制基础，而不需要将每种设备类型硬编码到 RLark 核心。
+
+- **设备模型与资产目录**：提供稳定身份、归属与作用域、位置与拓扑、设备类别、厂商信息、结构化能力、访问端点和生命周期状态。
+- **发现与同步**：通过服务化 Device Provider 发现物理或虚拟设备，并持续协调其健康、可用性、能力和连接状态。
+- **观察能力**：通过统一 API 与事件获取状态、遥测、日志、告警和能力变化。
+- **受控操作**：根据设备能力描述开放摄像头、抓取图像、切换机械臂模式或重置设备等命令，并提供参数校验、超时、幂等和结果回报。
+- **数据通道**：在完成授权后访问实时流和大体量设备数据，避免将视频帧或高频遥测数据写入控制面资源存储。
+- **分配与租约**：支持独占或共享预约、过期、续期、释放和冲突处理，避免交互式控制与调度任务同时不安全地使用同一设备。
+- **安全与授权**：提供操作级权限、策略校验、设备支持范围内的急停等安全边界、完整审计，并明确区分只读观察与物理控制权限。
+
+现有 embodied-runtime 的机器人和摄像头 API 可以作为首批验证场景，但平台协议应能够扩展到新的设备类别和厂商专有操作。通用元数据与能力 Schema 可以保证开放性；对于涉及安全或互操作性的操作，则应采用强类型、版本化的接口。
+
+## 第二阶段：身份、授权与资源作用域
+
+完善账号管理并提供可复用的安全原语，但不限定唯一的租户组织方式：
+
+- 用户、服务账号、机器身份、角色与权限。
+- Domain、Node、Job、Workflow、Task、Runtime、凭证、策略和配额的明确归属与作用域。
+- 面向 Agent、扩展服务和外部 Controller 的可委托、短期、可续期和可吊销凭证。
+- 经过验证的身份与作用域上下文贯穿 Gateway、Controller、数据库、Server、Agent 和扩展服务。
+- API、List/Watch、日志、Exec、代理、事件和 Secret 的作用域级授权。
+- 特权操作审计，并按需对接 OIDC 等外部身份提供方。
+
+这些原语应允许部署者按需建立用户、项目、Workspace、组织或其他安全边界，而不要求 RLark 固化唯一的租户流程。核心授权与资源隔离仍必须由 RLark 强制保证，不能完全委托给调度器或策略扩展。
+
+## 第三阶段：服务化扩展框架
+
+为独立部署的扩展服务提供统一的运行机制，同时为不同扩展领域保留独立协议。
+
+首批扩展类型包括：
+
+- **Runtime Provider**：在 Kubernetes、Docker、Raw 主机或其他运行环境中执行 Task。
+- **Workload Adapter**：转换或委托自定义任务的完整生命周期。
+- **Device Provider**：发现设备、发布结构化能力与状态，并开放经过授权的观察和控制操作。
+- **Admission 与 Policy Service**：根据部署环境定义的策略校验或修改请求。
+- **Event Consumer**：通过 Webhook 或事件流接入审批、通知、计量、审计和自动化流程。
+
+扩展框架应提供：
+
+- 声明式扩展注册、健康检查、API 版本协商和能力发现。
+- mTLS 服务身份、最小权限授权、调用审计和安全的 Secret 配置方式。
+- 超时、重试、幂等键、熔断，以及在事件丢失或部分失败后的状态协调。
+- `ExtensionDefinition` 和限定作用域的 `ExtensionBinding`，将扩展安装及信任决策与具体作用域配置分离。
+- 协议规范、便捷使用的 SDK、一致性测试和参考实现。
+
+初期扩展服务地址应由平台管理员审核。允许普通用户注册任意地址前，需要防范数据外泄、凭证滥用和服务端请求伪造等风险。
+
+## 第四阶段：开放集成与二次开发能力
+
+提供稳定、可组合的开放接口，使部署者和二次开发者无需修改 RLark 核心代码即可扩展领域能力：
+
+- **扩展资源与外部 Controller**：允许定义 `TrainingPipeline`、`RobotMission` 等领域资源，并通过稳定 API 将其协调为 RLark Job 和 Task。
+- **稳定的公开 API 与事件**：避免外部系统和扩展依赖内部数据库结构或实现细节。
+- **设备 API 与数据面访问**：通过稳定的发现、控制、事件和流式传输协议，支持设备资产目录、状态视图、远程操作和实时摄像头等集成场景。
+- **Schema 驱动的 UI 扩展**：优先通过 Schema 生成表单和资源视图，而不是直接引入任意前端代码或微前端。
+- **配额、策略和计量事件**：为审批、资源治理、用量核算和外部系统集成提供基础能力。
+
+RLark 应提供可组合的基础原语，而不是固化某一种应用流程。具体的用户接入、界面、审批和领域资源可根据部署需求在 RLark 之外实现。
+
+## 后续扩展方向
+
+在基础协议得到实际验证后，再根据具体场景评估以下服务化扩展接口：
+
+- 基于拓扑、数据位置、网络时延、成本、配额或领域约束的调度过滤与评分。
+- 数据集、模型、制品、对象存储和 Secret Provider。
+- 基于开放可观测标准的日志、指标、链路追踪和告警集成。
+
+这些接口应从真实用例出发逐步引入，而不是通过一个通用插件抽象提前覆盖所有场景。