Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions agent_context/MAP.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,8 @@ topics:
- toppra
- motion generator
- trajectory planning
- curobo collision world
- dynamic collision integration
- 运动生成
- 运动规划
- 轨迹规划
Expand All @@ -296,6 +298,16 @@ topics:
- waypoint
- velocity
- acceleration
- canonical obstacle ID
- logical collision source ID
- physical YAML obstacle name
- sphere derived obstacle names
- empty collision mesh
- dynamic_collision_entity_ids
- collision_world_entity_ids
- collision_world_batch_mode
- collision_geometry_by_id
- make_planning_scene_provider
paths:
- topics/motion-planning/motion-planning.md
source_of_truth:
Expand All @@ -304,9 +316,11 @@ topics:
- embodichain/lab/sim/planners/curobo/curobo_planner.py
- embodichain/lab/sim/planners/curobo/curobo_yaml.py
- embodichain/lab/sim/planners/motion_generator.py
- embodichain/lab/sim/skills/scene.py
related_topics:
- robot-system
- ik-solvers
- atomic-actions
status: active

- id: rl-learning
Expand Down Expand Up @@ -429,6 +443,8 @@ topics:
- action primitive
- object semantics
- scene grounding
- scene registry
- semantic scene
- AtomicAction
- ActionInvocation
- AtomicActionEngine
Expand All @@ -453,6 +469,28 @@ topics:
- SceneSnapshotSupplier
- SceneProvider
- RigidObjectSceneProvider
- SceneRegistry
- RegistrySceneProvider
- SceneEntityRef
- SceneObjectRef
- SceneArticulationRef
- SceneLinkRef
- SceneAffordanceRef
- SceneEntityRegistration
- SceneCollisionRole
- SceneCollisionWorldMode
- from_simulation
- validate_collision_integration
- make_planning_scene_provider
- collision_geometry_by_id
- collision_world_entity_ids
- canonical scene ID
- logical collision source ID
- sphere derived obstacle names
- empty collision mesh
- flat entity namespace
- native_name
- parent native source identity
- ObjectSemantics
- entity_id
- frozen ObjectSemantics
Expand Down Expand Up @@ -514,6 +552,8 @@ topics:
- embodichain/lab/sim/atomic_actions/trajectory_ops.py
- embodichain/lab/sim/atomic_actions/primitives/
- embodichain/lab/sim/atomic_actions/__init__.py
- embodichain/lab/sim/skills/scene.py
- embodichain/lab/sim/skills/__init__.py
related_topics:
- motion-planning
- robot-system
Expand Down
110 changes: 89 additions & 21 deletions agent_context/topics/atomic-actions/atomic-actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,23 +77,81 @@ migrate an older custom action by renaming that implementation to `_plan()`.

## Object identity and pose grounding

`ObjectSemantics.entity_id` is the canonical pre-registry snapshot key. It is
optional for direct-core compatibility but, when supplied, must be a non-empty
string. Pose grounding with an explicit ID is strict: resolve it only from the
current `PlanningContext.scene`; a missing snapshot entry is an error and never
falls back to the live `entity`. Only when no ID is supplied may the core read
`ObjectSemantics.entity`; that path emits `DeprecationWarning`, reads live state,
and cannot declare a scene-motion dependency.
`ObjectSemantics.entity_id` is the typed core's canonical snapshot-key lowering
target. The registry-backed path obtains it from a resolved `SceneEntityRef`.
It remains optional for advanced direct-core compatibility but, when supplied,
must be a non-empty string. Pose grounding with an explicit ID is strict:
resolve it only from the current `PlanningContext.scene`; a missing snapshot
entry is an error and never falls back to the live `entity`. Only when no ID is
supplied may the core read `ObjectSemantics.entity`; that path emits
`DeprecationWarning`, reads live state, and cannot declare a scene-motion
dependency.

`ObjectSemantics` is shallow-frozen. Top-level fields such as `entity_id`,
`entity`, and `label` cannot be rebound after construction; create a new
semantics value to change identity. Nested affordance and metadata objects may
remain mutable, but they never establish identity.

`SceneSnapshot` owns copies of its input poses, but `EntityState.pose` tensors
are not deeply read-only. Callers must treat published snapshot values as
immutable and publish a newer scene version for changes. Enforced deep
immutability is deferred to the SceneRegistry/snapshot hardening phase.
`SceneSnapshot` owns copies of input entity states and returns a defensive
`EntityState`/pose copy on every public mapping lookup. Mutating an input tensor
or a previously returned pose cannot change the published snapshot. Publish a
new scene version for every material dynamic-state change.

## Scene registry integration

`embodichain.lab.sim.skills.SceneRegistry` is the canonical integration catalog.
It owns immutable registration metadata: typed identity, aliases, pose source,
parent relationships, backend-local names, dynamics, geometry, collision role,
semantic type, and affordance data. A `SceneSnapshot` does not duplicate that
catalog; it contains only versioned dynamic pose/confidence and collision
revision state.

All object, articulation, link, and affordance IDs occupy one flat globally
unique namespace. Store link/affordance ancestry in
`SceneEntityRegistration.parent`, not by nesting or qualifying the ID. String
lookups may resolve aliases once to a canonical typed reference. An already
typed ref must contain a canonical ID and match the registered ref class.
Duplicate IDs, ambiguous aliases, alias/canonical collisions, missing parents,
and type mismatches fail at construction or lookup. Within one reference type,
the same `(parent, native_name)` physical source cannot be assigned multiple
canonical IDs; the same local name remains valid under different parents or
for different reference types.

`SceneRegistry.from_simulation()` is explicit opt-in. Its `rigid_objects` and
`articulations` mappings are `registry_id -> simulation_uid`; selected UIDs are
installed as aliases, and unlisted simulation entities are never scanned.
Collision participation defaults to `NONE`, and every static/dynamic collision
registration requires a geometry provider.

`registry.make_planning_scene_provider(motion_generator, batch_size=...)`
returns a fresh `RegistrySceneProvider` with independent baselines and revision
counters after eager registry/provider/planner validation. Snapshots expose
canonical IDs only. The provider requires stable ordered `env_ids` and
monotonic timestamps, derives relative affordance poses from the same
observation, compares movement against the last materially published pose, and
maintains per-row collision revisions. Plain `make_scene_provider()` is only
for perception and advanced direct-core consumers without planner agreement.

For an external perception/hardware provider, call
`registry.validate_collision_integration(..., scene_provider=provider)`
directly. The registry's complete `STATIC ∪ DYNAMIC` ID set must exactly
match `MotionGenerator.collision_world_entity_ids`; separately, the registry,
provider, and planner dynamic ID sets must match exactly in the canonical
namespace. The planner must support live updates for a non-empty dynamic set,
and planner/registry batch mode must agree. With dynamic entities, one
environment may infer `SHARED`; multiple environments must explicitly select
`SceneCollisionWorldMode.SHARED` or `PER_ENV`.

Construct a registry-backed cuRobo world with
`registry.collision_geometry_by_id()`. Its default mapping includes only
`STATIC` and `DYNAMIC` registrations and excludes `NONE`. Mapping keys are
canonical logical/source IDs for cache identity and full-world validation. With
`cuboid` or `mesh`, they are also the physical YAML and runtime-update keys.
Static `sphere` sources expand to backend names such as `id_0`; dynamic sphere
configuration is rejected, while cache/full-world identity stays on `id`.
Registry mappings fail fast when a source lacks geometry required by the chosen
representation. List-valued cuRobo worlds and `RigidObjectSceneProvider` remain
advanced direct-core paths.

Stable object identity follows these exact rules:

Expand All @@ -108,11 +166,10 @@ Stable object identity follows these exact rules:
to the same live entity handle. `label` is descriptive and never establishes
identity.

This is a snapshot/identity bridge, not alias resolution. A future
`SceneRegistry` owns uniqueness, aliases, normalization, and authoritative
registry IDs. Partial-batch `StateDelta` attachment merges use the same stable
identity rules, so equivalent semantic wrappers update one held object instead
of creating label-based duplicates.
The direct-core identity rules do not perform alias resolution; normalization
belongs only to `SceneRegistry`. Partial-batch `StateDelta` attachment merges
use the same stable identity rules, so equivalent semantic wrappers update one
held object instead of creating label-based duplicates.

For both individual and coordinated attachments, a same-identity partial merge
preserves scalar metadata: if any previously active environment row remains,
Expand Down Expand Up @@ -251,11 +308,14 @@ acknowledgement timeout in their transport/controller layer.
boundary used by execution adapters. `SceneSnapshot.collision_entity_ids`
identifies obstacle poses consumed by a planner, while
`collision_world_revision` is either global or per environment. A newer
revision invalidates only affected batch rows. `RigidObjectSceneProvider`
tracks live simulation objects, filters sub-threshold pose noise, advances the
general scene version, and maintains per-environment collision revisions.
Thresholds are measured from the last materially published pose per entity and
environment, so repeated sub-threshold motion eventually becomes observable.
revision invalidates only affected batch rows. `RegistrySceneProvider` is the
canonical provider and derives its entity/collision sets from one immutable
`SceneRegistry`. It filters sub-threshold pose noise, advances the general scene
version, and maintains per-environment collision revisions. Thresholds are
measured from the last materially published pose per entity and environment, so
repeated sub-threshold motion eventually becomes observable.
`RigidObjectSceneProvider` retains that lower-level revision behavior for
advanced direct-core integrations.
For lightweight sources that do not need environment correlation IDs,
`SimulationExecutionAdapter` also accepts a mutually exclusive
`SceneSnapshotSupplier(timestamp)` callback.
Expand All @@ -271,6 +331,14 @@ parameters to each skill. Add/remove/geometry mutations are not yet supported
by this pose-update path; providers should revision only pose-updatable
registered obstacles.

`BasePlanner.collision_world_entity_ids`, `dynamic_collision_entity_ids`, and
`collision_world_batch_mode` expose the backend's complete world, dynamic
subset, and batching contract. `MotionGenerator` validates and forwards those
properties for `SceneRegistry.make_planning_scene_provider()`. External
providers call `validate_collision_integration(..., scene_provider=...)`.
These construction checks are separate from per-plan
`bind_collision_world()`.

Runnable closed-loop examples live under `scripts/tutorials/atomic_action/`:
`tracking_error_recovery.py`, `moving_target_recovery.py`, and
`dynamic_obstacle_recovery.py`. Each injects one disturbance, reports the
Expand Down
41 changes: 41 additions & 0 deletions agent_context/topics/motion-planning/motion-planning.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,19 @@ Learning-based EEF waypoint planner. Franka Panda only.

### CuroboPlanner collision worlds

`CuroboWorldCfg.rigid_objects` accepts either a mapping or a sequence. Use
`Mapping[registry_id, RigidObject]` for a registry-backed integration. The
mapping key is the authoritative logical/source obstacle ID used by the
content-cache key, `collision_world_entity_ids`, and registry validation. For
`cuboid` and `mesh`, it is also the physical YAML obstacle name and dynamic
update key. For `sphere`, one static source expands to physical YAML names such
as `registry_id_0`; dynamic sphere configuration is rejected, while cache and
full-world identity remain keyed by `registry_id`. A registry mapping whose
source lacks mesh geometry required by the selected representation fails fast
instead of silently dropping the source. The sequence form is an advanced
direct-core path that derives names from each object's `uid` or an
`obstacle_<index>` fallback.

`CuroboWorldCfg.multi_env` controls collision-world batching, not whether robot
states or goals are batched:

Expand Down Expand Up @@ -126,10 +139,33 @@ them into `CuroboPlanOptions.dynamic_obstacle_poses`.
to the backend hook. Atomic actions use that facade from their framework-owned
`plan()` template when a `SceneSnapshot` declares collision entities;
individual skills must not construct backend obstacle options themselves.
`BasePlanner.collision_world_entity_ids`, `dynamic_collision_entity_ids`, and
`collision_world_batch_mode` expose the complete canonical world, its dynamic
subset, and the `"shared"` / `"per_env"` mode. `MotionGenerator` validates and
forwards those properties to the integration layer. For cuRobo, the complete
set is every mapping key (or inferred sequence name), while the dynamic set is
exactly `CuroboWorldCfg.dynamic_obstacle_names`. Sphere-expanded physical YAML
names are not part of either logical ID declaration.
`CuroboWorldCfg` rejects duplicate obstacle names and requires every
`dynamic_obstacle_name` to match an object registered in `rigid_objects`, so a
planner-local mismatch fails before backend construction.

For the canonical path, pass `SceneRegistry.collision_geometry_by_id()` into
`CuroboWorldCfg.rigid_objects`, derive dynamic names from the registry, and call
`SceneRegistry.make_planning_scene_provider(motion_generator, batch_size=...)`
before execution. The geometry mapping excludes `NONE` registrations. The
factory first requires the registry's complete `STATIC ∪ DYNAMIC` set to
equal `MotionGenerator.collision_world_entity_ids`, then requires exact
registry/derived-provider/planner dynamic-subset agreement. It also checks
update capability for a non-empty dynamic set and the same collision-world
batch mode. An external perception/hardware provider instead uses
`validate_collision_integration(..., scene_provider=provider)`.

One environment may infer `SHARED`; a multi-environment registry with dynamic
entities must explicitly choose `SHARED` or `PER_ENV`. Alias normalization
happens before planner construction, so planner IDs must never be simulator
UIDs unless that string is also the chosen canonical registry ID.

`MotionGenerator.resolve_plan_options()` is the corresponding option-ownership
boundary. It copies caller-supplied typed options, otherwise obtains backend
defaults; for TOPPRA it maps the requested sample count and generic
Expand Down Expand Up @@ -271,3 +307,8 @@ The decorator checks that every `PlanState` in `target_states` shares the same l
- **Fork safety with GPU sim** — `ToppraPlannerCfg.mp_context=None` defaults to `spawn` on GPU to avoid fork-after-CUDA-init hazards. Force `fork` only when the sim device is CPU or you have verified it is safe.
- **cuRobo shared-world mismatch** — World-frame poses may differ solely because replicated arenas are offset. Compare poses after robot-base rebasing: keep `multi_env=False` if they match, and enable it only when robot-relative layouts differ.
- **Dynamic obstacles silently stale** — A planner participates in atomic-action collision revision recovery only when it declares `supports_collision_world_updates=True`; its hook must bind every `collision_entity_id` pose into the current planning attempt.
- **Registry/planner identity drift** — Registry-backed cuRobo worlds must use a
canonical-ID mapping, not a list whose names are inferred from UIDs. Validate
exact full registry/planner collision-world agreement, dynamic
registry/provider/planner agreement, and batch-mode agreement through
`SceneRegistry` before starting execution.
Loading