Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 42 additions & 56 deletions demo/carla/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,9 @@
Drive simulated trucks from a browser over MoQ. Trucks share one
[CARLA](https://carla.org) world and drive themselves until someone takes
control. Each truck publishes its cameras and a status track; a web player
watches them on a map, and a driver steers one by dialing the truck's
[socket](../../doc/concept/socket.md) endpoint: its controls and the truck's
acknowledgements travel on a session only the two of them read, while what
every viewer shows stays on `status`.
watches them on a map, and a driver steers one by publishing a **control**
track back. The truck follows every driver's broadcast and echoes what it
accepted on `status`, which every viewer reads.

```text
├── web/ # the player: the <moq-carla> element, with map, video, keyboard and steering-wheel input (@moq/demo-carla)
Expand Down Expand Up @@ -45,29 +44,24 @@ The player alone runs against any relay with `just carla web <url>`. Append
`?preview` to the local URL for a fixture-driven layout review that opens no
connection; it is removed from production builds.

## Broadcasts and sessions
## Broadcasts

```text
carla/{truck}/{camera}.hang <- one media broadcast per camera (hang catalog)
sd 240p H.264, for map previews
hd 720p H.264, for the focused view
status JSON Snapshot telemetry

carla/.server/{truck} <- the truck's server socket endpoint, which drivers dial
carla/.client/{id} <- one client endpoint per driver lease
viewer/carla/{truck}/{driverId} <- one broadcast per driver lease
control one JSON command per group
```

Each truck accepts `track` sessions on its server endpoint. A driver dials from
a client endpoint of its own, one session per lease, and sends one control per message;
the truck answers each control it accepted from the lease holder on the same
session. Cameras are sibling broadcasts, so switching views never creates a
second claim on the same wheel.
The worker follows the announcements under `viewer/carla/{truck}/` for each
truck and fans every driver's `control` track into that truck's gate. Cameras
are sibling broadcasts, so switching views never creates a second claim on the
same wheel, and the driver scope is disjoint from every media path.

Sockets need moq-lite-07 or moq-transport on every hop, so the worker and the
player offer only `moq-lite-07-wip` and `moq-transport-22`; a relay must
accept one of them.

### Control (player to truck, on the driver's session)
### Control (player to truck)

```json
{ "type": "drive", "sequence": 42, "steering": -1.0..1.0,
Expand All @@ -78,27 +72,18 @@ accept one of them.
{ "type": "respawn", "sequence": 45, "timestamps": [...] }
```

The truck reads them oldest first, skipping one that falls a heartbeat (250 ms)
behind the newest, so a late control never holds up a newer one, while a burst,
such as a reset right before a drive, still arrives whole.

### Acknowledgement (truck to player, on the driver's session)

```json
{ "sequence": 44, "media_pipeline": [ { "label": "received→decoded", "ms": 12 } ] }
```

Sent for each `drive`, `estop`, and `reset_estop` the truck accepted from the
lease holder, so the player measures its round trip and its own media pipeline.
A bystander's e-stop, a command the gate refused, or an accepted `respawn`
(confirmed from `status` instead) gets none.
Each command is a group of its own, and a superseded one stays fresh for a
heartbeat (250 ms, the track's max age). The truck reads them oldest first with
the same delay budget, so a late control never holds up a newer one, while a
burst, such as a reset right before a drive, still arrives whole.

### Status (truck to every viewer)

```json
{ "accepted": { "steering": .., "throttle": .., "brake": .., "gear": ".." },
"autonomy": "autonomous|manual", "estopped": true,
"driver": "driver-3eb1bd43", "estop_by": "driver-9c0a7f12",
"driver": "3eb1bd43c0a7", "estop_by": "safety-9c0a7f12e4d1", "sequence": 44,
"media_pipeline": { "3eb1bd43c0a7": [ { "label": "received→decoded", "ms": 12 } ] },
"map": "Town10HD_Opt",
"pose": { "x": 12.35, "y": -78.9, "yaw": 180.1, "speed": 9.88 },
"measured": { "steering": .., "throttle": .., "brake": .., "gear": ".." },
Expand All @@ -111,6 +96,11 @@ autonomous truck changes no accepted state as it moves. `pose` is in CARLA
world metres (x east, y south, yaw clockwise from +x), which is what the
player's baked road maps use.

`sequence` and `media_pipeline` acknowledge the lease holder: the newest
`drive`, `estop`, or `reset_estop` the truck accepted from it, which the player
times its round trip by, and the media pipeline stages it reported. A
bystander's e-stop or a refused command changes neither.

`accepted` is the gate repeating what it let through. `measured`, `motion`, and
`collision` come from the vehicle itself, so the player can draw requested
against measured steering and show the gap. `collision.at` is the unix
Expand All @@ -124,15 +114,17 @@ collision.
The worker's gate (`sim/moq_carla_sim/drive.py`) owns every truck's controls:

- **One driver at a time.** The first driver to send a command holds the
lease until its session ends or its 1 s watchdog expires; the player
heartbeats every 250 ms. The lease is bound to the driver's session, so a
driver that redials, even from the same endpoint id, is a new claimant
rather than a continuation.
lease until its broadcast ends or its 1 s watchdog expires; the player
heartbeats every 250 ms, and lets go once `status` stops naming it. The
lease is bound to the driver's path and the publisher instance serving it,
so a driver that republishes at the same path is a new claimant rather than
a continuation.
- **E-stop latches** until an explicit reset, from any viewer. The lease
holder sends it on its session; any other viewer dials a short-lived session
of its own, so a bystander's stop neither takes the lease nor waits behind
the holder's controls. A latched truck is also cleared when it respawns as
stopped, below.
holder sends it on its control track; any other viewer publishes a
short-lived `safety-{id}` broadcast of its own until `status` echoes it, so
a bystander's stop neither takes the lease nor waits behind the holder's
controls. A latched truck is also cleared when it respawns as stopped,
below.
- **Gear changes are interlocked**: the gate keeps the current gear unless the
command releases throttle and applies some brake.
- **Takeover starts braking**: the player keeps measured steering and gear,
Expand All @@ -148,20 +140,15 @@ The worker's gate (`sim/moq_carla_sim/drive.py`) owns every truck's controls:
- **A stopped truck respawns**: any truck that has not moved 2 m in 2 minutes
of a running world, whether autonomous, e-stopped, or parked under a driver,
returns to a free spawn point under autonomy, its e-stop cleared. The worker
ends the holder's driving session, so the player lets go instead of
ignores the rest of the holder's run, so the player lets go instead of
reclaiming the fresh spawn.

The lease is cooperative. Anonymous drivers share one grant to publish
`carla/.client/*`, so an endpoint id is all it takes to announce in a driver's
place and win the relay's route to its next session. Each driver's id is
therefore 128 random bits, and `status` names the holder and the e-stop's
sender by a handle instead: `driver-` and 8 hex digits of the SHA-256 of the
session's secret id, which the driver derives the same way to recognize its
own lease. The grant covers only client endpoints, so no driver can announce a
truck's server endpoint, whatever its name. Anyone may still claim a free truck
or press its e-stop, so the lease stops two honest drivers fighting over a
truck, not a deliberate disruption. Anything beyond a simulator needs
per-driver tokens scoped to the driver's own endpoint.
The lease is cooperative. Anonymous drivers share one grant to publish under
`viewer/**`, and `status` names the holder, so anyone can announce at the
holder's path under another instance, which ends the holder's run and frees
the lease. It stops two honest drivers fighting over a truck, not a deliberate
takeover. Anything beyond a simulator needs per-driver tokens scoped to the
driver's own path.

## The simulator

Expand Down Expand Up @@ -223,10 +210,9 @@ checkout.
just carla venv
just carla service-install

# The units read this. MOQ_URL carries a token allowed to publish `carla/**`,
# which covers the trucks' server endpoints, and subscribe `carla/.client/**`,
# the endpoints drivers dial from; keeping it here keeps it off the command
# line. CARLA_IMAGE is the tag the moq-dev/carla build printed, since the worker
# The units read this. MOQ_URL carries a token allowed to publish `carla/**`
# and subscribe `viewer/carla/**`, where drivers publish; keeping it here keeps
# it off the command line. CARLA_IMAGE is the tag the moq-dev/carla build printed, since the worker
# refuses a stock simulator.
cat >~/.config/moq/carla.env <<'EOF'
MOQ_URL=https://cdn.moq.dev/demo?jwt=...
Expand Down
80 changes: 40 additions & 40 deletions demo/carla/sim/moq_carla_sim/drive.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@

from __future__ import annotations

import hashlib
from dataclasses import dataclass, field, replace
from enum import Enum
from typing import Literal
Expand All @@ -24,16 +23,6 @@ class Gear(str, Enum):
REVERSE = "reverse"


def handle(session: str) -> str:
"""The name `status` gives the lease or e-stop a session carried.

A hash of the session id, which only that session's two ends know, so the
driver recognizes its own lease while no viewer learns its endpoint id,
which anyone could announce to take the route to that driver's next dial.
"""
return "driver-" + hashlib.sha256(session.encode()).hexdigest()[:8]


def _clamp(value: float, low: float, high: float) -> float:
return max(low, min(high, value))

Expand Down Expand Up @@ -105,17 +94,15 @@ class Command:
`drive`, `estop`, `reset_estop`, and `respawn` arrive on the wire;
`driver_left` and `watchdog_expired` are synthesized locally.

`driver_id` is the driver's socket endpoint id, which stays private.
`session` is the socket session that carried it, one per lease the player
takes: the same id on another session is a different driver for the lease.
The session id is the secret that keeps a session to its two ends, so only
its `handle` leaves this process. Only a locally synthesized command has
none.
`driver_id` is the driver's path segment, which `status` echoes. `instance`
names the run that sent it, one per publisher instance serving the path: the
same id under another instance is another run of the driver, so it is a
different driver for the lease.
"""

kind: CommandKind
driver_id: str = ""
session: str | None = None
instance: str | None = None
sequence: int = 0
state: DriveState | None = None
timestamps: tuple[TimestampEntry, ...] = ()
Expand All @@ -127,30 +114,37 @@ class Drive:
The first driver to send a command owns the truck until it disconnects or
its command lease expires. Other drivers cannot silently steal control.

The lease binds to the driver's socket session, so a driver whose session
ends loses it, and another session claiming the same driver id is another
driver. `driver` and `estop_by` are session handles, never endpoint ids, so
the public `status` track gives nobody an id to announce in the holder's
place.
The lease binds to the driver's broadcast path and the run serving it, so a
driver whose broadcast ends or restarts loses it.
It is still cooperative, not enforced: the holder's id is echoed on the
public `status` track, so anyone allowed to publish there can announce at
that path under another instance, which ends the holder's broadcast and frees
the lease for the newcomer. This stops two honest drivers colliding; it
does not stop a deliberate takeover.
"""

def __init__(self) -> None:
self._driver: str | None = None
self._session: str | None = None
self._instance: str | None = None
self._sequence: int | None = None
self._state = DriveState.stopped()
self._takeover_pedals: tuple[float, float] | None = None
self._estopped = False
self._estop_by: str | None = None

@property
def driver(self) -> str | None:
"""The lease holder's handle, or None while nobody holds it."""
return None if self._session is None else handle(self._session)
return self._driver

@property
def holder(self) -> tuple[str, str | None] | None:
"""The lease holder's driver id and run, or None while nobody holds it."""
return None if self._driver is None else (self._driver, self._instance)

@property
def holder(self) -> tuple[str, str] | None:
"""The lease holder's driver id and session, or None while nobody holds it."""
return None if self._driver is None or self._session is None else (self._driver, self._session)
def sequence(self) -> int | None:
"""The newest sequence the holder's run had accepted, which `status` echoes."""
return self._sequence

@property
def state(self) -> DriveState:
Expand All @@ -162,25 +156,24 @@ def estopped(self) -> bool:

@property
def estop_by(self) -> str | None:
"""The handle of the session that latched the e-stop."""
return self._estop_by

def holds(self, command: Command) -> bool:
"""Whether `command` comes from the session holding the lease."""
return self._driver is not None and (self._driver, self._session) == (command.driver_id, command.session)
"""Whether `command` comes from the run of the driver holding the lease."""
return self._driver is not None and (self._driver, self._instance) == (command.driver_id, command.instance)

def accepted(self) -> Accepted:
return Accepted(state=self._state, estopped=self._estopped)

def apply(self, command: Command) -> bool:
"""Apply a command, returning whether it changed or refreshed state."""
if command.kind == "drive":
# A lease binds to a session, so a command without one cannot hold it.
if command.session is None or self._estopped or not (self._driver is None or self.holds(command)):
if self._estopped or not (self._driver is None or self.holds(command)):
return False
claiming = self._driver is None
self._driver = command.driver_id
self._session = command.session
self._instance = command.instance
self._sequence = command.sequence
state = (command.state or DriveState.stopped()).sanitized()
pedals = (state.throttle, state.brake)
if claiming:
Expand All @@ -201,17 +194,23 @@ def apply(self, command: Command) -> bool:
# latches and brings the truck to a stop, but it must never transfer
# the driving lease to its sender. Otherwise a bystander could
# e-stop, then reset_estop, and drive off with a lease they never
# legitimately claimed.
# legitimately claimed. Only refresh the acknowledgement sequence
# when the sender already holds the lease, so a bystander's e-stop
# cannot masquerade as the authoritative command either.
if self.holds(command):
self._sequence = command.sequence
self._state = DriveState.stopped()
self._estopped = True
if not was_estopped and command.session is not None:
self._estop_by = handle(command.session)
if not was_estopped:
self._estop_by = command.driver_id

elif command.kind == "reset_estop":
if not self._estopped:
return False
# Reset clears the global latch without claiming or transferring the
# lease.
# lease. Only the holder's sequence is authoritative for that lease.
if self.holds(command):
self._sequence = command.sequence
self._state = DriveState.stopped()
self._estopped = False
self._estop_by = None
Expand All @@ -235,6 +234,7 @@ def _can_shift(state: DriveState) -> bool:

def _clear_driver(self) -> None:
self._driver = None
self._session = None
self._instance = None
self._sequence = None
self._state = DriveState.stopped()
self._takeover_pedals = None
25 changes: 12 additions & 13 deletions demo/carla/sim/moq_carla_sim/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,6 @@
# A local development relay, which needs no token.
DEFAULT_URL = "http://localhost:4443"

# Drivers dial each truck's socket endpoint, which only these versions carry:
# an endpoint is a hidden broadcast, and moq-lite before 07 hides it from every
# peer. The web player offers the same list.
VERSIONS = ("moq-lite-07-wip", "moq-transport-22")


def default_trucks() -> list[str]:
trucks = json.loads(TRUCKS_FILE.read_text())
Expand Down Expand Up @@ -115,6 +110,11 @@ def broadcast_path(truck: str, camera: str) -> str:
return f"carla/{truck}/{camera}.hang"


def driver_prefix(truck: str) -> str:
"""Where a truck's drivers publish, each at `{prefix}/{driverId}`."""
return f"viewer/carla/{truck}"


# The one task allowed to end the run without failing it.
SHUTDOWN = "shutdown"

Expand Down Expand Up @@ -238,7 +238,7 @@ async def run(args: argparse.Namespace) -> None:
# shares the connection.
events = asyncio.create_task(bridge.run(), name="bridge")
try:
async with moq.connect(args.url, versions=VERSIONS) as client:
async with moq.connect(args.url) as client:
try:
for truck in world.trucks:
route = client.route(broadcast_path(truck.name, args.camera), hidden=True)
Expand All @@ -248,17 +248,16 @@ async def run(args: argparse.Namespace) -> None:
session = TruckSession(truck, route, video, args.location, map_name, world.clock)
sessions.append(session)
log.info(
"publishing %s, accepting drivers at %s",
"publishing %s, consuming %s/*",
broadcast_path(truck.name, args.camera),
session.drivers.path,
driver_prefix(truck.name),
)

relay = client.session
if relay is None:
raise RuntimeError("relay session is not connected")
origin = relay.publish()
tasks = [
asyncio.create_task(session.run(origin), name=f"truck:{truck.name}")
asyncio.create_task(
session.run(client, driver_prefix(truck.name)),
name=f"truck:{truck.name}",
)
for session, truck in zip(sessions, world.trucks)
]

Expand Down
Loading