Skip to content
Merged
81 changes: 54 additions & 27 deletions demo/carla/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@
Drive simulated trucks from a browser over MoQ. Trucks share one
[CARLA](https://carla.org) world and drive themselves until someone takes
control. Each truck publishes its cameras and a status track; a web player
watches them on a map and publishes a steering **control** track back. Control
follows the [moq-boy](../boy) pattern: the viewer publishes, the truck
subscribes, and the truck echoes what it accepted on `status`.
watches them on a map, and a driver steers one by dialing the truck's
[socket](../../doc/concept/socket.md) endpoint: its controls and the truck's
acknowledgements travel on a session only the two of them read, while what
every viewer shows stays on `status`.

```text
├── web/ # the player: the <moq-carla> element, with map, video, keyboard and steering-wheel input (@moq/demo-carla)
Expand Down Expand Up @@ -44,24 +45,29 @@ The player alone runs against any relay with `just carla web <url>`. Append
`?preview` to the local URL for a fixture-driven layout review that opens no
connection; it is removed from production builds.

## Broadcasts
## Broadcasts and sessions

```text
carla/{truck}/{camera}.hang <- one media broadcast per camera (hang catalog)
sd 240p H.264, for map previews
hd 720p H.264, for the focused view
status JSON Snapshot telemetry

viewer/carla/{truck}/{driverId} <- one broadcast per driver
control JSON Snapshot commands
carla/.socket/{truck} <- the truck's socket endpoint, which drivers dial
carla/.socket/driver-{id} <- one endpoint per driver lease
```

The worker consumes `viewer/carla/{truck}/` for each truck, fanning in every
driver's `control` track. Cameras are sibling broadcasts, so switching views
never creates a second claim on the same wheel, and the driver scope is
disjoint from every media path.
Each truck accepts `track` sessions on its endpoint. A driver dials from an
endpoint of its own, one session per lease, and sends one control per message;
the truck answers each control it accepted from the lease holder on the same
session. Cameras are sibling broadcasts, so switching views never creates a
second claim on the same wheel.

### Control (player to truck)
Sockets need moq-lite-07 or moq-transport on every hop, so the worker and the
player offer only `moq-lite-07-wip` and `moq-transport-22`; a relay must
accept one of them.

### Control (player to truck, on the driver's session)

```json
{ "type": "drive", "sequence": 42, "steering": -1.0..1.0,
Expand All @@ -72,13 +78,27 @@ disjoint from every media path.
{ "type": "respawn", "sequence": 45, "timestamps": [...] }
```

### Status (truck to player)
The truck reads them oldest first, skipping one that falls a heartbeat (250 ms)
behind the newest, so a late control never holds up a newer one, while a burst,
such as a reset right before a drive, still arrives whole.

### Acknowledgement (truck to player, on the driver's session)

```json
{ "sequence": 44, "media_pipeline": [ { "label": "received→decoded", "ms": 12 } ] }
```

Sent for each `drive`, `estop`, and `reset_estop` the truck accepted from the
lease holder, so the player measures its round trip and its own media pipeline.
A bystander's e-stop, a command the gate refused, or an accepted `respawn`
(confirmed from `status` instead) gets none.

### Status (truck to every viewer)

```json
{ "accepted": { "steering": .., "throttle": .., "brake": .., "gear": ".." },
"autonomy": "autonomous|manual", "estopped": true,
"driver": "ab12cd", "estop_by": "ef34gh", "sequence": 44,
"media_pipeline": { "ab12cd": [ { "label": "received→decoded", "ms": 12 } ] },
"driver": "driver-ab12cd", "estop_by": "driver-ef34gh",
"map": "Town10HD_Opt",
"pose": { "x": 12.35, "y": -78.9, "yaw": 180.1, "speed": 9.88 },
"measured": { "steering": .., "throttle": .., "brake": .., "gear": ".." },
Expand All @@ -104,13 +124,14 @@ collision.
The worker's gate (`sim/moq_carla_sim/drive.py`) owns every truck's controls:

- **One driver at a time.** The first driver to send a command holds the
lease until its broadcast ends or its 1 s watchdog expires; the player
heartbeats every 250 ms. The lease is bound to the driver's path and the
publisher instance serving it, so a driver that reconnects at the same
path is a new claimant rather than a continuation.
- **E-stop latches** until an explicit reset, from any viewer: the player
latches and resets it through a short-lived safety broadcast without taking
the lease. A latched truck nobody drives is also cleared when it respawns
lease until its session ends or its 1 s watchdog expires; the player
heartbeats every 250 ms. The lease is bound to the driver's session, so a
driver that redials, even from the same endpoint id, is a new claimant
rather than a continuation.
- **E-stop latches** until an explicit reset, from any viewer. The lease
holder sends it on its session; any other viewer dials a short-lived session
of its own, so a bystander's stop neither takes the lease nor waits behind
the holder's controls. A latched truck nobody drives is also cleared when it respawns
as wedged, below.
- **Gear changes are interlocked**: the gate keeps the current gear unless the
command releases throttle and applies some brake.
Expand All @@ -128,10 +149,15 @@ The worker's gate (`sim/moq_carla_sim/drive.py`) owns every truck's controls:
e-stop latched, that has not moved 2 m in 5 minutes of a running world
returns to a free spawn point under autonomy, its e-stop cleared.

The lease is cooperative. Anyone who can publish under `viewer/carla/` can
announce a driver path, so it stops two honest drivers fighting over a truck,
not a deliberate takeover. Anything beyond a simulator needs per-driver tokens
scoped to the driver's own path.
The lease is cooperative. Anonymous drivers share one grant, publish
`carla/.socket/driver-*`, so anyone can announce an endpoint under the id that
`status` names as the holder. That never continues the holder's lease, since a
lease is one session, but it can disrupt it: the relay may route the truck's
accept of the holder's next session to the impostor. Truck names must not look
like driver ids, or the same grant would let anyone impersonate a truck; the
worker refuses one. So the lease stops two honest drivers fighting over a
truck, not a deliberate disruption. Anything beyond a simulator needs
per-driver tokens scoped to the driver's own endpoint.

## The simulator

Expand Down Expand Up @@ -193,8 +219,9 @@ checkout.
just carla venv
just carla service-install

# The units read this. MOQ_URL carries a token allowed to publish `carla/**`
# and subscribe `viewer/carla/**`; keeping it here keeps it off the command
# The units read this. MOQ_URL carries a token allowed to publish `carla/**`,
# which covers the trucks' endpoints, and subscribe `carla/.socket/driver-*`,
# the endpoints drivers dial from; keeping it here keeps it off the command
# line. CARLA_IMAGE is the tag the moq-dev/carla build printed, since the worker
# refuses a stock simulator.
cat >~/.config/moq/carla.env <<'EOF'
Expand Down
53 changes: 21 additions & 32 deletions demo/carla/sim/moq_carla_sim/drive.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,15 +94,16 @@ class Command:
`drive`, `estop`, `reset_estop`, and `respawn` arrive on the wire;
`driver_left` and `watchdog_expired` are synthesized locally.

`driver_id` is the driver's path segment, which `status` echoes. `instance`
names the run that sent it, one per publisher instance serving the path: the
same id under another instance is another run of the driver, so it is a
different driver for the lease.
`driver_id` is the driver's socket endpoint id, which `status` echoes.
`session` is the socket session that carried it, one per lease the player
takes: the same id on another session is a different driver for the lease.
The session id is the secret that keeps a session to its two ends, so it
never leaves this process.
"""

kind: CommandKind
driver_id: str = ""
instance: str | None = None
session: str | None = None
sequence: int = 0
state: DriveState | None = None
timestamps: tuple[TimestampEntry, ...] = ()
Expand All @@ -114,19 +115,19 @@ class Drive:
The first driver to send a command owns the truck until it disconnects or
its command lease expires. Other drivers cannot silently steal control.

The lease binds to the driver's broadcast path and the run serving it, so a
driver whose broadcast ends or restarts loses it.
It is still cooperative, not enforced: the holder's id is echoed on the
public `status` track, so anyone allowed to publish there can announce at
that path under another instance, which ends the holder's broadcast and frees
the lease for the newcomer. This stops two honest drivers colliding; it
does not stop a deliberate takeover.
The lease binds to the driver's socket session, so a driver whose session
ends loses it, and another session claiming the same driver id is another
driver. It is still cooperative, not enforced: anonymous drivers share one
grant, so anyone can announce an endpoint under the holder's id, which the
public `status` track echoes. Their own sessions never continue the
holder's lease, but they can disrupt it: a relay may route the truck's
accept of the holder's next session to the impostor instead. This stops two
honest drivers colliding; it does not stop a deliberate disruption.
"""

def __init__(self) -> None:
self._driver: str | None = None
self._instance: str | None = None
self._sequence: int | None = None
self._session: str | None = None
self._state = DriveState.stopped()
self._takeover_pedals: tuple[float, float] | None = None
self._estopped = False
Expand All @@ -136,10 +137,6 @@ def __init__(self) -> None:
def driver(self) -> str | None:
return self._driver

@property
def sequence(self) -> int | None:
return self._sequence

@property
def state(self) -> DriveState:
return self._state
Expand All @@ -153,8 +150,8 @@ def estop_by(self) -> str | None:
return self._estop_by

def holds(self, command: Command) -> bool:
"""Whether `command` comes from the run of the driver holding the lease."""
return self._driver is not None and (self._driver, self._instance) == (command.driver_id, command.instance)
"""Whether `command` comes from the session holding the lease."""
return self._driver is not None and (self._driver, self._session) == (command.driver_id, command.session)

def accepted(self) -> Accepted:
return Accepted(state=self._state, estopped=self._estopped)
Expand All @@ -166,8 +163,7 @@ def apply(self, command: Command) -> bool:
return False
claiming = self._driver is None
self._driver = command.driver_id
self._instance = command.instance
self._sequence = command.sequence
self._session = command.session
state = (command.state or DriveState.stopped()).sanitized()
pedals = (state.throttle, state.brake)
if claiming:
Expand All @@ -188,11 +184,7 @@ def apply(self, command: Command) -> bool:
# latches and brings the truck to a stop, but it must never transfer
# the driving lease to its sender. Otherwise a bystander could
# e-stop, then reset_estop, and drive off with a lease they never
# legitimately claimed. Only refresh the acknowledgement sequence
# when the sender already holds the lease, so a bystander's e-stop
# cannot masquerade as the authoritative command either.
if self.holds(command):
self._sequence = command.sequence
# legitimately claimed.
self._state = DriveState.stopped()
self._estopped = True
if not was_estopped:
Expand All @@ -202,9 +194,7 @@ def apply(self, command: Command) -> bool:
if not self._estopped:
return False
# Reset clears the global latch without claiming or transferring the
# lease. Only the holder's sequence is authoritative for that lease.
if self.holds(command):
self._sequence = command.sequence
# lease.
self._state = DriveState.stopped()
self._estopped = False
self._estop_by = None
Expand All @@ -228,7 +218,6 @@ def _can_shift(state: DriveState) -> bool:

def _clear_driver(self) -> None:
self._driver = None
self._instance = None
self._sequence = None
self._session = None
self._state = DriveState.stopped()
self._takeover_pedals = None
32 changes: 20 additions & 12 deletions demo/carla/sim/moq_carla_sim/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,15 @@
# A local development relay, which needs no token.
DEFAULT_URL = "http://localhost:4443"

# Drivers dial each truck's socket endpoint, which only these versions carry:
# an endpoint is a hidden broadcast, and moq-lite before 07 hides it from every
# peer. The web player offers the same list.
VERSIONS = ("moq-lite-07-wip", "moq-transport-22")

# The id form anonymous drivers may publish endpoints under. A truck named like
# one could be impersonated by any of them.
DRIVER_ID_PREFIX = "driver-"


def default_trucks() -> list[str]:
trucks = json.loads(TRUCKS_FILE.read_text())
Expand Down Expand Up @@ -80,6 +89,9 @@ def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
args = parser.parse_args(argv)
if not args.trucks:
args.trucks = default_trucks()
for truck in args.trucks:
if truck.startswith(DRIVER_ID_PREFIX):
parser.error(f"truck {truck!r} is named like a driver, whose endpoints anyone may publish")
return args


Expand Down Expand Up @@ -110,11 +122,6 @@ def broadcast_path(truck: str, camera: str) -> str:
return f"carla/{truck}/{camera}.hang"


def driver_prefix(truck: str) -> str:
"""Where a truck's drivers publish, each at `{prefix}/{driverId}`."""
return f"viewer/carla/{truck}"


# The one task allowed to end the run without failing it.
SHUTDOWN = "shutdown"

Expand Down Expand Up @@ -238,7 +245,7 @@ async def run(args: argparse.Namespace) -> None:
# shares the connection.
events = asyncio.create_task(bridge.run(), name="bridge")
try:
async with moq.connect(args.url) as client:
async with moq.connect(args.url, versions=VERSIONS) as client:
try:
for truck in world.trucks:
route = client.route(broadcast_path(truck.name, args.camera), hidden=True)
Expand All @@ -248,16 +255,17 @@ async def run(args: argparse.Namespace) -> None:
session = TruckSession(truck, route, video, args.location, map_name, world.clock)
sessions.append(session)
log.info(
"publishing %s, consuming %s/*",
"publishing %s, accepting drivers at %s",
broadcast_path(truck.name, args.camera),
driver_prefix(truck.name),
session.drivers.path,
)

relay = client.session
if relay is None:
raise RuntimeError("relay session is not connected")
origin = relay.publish()
tasks = [
asyncio.create_task(
session.run(client, driver_prefix(truck.name)),
name=f"truck:{truck.name}",
)
asyncio.create_task(session.run(origin), name=f"truck:{truck.name}")
for session, truck in zip(sessions, world.trucks)
]

Expand Down
Loading
Loading