Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Build a capability
description: Write your own reusable Lifecycle capability with LifecycleCapability, including storage, durable jobs, request handling, events, and user callbacks.
pcx_content_type: how-to
sidebar:
order: 7
order: 10
products:
- agents
---
Expand Down
16 changes: 8 additions & 8 deletions src/content/docs/agents/runtime/capabilities/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -128,14 +128,14 @@ These capabilities ship with the `agents` package and are stable:

These capabilities are experimental. Their APIs may change between releases:

| Capability | Import | What it does |
| ---------------------------------------------------- | ----------------- | ------------------------------------------------------------------- |
| [`Streams`](/agents/runtime/capabilities/streams/) | `agents/streams` | Stores incremental output that readers replay and follow |
| [`Sessions`](/agents/runtime/capabilities/sessions/) | `agents/sessions` | Stores conversation history as a tree of messages |
| `Queue` | `agents/queue` | Runs background callbacks one at a time, in push order |
| `Tasks` | `agents/tasks` | Runs replayable background work with journaled steps |
| `RoutedAgents` | `agents/routing` | Keeps a catalog of independent agents and forwards requests to them |
| `Browser` | `agents/browser` | Keeps one persistent Browser Run session open for the agent |
| Capability | Import | What it does |
| ------------------------------------------------------------- | ----------------- | ------------------------------------------------------------------- |
| [`Streams`](/agents/runtime/capabilities/streams/) | `agents/streams` | Stores incremental output that readers replay and follow |
| [`Sessions`](/agents/runtime/capabilities/sessions/) | `agents/sessions` | Stores conversation history as a tree of messages |
| [`Queue`](/agents/runtime/capabilities/queue/) | `agents/queue` | Runs background callbacks one at a time, in push order |
| [`Tasks`](/agents/runtime/capabilities/tasks/) | `agents/tasks` | Runs multi-step background work that resumes where it stopped |
| [`RoutedAgents`](/agents/runtime/capabilities/routed-agents/) | `agents/routing` | Keeps a catalog of independent agents and forwards requests to them |
| [`Browser`](/agents/tools/browser/) | `agents/browser` | Keeps one persistent Browser Run session open for the agent |

A [harness](/agents/harnesses/), such as `PiHarness`, is a capability that runs a whole agent loop.

Expand Down
134 changes: 134 additions & 0 deletions src/content/docs/agents/runtime/capabilities/queue.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
---
title: Queue
description: Run background work after a request returns with the Queue capability. Items run one at a time in push order, retry on failure, and survive restarts.
pcx_content_type: reference
sidebar:
order: 7
badge:
text: Experimental
products:
- agents
---

import { Render, TypeScriptExample } from "~/components";

<Render
file="experimental"
product="agents"
params={{ name: "Queue", module: "agents/queue" }}
/>

`Queue` lets your agent answer a request right away and do the slow part afterwards. Push an item, return a response, and the queue runs the item's callback in the background. If the callback throws, the queue retries it. If the Durable Object restarts before an item runs, the item is still there and runs when the object wakes.

`Queue` from `agents/queue` stores each item as a callback name and a payload. Items run one at a time, in the order you pushed them. `Queue` is not [Cloudflare Queues](/queues/). It does not need a binding, and its items stay inside one Durable Object.

## Install the capability

Register callbacks by name, then install the queue on the [Lifecycle](/agents/runtime/lifecycle/):

<TypeScriptExample filename="src/index.ts">

```ts
import { DurableObject } from "cloudflare:workers";
import { getCurrentAgent, Lifecycle } from "agents/lifecycle";
import { Queue, type QueueItem } from "agents/queue";

type Upload = { fileName: string; bytes: number };

export class UploadAgent extends DurableObject<Env> {
readonly queue = new Queue({
callbacks: {
processUpload: (payload: Upload, item: QueueItem<Upload>) => {
const { agent } = getCurrentAgent<UploadAgent>();
console.log(
`${agent?.lifecycle.name}: processed ${payload.fileName} (${item.id})`,
);
},
},
retry: { maxAttempts: 5, baseDelayMs: 500, maxDelayMs: 10_000 },
onError: (error) => console.error("Upload failed", error),
});
readonly lifecycle = Lifecycle.install(this).use(this.queue);

async onRequest(request: Request): Promise<Response> {
const upload = await request.json<Upload>();
const item = await this.queue.push("processUpload", upload);
return Response.json({ id: item.id }, { status: 202 });
}
}
```

</TypeScriptExample>

`push()` returns once the item is stored, so the request does not wait for the callback. The callback runs later, in its own invocation. It does not see the request or the connection that pushed the item, but it runs inside your object's context, so [`getCurrentAgent()`](/agents/runtime/lifecycle/get-current-agent/) returns the object.

Each item stores the callback's name, not the function. Register every callback in the field initializer, so an item pushed before a restart finds its callback after the object wakes. `push()` checks the callback name and the payload type against the map.

## Options

| Option | Description |
| ----------- | ------------------------------------------------------------------------------------------------------------------ |
| `callbacks` | Named callbacks `(payload, item) => unknown` |
| `retry` | Default retry policy for callbacks: `{ maxAttempts, baseDelayMs, maxDelayMs }`. Defaults to `3`, `100`, and `3000` |
| `onError` | `(error) => void`. Observes an item's failure after its last attempt |

## Push items

`push(callback, payload?, options?)` stores one item and returns it:

```ts
await this.queue.push("processUpload", upload, {
id: `upload:${upload.fileName}`,
retry: { maxAttempts: 1 },
});
```

| Option | Description |
| ------- | ----------------------------------------------------------------------------------------------------- |
| `id` | A stable ID. Pushing an ID that is already queued replaces that item and keeps its place in the queue |
| `retry` | Overrides the queue's retry policy for this item |

Use `id` when the same work might be pushed twice, such as a retried upload request. The second push replaces the first instead of queueing a duplicate.

Once the Lifecycle has started, `push()` writes the item before it returns its promise. A push in the same synchronous block as your own SQL writes commits with them, so the item and your data are stored together or not at all.

## Manage items

| Method | Description |
| ---------------------- | ----------------------------------------------------------------------------- |
| `get(id)` | Returns one pending item, or `undefined` |
| `list(criteria?)` | Returns pending items in push order. Filter by callback with `{ callback }` |
| `cancel(id)` | Removes a pending item. Returns `true` if one was removed |
| `cancelAll(callback?)` | Removes every pending item, or every item for one callback. Returns the count |

A `QueueItem` has these fields:

| Field | Description |
| ----------- | -------------------------------------------------------- |
| `id` | The item's ID |
| `callback` | The name of the callback that runs the item |
| `payload` | The data passed to the callback |
| `createdAt` | When the item was pushed, as a Unix timestamp in seconds |
| `retry` | The retry policy for this item, if one is set |

## Failures and retries

A callback that throws is retried with exponential backoff, up to `maxAttempts`. After the last attempt, the queue drops the item and calls `onError`. There is no dead-letter queue, so record anything you need to keep inside `onError` or the callback.

If the object is evicted while a callback runs, the Lifecycle wakes it again and the callback runs again. Make external effects idempotent.

## Limits

- Items run one at a time, not in parallel. A slow callback delays every other item, and every other [job](/agents/runtime/lifecycle/job-queue/) on the object. Keep callbacks short.
- There is no priority. Items run in push order.
- Payloads must be JSON-serializable.

For a callback at a set time, on a cron, or on an interval, use [`Scheduler`](/agents/runtime/capabilities/scheduler/). For work with several steps that should resume where it stopped, use [`Tasks`](/agents/runtime/capabilities/tasks/).

## Observability

The queue publishes `queue:create`, `queue:retry`, and `queue:error` events to the `agents:schedule` [diagnostics channel](/agents/runtime/observability/diagnostics-channels/#schedule-and-queue-events).

## On the Agent class

The [`Agent` class](/agents/agent-class/) installs a `Queue`. Its `this.queue()`, `this.dequeue()`, and `this.getQueue()` methods call it, and callback names resolve to methods on the Agent. Those methods are stable. For more information, refer to [Queue tasks](/agents/agent-class/queue-tasks/).
132 changes: 132 additions & 0 deletions src/content/docs/agents/runtime/capabilities/routed-agents.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
---
title: Routed agents
description: Give each user an open-ended set of agents, such as one per chat, with the RoutedAgents capability. A hub Durable Object keeps the list and forwards requests and WebSockets to each agent.
pcx_content_type: reference
sidebar:
order: 9
badge:
text: Experimental
products:
- agents
---

import { Render, TypeScriptExample } from "~/components";

<Render
file="experimental"
product="agents"
params={{ name: "RoutedAgents", module: "agents/routing" }}
/>

Many applications give each user a growing set of agents, such as one per chat, document, or task. `RoutedAgents` lets one hub Durable Object per user own that set. The hub creates agents, lists them for a sidebar, and deletes them. Clients reach each agent through the hub's URL, so the hub decides which agents a user can reach.

Each routed agent is a separate Durable Object, with its own storage, alarm, and WebSocket connections. The hub keeps a list of entries, each with an ID and your own metadata, such as a chat title. Listing, renaming, and deleting entries only touch the hub, so a user with hundreds of chats does not wake hundreds of agents to draw a sidebar.

`RoutedAgents` is exported from `agents/routing`, alongside [`routeAgentRequest()`](/agents/runtime/routing/).

## Install the capability

The hub can be a plain Durable Object. The routed agents must extend the [`Agent` class](/agents/agent-class/), which includes [Think](/agents/harnesses/think/) and `AIChatAgent` agents, because deleting an entry uses the `Agent` class's teardown to delete the agent's storage.

This hub keeps one `ChatAgent` per chat. `ChatAgent` is defined in another module:

<TypeScriptExample filename="src/index.ts">

```ts
import { DurableObject } from "cloudflare:workers";
import { Lifecycle } from "agents/lifecycle";
import { RoutedAgents, routeAgentRequest } from "agents/routing";
import type { ChatAgent } from "./chat-agent";

type ChatMetadata = { title: string };

export class UserHub extends DurableObject<Env> {
readonly chats = new RoutedAgents<ChatAgent, ChatMetadata>({
namespace: this.env.ChatAgent,
route: "chats",
});
readonly lifecycle = Lifecycle.install(this).use(this.chats);

async onRequest(request: Request): Promise<Response> {
if (request.method === "POST") {
const { title } = await request.json<ChatMetadata>();
const entry = await this.chats.create({ metadata: { title } });
return Response.json(entry, { status: 201 });
}
return Response.json(await this.chats.list());
}
}

export { ChatAgent } from "./chat-agent";

export default {
async fetch(request: Request, env: Env): Promise<Response> {
return (
(await routeAgentRequest(request, env)) ??
new Response("Not found", { status: 404 })
);
},
} satisfies ExportedHandler<Env>;
```

</TypeScriptExample>

| Option | Description |
| ----------- | ------------------------------------------------------------------------------------------- |
| `namespace` | The Durable Object binding for the routed agents. It can point at a class in another Worker |
| `route` | One URL-safe path segment that the capability claims, such as `"chats"` |

Install `RoutedAgents` before [`WebSockets`](/agents/runtime/capabilities/websockets/). `WebSockets` accepts every WebSocket upgrade, so an upgrade meant for a routed agent must reach `RoutedAgents` first:

```ts
readonly lifecycle = Lifecycle.install(this).use(this.chats).use(this.webSockets);
```

## Route requests

`routeAgentRequest()` sends requests to the hub. The hub forwards requests under `/{route}/{id}` to the routed agent behind that entry, and keeps the rest of the path:

```txt
/agents/user-hub/alice -> UserHub "alice"
/agents/user-hub/alice/chats/{id} -> the ChatAgent behind entry {id}
/agents/user-hub/alice/chats/{id}/history -> the same ChatAgent, which sees /history
```

An unknown or deleted ID returns `404`, and the hub's `onRequest()` never sees it. Authenticate the user before you route the request, so one user cannot reach another user's hub. For more information, refer to [Authentication](/agents/runtime/routing/authentication/).

The routed agent answers a forwarded WebSocket upgrade and then owns the socket, so chat messages do not wake the hub. Connect to a routed agent from React with `basePath`:

```tsx
const chat = useAgent({
agent: "ChatAgent",
basePath: `agents/user-hub/alice/chats/${encodeURIComponent(chatId)}`,
});
```

Choose a `route` that cannot appear anywhere else in the hub's paths, including the hub's own name. The capability matches the segment anywhere in the path, and a match with no entry behind it returns `404` instead of reaching your `onRequest()`.

## Manage entries

| Method | Description |
| --------------------------- | ---------------------------------------------------------------------------------------------- |
| `create({ metadata? })` | Creates an entry without waking the routed agent. Returns the entry |
| `get(id)` | Returns a typed stub for RPC to the routed agent, or `null` for an unknown or deleted entry |
| `list()` | Returns entries, most recently updated first |
| `setMetadata(id, metadata)` | Replaces an entry's metadata. Returns `false` for an unknown entry |
| `delete(id)` | Removes the entry and deletes the routed agent's storage. Returns `false` for an unknown entry |

An entry has an `id`, your `metadata`, and `createdAt` and `updatedAt` times in Unix milliseconds. Clients only see entry IDs. The hub gives each routed agent a random Durable Object name that never leaves the hub, so a client cannot guess its way to another user's agent.

Keep conversation data in the routed agent, and keep what a list or search needs, such as a title or a last message, in the entry's metadata. To keep the metadata current, have the routed agent call an RPC method on its hub that calls `setMetadata()`.

## Delete entries

`delete(id)` hides the entry first, then tells the routed agent to delete itself. The routed agent deletes its storage the next time it wakes. If the call fails partway, the entry stays hidden, and calling `delete(id)` again finishes the job.

To delete the hub itself, call `await this.lifecycle.dispose()` before `this.ctx.storage.deleteAll()`. Disposal tells every remaining routed agent to delete itself, retrying each one a few times. A routed agent that is still unreachable after those retries is left behind, and the hub no longer has an entry to retry from.

## On the Agent class

An `Agent` can be the hub too. Construct `RoutedAgents` as a field and install it with `this.lifecycle.use()` in the constructor. `this.destroy()` disposes the capability before it deletes the hub's storage. For routing on the `Agent` class, refer to [Routing](/agents/agent-class/routing/).

The [routing example](https://github.com/cloudflare/agents/tree/main/examples/next/routing) runs one `ChatAgent` per chat behind a plain Durable Object hub, with `WebSockets` serving the hub's own methods.
2 changes: 2 additions & 0 deletions src/content/docs/agents/runtime/capabilities/sessions.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ description: Store durable conversation history in a Durable Object with the Ses
pcx_content_type: reference
sidebar:
order: 6
badge:
text: Experimental
products:
- agents
---
Expand Down
2 changes: 2 additions & 0 deletions src/content/docs/agents/runtime/capabilities/streams.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ description: Store incremental output, such as model tokens, in a durable chunk
pcx_content_type: reference
sidebar:
order: 5
badge:
text: Experimental
products:
- agents
---
Expand Down
Loading
Loading